Reality Check: When Generative AI Projects Hit a Roadblock

Generative AI’s potential is nearly limitless, but that freedom creates its own challenges. When everything seems possible, figuring out which paths are best for your company can seem impossible. Inevitably, you’ll go a ways down the road before realizing you need to turn around. This is a tale of taking the wrong path — and why initial setbacks don’t have to be the end of the road for your GenAI dreams.

Experimentation and innovation are key to unlocking GenAI’s potential, and the real opportunity is for companies that stay positive, learn from missteps, and enlist trusted partners to tackle thorny problems. Learn how we worked with a company to overcome a flawed GenAI project and deliver meaningful innovation. 

Don’t Get Ahead of Your GenAI Dreams

Before this customer came to us, they had sky-high GenAI ambitions. Specifically, they wanted to transform document processing, not just make it faster or more effective. The company envisioned a system smart enough to automatically navigate legal filings, insurance claims, and medical reports‌ — documents packed with crucial details yet incredibly cumbersome to handle manually.

Imagine being able to summarize and extract data from hundreds of pages of complex documents at lightning speed. That was the dream.

But with nascent technologies like GenAI, there can be a tendency to focus on the end result and overlook the process. For example, the company provided preprocessed data in JSON format, thinking it would make the project easier and more secure.

But converting data to JSON format wasn’t simply a swap; it was a material change in the information’s structure. Instead of a rich, multilayered cake (the original documents), the company created a smoothie (JSON). The GenAI model needed the original data’s full flavor and complexity, not a derivative.

Mission Cloud soon realized that relying on JSON extraction wasn’t going to work, much less transform the document processing experience. Mission Cloud recognized the need for an alternative text extraction and began work on a proof of concept.

The proof of concept was narrow in scope, focusing on certain medical visits that were part of the massive data set. This wasn’t a retreat; it was a smart recalibration. This approach helped the client see what proper processing looked like — and just how intricate automating document analysis with GenAI was.

The architecture involved several key AWS services, starting with Amazon S3 for persisting the documents. Amazon Textract was used for text extraction, while Amazon DynamoDB was employed to store both the interim processing steps and the configuration details. Mission Cloud also used Amazon Comprehend to classify the sub-documents within the larger dataset.

Amazon A2I was integrated into the architecture to address any documents that failed to meet extraction thresholds or required human-in-the-loop processing. AWS Lambda played a crucial role throughout the process, handling tasks such as pre- and post-processing of the extracted text. Amazon API Gateway facilitated the interaction between different components, and DynamoDB was used again for data storage.

The architecture also incorporated AWS CloudFormation for building and deploying the stack, ensuring a streamlined and reproducible environment. Finally, Amazon Bedrock and Claude 2 were leveraged for querying and refining the extracted text, while OpenSearch was used as the vector store.

The customer appreciated the detailed breakdown of the various AWS services involved, which helped them comprehend the scale and complexity of the undertaking. By taking a breath and offering proof of concept, Mission Cloud helped them avoid any further wasted effort while proving that GenAI wasn’t the problem. This seeming step back was actually a step forward, showing the power of GenAI and of properly working with data.

Turning Lessons Into Actionable Advice 

What can your company learn from this cautionary tale while still embracing the massive potential of GenAI with AWS? Here are several tips.

Embrace Raw Data

The company initially provided data in a preprocessed JSON format, hoping to streamline the work. This decision, while well-intentioned, proved to be a significant misstep. The preprocessing acted like a filter, diluting the crucial details and unique characteristics of the original documents.

Raw data contains the complexities, nuances, and contexts necessary for GenAI to properly analyze and draw conclusions. But without feeding original information to the AI model, the company couldn’t realistically move forward.

Assess Your Capabilities

The initial data-extraction process was difficult, indicating a gap in the customer’s internal capabilities. This scenario underscores the importance of having robust internal processes for data handling before integrating external GenAI solutions. A thorough internal assessment can help identify these gaps early, ensuring you're prepared to take advantage of AI’s capabilities.

Define Your Goals 

From the outset, the company had a broad goal to improve intelligent document processing with GenAI. However, as the project evolved, it became clear that a more defined and focused objective — like honing in on medical visit documentation — was the fastest path to practical application.

The company initially based all its efforts on a goal that didn’t reflect on-the-ground conditions. But with Mission Cloud’s help, including the proof of concept, a new strategy was created that was more realistic and quickly achieved tangible results.

Ensure Consistent Communication

Changing course mid-project is a stressful decision, and conflict is often the result. This project, however, successfully pivoted because of the constant, productive dialogue between the company and Mission Cloud.

Both parties had clear points of contact and a framework for managing changes, ensuring that everyone felt listened to, included, and in the loop throughout the project life cycle.

Set Realistic Project Scopes

The project’s initial ambition was to automate the analysis of an extensive range of documents, which was admirable but ultimately untenable. The revised scope focused on a manageable segment of the documents.

This adjustment didn’t signal failure; instead, it was a strategic realignment based on the complexities uncovered and what could feasibly be achieved. Setting a realistic scope requires hard conversations about what’s possible, but it can boost your team’s confidence and focus everyone’s effort on attaining achievable outcomes.

Stay Flexible and Adaptable

Mission Cloud’s pivot to a proof of concept for analyzing document segments exemplifies why you must be flexible during GenAI projects. Like any change management effort, you must be open to changing course based on project findings and technological capabilities.

The Importance of Trust and Transparency

For any company, persevering through challenges and collaborating with an external partner requires deep trust and open dialogue. A pivotal moment was when the customer gave Mission Cloud access to the raw, unprocessed data. This handover wasn't just about data formats‌ — ‌it was a testament to the company's trust in Mission Cloud's expertise with GenAI.

Having a partner with a proven track record in GenAI, like Mission Cloud, brings immediate advantages. Your company taps into deep knowledge and experience that drastically reduces the learning curve. Mission Cloud's insights into GenAI can help customers overcome immediate technical hurdles while keeping in mind the strategic adjustments necessary for long-term success.

This depth of expertise extends beyond technical knowledge. Mission Cloud brings a strategic lens to projects, identifying challenges and opportunities that might not be immediately apparent. In the case of this company's GenAI initiative, Mission Cloud didn't just troubleshoot; it realigned the project's focus toward more achievable and impactful goals. By having open dialogue about the project's direction, everyone became comfortable acknowledging the project’s flaws and making the hard decisions required to recalibrate and innovate.

As you continue to invest in GenAI to drive innovation and transformation, look at where a partner like Mission Cloud can offer technical solutions, strategic insight, and a sense of partnership built on principles of trust, transparency, and expertise.

Ready to elevate your generative AI experience? Connect with us to discover how Mission Cloud can help put you on a path to success.