Accelerating Data Processing 1,000x with ETL and QuickSight

Inefficient and Excel-based legacy processes for data collection, ingestion, and analysis were hindering how quickly this firm could make actionable decisions and develop new iterations of its time-sensitive pharmaceutical tests. The fast-scaling biotech company tapped Mission Cloud and its data, analytics, and machine learning (DAML) services to deliver AWS modernization that could provide significant and immediate improvements. Mission Cloud successfully transformed the biotech’s data and analytics infrastructure by eliminating previously-manual requirements, improving time-to-analytics from days to minutes, and giving staff a near-instant and 360-degree view of its test results utilizing Amazon Quicksight as a visualization engine. The existing process which once took weeks now only takes minutes, transforming the time-to-market and development pipeline.

The Challenge

The data from ongoing medical testing is the lifeblood of a life sciences company; in this case, the biotech firm had become hampered by an unsophisticated approach to its data collection and analysis. Data was being collected in Excel spreadsheets, which provided only a flat view. The process from data ingestion to analysis in order to make critical business decisions was also manual and slow. The biotech had started with a small on-premises IT footprint but quickly determined that a cloud-based solution would prove necessary as it scaled operations.

But the company did not have the existing data management and engineering resources to execute a data and analytics transformation in-house. It understood that doing so would require either adding more talent or a partnership with a cloud services provider that had the requisite AWS and data engineering expertise to integrate its tools, teams, and workflows into AWS and enable the company to scale with more efficiency.

To remedy this, the company sought to develop a superior cloud-centric system of delivering its testing data back to its team. The company’s datasets consist of both tabular data and multimedia files (such as videos of pill interactions within the body) that needed to be quickly ingested and accurately time-stamped with one another. And the company needed to pool the test results from various labs and simulators into a single place, and thus make an end-to-end data analysis procedure considerably less time consuming.
After its vetting process, the biotech chose AWS as its cloud provider because of the relative ease to get started on the platform, and because the company determined it would be the easiest to work with going forward as it scaled.

Why Mission Cloud

The AWS representative helping the biotech to adopt a cloud-native approach to data recommended Mission Cloud. As the biotech came to understand that AWS was going to be the right cloud platform, it also realized it could leverage an AWS partner to execute its vision with more confidence – and less cost – than attempting to build the entire solution internally. So the biotech chose to work with Mission Cloud because of the AWS partner’s deep experience with healthcare and life sciences organizations. Also important to the biotech was that Mission Cloud holds a large number of AWS Competencies, including the AWS Healthcare Consulting Competency and the AWS Life Sciences Consulting Competency, the QuickSight Service Delivery Designation, and has proven acumen working on projects where ensuring HIPAA-and PII-compliance is paramount.

The Solution

Mission Cloud worked with the biotech to identify the pain points within the existing analytical systems and data collection methodology. The solution that Mission Cloud designed and implemented is a data analytics infrastructure on AWS capable of supporting robust analysis without the need for writing extensive code, Excel formulas, or deploying or managing services. The Mission-built solution also incorporates automated ETL recipes.

Mission Cloud’s team of data experts implemented a serverless data and analytics workflow that allows the biotech to ingest the results from its testing with a single upload to S3. This then triggers automated processes for ETL and data enrichment. It also allows data to flow end-to-end to AWS QuickSight for visualization, and with a single click in QuickSight, newly arriving data is displayed in dashboards for all the company's users. The biotech also takes advantage of QuickSight email capabilities to distribute updates to all stakeholders and even regulators as required.

The Results

The biotech now has a full 360-degree view of its test results. Migrating from Excel and workbooks has taken the firm from a rudimentary data ingestion and analysis process to a streamlined, end-to-end workflow with minimal touchpoints. Now, it only needs to upload test data into a specific S3 bucket and its new process takes care of the rest.

The new process has saved the staff an immense amount of time. Staff can now have a test result uploaded, visualized, and analyzed in a matter of minutes with the transition to S3; this process had taken a week under its incumbent workflow with additional work required for each step.

These differences mean that the biotech can now iterate far faster on its experimentation and make data-backed chemical and engineering adjustments as research workstreams progress. Doing so meaningfully accelerates time-to-approval for new medications, as the data is now immediately available for the FDA as needed. In short, the new data ingesting and analysis structure enables the biotech to reduce the time-to-market across its wide range of therapies and meaningfully accelerate all new development efforts.

As part of every engagement, Mission Cloud ensures all solutions and infrastructure delivered are thoroughly documented so that the customer can understand, use, and modify the solution as needed as they continue to scale. In this instance, our documentation included the resources needed to enable the biotech's existing team to now add new data, new tables, and other functions as necessary.