Resource
Executive Summary
MagellanTV Uses Generative AI for International Expansion
In an effort to expand their global reach, MagellanTV needed to internationalize its content, including foreign-language captioning and foreign-language voice overdubbing. The company enlisted Mission Cloud to leverage the AWS Migration Acceleration Program (MAP) and AWS-native AI services so it could automate the existing manual process of creating and reviewing captions, and add the capability to automatically translate content for dubbing in any language desired.
Mission Cloud conducted an in-depth assessment of the customer’s existing environment and workflow. The company then developed a target-state architecture diagram to upload the data, used AWS services such as Transcribe, Translate and Polly, and integrated with a large language model and Amazon SageMaker for updated and translated files.
Mission Cloud’s solution leverages MAP to automate and accelerate execution. AWS funding was secured for both the assessment and implementation phases of the project. Mission Cloud's expertise in generative AI enables MagellanTV to continue delivering high-quality, accessible content to a broader audience.
How MagellanTV Used Generative AI for International Expansion
Background: Creating Content for an International Audience
MagellanTV’s existing offering focused on English-language content, but the company wanted to also provide documentaries in other foreign languages, such as Spanish and Mandarin to better serve its growing customer base and attract international audiences. MagellanTV’s initial expansion focused on content delivery to Mexico and Taiwan. Because MagellanTV already had access to a massive library, it started internationalizing content, including foreign-language captioning and, eventually, foreign-language voice overdubbing.
AWS initially referred MagellanTV to Mission Cloud for Cloud Foundation services to help the company optimize costs. When Mission Cloud realized that MagellanTV wanted to create an in-house captioning and dubbing solution, it quickly demonstrated fluency in the space, including in-depth generative AI expertise and similar project successes. With the client confidently on board, Mission Cloud began working on the initial phase of the project — the assessment.
Challenge: Leveraging Native AWS AI Services to Automate Content Updates
At the time, MagellanTV’s workflow required partnering with a third-party service, Papercup, to translate and transcribe text. This incurred per-minute charges across thousands of hours of video content, making it a high-cost, unsustainable method.
The company was also using an in-house video editor to do manual editing work. Not only was this process expensive, it was incredibly time-consuming, potentially taking up to 60 hours to dub one hour of content and consuming about 70% of the video editor’s time. MagellanTV needed a way to automate its pipeline to translate, transcribe and dub content, with minimal human-in-the-loop effort.
Designing an effective solution would involve an in-depth assessment of how well AWS-managed AI services performed on video dubbing (using Polly), translation (using Translate) and transcription (using Transcribe). Mission Cloud also needed to identify challenges, possible solutions, workflows and scope for the mobile phase of the solution.
Solution: Conducting an In-Depth Environment and Workflow Assessment
Mission Cloud developed a solution that leveraged the AWS Migration Acceleration Program (MAP) to reduce costs while automating and accelerating execution. The priority for this first phase of the project was to assess MagellanTV’s existing environment and workflow. Mission Cloud’s work included understanding the client’s main pain points and the current process for transcribing, translating and dubbing.
One of the first steps taken was assessing native AWS services to determine whether they were suitable for the client’s intended purpose. One example was assessing the language limitations of Polly and the ability to adjust speed to match dubbing. A sentence might only be a few words in English but much longer when translated to Spanish, requiring a speed adjustment to accommodate the translation without interrupting the viewing experience. In these cases, the goal was to find a way to automatically update those lines through the pipeline.
In addition to the capability evaluation process, Mission Cloud determined the best way to use the large language model (LLM) to resolve MagellanTV’s challenges. Slang detection, for example, would spot English-language slang and colloquialisms and properly translate that content into Spanish.
Sample analyses revealed that even though content might include a correct caption file in English, the inclusion of slang terms and other informal English could reduce translation accuracy. Mission Cloud used the LLM for slang detection and replacement to demonstrate the capability of integrating the model with AWS native services.
Results: Improving Content Workflow for International Expansion
After the assessment, Mission Cloud developed a target-state architecture diagram. This indicates how to upload data, use Transcribe, Translate and Polly, and integrate with the LLM and SageMaker for updated and translated files. This diagram also identifies points of the workflow with challenges that will require future development time to resolve.
MagellanTV now has a well-designed diagram illustrating the future process it can leverage to translate and transcribe content once phase two is complete. The process developed by Mission Cloud is as follows:
- Uploading the original English video as an .mp4 file to Transcribe, which then generates an English transcription SRT file.
- Translate uses this file to generate an updated SRT file in the new language.
- A Python algorithm identifies lines that need to be adjusted.
- Polly’s speed-rate capability is used for adjusting the speed to accommodate length.
- At this point, some lines might still be too fast or sound unnatural. To address this, an LLM from Bedrock is used to summarize each identified sentence based on the length of the original English version and match the appropriate speed.
- Polly creates a synthetic voice and audio file of the translation.
- Finally, the new file is integrated with the original video and the background soundtrack.
AWS funded the assessment work as part of the MAP, delivering additional value for the client. The mobilize phase of MAP will also fund the second phase, implementation. After implementation, MagellanTV will leverage Cloud Foundation for cost optimization and best practices guidance. After the assessment, MagellanTV showed a strong interest in migrating its content delivery management system to AWS instead of using a third-party tool.
“We were immediately impressed with Mission Cloud’s expertise both on AWS and with generative AI,” said Thomas Lucas, Co-Founder and Director of MagellanTV. “Working with Mission has not only provided much needed AWS expertise to ensure our business is secure, reliably online and optimized, but also to build out a bleeding-edge machine learning pipeline on top of AWS cognitive services. Partnering with Mission Cloud will enable us to achieve a long-held goal for MagellanTV, which is to massively expand the reach of our documentary platform and strengthen our position as a truly global company.”
Ultimately, Mission Cloud developed a solution so MagellanTV could expand its international presence and audience while delivering high-quality, accessible content. The results of this project informed Mission Cloud about the capabilities and limitations of LLMs and how they can be used to guarantee results for customers.
AWS Services Used
During the engagement with Mission Cloud, MagellanTV leveraged multiple AWS services to take advantage of cost-saving opportunities and support.
- Generative AI
- AI21
- Anthropic's Claude 2.1
- Transcribe
- Translate
- Bedrock
- Comprehend
- Polly
- SageMaker
- Migration Acceleration Program
