JoVE Boosts Science Video Streaming Performance Via AWS Migration

As the business grew, the data center performance began to lag, risking the sustained future growth of the subscriber base. That was the situation JoVE faced. The company streams high-quality science education and laboratory experiment video articles to researchers, students, and teachers worldwide. Previously, JoVE relied on a co-location data center partner to host its video delivery application and database. The IT infrastructure also connected to a content delivery network to speed up video file provisioning.

“Mission generated a clear lockdown plan, which was key. We did not want any surprises and knew Mission could deliver on its plan. Mission further earned our trust by providing advice on how to set up the environment—even before we became a client,”

Chris Macdonald
Director of IT

Highlights

• Applied infrastructure-as-code to enable automatic deployment of networking, services and servers.
• Built a continuous integration/continuous deployment (CI/CD) pipeline to streamline application and database code updates.
• Ran load-balancing and auto-scaling to build servers on-demand and to handle activity spikes.
• Shut down server instances automatically when they aren’t needed.
• Monitored and stored logs to document transactions after server instances are wiped.
• Provided high-availability failover and system backups for disaster protection.
• Optimized costs of server and storage resources.

The Challenge

As JoVE expanded its subscriber base, the data center began to regularly see application performance issues—particularly when viewer usage spiked. Additionally, it took a long time for developers to deploy application updates, which were uploaded to three data centers globally—one at a time. Adding to the challenge, timely technical support was not consistently available.

“We knew we needed to make a change in how our IT infrastructure was set up and supported,” says Director of JoVE IT Chris Macdonald. “The level of instability we experienced was certain to hamper our future growth.”

After considering Google Cloud, JoVE decided to test the Amazon Web Services (AWS) cloud environment. JoVE wanted to see if it could deliver the required application performance and deployment speed. JoVE also hoped to work with experts to design (and provide responsive support for) the cloud infrastructure.

The Solution

JoVE first partnered directly with AWS to run a proof-of-concept (PoC) that would support a planned new release of the video delivery application. As the project unfolded, Macdonald realized that additional external experts were needed to efficiently migrate the legacy application and infrastructure from the co-located facility and into AWS.

“Our internal IT team focuses on application development and ensuring our application and database work well,” Macdonald says. “We knew we could benefit from the guidance, expertise and engineering resources of a leading AWS solution provider, so that our application could fully leverage the cloud’s benefits.”

After meeting with two AWS solution providers, JoVE chose to partner with Mission. JoVE saw Mission had demonstrable technical expertise for designing cloud architectures and in providing ongoing, fast-response support. Mission also showed JoVE how to optimize cloud resource usage and reduce IT costs. “Mission further earned our trust by providing advice on how to set up the environment—even before we became a client,” Macdonald adds.

The Implementation

Mission consulted with Macdonald and the internal JoVE IT team to produce a multi-phase strategic technology roadmap—one that would support migrating from the co-located data center to AWS. The project began with an infrastructure-as-code project, using Terraform to automatically deploy networking, services and servers in AWS.

Subsequent phases provided auto-scaling, high-availability failover, log monitoring, backups, cost optimization, and the launch of JoVE’s new video application on the AWS infrastructure. While configuring the AWS environment for the JoVE database, Mission built a MySQL open-source relational database server—and then replicated the database to AWS. Later, Mission secured the cloud infrastructure by deploying AWS Web Application Firewall.

In addition to building a United States-based environment at an AWS Virginia data center (US-East-1), Mission also set up an identical environment at an AWS data center in South Korea (Asia-Pacific-Seoul). This second data center delivers videos to Asian clients more efficiently than the previous co-location data center did.

JoVE previously loaded videos into one server and then mirrored the videos to other data centers. Now, videos are hosted in AWS S3. So, when subscribers must access a video via JoVE’s website, they can connect to Cloudflare, the content delivery network that Mission integrated with AWS. Cloudflare automatically accesses the AWS data center closest to the viewer.

“Mission also provides IT system admin services, which lightens the load on our internal IT team. They created a custom DevOps-managed service to help us maintain the AWS environment, which means we do not need to hire an extra person for our IT team.”

Chris Macdonald
Director of IT

The Results

• Ensured consistent application performance for subscribers accessing videos.
• Reduced IT costs by approximately 50%.
• Reduced the time needed for developers to update static pages from 45 minutes to two minutes.
• Synced changes to applications and databases automatically among the various data centers.
• Freed developers from having to update applications and databases in multiple data centers.

Previously, when application and database changes were required, JoVE successively updated the code base in each data center. But syncing video files between data centers was time consuming. To address this, Mission designed a CI/CD pipeline that automates application deployments and patch updates, all with the push of a button.

When JoVE updates the production codebase, the deployment happens immediately. The AWS environment automatically builds new servers from scratch in the Virginia data center; patches and configures the servers; and then creates Amazon Machine Images (AMI) that are then rolled out to both AWS regions. The automated process places servers into an auto-scaling group and launches the new instances.

When JoVE wants to update static content, developers can use the AWS Simple Storage Service (S3) static content solution (designed by Mission). The web servers sync with S3 every 10 minutes, and if there is a static page to update, they download content from the JoVE source control system via S3. This has cut down the deployment of static pages from 45 minutes to two minutes.

“With S3, we also have just one file system that goes to all the web servers, which reduces our storage costs,” Macdonald adds. “The faster deployment process is one of the key benefits we were looking for, as our videos increased to multiple terabytes of data, and caused deployments and downloads to take too long, depending on where users were located.”

Migrating the infrastructure from the co-location data center to the AWS environments has reduced the overall monthly IT spend for JoVE by approximately 50%. It also delivered superior performance to JoVE developers and clients—even when viewing volume spikes occur. Additionally, JoVE gains increased insight into the performance of the environment. This streamlines issue analysis, so IT doesn’t need to cycle through frequent break-fix processes.

Another key benefit comes from Mission showing JoVE how to start/stop server instances, and how to optimize large AWS Elastic Block Storage (EBS) volumes, which further reduce cloud costs. “These are both good examples of Mission’s knowledge of all AWS services and how to synchronize them to maximize performance and optimize costs,” Macdonald says. “They are a great compliment to a company like ours, with an internal team that focuses on development.”