Blue/Green Deployments on AWS
Introduction
Designing and building your application to run on AWS is a crucial step in successful deployment. It involves evaluating several essential factors, including selecting appropriate deployment models, selecting the right AWS services, understanding overall business needs, and translating them into your technical requirements.
Application Deployment, Application Release Process, Deployment Models, Deployment Strategies, and Deployment Design and Deployment Patterns all aim to provide high availability, faster upgrades, and lower downtime with negligible traffic/user impacts.
The deployment process would be different for applications that ran on-premises (traditional manner). They would have a stable version of the application used for production. Any production failure would result in a rollback to the previous stable versions, which meant applications went offline with extended downtimes, deployment fixes in the middle of the night, and business loss.
As organizations shift toward the cloud, the deployment process has adopted a DevOps practice to deliver applications and services at great speed. DevOps includes:
- Continuous Integration and Continuous Delivery (CI/CD)
- Microservices
- Infrastructure as Code (IaC)
- Observability (monitoring and logging)
- Communication and Collaboration
These aspects of DevOps help automate application build, test, and deployment, and help keep environments up as much as possible and speed up the deployment process.
AWS supports several deployment strategies. We will discuss the most common deployment technique that achieves agility, scalability, and automation capabilities: the blue/green deployment methodology. We will also look at some of the benefits, trade-offs, requirements, and advantages of choosing other methods.
Blue/Green Deployment Methodology
A blue/green deployment method requires creating two similar but separate environments. The primary one (blue) runs the current application version, and one environment (green) runs the newer version. Both the environments run in production. The application release model gradually directs traffic from an older version (blue environment) of an application or microservice to a nearly identical (ready and tested) new release (green environment), providing much lower downtime and rollback ability.
As the release team prepares for a newer software version, the final testing stage does not occur in a live environment (blue). Once thoroughly tested, the software is in green, and all incoming requests get directed to flow towards green instead of blue.
Green is now live, and blue is idle.
Benefits of Blue/Green Deployment
Environmental Isolation:
- Blue/green deployments provide isolation between blue and green application environments, allowing you to run parallel green environments without affecting resources under the blue environment.
- Blue/green deployments allow deploying immutable infrastructure by deploying a new environment when needed.
Quicker Rollbacks:
- Blue/green deployment provides the ability to roll traffic back to the operational production environment quickly.
- Total time of operations impaired can be reduced to the minimum when an issue is detected under a green environment and re-sending the traffic to the blue environment.
Canary Testing:
- Blue/green deployments provide a mechanism to validate the green environment before going production live or sending a smaller percentage of production traffic to model the actual load, also known as canary analysis or canary testing. Post validation, you can either decide to send all traffic to the green environment or keep all traffic at blue, reducing the downtime and blast radius.
- QA/Testing team can perform testing on the green environment without disrupting the blue environment.
Disaster Recovery:
- Blue/green deployments allow quickly switching and deploying resources under a production environment, providing much better recovery time during a disaster or outage affecting a service/feature/availability zone (AZ)/region on AWS.
Improve CI/CD Workflows:
- Blue/green deployments improve CI/CD workflow by deploying resources in a newer environment (green) with little dependencies on existing systems.
Cost Optimization:
- Blue/green deployments provide a measure to optimize cost by launching optimized resources for the newer optimized environment (application version).
- It also allows to scale out traffic eventually from a blue to green environment and later decommission the blue environment.
AWS Services for Blue/Green Deployment
AWS provides several native tools and services to automate and streamline deployments and infrastructure.Following is the list of a few:
Amazon Route 53:
Amazon Route 53 is a highly available and scalable cloud Domain Name System (DNS) web service. Amazon Route 53 runs on a global network of DNS servers, and it provides a highly reliable and cost-effective way to route user requests on IP resources to appropriate destinations.
AWS Elastic Load Balancer:
AWS Elastic Load Balancer distributes incoming application traffic across multiple targets and virtual appliances in one or more AZs and provides high availability and greater fault tolerance.
AWS Auto Scaling:
AWS Auto Scaling provides monitoring capabilities to your environment and automatically adjusts capacity to maintain steady, predictable performance at the lowest possible cost.
AWS CloudFormation:
AWS CloudFormation allows the provisioning and managing of AWS and third-party resources by using IaC.
Amazon CloudWatch:
Amazon CloudWatch provides a monitoring service on AWS resources and applications. Amazon CloudWatch collects and visualizes metrics, ingests and monitors log files, and defines alarms.
Amazon CloudWatch provides system-wide visibility into resource utilization, application performance, and operational health, which is crucial to early detection of application health in blue/green deployments.
AWS CodeDeploy:
AWS CodeDeploy provides a deployment service that automates deployments to various compute types such as EC2 instances, on-premises instances, Lambda functions, or Amazon ECS services.
Blue/green deployment is a feature of CodeDeploy. CodeDeploy can also roll back deployment in case of failure. You can also use CloudWatch alarms to monitor the state of deployment and utilize CloudWatch Events to process the deployment or instance state change events.
Amazon ECS:
Amazon ECS provides a fully managed container orchestration service that makes it easy to deploy, manage, and scale containerized applications.Amazon ECS allows traffic to be shifted during a deployment mainly in three ways:
- Canary: Traffic is redirected in two increments.
- Linear: Traffic is redirected in equal increments.
- All-at-once: All traffic is redirected to the updated tasks.
AWS Lambda Hooks: With AWS Lambda hooks, CodeDeploy can call the Lambda function during the various lifecycle events, including deployment of ECS, Lambda function deployment, and ECC2/On-premise deployment. The hooks help create a deployment workflow for applications.
Implementation Techniques for Blue/Green Deployments
Several techniques exemplify how you can implement blue/green on AWS Cloud, but final considerations mainly depend upon overall architecture and application design.
Amazon Route 53 - DNS Entry:
Customers can use Amazon’s Route 53 DNS service, to manage incoming traffic between the environments (blue to green) and perform switchover and rollback accordingly. This technique can be applied to compute (EC2 instances, Auto Scaling groups, load balancers, services on Amazon ECS), or networking (Route 53, IP, and DNS endpoints)
AWS Auto Scaling Behind a Load Balancer:
AWS Auto Scaling is used to scale and manage compute instances behind a load balancer. Each environment (blue & green) would have a separate Auto Scaling group. A simple swap of Autocaling groups can allow the load balancers to send traffic to either one based on preference.
Auto Scaling Launch Configurations:
Each Auto Scaling group would have its launch configurations that can be added/edited to other Auto Scaling groups but not deleted. An Auto Scaling group can scale to twice its original size during newer deployments and shrink down to remove old deployments or configurations.
AWS Elastic Beanstalk URL Swap:
AWS Elastic Beanstalk allows deploying several application versions using an application bundle. Each environment (blue & green) can have its application bundle deployed with a separate URL. When required, Elastic Beanstalk allows to SWAP environment URL, which performs a DNS switch and promotes the green environment to production. The same SWAP feature can allow to roll back and bring a blue environment in production.
When Blue/Green Deployment is Not Recommended
The blue/green deployment model is not suited for every workload or use case. It’s not recommended when there are high levels of complexities, multiple failure points, process breakdown, and others.
Below are a few circumstances when blue/green deployment is not very well suited:
- Schema changes are too complex to decouple.
- Performance degradation due to data not available locally but in separate geographic locations (blue/green).
- Irrespective of running on physical servers, cloud VMs, or Containers on Kubernetes, running two identical environments is expensive.
- Demanding requirements to produce continuous data consistency, data synchronization, reliability, etc.
- Challenges around database mirroring and database schema management introduce additional complexities.
- An application needs to be deployment aware.
- Working around commercial off-the-shelf (COTS) applications for blue/green deployment generates additional complexities.
Conclusion
AWS provides several services, features, and tools to simplify and automate the provisioning of infrastructure and applications. The blue/green deployment strategy offers a great way to achieve a seamless deployment process, mitigate risk, reduce human errors and downtime, and provide a roll-back capability.
In order to achieve the best results in application deployment, you should evaluate the available features of each service against the needs of your application and your organization.
FAQ
-
How can custom domains and SSL certificates be managed during a blue/green deployment, ensuring a seamless transition without downtime or security warnings?
For managing custom domains and SSL certificates during blue/green deployments, it's crucial to ensure that both environments are configured to handle the same domain and certificate setup. This often involves using DNS management strategies to switch traffic seamlessly between environments without user disruption.
-
What strategies are recommended for managing database schema changes during blue/green deployments to ensure data integrity and consistency across blue and green environments?
Adopting backward-compatible changes regarding database schema changes ensures both environments can operate with the current and new schema versions. This might include using feature toggles or maintaining additional compatibility layers in the database.
-
How do the costs of running parallel environments during blue/green deployments compare to other deployment strategies, and what measures can be taken to minimize these costs?
- To minimize costs associated with running parallel environments in blue/green deployments, consider leveraging cloud services that allow flexible resource allocation and scaling. This includes using on-demand or spot instances, closely monitoring resource usage, and scaling down the inactive environment to a minimal footprint when not used.
References:
1. Quick Start Reference Deployment:
2. Deployment Strategies:
3. Blue/Green Deployment AWS whitepaper:
Author Spotlight:
Hozefa Bata
Keep Up To Date With AWS News
Stay up to date with the latest AWS services, latest architecture, cloud-native solutions and more.