Understanding ECS Anywhere and EKS Anywhere

Container orchestration services like ECS and EKS are incredibly popular, with good reason. They reduce both the friction of container adoption and ongoing operational overhead by  being production-ready managed services. One of the primary benefits of running containers is that they are highly portable and allow for DevOps teams to focus on application functionality over environmental configuration. 

However, all container orchestration platforms also have implementation-specific requirements and feature sets. While one may be running the same container image across multiple platforms, the configuration, deployment, discovery and communication patterns will often differ depending on the destination. 

While the portability of containers makes it easier to get an application running in the cloud, they still require expertise in a given platform’s minutiae  to realize high-performing and resilient architectures.Or they require a tradeoff in administrative/management overhead (i.e., running bare-metal Kubernetes). 

These issues only multiply when considering hybrid or multi-cloud architectures, regardless of the business driver. Whether there are specific workloads that the organization wants to run on-premises due to compliance or security requirements, leveraging existing data center hardware investment for cost optimization, or running a hot-standby DR environment for availability, chances are that context or platform dependent configuration will be necessary. By nature, running multi-cloud or hybrid architecture causes problems because it leaves the organization with multiple distinct infrastructures to manage, requiring different skills, expertise, and functional roles. 

Managing unified infrastructure is far easier, one of the cloud’s core value propositions. Fortunately, AWS has offered a solution to running disparate container orchestration platforms: ECS Anywhere and EKS Anywhere! The “Anywhere” platform allows one to run ECS and EKS clusters using any hardware or cloud provider by unifying the control plane across multiple networks through agents. This vision of hybrid architecture enables us to actually run a container outside of AWS using the same configuration and patterns as within, without sacrificing the benefits and support of a robust, SLA-backed managed service. 

Although the names are similar, there is a significant difference between the two offerings. Implementing ECS is far more accessible yet far less flexible than EKS, and their respective Anywhere versions pose similar dynamics.

ECS Anywhere

ECS Anywhere enables us to deploy and manage containerized workloads on any infrastructure, providing a consistent user and API experience expected from ECS while eliminating the need to manage different architectures. Both on-premises and cloud implementations have similar tools for cluster and workload management, scheduling, and monitoring. 

We can define ECS clusters, whether ECS or ECS Anywhere, in the control panel, via API, or using standard IAC (Infrastructure-As-Code) tooling such as Terraform. In order to join an ECS Anywhere cluster, there is an additional prerequisite outside of the typical ECS Agent, and Docker: the AWS Systems Manager Agent (SSM Agent). This agent-based software is used to run Amazon EC2 at the edge, and supports hybrid deployments- allowing Systems Manager to perform unified instance management, configuration, and patching operations, essentially turning any machine in any location into an SSM Managed Instance.

With ECS Anywhere, the Amazon ECS control plane remains in the region. Users familiar with ECS can use the same interface for ECS Anywhere. More importantly, ECS Anywhere does not affect the on-premises infrastructures needed for application deployment, operations, or maintenance. It sends only the information necessary to manage a task to the Amazon control plane. All other information remains on-premises, allowing for unchanged security and governance processes. 

ECS Anywhere is compatible with any infrastructure and it can work with virtual machines, bare metal, Raspberry Pi, or any hardware running supported operating systems and architecture. If the non-AWS infrastructure loses connectivity, ECS Anywhere tasks continue to run until manually stopped. Connectivity is only required to update or scale tasks or connect other in-region AWS services, but this can have implications for commonly used additional services like ECR or CloudWatch. When network connectivity is restored, the node credentials are automatically renewed and normal functionality resumes. As with any service, particularly ones geared towards hybrid infrastructure, there are some prerequisites that should be considered before adoption, as they may not fit your particular use case.

ECS Anywhere allows us to leverage existing infrastructure while maintaining the simplicity of its traditional container management orchestration and control. ECS Anywhere is a fitting choice for companies that want to manage ECS instances running on-premises but do not want  to incur the overhead of setting up and managing Kubernetes clusters. That being said, fortunately Amazon has a solution for those that do!

EKS Anywhere

While ECS is far easier to adopt and manage, EKS provides a (mostly) blank canvas with a multitude of extensible configuration options, and is often the tool of choice for customers focused on building scalable, complex, and distributed platforms. While certainly not every workload can or should be run on Kubernetes, most of the traditional counter-arguments and risks from even a few years ago have been addressed. In my opinion, EKS offers more workload and configuration flexibility than any other single AWS managed service- but these capabilities come at an additional cost. Kuberetes necessitates a great deal of domain expertise, foresight, and planning, on top of the usual AWS compute/service charges.

EKS Anywhere ostensibly provides “centralized” visibility of multiple Kubernetes clusters via the EKS Connector while leveraging the open-source EKS Distro. This distro provides some opinionated component configurations, integration testing, and extends support for legacy Kubernetes versions no longer maintained by the community. However, it does make some limiting assumptions about the cluster's configuration and the infrastructure it expects. 

At this time, EKS Anywhere provides the ability to run EKS Anywhere primarily using VMware vSphere, but does support the creation of local clusters using docker- a boon for local testing- as it does support IRSA (with a little bit of effort). By the end of the year, Amazon is planning support for other deployment options, including bare metal. Because of the EKS distro’s opinionated configuration, and platform requirements, EKS Anywhere is not yet a sufficient turnkey solution for heavily customized clusters.  For standard operations, the end result is simplified cluster creation, fewer architectural decisions, with the added ability of a centralized view into all clusters, whether on cloud or on-premises.

EKS Anywhere falls under the common structure of Amazon’s “Shared Responsibility” model, and AWS provides a high-level comparison of the differences between the managed and “anywhere” versions and their respective support. Obviously, from a hardware perspective, an EKS Anywhere cluster has components outside of AWS support purview, but customers with an Enterprise Support Agreement can rely on AWS for support with EKS Anywhere clusters. In addition, one can use helpful tools like Weave’s eksctl which can go a long way in simplifying provisioning and “Day 2” operations. I was also thrilled to see EKS Anywhere utilizes Cillium as the default CNI- I’ve been getting into BPF lately for its observability capabilities, which Cillium combines with extensive security and encryption options, plus easy to manage policy enforcement (I’ve been running it on my personal clusters and am a big fan). 

Does EKS Anywhere solve all of the difficulties in running multiple or federated clusters in a hybrid architecture? Not by a longshot- but, it does simplify some of the design and implementation of the model, and as someone who loves EKS, I’m excited for the future of this product. For more information about EKS Anywhere, please see the FAQs and supporting documentation.

Summary

There are many factors such as regulatory and compliance mandates, security requirements, or integration with existing systems that might compel a company to maintain a hybrid Kubernetes strategy. They may simply want to leverage an existing infrastructure investment to reduce their cloud costs, or to add additional resiliency. Whether running containers on ECS or EKS, these requirements used to necessitate maintaining multiple, distinct, operating environments, typically with differing configuration and deployment options. The new AWS Anywhere features for ECS and EKS are a good first step towards unifying and solving some of the resulting headaches- but there still are plenty of design choices left to contend with, such as federation, service discovery, unified monitoring, and secrets management- among others.  

AWS ECS Anywhere allows us to orchestrate our ECS containers from the ECS control plane, regardless of which worker is running the container. It can be in AWS, on-premises, or with another cloud provider, allowing us to view and maintain our container workloads through a single pane of glass. 

Due to the extensibility and sheer number of build and configuration options, Kubernetes presents more challenges and nuances for a unified hybrid experience, let alone an easily managed one. EKS Anywhere is Amazon’s first step toward realizing this vision (I would, however, love to see better incorporation of configuration management features, ala Anthos). EKS Anywhere allows us to build Kubernetes clusters in an opinionated, best practices configuration and manage via the EKS Connector within the AWS Console. 

Hybrid architectures are, by nature, extremely complicated- there are far more risks and design decisions to be carefully reasoned and made. What might be a straightforward architecture or tooling choice in a cloud-native environment can quickly become a web of different requirements, management overhead and failure points to consider. ECS Anywhere, and to a lesser extent EKS Anywhere, provide a framework to lessen the toil of living in a hybrid world. I’m excited to see how these new offerings mature, given the growth and improvements I’ve seen within EKS since its initial release.

Ready to unlock the benefits of containerization? Learn about the intricacies of successful container architectures and how Mission can help you leverage containers to meet your specific needs and deliver high value for your team and end users today.

FAQ

1. What are the specific security measures and best practices recommended for ECS Anywhere and EKS Anywhere deployments?

For ECS Anywhere and EKS Anywhere deployments, AWS recommends utilizing its native security tools and features, such as AWS Identity and Access Management (IAM) for fine-grained access control and AWS Key Management Service (KMS) for encryption needs. Best practices also include regular security assessments, adherence to the principle of least privilege, and enabling logging and monitoring through AWS CloudTrail and Amazon CloudWatch.

1. How does application performance management work with ECS Anywhere and EKS Anywhere, particularly in hybrid environments?

Application performance management (APM) in hybrid environments using ECS Anywhere or EKS Anywhere relies on consistent monitoring and optimization strategies. Tools like Amazon CloudWatch can be utilized for monitoring container health, performance metrics, and logging. Additionally, integrating third-party APM solutions that support containerized environments can provide deeper insights into application performance and user experience.

1. Can ECS Anywhere and EKS Anywhere be used with other AWS services to create a comprehensive disaster recovery strategy?

ECS Anywhere and EKS Anywhere can integrate with AWS services like Amazon RDS for database needs, Amazon S3 for storage, and AWS Lambda for serverless compute capabilities, forming a comprehensive disaster recovery strategy. Leveraging AWS CloudFormation for infrastructure as code (IaC) also aids in rapidly deploying and managing resources across hybrid environments, ensuring resilience and continuity.