AWS CloudFormation vs. Terraform: Which One Should You Choose?

Infrastructure as code [IaC] is a big buzzword these days, and it’s no mystery why. IaC, referred to as software-defined infrastructure, is defined as an IT setup wherein developers or operations teams automatically manage and provision the technology stack for an application through software, rather than using a manual process to configure discrete hardware devices and operating systems. It’s a cornerstone of DevOps, designed to boost the agility, productivity and quality of work within organizations.

When people look for tools to implement infrastructure as code on AWS, they often narrow the choice between AWS CloudFormation or the open-source tool Terraform. Let’s review the differences between the two so you can determine which is right for your needs.

CloudFormation

AWS CloudFormation is a managed AWS service with a common language for you to model and provision AWS and third-party application resources for your cloud environment in a secure and repeatable manner. This provides a single reference point for both AWS infrastructure mapping and development.  

CloudFormation abstracts away many of the subtleties involved in managing dependencies between AWS resources. Additionally, CloudFormation allows for the modification and destruction of provisioned resources in a prescribed and predictable manner, making versioning and iterating on your infrastructure much more accessible. CloudFormation is AWS-focused, and AWS-native. 

With CloudFormation, you don’t need to figure out which AWS services need to be provisioned or the subtleties of how to make these dependencies connect together  — CloudFormation takes care of this for you in a siloed kind of manner. Once deployed, you can destroy and modify AWS resources in a controlled and predictable way within CloudWatch, allowing you to version control your own AWS infrastructure. This allows for iterative testing, as well as rollback (when configured properly).  

Terraform‍

Created by HashiCorp, Terraform is an open-source infrastructure-as-code software tool that helps users with the task of setting up and provisioning datacenter infrastructure. A cloud-agnostic tool, Terraform codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.

Now that we’ve defined these two IaC platforms, let’s review some of the key differences in more depth.

State Management

With both CloudFormation and Terraform, you need to keep track of all resources under management.

With CloudFormation, users can perform regular drift detection on their entire provisioned infrastructure, and receive detailed responses if anything has changed. Some resources in a CloudFormation stack are able to have parameters changed without destroying and rebuilding the targeted resource, while others are considered immutable and will be rebuilt.  Additionally, before CloudFormation will delete a resource, it will determine dependencies and fail the command if any exist (which would remain after resource removal). 

Terraform stores the state of the infrastructure on the provisioning computer, or in a remote site (for team use). This state file is a custom JSON format which serves as a map for Terraform, describing which resources it manages, and how those resources should be configured.

Since CloudFormation is a managed AWS service, it does this for you. CloudFormation will consistently check infrastructure it has provisioned to detect if it is maintaining that state and configuration.  If you’re using Terraform, it stores its state on a local disk, and there is a remote state option, which writes the state data to a remote data store, to be shared between all members of a team. Remote state supports Amazon S3, but you need to configure it yourself.

Now that we’ve covered differences in state management, let’s move on to the topic of modularity.

Modularity

In the constantly changing world of infrastructure requirements,  flexibility is key. CloudFormation and Terraform have unique ways of addressing this need.

Terraform has a modules, which are containers for multiple resources that are used together. Modules allow developers to abstract their infrastructure into reusable, shareable code and increases iteration speed for teams (much like functions do in a programming language like Ruby).

CloudFormation utilizes a system called “nested stacks.” That is, CloudFormation  templates being called from within CloudFormation templates. These nested stacks can further be abstracted into StackSets. It should be noted that StackSets require additional permissions, beyond those of normal AWS CloudFormation.  

A benefit of Terraform is increased flexibility over CloudFormation with regards to modularity. Terraform modules can be pulled in for any provider supported, or organizations can roll their own.

Conversely, in a multi-cloud or hybrid environment, CloudFormation doesn’t easily allow users to provision or natively coordinate non-AWS resources. It’s not impossible, as there is a custom resources feature in CloudFormation, but it requires additional templating and design to bring in third party resources, or those AWS services not available organically.

Configuration

CloudFormation and Terraform differ in how they handle configuration and parameters.

Terraform uses provider specific data sources. The implementation is in a modular fashion, allowing data to be fetched or computed for use elsewhere in a Terraform configuration. This lets a Terraform configuration make use of information defined outside of Terraform (such as an Elastic IP address), to update or provision infrastructure.

CloudFormation uses parameters, and has a maximum of 60 parameters per template. Each parameter must have a logical and unique ID among all others in the template. The parameters must be of a type supported by CloudFormation and they have to be provided at the stack’s runtime. Additionally, each parameter must be declared and referenced from within the same template. CloudFormation does have the capability to use Dynamic References to retrieve parameters at runtime from AWS Systems Manager parameter store, if one has been configured properly. 

Language‍

Terraform uses HashiCorp Configuration Language (HCL), a language built by HashiCorp. It is fully compatible with JSON, and was created to strike a balance between human-friendly and machine-friendly languages, while remaining interpretable to humans.

AWS CloudFormation utilizes either JSON or YAML, with the YAML version being slightly easier to read (as well as more compact). CloudFormation also has a limit of 51,000 bytes for the template body itself. If a larger template is needed, AWS advises developers to separate resources into nested stacks.

With those distinctions established, let’s look at a couple of Mission use cases where we employed Terraform as a solution for customers.

Customer Use Case: JoVE

JoVE is the leading producer and publisher of videos that increase the productivity and efficacy of scientific, medical, and engineering research and education. As their business grew, their data center performance began to lag. They decided to move to the AWS cloud, working with Mission as a managed service provider. Mission produced a multi-phase technology roadmap that supported the move from the co-located data center to AWS, beginning with an IaC project that used Terraform to automatically deploy networking, services, and servers in AWS. As a result, the company was subsequently able to ensure consistent application performance for subscribers accessing videos.

Customer Use Case: Your Call Football

Your Call Football (YCF) offers a wholly unique football experience where fans get to call plays in real time, then see them run on the field by real players. YCF’s app saw intermittent bursts of intense activity that required real-time high availability to ensure an issue-free fan experience. Mission helped to support this by building out a backend utilizing infrastructure-as-code, leveraging Terraform, Elastic Beanstalk, and a queuing service to complete the IaC strategy. In addition, Mission built an elastic load balancing system to prepare load balancers ahead of massive load spikes, further ensuring a smooth user experience.

The Bigger Picture — CloudFormation or Terraform‍

Both are powerful cloud infrastructure management tools. If you’re already on AWS and using all AWS tools, CloudFormation may be more convenient, especially if you have no external tie ins from 3rd parties. However, if you’re looking for additional flexibility from a cloud-agnostic platform that integrates with AWS services (and those of all other popular providers), Terraform might be of greater utility for your organization. It’s important to not only look at what is working currently - but what the future state of any infrastructure might be. There are long-term benefits with both toolsets, and the final decision may come down to simple usability and developer comfort. Whichever choice you make, using either of these “swiss army knives” of AWS infrastructure is sure to increase productivity, portability, and efficiency.

FAQ

How do AWS CloudFormation and Terraform integrate with other DevOps and CI/CD tools?

When considering the integration and compatibility of AWS CloudFormation and Terraform with other DevOps and CI/CD tools, it's important to recognize that both are designed with automation and collaboration in mind. Terraform, being an open-source tool, generally has a broader scope of integration capabilities with a wide range of DevOps tools due to its extensive plugin system and community contributions. It works seamlessly with CI/CD tools like Jenkins, GitLab, and GitHub Actions, allowing for infrastructure changes to be part of the CI/CD pipelines. AWS CloudFormation, while offering deep integration within the AWS ecosystem, including direct integration with AWS CodePipeline and other AWS services, might require additional tools or custom scripting to integrate with third-party CI/CD platforms.

What is the learning curve associated with AWS CloudFormation and Terraform for beginners, and where can I find support for these technologies?

The learning curve for both AWS CloudFormation and Terraform can vary depending on the user's background. Terraform's use of HCL (HashiCorp Configuration Language) is often praised for its readability and simplicity, which might make it easier for beginners to grasp. The extensive documentation, tutorials, and community forums contribute to a supportive learning environment. CloudFormation uses JSON or YAML, which might be more familiar to those already accustomed to working with AWS services, but the overall complexity of AWS-specific configurations can present a steeper learning curve. The community support for CloudFormation is robust within the AWS user community, though Terraform's open-source nature has fostered a larger, more diverse community support network.

How do AWS CloudFormation and Terraform perform at scale, especially in large, complex environments with hundreds or thousands of resources?

In terms of performance and scalability, both tools are designed to handle large-scale infrastructure deployments, but their approaches differ. Terraform's state management allows it to efficiently manage and track the state of large numbers of resources across multiple providers, which is beneficial in complex environments. However, managing Terraform state at a very large scale can become challenging and may require careful state management practices, such as state locking and remote state storage. CloudFormation is deeply integrated with AWS and optimized for managing AWS resources, which can make it highly efficient within AWS environments. However, managing very large deployments with CloudFormation can require meticulous template organization and might encounter service-specific limits.