AWS Cloudwatch Metrics: Leveraging an Essential Performance Monitoring Service
Amazon Web Services CloudWatch is a powerful cloud-monitoring tool, but cutting through its complexity may require the assistance of cloud experts.
Infrastructure as code [IaC] is a big buzzword these days, and it’s no mystery why. IaC, referred to as software-defined infrastructure, is defined as an IT setup wherein developers or operations teams automatically manage and provision the technology stack for an application through software, rather than using a manual process to configure discrete hardware devices and operating systems. It’s a cornerstone of DevOps, designed to boost the agility, productivity and quality of work within organizations.
When people look for tools to implement infrastructure as code on AWS, they often narrow the choice between AWS CloudFormation or the open-source tool Terraform. Let’s review the differences between the two so you can determine which is right for your needs.
AWS CloudFormation is a managed AWS service with a common language for you to model and provision AWS and third-party application resources for your cloud environment in a secure and repeatable manner. This provides a single reference point for both AWS infrastructure mapping and development.
CloudFormation abstracts away many of the subtleties involved in managing dependencies between AWS resources. Additionally, CloudFormation allows for the modification and destruction of provisioned resources in a prescribed and predictable manner, making versioning and iterating on your infrastructure much more accessible. CloudFormation is AWS-focused, and AWS-native.
With CloudFormation, you don’t need to figure out which AWS services need to be provisioned or the subtleties of how to make these dependencies connect together — CloudFormation takes care of this for you in a siloed kind of manner. Once deployed, you can destroy and modify AWS resources in a controlled and predictable way within CloudWatch, allowing you to version control your own AWS infrastructure. This allows for iterative testing, as well as rollback (when configured properly).
Created by HashiCorp, Terraform is an open-source infrastructure-as-code software tool that helps users with the task of setting up and provisioning datacenter infrastructure. A cloud-agnostic tool, Terraform codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.
Now that we’ve defined these two IaC platforms, let’s review some of the key differences in more depth.
With both CloudFormation and Terraform, you need to keep track of all resources under management.
With CloudFormation, users can perform regular drift detection on their entire provisioned infrastructure, and receive detailed responses if anything has changed. Some resources in a CloudFormation stack are able to have parameters changed without destroying and rebuilding the targeted resource, while others are considered immutable and will be rebuilt. Additionally, before CloudFormation will delete a resource, it will determine dependencies and fail the command if any exist (which would remain after resource removal).
Terraform stores the state of the infrastructure on the provisioning computer, or in a remote site (for team use). This state file is a custom JSON format which serves as a map for Terraform, describing which resources it manages, and how those resources should be configured.
Since CloudFormation is a managed AWS service, it does this for you. CloudFormation will consistently check infrastructure it has provisioned to detect if it is maintaining that state and configuration. If you’re using Terraform, it stores its state on a local disk, and there is a remote state option, which writes the state data to a remote data store, to be shared between all members of a team. Remote state supports Amazon S3, but you need to configure it yourself.
Now that we’ve covered differences in state management, let’s move on to the topic of modularity.
In the constantly changing world of infrastructure requirements, flexibility is key. CloudFormation and Terraform have unique ways of addressing this need.
Terraform has a modules, which are containers for multiple resources that are used together. Modules allow developers to abstract their infrastructure into reusable, shareable code and increases iteration speed for teams (much like functions do in a programming language like Ruby).
CloudFormation utilizes a system called “nested stacks.” That is, CloudFormation templates being called from within CloudFormation templates. These nested stacks can further be abstracted into StackSets. It should be noted that StackSets require additional permissions, beyond those of normal AWS CloudFormation.
A benefit of Terraform is increased flexibility over CloudFormation with regards to modularity. Terraform modules can be pulled in for any provider supported, or organizations can roll their own.
Conversely, in a multi-cloud or hybrid environment, CloudFormation doesn’t easily allow users to provision or natively coordinate non-AWS resources. It’s not impossible, as there is a custom resources feature in CloudFormation, but it requires additional templating and design to bring in third party resources, or those AWS services not available organically.
CloudFormation and Terraform differ in how they handle configuration and parameters.
Terraform uses provider specific data sources. The implementation is in a modular fashion, allowing data to be fetched or computed for use elsewhere in a Terraform configuration. This lets a Terraform configuration make use of information defined outside of Terraform (such as an Elastic IP address), to update or provision infrastructure.
CloudFormation uses parameters, and has a maximum of 60 parameters per template. Each parameter must have a logical and unique ID among all others in the template. The parameters must be of a type supported by CloudFormation and they have to be provided at the stack’s runtime. Additionally, each parameter must be declared and referenced from within the same template. CloudFormation does have the capability to use Dynamic References to retrieve parameters at runtime from AWS Systems Manager parameter store, if one has been configured properly.
Terraform uses HashiCorp Configuration Language (HCL), a language built by HashiCorp. It is fully compatible with JSON, and was created to strike a balance between human-friendly and machine-friendly languages, while remaining interpretable to humans.
AWS CloudFormation utilizes either JSON or YAML, with the YAML version being slightly easier to read (as well as more compact). CloudFormation also has a limit of 51,000 bytes for the template body itself. If a larger template is needed, AWS advises developers to separate resources into nested stacks.
With those distinctions established, let’s look at a couple of Mission use cases where we employed Terraform as a solution for customers.
JoVE is the leading producer and publisher of videos that increase the productivity and efficacy of scientific, medical, and engineering research and education. As their business grew, their data center performance began to lag. They decided to move to the AWS cloud, working with Mission as a managed service provider. Mission produced a multi-phase technology roadmap that supported the move from the co-located data center to AWS, beginning with an IaC project that used Terraform to automatically deploy networking, services, and servers in AWS. As a result, the company was subsequently able to ensure consistent application performance for subscribers accessing videos.
Your Call Football (YCF) offers a wholly unique football experience where fans get to call plays in real time, then see them run on the field by real players. YCF’s app saw intermittent bursts of intense activity that required real-time high availability to ensure an issue-free fan experience. Mission helped to support this by building out a backend utilizing infrastructure-as-code, leveraging Terraform, Elastic Beanstalk, and a queuing service to complete the IaC strategy. In addition, Mission built an elastic load balancing system to prepare load balancers ahead of massive load spikes, further ensuring a smooth user experience.
Both are powerful cloud infrastructure management tools. If you’re already on AWS and using all AWS tools, CloudFormation may be more convenient, especially if you have no external tie ins from 3rd parties. However, if you’re looking for additional flexibility from a cloud-agnostic platform that integrates with AWS services (and those of all other popular providers), Terraform might be of greater utility for your organization. It’s important to not only look at what is working currently - but what the future state of any infrastructure might be. There are long-term benefits with both toolsets, and the final decision may come down to simple usability and developer comfort. Whichever choice you make, using either of these “swiss army knives” of AWS infrastructure is sure to increase productivity, portability, and efficiency.
Find out how much you could save on managing your AWS infrastructure.