Announcing Coherence 2.0 and CNC, the first open source IaC framework
All posts

Infrastructure Provisioning with Terraform: Best Practices

Learn best practices for infrastructure provisioning with Terraform, including modular design, remote state collaboration, and security measures. See real-world examples and gain actionable tips.

Zan Faruqui
May 16, 2023

Most IT professionals would agree:
managing cloud infrastructure can be complex and time-consuming without the right tools.

Using Terraform for infrastructure provisioning introduces a reliable, scalable approach for defining and deploying infrastructure as code. And following key best practices takes your Terraform usage to the next level.

In this post, you'll get an overview of Terraform capabilities and workflow. We'll then dive into infrastructure provisioning best practices like:

  • Structured project layout
  • Modular design
  • Remote state collaboration
  • Testing/validation
  • Execution plans
  • Secure provisions

You'll see real-world examples and gain actionable tips for leveraging Terraform to efficiently manage infrastructure across the full lifecycle - from initial provisioning to ongoing updates and maintenance.

Introduction to Infrastructure Provisioning with Terraform

Is Terraform an infrastructure provisioning tool?

Terraform Open Source is indeed an infrastructure provisioning tool that allows you to safely and efficiently manage infrastructure deployments across multiple cloud providers.

Some key things to know about using Terraform for infrastructure provisioning:

  • Automated infrastructure management: Terraform lets you define your infrastructure as code and provisions resources based on your configuration files. This automates manual processes and ensures consistency across environments.
  • Supports major cloud providers: Terraform has providers for most major cloud platforms like AWS, Azure, and GCP. This allows you to standardize infrastructure provisioning across multiple cloud accounts.
  • Configuration management: Terraform tracks infrastructure changes in a state file which serves as the source of truth for your environment. This helps with change management and lets you roll back configurations if needed.
  • Open source and free: As an open source tool, Terraform benefits from contributions from a large community. And the open source version can be used free of charge to provision infrastructure.

So in summary - yes, Terraform is a leading infrastructure as code tool purpose-built for automating provisioning and management of infrastructure across public and private clouds. Its capabilities for configuration, change control and multi-cloud support make it a great choice for streamlining infrastructure workflows.

How do I provision AWS infrastructure with Terraform?

Terraform is an open-source infrastructure as code (IaC) tool that allows you to safely and efficiently build, change, and version infrastructure. With Terraform, you can declaratively define your infrastructure across multiple cloud providers like AWS, Azure, or GCP, using a high-level configuration syntax.

Here are some key steps to get started with provisioning AWS infrastructure using Terraform:

Prepare Your System

  • Install Terraform on your local machine or CI/CD server. Terraform is available for Windows, Linux and macOS.
  • Set up credentials for Terraform to access your AWS account. The easiest method is to export your AWS access keys as environment variables.

Download, Install and Initialize Terraform

  • Download the Terraform binary from terraform.io and install it on your system.
  • Navigate to the directory where your Terraform configuration files are located and run terraform init. This will download any necessary plugins and modules.

Generate AWS Access Keys

  • In the AWS Management Console, go to IAM and create a user with programmatic access.
  • Generate a new access key and secret access key for that IAM user.
  • Export those keys as AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables on your system.

Create EC2 Instance with Terraform

  • Author a Terraform configuration file (with .tf extension) that describes the components you want to deploy on AWS.
  • For example, you can define an AWS EC2 instance resource with attributes like AMI ID, instance type, VPC settings, tags, etc.
  • Run terraform plan to verify the resources to be created.
  • Run terraform apply to provision the resources on AWS.

And that's it! In just a few steps you can start automating your cloud infrastructure deployments using Terraform. The key benefits are reproducibility, versioning and collaboration for your entire stack.

Is Terraform used for infrastructure?

Terraform is HashiCorp's open source infrastructure as code tool that provides a consistent CLI workflow to manage hundreds of cloud services. With Terraform, infrastructure can be described in human-readable configuration files that define your resources and provision them in an automated, secure, and efficient way.

Some key things to know about using Terraform for infrastructure provisioning:

  • Terraform lets you define both cloud infrastructure (like compute instances, databases, networking, storage) as well as custom in-house solutions. This consistent workflow helps unify cross-cloud and on-prem environments.
  • Terraform configuration files describe the desired end-state infrastructure. When applied, Terraform plans out changes needed to reach desired state and then provisions infrastructure efficiently.
  • Infrastructure described with Terraform can be version controlled and shared, enabling codified workflows around provisioning. Teams can collaborate effectively.
  • Users can create reusable infrastructure components that can be composited into more complex solutions. This helps promote modularity and separation of concerns.
  • Terraform integrates with popular cloud platforms like AWS, Azure, GCP enabling simplified and automated provisioning across services.

In summary, Terraform is considered a standard for provisioning and managing infrastructure across a breadth of environments. Its declarative approach, reusable components and cloud integrations help solve major pain points around managing infrastructure at scale.

How do you do infrastructure as code using Terraform?

Terraform allows developers to define their infrastructure configurations in a declarative language called HCL (HashiCorp Configuration Language). This infrastructure as code approach has several key benefits:

It's Platform Agnostic

Once you define the resources you need in HCL, Terraform can provision them on many different cloud platforms like AWS, Azure, or GCP. So you don't have to learn platform-specific tools.

It Manages Infrastructure Lifecycles

Terraform not only provisions infrastructure, but can also update and destroy it when configurations change. This is great for testing environments.

It Can Validate Plans

Before making any infrastructure changes, Terraform generates an execution plan so you can preview what it will do. This allows validation before any real resources are created.

It Integrates Well Into CI/CD Pipelines

Infrastructure as code integrates smoothly into continuous integration and continuous delivery (CI/CD) workflows. Teams can commit their HCL configs to source control and have Terraform automatically apply changes at various pipeline stages.

To get started with infrastructure provisioning with Terraform, developers simply need to:

  • Download the Terraform binary
  • Choose a cloud provider like AWS, Azure, or GCP
  • Author HCL configuration files describing the desired resources
  • Run terraform plan to preview infrastructure changes
  • Run terraform apply to provision the resources

With this infrastructure as code approach, teams can easily version, test, and automate their cloud infrastructure deployments.

Terraform Best Practices for Cloud Infrastructure

Terraform is a powerful infrastructure as code (IaC) tool that allows you to define, provision, and manage infrastructure in a safe, consistent, and repeatable way. By codifying your infrastructure, you can treat it as you would any other piece of software. Here are some best practices to follow when using Terraform to provision cloud infrastructure.

Structured Project Layout

A structured project layout promotes consistency, makes collaboration easier, and sets up a Terraform project for success as it scales over time. Here is a recommended file/folder structure:

  • main.tf - Contains core infrastructure configuration.
  • variables.tf - Declares input variables to make configuration reusable.
  • outputs.tf - Declares output values to import into other configurations.
  • modules/ - Contains reusable, encapsulated components.
  • environments/ - Directory for environment-specific Terraform configs.

Modular Design Principles

Breaking your infrastructure into reusable, composable modules simplifies management and allows you to combine components in flexible ways. Some best practices:

  • Encapsulate distinct components into their own modules (e.g. networking, services).
  • Make modules as self-contained and decoupled as possible.
  • Use input variables and outputs to connect modules.

Collaboration with Remote State Storage

Using a remote state backend enables teams to share state data and coordinate changes safely:

  • Persistence - State preserved beyond just one engineer's working directory.
  • Locking - State file locking prevents concurrent modifications.
  • Access control - Manage read/write permissions across team.

Popular backends include S3, Terraform Cloud, and Google Cloud Storage.

Testing and Validation with CI/CD

Incorporating automated testing into your CI/CD pipeline reduces errors and risk when provisioning infrastructure:

  • Syntax check - Validates all Terraform config files in a directory.
  • Format check - Identifies formatting issues according to Terraform style conventions.
  • Plan check - Performs a dry run to see planned infra changes.

Running plan checks before applying changes is crucial.

Infrastructure Provisioning with Terraform AWS

Here are some Terraform tips specific to infrastructure provisioning with AWS:

  • Leverage AWS provider resources for services like EC2, VPC, IAM, etc.
  • Use remote state stored in S3 for shared data and locking.
  • Tag resources consistently for easier searching/filtering.
  • Define IAM roles for Terraform to manage permissions.

Following these best practices will lead to more scalable, secure, and collaborative Terraform usage within your organization.

Terraform Execution Plans and Real-World Application

Terraform execution plans, generated with terraform plan, are a critical component of managing infrastructure as code. By previewing changes before applying them, execution plans empower developers to iteratively build and evolve infrastructure with confidence.

The Importance of Execution Plans

Execution plans allow developers to safely preview exactly what resources Terraform will create, update or destroy before making changes. This protects against inadvertent modifications and provides transparency into proposed infrastructure changes.

Key benefits of Terraform execution plans include:

  • Risk mitigation - Review changes in advance to prevent unintentional modifications
  • Change visibility - Clearly see adds, changes and destroys before applying
  • Collaboration - Share plans with teammates for feedback before applying
  • Iteration - Incrementally develop infrastructure plans and changes

With execution plans, infrastructure changes become low-risk, visible, collaborative processes instead of dangerous black boxes.

Terraform Plan Example

Here is an example Terraform configuration with an AWS provider and S3 bucket resource:

provider "aws" {
 region = "us-east-1"
}

resource "aws_s3_bucket" "data" {
 bucket = "my-data-bucket"
 acl    = "private"

 tags = {
   Name        = "My bucket"
   Environment = "Dev"
 }
}

Running terraform plan generates this execution plan:

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
 + create

Terraform will perform the following actions:

 # aws_s3_bucket.data will be created
 + resource "aws_s3_bucket" "data" {
     + acceleration_status         = (known after apply)
     + acl                         = "private"
     + arn                         = (known after apply)
     + bucket                      = "my-data-bucket"
     + bucket_domain_name          = (known after apply)
     + bucket_regional_domain_name = (known after apply)
     + force_destroy               = false
     + hosted_zone_id              = (known after apply)
     + region                      = (known after apply)
     + request_payer               = (known after apply)
     + tags                        = {
         + "Environment" = "Dev"
         + "Name" = "My bucket"
       }
     + website_domain              = (known after apply)
     + website_endpoint            = (known after apply)
   }

Plan: 1 to add, 0 to change, 0 to destroy.

This execution plan clearly shows the S3 bucket resource that will be created, including the settings that will be configured. Reviewing the plan before applying allows validation of the expected changes.

Applying Changes with Terraform

Once an execution plan has been reviewed and validated, the changes can be applied with:

terraform apply

For the above example, Terraform will create the S3 bucket according to the plan:

aws_s3_bucket.data: Creating...
aws_s3_bucket.data: Creation complete after 1s (ID: my-data-bucket)

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

The apply command executes the proposed actions from the execution plan, provisioning or changing infrastructure as outlined.

Iterative Development with Terraform

An iterative plan and apply cycle allows infrastructure to be built incrementally:

  1. Initial plan - outlines baseline resources
  2. Apply - provisions initial resources
  3. Modified plan - previews changes to baseline
  4. Apply - provisions incremental changes

This workflow allows large infrastructure environments to be constructed iteratively while maintaining control and visibility at each step.

Execution plans are integral to real-world Terraform usage. By previewing changes via terraform plan, infrastructure can be built collaboratively and iteratively, mitigating risk in the provisioning process.

Infrastructure Provisioning with Terraform Examples

Basic Terraform Apply Example

Here is a simple example to demonstrate using terraform apply to provision infrastructure:

# Configure AWS provider  
provider "aws" {
 region = "us-east-1"
}

# Create VPC
resource "aws_vpc" "example" {
 cidr_block = "10.0.0.0/16"
}

# Create subnet
resource "aws_subnet" "example" {
 vpc_id     = aws_vpc.example.id
 cidr_block = "10.0.1.0/24"
}

# Run terraform apply to provision resources
terraform apply

This applies the Terraform configuration to create a new VPC and subnet in AWS. When terraform apply is executed, the resources are created in the desired region. This demonstrates the simplicity of using Terraform to manage infrastructure as code.

Key things to note:

  • Provider is configured for AWS access
  • Resources like VPC and subnet are defined
  • terraform apply provisions the resources

By codifying infrastructure in this way, it can easily be shared, reused, and automated.

Complex Infrastructure Scenarios

For more complex setups with multiple environments and resources, Terraform modules can help organize the configuration while allowing reuse:

# Module to create VPC
module "vpc" {
 source = "./modules/vpc"
 
 name = "main-vpc"
 cidr = "10.0.0.0/16"
}

# Module to create web subnet
module "web_subnet" {
 source = "./modules/subnet"

 name = "web"
 vpc_id = module.vpc.id  
 cidr = "10.0.1.0/24"
}

Additional Terraform capabilities like count, splat expressions, and functions allow infrastructure patterns to be dynamically created:

# Create multiple subnets
resource "aws_subnet" "subnets" {
 count = 3
 
 vpc_id = module.vpc.id
 cidr_block = "10.0.${count.index}.0/24"
}

This allows for reusable, maintainable, and scalable infrastructure across environments.

Version Control in Infrastructure Changes

Using Git for version control is critical when using Terraform to manage infrastructure:

# Make changes locally

# Add and commit changes
git add .
git commit -m "Create additional subnet"

# Push changes to master
git push origin master

# Run terraform apply to deploy changes  
terraform apply

This helps coordinate changes made by multiple team members and provides an audit log of all infrastructure changes. Other best practices like pull requests and policy enforcement can further improve infrastructure management.

Managing and Updating Infrastructure with Terraform

Terraform allows you to manage and update your infrastructure in a controlled, predictable way. Here are some best practices for changing infrastructure that's already deployed.

Infrastructure Lifecycle Concepts

Terraform includes some basic commands for managing infrastructure changes:

  • terraform destroy completely removes deployed infrastructure
  • terraform taint marks a resource as tainted so it will be destroyed and recreated on the next apply
  • terraform untaint reverses a terraform taint command

These provide low-level control for updating infrastructure.

Automated Policy Enforcement

Sentinel policies let you codify organizational rules that control and govern infrastructure changes. For example, you could restrict production infrastructure changes to certain times or require peer approval for modifications. Defining these rules upfront ensures consistency.

Upgrade Strategies for Zero Downtime

When upgrading infrastructure like databases, you want to avoid downtime. Techniques like blue-green deployment, canary releases, and staged rollouts allow phased upgrades to prevent disruption. Terraform facilitates these patterns.

Performing Safe Infrastructure Changes with Terraform Plan

terraform plan lets you preview exactly what changes will occur before applying them. This "dry run" capability ensures infrastructure updates happen as expected without surprises. Review the plan output to validate no unintended changes will happen during the next terraform apply.

Leveraging these Terraform best practices will enable your team to safely modify existing infrastructure. Changes can be implemented in a controlled, predictable way to avoid unexpected errors or service disruption.

Security Best Practices with Terraform

Terraform enables secure infrastructure provisioning and management by codifying configurations into human-readable files that can be collaboratively edited, reviewed, and validated before deployment. However, additional measures should be taken to harden security. Here are some best practices:

Secret Management Options

Sensitive data like passwords and API keys should never be stored directly in Terraform configuration files. Instead, integrate Terraform with secret management solutions like:

  • HashiCorp Vault to dynamically fetch secrets during provisioning.
  • AWS SSM Parameter Store for retrieving parameters.
  • AWS KMS for encryption/decryption.

By keeping secrets in a separate system, you avoid accidental exposure in your code repository.

Hardening Infrastructure Configurations

Security focused configuration baselines like CIS Benchmarks can be integrated into Terraform modules to auto-remediate non-compliant settings and enforce organizational policies.

Audit Logging and Monitoring

Enable detailed logging and monitoring around Terraform runs to track changes, access attempts, and errors. Solutions like AWS CloudTrail, CloudWatch, and third-party SIEMs provide auditing capabilities.

Ensuring Secure Provisions with Execution Plans

Always generate and inspect Terraform execution plans before applying changes. The human-readable plan output lets you double check that the right resources are being created, updated or destroyed per your expectations. This prevents unwanted modifications to critical infrastructure.

Conclusion

Terraform is a powerful infrastructure as code tool that can greatly simplify provisioning and managing infrastructure. By allowing you to define infrastructure in configuration files, Terraform enables consistent and repeatable deployments across environments.

Here are some key takeaways for using Terraform effectively:

  • Use modules to encapsulate infrastructure components and promote reusability. Breaking configurations into modules makes them easier to maintain and share.
  • Implement automated testing for Terraform code to validate changes and catch issues early. Unit tests and integration tests are critical for production-grade infrastructure.
  • Use remote backends to enable collaboration and protect the state file. The remote backend improves Terraform workflow for teams.
  • Manage Terraform state carefully. Don't modify it manually and take backups before making impactful changes.
  • Tag resources properly for better organization. Tags make it easier to manage infrastructure across accounts and environments.

With some planning and best practices, Terraform can enable infrastructure provisioning at scale while minimizing risks. It's a reliable platform for delivering production-ready infrastructure repeatedly and efficiently.