Test-driven development with Terraform

Published in

worldwide-writing-network

6 min readJun 11, 2021

The evolution of public cloud and cloud agnostic nature of Terraform has transformed the way we provision infrastructure in today’s world. Infra automation has given power to developers to automate their infra requirements. They can simply write code for what they need. If requirement changes, the code is adjusted to reflect the new demand. And at last if something is not needed it can be destroyed. Pretty fast right? But there is one bottleneck in this process. Every request for change has to go through a manual review process and it slows down the entire infra provisioning lifecycle.

Connect me on LinkedIn

What if we could do some automation around process to eliminate the manual review ? What if the code being written can be applied against a sandbox environment and verified against a set of test suites defined by the ops team for each resource type? And if the test cases pass then pipeline continues else it stops there. This is exactly what we will be discussing in this article. We will talk about automating terraform testing using tools like kitchen and Inspec to support test driven development in Terraform and eradicate any manual review process.

Imp Points

The setup uses 2 github repos and workflows.
Repo-1 contains the terraform code and CI/CD workflow that auto triggers with pull request to main branch. This workflow calls workflow -2 to provision and test infra against the sandbox environment. Workflow -1 waits for a response from workflow-2 before PR to main can be merged.
Repo-2 contains code for kitchen setup and CI/CD workflow which provisions the infra against sandbox environment using the code written in Repo-1 and run automated test against this infra. This workflow responds back to workflow-1 with the information whether code written in Repo-1 passes against the test suits enforced by ops team or some rework is needed.

Infra provisioning CI/CD workflow

Let’s look into the infra provisioning CI/CD workflow as per image below and understand what part we are trying to automate.

Terms Used ( Text highlighted in yellow in the image )

Landing Zone — it’s a repo ( in this example we are using github repo ) where we will write the terraform code to provision our infra.
TFE — Stands for Terraform Enterprise. This can be replaced by CLI or Terraform Cloud.

The scope of this article is to talk about the blocks in YELLOW in below image. These blocks facilitates the automation of infra provisioning along with automated testing using test suits against a sandbox environment using Kitchen and Inspec before the pull request can be merged and the code can be applied to application infrastructure. This will give us enough confidence to remove the manual review step as shown in below image.

Infra provisioning CI/CD setup using Terraform against GCP

Automated testing CI/CD workflow

Let’s see how this automated setup looks like. As shown in the image below, automated testing setup has 2 github repos and workflow.

Repo/workflow — 1 : This is the repo where developer will write the terraform code to provision infrastructure. When developer will raise a PR to merge to main brancj, the CI/CD build will trigger for automated testing.

Repo/workflow — 2 : This repo has environment to run kitchen. It also has test suites to run them against the infrastructure being provisioned by kitchen.

Note : workflow — 1 will pass the terraform code to workflow -2 and wait for the outcome. Based on outcome, the PR merge process in workflow -1 will continue further.

Tools used

Terraform CLI — This is used by kitchen terraform to provision infrastructure.
gcloud CLI — to run gcloud commands as part of testing.
Kitchen Terraform — Tool to automate testing with terraform.
inspec-gcp —Runs automated test cases against the infra.

Learn more about kitchen here.
Github repo for kitchen code here.
More about inspec-gcp here

Kitchen commands

We used below kitchen commands to automate the provisioning and testing the infra. Please check the usage in the code here.

kitchen create - This equates to terraform init
kitchen converge - This equates to terraform apply --auto-approve   
kitchen verify - Verifier(inspec.yml + controls)
kitchen destroy - This equates to terraform destroyPlease check README under amitkumardube/kitchen-terraform (github.com) for more details.

How to setup the kitchen environment?

We are using docker to setup the kitchen environment. Check this link to copy the Dockerfile. Please click here for instructions on how to build the image from the dockerfile provided. If you don’t want to build the docker image yourself , then please use the image available on public docker registry created by me.

# this is the base image 
FROM ruby:2.7.2 # Terraform client CLI version
ARG terraform_cli_version=0.14.9  # Copying Gemfile The Gemfile has instructions for bundler to install terraform-kitchenCOPY Gemfile . # installing all the dependencies as per the Gemfile
RUN bundle install # Adding Terraform to the image as kitchen requires terraform CLI to workADD https://releases.hashicorp.com/terraform/${terraform_cli_version}/terraform_${terraform_cli_version}_linux_amd64.zip . # Unzipping the zip terraform binary fileRUN unzip terraform_${terraform_cli_version}_linux_amd64.zip -d /usr/local/bin # Removing the compressed fileRUN rm terraform_${terraform_cli_version}_linux_amd64.zip

Kitchen-Terraform Configuration

Kitchen-Terraform works with the configuration in kitchen.yml file. kitchen.yml file has following structure. The each section is mentioned below along with the usage.

Please click here to explore the complete file.

Kitchen/Inspec file structure/organization

The infra that needs to be provisioned should be defined under fixtures. We can choose any other name if you like. The path is mentioned as here in kitchen.yml. In this example we are creating secrets and bigquery dataset and that’s why we have 2 folders under fixures.
For test cases, the structure is quite conventional. For each test suite, kitchen looks for a file called inspec.yml and folder called controls under directory test/integration/<test_suite_name>. In our example we have test suites big_query and secrets and that’s why we have created folders secrets and big_query under test/integration.
inspec.yml defines configuration for inspec and input variables for our test cases.
The test cases are written under controls folder. Example here. The test cases are written in Ruby. inspec-gcp plugin provides pre-defined resources which can be used with the ruby code to query the infrastructure. Example here.
We can also use gcloud with ruby code to query our infrastructure. Example here.