Dynamic Terraform Backend (in Azure)

I am a certified Terraform Associate engineer, yet I recently came across a something I was not aware of. How did I become certified without knowing everything you ask? The certification simply does not cover everything, and to be fair what I will explain in this article should be considered advanced functionality, outside of the core of what Terraform Associate engineers do.

Imagine you want to set up a remote backend for your Terraform configuration. A remote backend is a location where you will store your Terraform state file(s) that is not on your local computer. To make it more concrete: you want to store your remote state in a storage account in Azure. You would add a Terraform block looking something like the following (skipping the nitty-gritty details):

terraform {
  required_version = "..."
  required_providers {
    ...
  }
  backend "azurerm" {
    ...
  }
}

Now the question is, how do you configure the backend block in the above code snippet?

One major problem is that you can’t use variables or any other dynamic behavior in this block. Why is this a problem? Among the things you configure for this block when Azure is your backend of choice is a resource group name, a storage account name, a blob container name, and most importantly a key that represents the filename of the state file. Hard-coding the value of the key is the real problem, because that means your Terraform configuration will always use the same state file. This might not seem like a problem until you commit your Terraform configuration to GitHub and build workflows that should deploy your infrastructure. E.g. you might need a way to define different keys for different git branches.

That is where the following command flag (-backend-config) comes in handy:

terraform init \
  -backend-config="resource_group_name=<name>" \
  -backend-config="storage_account_name=<name>" \
  -backend-config="container_name=<name>" \
  -backend-config="key=${{ github.ref_name }}/my/path/terraform.tfstate"

What we have now achieved is a dynamic backend block. The provided -backend-config flags will set the corresponding properties of the backend configuration block. The important part in the above snippet is how the key is set: it will create a unique state file for each git branch. This means that you can spin up infrastructure for multiple branches at the same time, while avoiding any conflict in the state files. Note that the example above is intended to be part of a GitHub Actions workflow since I use the ${{ github.ref_name }} syntax, which is replaced by the current GitHub branch name. You can of course use any variable or reference to make it dynamic according to your needs.

I though that was very useful!