Terraform Variable Cross Validation

Introduction

#

I have previously written a few posts on the topic of testing and validation with Terraform:

One part of validation in Terraform is the use of validation blocks in your variable blocks. A variable block can contain zero, one, or many validation blocks. A validation block allows you to verify that the provided value for the variable fulfills a condition. If it does, Terraform will allow the plan and apply phases to continue. If it does not fulfill the condition the validation fails and Terraform stops the plan and apply phases.

In this post we will learn about a new feature available in Terraform 1.9 that allows us to reference any type of object in our variable validation blocks.

Validation in Terraform 1.8 and earlier

#

You should use validation blocks to help users of your modules to avoid making unnecessary mistakes that might cause an apply to fail later on. You should not use validation blocks to enforce your organization policies, use a policy as code framework for that!

An example of a good use of a validation block is if you are using the value of a variable as part of naming a resource, and this resource has constraints for how long (or short) the name can be. You can express this in Terraform like this:

variable "name_suffix" {
  type = string

  validation {
    condition     = length(var.name_suffix) <= 20
    error_message = "Name suffix should be 20 characters or less"
  }
}

In the validation block for the name_suffix variable we can only reference the variable value (using var.name_suffix) and hard-coded data (like the value 20), but nothing else.

What if, instead of hard-coding the length constraint for the name_suffix variable we instead had another variable for that value?

variable "name_suffix_max_length" {
  type    = number
  default = 20
}

variable "name_suffix" {
  type = string

  validation {
    condition     = length(var.name_suffix) <= 20
    error_message = "Name suffix should be at most 20 characters long"
  }
}

How can we validate that the name_suffix variable respects the value of the name_suffix_max_length variable? In Terraform 1.8 or earlier, this is impossible.

Enter Terraform 1.9.

Validation in Terraform 1.9

#

A highlight from Terraform 1.9¹ is that validation blocks can now reference other objects. This includes other variables, data sources, local values, and resources. This is a game-changer for variable validation!

This feature is available in Terraform 1.9.0 and forwards. If you are planning on using this feature make sure your Terraform version constraint is not blocking you:

terraform {
  required_version = "~> 1.9.0"
}

Reference to other variables

#

My guess is that referencing other variables in the validation block is what this feature will be most used for.

Remember that being able to reference other objects does not remove the requirement to also refer to the current variable in the validation block. This means you can’t write a condition that is completely external to the variable you are validating, i.e. something like this is not allowed:

variable "my_variable" {
  type = string
}

variable "my_other_variable" {
  type = string

  validation {
    condition     = contains(["foo", "bar"], var.my_variable)
    error_message = "Error ..."
  }
}

Returning now to the example in the previous section, we can use the new feature to improve the validation of our name_suffix variable:

variable "name_suffix_max_length" {
  type    = number
  default = 20
}

variable "name_suffix" {
  type = string

  validation {
    condition     = length(var.name_suffix) <= var.name_suffix_max_length
    error_message = "Name suffix should be ${var.name_suffix_max_length} characters or less"
  }
}

Providing a default value for name_suffix_max_length keeps the validation as before, but now we have the possibility to change the max length by passing in a different value.

A similar example could be if we have a variable that should follow a given pattern, but we want to be able to change this pattern using another variable:

variable "pattern" {
  description = "Regex pattern for allowed characters"
  type        = string
  default     = "^[a-z][a-z0-9]+"
}

variable "name_suffix" {
  type = string

  validation {
    condition     = can(regex(var.pattern, var.name_suffix))
    error_message = "Invalid name suffix provided"
  }
}

The pattern variable is a regex pattern that the name_suffix variable uses in its validation block. The default value expects the name_suffix value to start with a small character, then contain any number of small characters or numbers.

Reference local values

#

Local values are most often used to convert data from one format to another, or to combine data into other data structures.

As an example, we have data stored in a CSV file describing what AWS EC2 instance types we allow for a given AMI ID:

instance_type,ami
t2.micro,ami-11a1a11a
t3.micro,ami-22b2b22b
t3.nano,ami-33c3c33c
m3.large,ami-44d4d44d

The data is not directly usable as-is so we would like to transform it. We can parse the data using csvdecode and then do the transformations we need:

locals {
  data        = csvdecode(file("data.csv"))
  amis        = [for i in local.data : i.ami]
  ami_to_type = tomap({ for i in local.data : i.ami => i.instance_type })
}

We end up with a local value amis which is a list of AMI IDs, and a local value ami_to_type which is a map with the following format:

{
  ami-11a1a11a = "t2.micro"
  ami-22b2b22b = "t3.micro"
  ami-33c3c33c = "t3.nano"
  ami-44d4d44d = "m3.large"
}

We can use these local values for our variable validations:

variable "ami" {
  type = string

  validation {
    condition     = contains(local.amis, var.ami)
    error_message = "Use an existing AMI ID"
  }
}

variable "instance_type" {
  type = string

  validation {
    condition     = local.ami_to_type[var.ami] == var.instance_type
    error_message = "Instance size not allowed for AMI ID ${var.ami}"
  }
}

For the ami variable we make sure an existing AMI ID is provided. In the instance_type variable we check that the provided value is allowed for the selected AMI.

This example is a bit contrived, but the general pattern of reading data, converting it, then using it for validations is widely applicable.

Reference to data sources

#

Being able to reference data sources in our validation block opens up some interesting use-cases.

A simple example that can be applied to many use-cases is if we want to validate a variable value using data stored in a file:

data "local_file" "prefixes" {
  filename = "prefixes.txt"
}

variable "name_prefix" {
  type = string

  validation {
    condition     = contains(split("\n", data.local_file.prefixes.content), var.name_prefix)
    error_message = "Use one of the allowed prefixes defined in ${data.local_file.prefixes.filename}"
  }
}

The example above has a text file containing potentially hundreds of lines of text. Using the local_file data source we can read this file and use the content in the validation block of the name_prefix variable.

Another example with a wide range of use-cases is to read some data from a cloud provider and use this data in a variable validation. Here is an example from the Azure world:

# read data for a storage account
data "azurerm_storage_account" "this" {
  name                = "tfvalidation"
  resource_group_name = "rg-storage"
}

# read data for all blob containers in the storage account
data "azurerm_storage_containers" "all" {
  storage_account_id = data.azurerm_storage_account.this.id
}

variable "container_name" {
  type = string

  validation {
    condition = contains(
      [for container in data.azurerm_storage_containers.all.containers : container.name],
      var.container_name
    )
    error_message = "Use an existing blob container"
  }
}

Here we have used the azurerm_storage_containers data source to read all blob containers in an Azure storage account. Then we use this data in the validation block of the container_name variable in order to make sure the provided value is the name of an existing blob container.

Reference to resources

#

You can also reference attributes of resources in variable validations. However, this is where things get tricky.

Resources have some attributes that are known directly at the plan phase, and some attributes that are only known after the apply phase. You can refer to both types of attributes in your variable validations.

When you craft your validation blocks you should understand if you are using plan phase attributes or apply phase attributes. This will determine when the variable validation will potentially stop Terraform from continuing. If it is OK for Terraform to create a few resources, then run the validation blocks and potentially stop, then it is OK to use apply phase attributes in your validation blocks. However, if you do not want Terraform to go ahead and create any resources before every validation has passed then you should make sure to only use plan phase attributes in your validation blocks.

There might be good use-cases for apply phase attributes, but I have currently not thought of any. If you have a killer use-case for using apply phase resource attributes in your variable validations, let me know!

Also, most plan phase resource attributes will most likely also be known in some other way: through variables or local values. This is because plan phase resource attributes are either attributes you provide yourself, or they are default values for the given resource type. So although you could use resource attributes in your variable validations there will most likely be a simpler way of doing it.

Just to illustrate that it is possible, consider this contrived example for an AWS VPC:

resource "aws_vpc" "this" {
  cidr_block = "10.0.0.0/16"

  tags = {
    Name  = "The VPC"
    Owner = "Team A"
  }
}

variable "team" {
  type = string

  validation {
    condition     = lookup(aws_vpc.this.tags, "Owner", "Invalid") == var.team
    error_message = "Provide the correct team name"
  }
}

The validation block for the team variable checks that the provided value is the same as the Owner tag on the aws_vpc resource. As I said, it is a contrived example but I am sure there will be legit use-cases for using resource attributes in variable validations showing up in the coming weeks.

How does this change your Terraform tests?

#

The new variable validation feature will have an impact on your Terraform tests. Testing your variable validation becomes more important than before.

Consider the following Terraform variable configuration:

variable "first" {
  type = string
}

variable "second" {
  type = string
}

variable "third" {
  type = string
}

variable "name" {
  type = string

  validation {
    condition     = length(var.name) + length(var.first) + length(var.second) + length(var.third) < 50
    error_message = "Combined length is too long!"
  }
}

There are four variables in total: first, second, third, and name. All variables will be combined somehow (not shown), and we have a validation block for the name variable that makes sure the combined length of all the string variables is less than 50 characters.

To test this validation we could write a few tests like this:

variables {
  first  = "firstname"
  second = "secondname"
  third  = "thirdname"
  name   = "theactualname"
}

run "invalid_first" {
  command = plan

  variables {
    first = "thisnameiswaytoolong"
  }

  expect_failures = [
    var.name,
  ]
}

run "invalid_second" {
  command = plan

  variables {
    second = "thisnameiswaytoolong"
  }

  expect_failures = [
    var.name,
  ]
}

run "invalid_third" {
  command = plan

  variables {
    third = "thisnameiswaytoolong"
  }

  expect_failures = [
    var.name,
  ]
}

The important thing to remember for your tests is to test all possible combinations (or all classes of possible combinations) of the variables that are used in a given validation block.

As a second example let’s return to our Azure Terraform configuration from above. In that example we had a variable named container_name that should be the value of an existing blob container in an Azure Storage Account. Using mocks for our data sources we can write tests for this:

# mocks for the azurerm provider
mock_provider "azurerm" {

  # mock the data source for the azure storage account
  mock_data "azurerm_storage_account" {
    defaults = {
      id = "/subscriptions/1/resourceGroups/2/providers/Microsoft.Storage/storageAccounts/3"
    }
  }

  # mock the data source for the blob containers
  mock_data "azurerm_storage_containers" {
    defaults = {
      containers = [
        {
          name = "first"
        },
        {
          name = "second"
        }
      ]
    }
  }
}

variables {
  container_name = "first"
}

# using the global variable value should result in a passing test
run "valid" {
  command = plan
}

# overriding the value with an invalid blob container
# should fail the validation
run "invalid" {
  command = plan

  variables {
    container_name = "third"
  }

  expect_failures = [
    var.container_name,
  ]
}

Summary

#

A highlighted feature of Terraform 1.9 covered in this blog post concerns variable validation, specifically the ability to reference other variables, local values, data sources, and resources in our variable validations.

After seeing variable validation in Terraform 1.9 one could almost argue that variable validation before Terraform 1.9 was broken. It will be interesting to see the adoption of this feature. I hope it will lead to increased use of validation blocks where it makes sense.