A few years ago when I was working exclusively with the AWS platform I was early to jump on the Cloud Development Kit (CDK) train. I had been using AWS CloudFormation and HashiCorp Terraform for a few years for all my infrastructure-as-code needs up until then. However, I never got comfortable using the CDK and abandoned it long before it reached 1.0.
Why was the CDK making me uncomfortable? To me it just did not provide any benefits over the traditional declarative approach. In the end the CDK code looked like slightly more complex declarative code. It could be mapped one-to-one with an equivalent declarative template. There were a few things that were better than the declarative approach though, but keep in mind that I was comparing CDK primarily to CloudFormation at that time. Perhaps the best benefit with the CDK was that you could do any kind of string and array manipulation your chosen programming language offered. This was (is) a desperately needed feature in CloudFormation.
Fast-forward a few years and enter Azure Bicep! Bicep provides many things that CloudFormation does not have (ignoring platform-specific things in this comparison of course). There is not much feature-wise I am missing from Azure Bicep. We can discuss what I do miss in Bicep in another post.
However, I recently tried to achieve something involving recursive module calls in Bicep and quickly realized that it is not possible. The Bicep language server even warns you in your editor if you are trying to create a recursive loop of module calls. I turned to Terraform to see if HashiCorp has introduced support for recursive module calls. Initially the HashiCorp Developer AI actually told me that it should indeed be possible, so I was hopeful. It turns out that you can try to make recursive module calls, there is no immediate warning in your editor. However, once you run a terraform init you realize that Terraform is trying to dig an infinitely deep hole of module reference in module reference and it eventually errors out.
So what is it I am trying to do with recursive module calls?
A use-case for recursive module calls#
The use-case I am trying to solve is that I want to define a structure of Azure management groups and subscriptions in a simple YAML file, or something similar. I picked YAML at first, but JSON would work too as well as defining the structure in the chosen declarative language (Bicep, Terraform).
For this post let’s concentrate only on management groups. My idea was to define the structure like this:
id: mg-root
name: Tenant Root Group
children:
- id: mg-contoso
name: Contoso
children:
- id: mg-platform
name: Platform
children:
- id: mg-identity
name: Identity
- id: mg-management
name: Management
- id: mg-connectivity
name: Connectivity
- id: mg-landing-zones
name: Landing Zones
children:
- id: mg-sap
name: SAP
- id: mg-corp
name: Corp
- id: mg-online
name: Online
- id: mg-decommissioned
name: Decommissioned
- id: mg-sandbox
name: Sandbox
The sample structure is fetched from the Azure Landing Zone documentation.
I would then like to read this file in Bicep (or Terraform) and through clever recursive module calls create this structure of management groups.
A proposed solution with Azure Bicep#
To solve this with Bicep my approach was to have the following main.bicep file:
targetScope = 'tenant'
var data = loadYamlContent('data.yaml')
resource root 'Microsoft.Management/managementGroups@2023-04-01' existing = {
name: data.id
}
module recursive 'modules/recursive.bicep' = [for (child, index) in data.children: {
name: 'child-module-${index}'
params: {
name: child.name
children: child.children
id: child.id
parentId: root.id
}
}]
The recursive module file modules/recursive.bicep looks like this:
targetScope = 'tenant'
param parentId string
param id string
param name string
param children array = []
resource mg 'Microsoft.Management/managementGroups@2023-04-01' = {
name: id
properties: {
details: {
parent: {
id: parentId
}
}
displayName: name
}
}
module childMgs 'recursive.bicep' = [for (child, index) in children: {
name: 'child-module-${id}-${index}'
params: {
name: child.name
children: child.children
id: child.id
parentId: mg.id
}
}]
I thought it was a good idea, but Bicep did not agree. I have submitted a proposal to the Bicep team for how this can be allowed. Vote for this issue if you agree!
A proposed solution with HashiCorp Terraform#
To solve this with Terraform my approach was to have the following main.tf file:
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
}
}
}
provider "azurerm" {
features {}
}
locals {
data = jsondecode(file("data.json"))
}
module "children" {
source = "./modules/recursive"
children = local.data.children
name = local.data.id
parent = local.data.parent
}
Terraform has no built-in support to read YAML so I converted the file to JSON and read it using jsondecode(...).
The recursive module file modules/recursive/main.tf looks like this:
terraform {
required_providers {
azurerm = {
source = "hashicorp/azurerm"
}
}
}
variable "name" {
type = string
}
variable "parent" {
type = string
}
variable "children" {
type = list(object({
id = string
name = string
children = list(any)
}))
}
resource "azurerm_management_group" "this" {
name = var.name
parent_management_group_id = var.parent
}
module "recursive" {
for_each = toset(var.children)
source = "./"
name = each.value.name
parent = azurerm_management_group.this.id
children = each.value.children
}
To be honest I am not sure this would have worked even if recursive module calls were allowed, but at least my editor is not complaining at this point. When I run terraform init however:
$ terraform init
Initializing the backend...
Initializing modules...
╷
│ Error: Failed to remove local module cache
│
│ Terraform tried to remove
│ .terraform/modules/children.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive
│ in order to reinstall this module, but encountered an error: unlinkat
│ .terraform/modules/children.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive:
│ file name too long
Terraform does not know I make recursive module calls, but it does its best to find the end of the recursive calls but eventually ends up complaining about the length of a filename.
Solving the problem using the Cloud Development Kit for Terraform#
In the introduction I spoke about the CDK. CDK is specifically for AWS infrastructure. However, a few years ago a new tool called Cloud Development Kit for Terraform (CDKTF) arrived. CDKTF follows the same structure as the CDK. I recommend that you read through the CDKTF documentation if you are interested to learn more, because I will not explain the details of CDKTF in this post.
I wrote my CDKTF code using TypeScript, but there are other alternatives available.
I defined the management group structure in TypeScript in managementGroups.ts:
export type ManagementGroupDefinition = {
name: string
parent?: string
children?: ManagementGroupDefinition[]
}
export const managementGroups: ManagementGroupDefinition = {
name: "Pseudo Root Group",
parent: "/providers/Microsoft.Management/managementGroups/<my tenant id>",
children: [
{
name: "Contoso",
children: [
{
name: "Platform",
children: [
{
name: "Identity"
},
{
name: "Management"
},
{
name: "Connectivity"
}
]
},
{
name: "Landing Zones",
children: [
{
name: "SAP"
},
{
name: "Corp"
},
{
name: "Online"
}
]
},
{
name: "Decommissioned"
},
{
name: "Sandbox"
}
]
}
]
}
You could define the structure in YAML as before, but for simplicity I defined it as TypeScript. One thing to note is that I have not included the id field in this structure. This is because Terraform does not allow me to define a custom id for my management groups. This is unfortunately not ideal, but I’ll let it slide for now. Another thing to note is that I have included a parent field in the root management group. The parent is my actual tenant root group, but I decided to create a pseudo root group instead of working directly in the actual root group.
Next I have my CDKTF application in main.ts:
import { Construct } from "constructs";
import { App, TerraformStack } from "cdktf";
import { AzurermProvider } from "@cdktf/provider-azurerm/lib/provider"
import { ManagementGroup } from "@cdktf/provider-azurerm/lib/management-group"
import { managementGroups, ManagementGroupDefinition } from "./managementGroups"
class Stack extends TerraformStack {
constructor(scope: Construct, id: string) {
super(scope, id);
new AzurermProvider(this, "azurerm", {
features: {}
})
this.makeLayer(managementGroups)
}
makeLayer(config: ManagementGroupDefinition) {
const parent = new ManagementGroup(this, config.name, {
displayName: config.name,
parentManagementGroupId: config.parent ?? undefined
})
config.children?.forEach( (child) => {
this.makeLayer({ ...child, parent: parent.id })
})
}
}
const app = new App();
new Stack(app, "cdktf");
app.synth();
The magic happens in the makeLayer method of my Stack class. This is where I create a new ManagementGroup with a given displayName and an optional parentManagementGroupId. Next I loop over each child to this management group and once again call makeLayer, and here we have the recursion!
CDKTF constructs a valid Terraform template from this main.ts file. No need for infinite recursive module calls because CDKTF knows the recursive loop has an end.
To be honest, the resulting Terraform configuration does not actually use modules. So the result is not a “solution” to the recursion problem. What CDKTF does here is generate a single configuration with all my management groups defined, it does not introduce a module and make recursive calls to it.
Summary#
Currently neither Azure Bicep or HashiCorp Terraform supports recursive module calls out of the box. This is one use-case where an imperative approach to infrastructure-as-code wins. I am still not convinced the imperative approach is worth it in the long run, I prefer the clarity of the declarative approach.
Of course there is the middle ground, you could use an imperative approach to generate a declarative template which you can then deploy. And this is what happens under the hood with CDKTF anyway.
As with everything else it depends on what you want to do. So far in my career this was the first time I tried to do something in a declarative way that was just not supported.
Oh and by the way, I know one could argue for that I am trying to write imperative code using a declarative language when I do recursive module calls - but that is another discussion.
