Want to Contribute to us or want to have 15k+ Audience read your Article ? Or Just want to make a strong Backlink?

The issue of recursive module calls in declarative infrastructure-as-code

Just a few years in the past once I was working solely with the AWS platform I used to be early to leap on the Cloud Growth Equipment (CDK) practice. I had been utilizing AWS CloudFormation and HashiCorp Terraform for a couple of years for all my infrastructure-as-code wants up till then. Nevertheless, I by no means obtained snug utilizing the CDK and deserted it lengthy earlier than it reached 1.0.

Why was the CDK making me uncomfortable? To me it simply didn’t present any advantages over the standard declarative method. In the long run the CDK code appeared like barely extra complicated declarative code. It could possibly be mapped one-to-one with an equal declarative template. There have been a couple of issues that had been higher than the declarative method although, however remember the fact that I used to be evaluating CDK primarily to CloudFormation at the moment. Maybe the most effective profit with the CDK was that you possibly can do any sort of string and array manipulation your chosen programming language provided. This was (is) a desperately wanted characteristic in CloudFormation.

Quick-forward a couple of years and enter Azure Bicep! Bicep supplies many issues that CloudFormation doesn’t have (ignoring platform-specific issues on this comparability in fact). There may be not a lot feature-wise I’m lacking from Azure Bicep. We will talk about what I do miss in Bicep in one other publish.

Nevertheless, I not too long ago tried to realize one thing involving recursive module calls in Bicep and shortly realized that it isn’t potential. The Bicep language server even warns you in your editor in case you are attempting to create a recursive loop of module calls. I turned to Terraform to see if HashiCorp has launched assist for recursive module calls. Initially the HashiCorp Developer AI really advised me that it ought to certainly be potential, so I used to be hopeful. It seems that you could attempt to make recursive module calls, there isn’t any fast warning in your editor. Nevertheless, when you run a terraform init you notice that Terraform is attempting to dig an infinitely deep gap of module reference in module reference and it will definitely errors out.

So what’s it I’m attempting to do with recursive module calls?



A use-case for recursive module calls

The use-case I’m attempting to unravel is that I need to outline a construction of Azure administration teams and subscriptions in a easy YAML file, or one thing comparable. I picked YAML at first, however JSON would work too in addition to defining the construction within the chosen declarative language (Bicep, Terraform).

For this publish let’s focus solely on administration teams. My concept was to outline the construction like this:

id: mg-root
identify: Tenant Root Group
youngsters:
  - id: mg-contoso
    identify: Contoso
    youngsters:
      - id: mg-platform
        identify: Platform
        youngsters:
          - id: mg-identity
            identify: Id
          - id: mg-management
            identify: Administration
          - id: mg-connectivity
            identify: Connectivity
      - id: mg-landing-zones
        identify: Touchdown Zones
        youngsters:
          - id: mg-sap
            identify: SAP
          - id: mg-corp
            identify: Corp
          - id: mg-online
            identify: On-line
      - id: mg-decommissioned
        identify: Decommissioned
      - id: mg-sandbox
        identify: Sandbox
Enter fullscreen mode

Exit fullscreen mode

The pattern construction is fetched from the Azure Landing Zone documentation.

I might then prefer to learn this file in Bicep (or Terraform) and thru intelligent recursive module calls create this construction of administration teams.



A proposed resolution with Azure Bicep

To resolve this with Bicep my method was to have the next major.bicep file:

targetScope="tenant"

var knowledge = loadYamlContent('knowledge.yaml')

useful resource root 'Microsoft.Administration/managementGroups@2023-04-01' current = {
  identify: knowledge.id
}

module recursive 'modules/recursive.bicep' = [for (child, index) in data.children: {
  name: 'child-module-${index}'
  params: {
    name: child.name
    children: child.children
    id: child.id
    parentId: root.id
  }
}]
Enter fullscreen mode

Exit fullscreen mode

The recursive module file modules/recursive.bicep seems to be like this:

targetScope="tenant"

param parentId string
param id string
param identify string
param youngsters array = []

useful resource mg 'Microsoft.Administration/managementGroups@2023-04-01' = {
  identify: id
  properties: {
    particulars: {
      mum or dad: {
        id: parentId
      }
    }
    displayName: identify
  }
}

module childMgs 'recursive.bicep' = [for (child, index) in children: {
  name: 'child-module-${id}-${index}'
  params: {
    name: child.name
    children: child.children
    id: child.id
    parentId: mg.id
  }
}]
Enter fullscreen mode

Exit fullscreen mode

I believed it was a good suggestion, however Bicep didn’t agree. I’ve submitted a proposal to the Bicep crew for a way this may be allowed. Vote for this difficulty in case you agree!



A proposed resolution with HashiCorp Terraform

To resolve this with Terraform my method was to have the next major.tf file:

terraform {
  required_providers {
    azurerm = {
      supply = "hashicorp/azurerm"
    }
  }
}

supplier "azurerm" {
  options {}
}

locals {
  knowledge = jsondecode(file("knowledge.json"))
}

module "youngsters" {
  supply   = "./modules/recursive"
  youngsters = native.knowledge.youngsters
  identify     = native.knowledge.id
  mum or dad   = native.knowledge.mum or dad
}
Enter fullscreen mode

Exit fullscreen mode

Terraform has no built-in assist to learn YAML so I transformed the file to JSON and skim it utilizing jsondecode(...).

The recursive module file modules/recursive/major.tf seems to be like this:

terraform {
  required_providers {
    azurerm = {
      supply = "hashicorp/azurerm"
    }
  }
}

variable "identify" {
  sort = string
}

variable "mum or dad" {
  sort = string
}

variable "youngsters" {
  sort = checklist(object({
    id       = string
    identify     = string
    youngsters = checklist(any)
  }))
}

useful resource "azurerm_management_group" "this" {
  identify                       = var.identify
  parent_management_group_id = var.mum or dad
}

module "recursive" {
  for_each = toset(var.youngsters)
  supply   = "./"
  identify     = every.worth.identify
  mum or dad   = azurerm_management_group.this.id
  youngsters = every.worth.youngsters
}
Enter fullscreen mode

Exit fullscreen mode

To be trustworthy I’m not certain this could have labored even when recursive module calls had been allowed, however at the very least my editor is just not complaining at this level. Once I run terraform init nonetheless:

$ terraform init
Initializing the backend...
Initializing modules...
╷
│ Error: Did not take away native module cache
│
│ Terraform tried to take away
│ .terraform/modules/youngsters.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive
│ to be able to reinstall this module, however encountered an error: unlinkat
│ .terraform/modules/youngsters.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive.recursive:
│ file identify too lengthy
Enter fullscreen mode

Exit fullscreen mode

Terraform doesn’t know I make recursive module calls, nevertheless it does its finest to search out the tip of the recursive calls however finally finally ends up complaining concerning the size of a filename.



Fixing the issue utilizing the Cloud Growth Equipment for Terraform

Within the introduction I spoke concerning the CDK. CDK is particularly for AWS infrastructure. Nevertheless, a couple of years in the past a brand new device referred to as Cloud Growth Equipment for Terraform (CDKTF) arrived. CDKTF follows the identical construction because the CDK. I like to recommend that you just learn by way of the CDKTF documentation in case you are to be taught extra, as a result of I cannot clarify the small print of CDKTF on this publish.

I wrote my CDKTF code utilizing TypeScript, however there are different options accessible.

I outlined the administration group construction in TypeScript in managementGroups.ts:

export sort ManagementGroupDefinition = {
    identify: string
    mum or dad?: string
    youngsters?: ManagementGroupDefinition[]
  }

export const managementGroups: ManagementGroupDefinition = {
    identify: "Pseudo Root Group",
    mum or dad: "/suppliers/Microsoft.Administration/managementGroups/<my tenant id>",
    youngsters: [
        {
            name: "Contoso",
            children: [
                {
                    name: "Platform",
                    children: [
                        {
                            name: "Identity"
                        },
                        {
                            name: "Management"
                        },
                        {
                            name: "Connectivity"
                        }
                    ]
                },
                {
                    identify: "Touchdown Zones",
                    youngsters: [
                        {
                            name: "SAP"
                        },
                        {
                            name: "Corp"
                        },
                        {
                            name: "Online"
                        }
                    ]
                },
                {
                    identify: "Decommissioned"
                },
                {
                    identify: "Sandbox"
                }
            ]
        }
    ]
}
Enter fullscreen mode

Exit fullscreen mode

You might outline the construction in YAML as earlier than, however for simplicity I outlined it as TypeScript. One factor to notice is that I’ve not included the id area on this construction. It’s because Terraform doesn’t permit me to outline a customized id for my administration teams. That is sadly not perfect, however I am going to let it slide for now. One other factor to notice is that I’ve included a mum or dad area within the root administration group. The mum or dad is my precise tenant root group, however I made a decision to create a pseudo root group as a substitute of working instantly within the precise root group.

Subsequent I’ve my CDKTF utility in major.ts:

import { Assemble } from "constructs";
import { App, TerraformStack } from "cdktf";
import { AzurermProvider } from "@cdktf/provider-azurerm/lib/supplier"
import { ManagementGroup } from "@cdktf/provider-azurerm/lib/management-group"
import { managementGroups, ManagementGroupDefinition } from "./managementGroups"

class Stack extends TerraformStack {
  constructor(scope: Assemble, id: string) {
    tremendous(scope, id);

    new AzurermProvider(this, "azurerm", {
      options: {}
    })

    this.makeLayer(managementGroups)
  }

  makeLayer(config: ManagementGroupDefinition) {
    const mum or dad = new ManagementGroup(this, config.identify, {
      displayName: config.identify,
      parentManagementGroupId: config.mum or dad ?? undefined
    })

    config.youngsters?.forEach( (baby) => {
      this.makeLayer({ ...baby, mum or dad: mum or dad.id })
    })
  }
}

const app = new App();
new Stack(app, "cdktf");
app.synth();
Enter fullscreen mode

Exit fullscreen mode

The magic occurs within the makeLayer methodology of my Stack class. That is the place I create a brand new ManagementGroup with a given displayName and an non-obligatory parentManagementGroupId. Subsequent I loop over every baby to this administration group and as soon as once more name makeLayer, and right here we have now the recursion!

CDKTF constructs a sound Terraform template from this major.ts file. No want for infinite recursive module calls as a result of CDKTF is aware of the recursive loop has an finish.

To be trustworthy, the ensuing Terraform configuration doesn’t really use modules. So the outcome is just not a “resolution” to the recursion drawback. What CDKTF does right here is generate a single configuration with all my administration teams outlined, it doesn’t introduce a module and make recursive calls to it.



Abstract

At present neither Azure Bicep or HashiCorp Terraform helps recursive module calls out of the field. That is one use-case the place an crucial method to infrastructure-as-code wins. I’m nonetheless not satisfied the crucial method is price it in the long term, I desire the readability of the declarative method.

In fact there’s the center floor, you possibly can use an crucial method to generate a declarative template which you’ll then deploy. And that is what occurs below the hood with CDKTF anyway.

As with every part else it relies on what you need to do. To date in my profession this was the primary time I attempted to do one thing in a declarative method that was simply not supported.

Oh and by the best way, I do know one may argue for that I’m attempting to put in writing crucial code utilizing a declarative language once I do recursive module calls – however that’s one other dialogue.

Add a Comment

Your email address will not be published. Required fields are marked *

Want to Contribute to us or want to have 15k+ Audience read your Article ? Or Just want to make a strong Backlink?