Terraform — flatten the collection type variable

Photo by Önder Örtel on Unsplash

Terraform let’s us manage our infrastructure with all the cloud providers. Recently I used terraform with GCP to deploy some complex infrastructure. While writing code I came across a complex collection type variable which I needed to loop through but it was not possible to do that in it’s existing form. Let’s examine the variable as mentioned below.

Connect me on LinkedIn

Important Points

• The code explained this example can be found at terraform-functions-as-modules/flatten at main · amitkumardube/terraform-functions-as-modules (github.com)
• The resources used in this example are related to GCP.

variable.tf

variable "bucket_user_roles" {    
               type = list(object({        
                   bucket_name = string        
                   roles = list(object({            
                     role_name = string            
                     members = list(string)        
                   }))    
               }))
}

terraform.tfvars — The value of the variable

bucket_user_roles = [
    {        
     bucket_name = "bucket-1"        
     roles = [            
           {                
                role_name = "roles/storage.admin"                
                members = ["abc@gmail.com","def@gmail.com"]            
           }, 
           {
                role_name = "roles/storage.legacyBucketReader"
                members = ["abc@gmail.com","def@gmail.com"]   
           },
           {
                role_name = "roles/storage.legacyBucketWriter"
                members = ["abc@gmail.com","def@gmail.com"]
            },
             ]
    },
    {        
     bucket_name = "bucket-2"        
     roles =  [
            {
                role_name = "roles/storage.admin"
                members = ["ghi@gmail.com","jkl@gmail.com"]
            },
            {
                role_name = "roles/storage.legacyBucketReader"
                members = ["ghi@gmail.com","jkl@gmail.com"]
            },
            {
                role_name = "roles/storage.legacyBucketWriter"
                members = ["ghi@gmail.com","jkl@gmail.com"]
            },     
             ]
    }
]

The challenge is that we need to find a way to loop through the roles list. roles is a nested block declared under variable bucket_user_roles.

The loop on roles list(nested block) is not possible in current form using count or for-each.

In order to achieve it we need to flatten this nested block into a flat list which we should be able to loop over using count or for-each. We will use a terraform function called flatten in order to achieve that.

Let’s see how flatten function can help to achieve what we need.

We will create a local variable with transformation applied using flatten. This creates a flat list of object with combination of bucket_name , role_name and members which we can use to loop over using count.

// flatten the collection variable into a list of object which can be used with count
 locals {
    local_user_role = flatten([
        for key ,value in var.bucket_user_roles : [
            for index , val in value.role : {
                bucket_name = value.bucket_name
                role_name = val.role_name
                members = val.members
            }
        ]
     ])
     }

With input provided as variable bucket_user_roles, it returns following output.

local_user_role = [
            {
                bucket_name = "bucket-1"
                role_name = "roles/storage.admin"
                members = ["abc@gmail.com","def@gmail.com"]
            },
            {
                bucket_name = "bucket-1"
                role_name = "roles/storage.legacyBucketReader"
                members = ["abc@gmail.com","def@gmail.com"]
            },
            {
                bucket_name = "bucket-1"
                role_name = "roles/storage.legacyBucketWriter"
                members = ["abc@gmail.com","def@gmail.com"]
            },                        
            {
                bucket_name = "bucket-2"
                role_name = "roles/storage.admin"
                members = ["ghi@gmail.com","jkl@gmail.com"]
            },
            {
                bucket_name = "bucket-2"
                role_name = "roles/storage.legacyBucketReader"
                members = ["ghi@gmail.com","jkl@gmail.com"]
            },
            {
                bucket_name = "bucket-2"
                role_name = "roles/storage.legacyBucketWriter"
                members = ["ghi@gmail.com","jkl@gmail.com"]
            },                        
]

Great. Now we have a local variable local_user_rolewhich is a flat list of objects. Let’s use this variable in terraform code to perform iam binding using count loop.

// using count on local variable to loop through it resource 
"google_storage_bucket_iam_binding" "binding" {
    count = length(local.local_user_role)
    bucket = local.local_user_role[count.index].bucket_name
    role = local.local_user_role[count.index].role_name
    members = local.local_user_role[count.index].members
}

Running terraform plan

We can see that it’s going through all the roles declared to create 6 resources for 6 roles.

terraform plan 
.............
Plan: 6 to add, 0 to change, 0 to destroy.
------------------------------------------------------------------------

Hurray! We have successfully looped through the roles nested block as was needed to complete our challenge.

Hope you find it useful.

Good Luck.