How to use dynamic block in Terraform

Have you ever found yourself writing the same Terraform configuration blocks over and over again, wishing there was a better way? If you’ve been working with Terraform for a while, you’ve probably encountered situations where you need to create multiple similar nested blocks based on a variable or conditional logic. That’s where dynamic blocks come to the rescue – they’re like the Swiss Army knife of Terraform configurations, making your infrastructure as code more flexible and maintainable.

What are dynamic blocks and why should you care?

Think of dynamic blocks as your personal assistant that can generate configuration blocks on the fly. Instead of hardcoding multiple similar blocks, you can use a single dynamic block that iterates over a collection and creates the necessary configuration automatically. It’s like having a template that adapts to your needs – pretty neat, right?

In traditional Terraform configurations, when you need multiple similar blocks, you’d typically write them out one by one. This approach works, but it’s about as efficient as writing the same email to 20 different people manually instead of using a mail merge. Dynamic blocks solve this problem by allowing you to generate nested blocks programmatically, making your code DRY (Don’t Repeat Yourself) and much easier to maintain.

Understanding the anatomy of a dynamic block

Before we dive into the practical examples, let’s break down how a dynamic block actually works. The basic syntax might look a bit intimidating at first, but once you get the hang of it, it becomes second nature.

dynamic "block_type" {
  for_each = var.some_collection
  content {
    # Configuration for each block
    property = block_type.value
  }
}

The structure consists of three main components that work together like gears in a machine. First, you have the block_type, which specifies what kind of nested block you’re creating. Then there’s for_each, which takes a collection (list, set, or map) to iterate over. Finally, the content block defines what goes inside each generated block.

Common use cases for dynamic blocks

Security group rules in AWS

One of the most common scenarios where dynamic blocks shine is when configuring security group rules. Let’s say you need to allow traffic from multiple IP ranges to your EC2 instances. Instead of writing individual ingress rules, you can use a dynamic block:

variable "allowed_ips" {
  type = list(object({
    from_port   = number
    to_port     = number
    protocol    = string
    cidr_blocks = list(string)
  }))
  default = [
    {
      from_port   = 80
      to_port     = 80
      protocol    = "tcp"
      cidr_blocks = ["10.0.0.0/8"]
    },
    {
      from_port   = 443
      to_port     = 443
      protocol    = "tcp"
      cidr_blocks = ["10.0.0.0/8", "192.168.0.0/16"]
    }
  ]
}

resource "aws_security_group" "example" {
  name        = "example-sg"
  description = "Example security group with dynamic rules"

  dynamic "ingress" {
    for_each = var.allowed_ips
    content {
      from_port   = ingress.value.from_port
      to_port     = ingress.value.to_port
      protocol    = ingress.value.protocol
      cidr_blocks = ingress.value.cidr_blocks
    }
  }
}

This approach is much cleaner than writing multiple ingress blocks manually, wouldn’t you agree? Plus, if you need to add or remove rules, you just update the variable – no need to touch the resource configuration itself.

Container definitions in ECS

Another fantastic use case is when working with AWS ECS task definitions. You might need to define multiple containers with different configurations:

variable "containers" {
  type = list(object({
    name   = string
    image  = string
    memory = number
    ports  = list(number)
  }))
}

resource "aws_ecs_task_definition" "app" {
  family                   = "my-app"
  requires_compatibilities = ["FARGATE"]
  network_mode            = "awsvpc"
  cpu                     = "256"
  memory                  = "512"

  container_definitions = jsonencode([
    for container in var.containers : {
      name      = container.name
      image     = container.image
      memory    = container.memory
      essential = true
      portMappings = [
        for port in container.ports : {
          containerPort = port
          protocol      = "tcp"
        }
      ]
    }
  ])
}

Advanced techniques with dynamic blocks

Conditional dynamic blocks

Sometimes you want to create blocks only when certain conditions are met. You can achieve this by using conditional expressions with your for_each argument:

variable "enable_logging" {
  type    = bool
  default = true
}

variable "log_buckets" {
  type = list(string)
  default = ["bucket1", "bucket2"]
}

resource "aws_s3_bucket" "example" {
  bucket = "my-example-bucket"

  dynamic "logging" {
    for_each = var.enable_logging ? var.log_buckets : []
    content {
      target_bucket = logging.value
      target_prefix = "logs/"
    }
  }
}

This pattern is incredibly useful when you want to make certain features optional in your infrastructure. It’s like having a switch that can turn features on or off based on your environment or requirements.

Nested dynamic blocks

Here’s where things get really interesting – you can nest dynamic blocks within other dynamic blocks. It’s like inception, but for infrastructure code:

variable "load_balancer_rules" {
  type = list(object({
    priority = number
    conditions = list(object({
      field  = string
      values = list(string)
    }))
    target_group_arn = string
  }))
}

resource "aws_lb_listener_rule" "example" {
  listener_arn = aws_lb_listener.main.arn

  dynamic "action" {
    for_each = var.load_balancer_rules
    content {
      type             = "forward"
      target_group_arn = action.value.target_group_arn
    }
  }

  dynamic "condition" {
    for_each = var.load_balancer_rules
    content {
      dynamic "host_header" {
        for_each = [for c in condition.value.conditions : c if c.field == "host-header"]
        content {
          values = host_header.value.values
        }
      }
    }
  }
}

Best practices and common pitfalls

When working with dynamic blocks, there are several best practices you should keep in mind to avoid common mistakes that can make your code harder to understand or debug.

First, always use meaningful variable names. Instead of using generic names like item or element, use descriptive names that indicate what the block represents. This makes your code self-documenting and easier for your team members (or future you) to understand.

Second, be careful with the complexity of your dynamic blocks. While it’s tempting to create highly sophisticated nested structures, remember that readability should be your priority. If a dynamic block becomes too complex, consider breaking it down into smaller, more manageable pieces or using local values to prepare your data.

Here’s an example of how to improve readability with local values:

locals {
  # Prepare and transform data before using in dynamic blocks
  formatted_rules = [
    for rule in var.security_rules : {
      from_port   = rule.port
      to_port     = rule.port
      protocol    = rule.protocol
      cidr_blocks = rule.allow_from
      description = "Allow ${rule.protocol} on port ${rule.port}"
    }
  ]
}

resource "aws_security_group" "main" {
  name = "main-sg"

  dynamic "ingress" {
    for_each = local.formatted_rules
    content {
      from_port   = ingress.value.from_port
      to_port     = ingress.value.to_port
      protocol    = ingress.value.protocol
      cidr_blocks = ingress.value.cidr_blocks
      description = ingress.value.description
    }
  }
}

Debugging dynamic blocks

Let’s face it – debugging dynamic blocks can sometimes feel like finding a needle in a haystack. But don’t worry, I’ve got some tricks up my sleeve that’ll make your life easier.

The terraform console command is your best friend when debugging dynamic blocks. You can test your expressions and see exactly what data is being generated:

terraform console
> var.allowed_ips
> [for rule in var.allowed_ips : rule.from_port]

Another helpful technique is to use the terraform plan command with the -out flag to save your plan to a file, then inspect it carefully to see what resources and blocks are being created:

terraform plan -out=tfplan
terraform show -json tfplan | jq '.planned_values'

Performance considerations

While dynamic blocks are powerful, they do come with some performance implications you should be aware of. Each time Terraform runs a plan or apply, it needs to evaluate all the dynamic blocks and their iterations. If you’re working with large datasets, this can slow down your Terraform operations.

To optimize performance, consider these strategies. First, use local values to pre-compute complex transformations instead of doing them inline within the dynamic block. Second, if you’re dealing with very large collections, consider whether all items really need to be in a single resource or if they could be split across multiple resources.

Real-world example: Building a flexible network architecture

Let’s put everything together with a real-world example that showcases the power of dynamic blocks. We’ll create a flexible network architecture that can adapt to different environments:

variable "environment" {
  type    = string
  default = "dev"
}

variable "network_config" {
  type = map(object({
    vpc_cidr = string
    subnets = list(object({
      cidr = string
      az   = string
      type = string # "public" or "private"
    }))
    enable_nat_gateway = bool
    enable_vpn_gateway = bool
  }))
  default = {
    dev = {
      vpc_cidr = "10.0.0.0/16"
      subnets = [
        { cidr = "10.0.1.0/24", az = "us-west-2a", type = "public" },
        { cidr = "10.0.2.0/24", az = "us-west-2b", type = "public" },
        { cidr = "10.0.10.0/24", az = "us-west-2a", type = "private" },
        { cidr = "10.0.11.0/24", az = "us-west-2b", type = "private" }
      ]
      enable_nat_gateway = false
      enable_vpn_gateway = false
    }
    prod = {
      vpc_cidr = "10.1.0.0/16"
      subnets = [
        { cidr = "10.1.1.0/24", az = "us-west-2a", type = "public" },
        { cidr = "10.1.2.0/24", az = "us-west-2b", type = "public" },
        { cidr = "10.1.3.0/24", az = "us-west-2c", type = "public" },
        { cidr = "10.1.10.0/24", az = "us-west-2a", type = "private" },
        { cidr = "10.1.11.0/24", az = "us-west-2b", type = "private" },
        { cidr = "10.1.12.0/24", az = "us-west-2c", type = "private" }
      ]
      enable_nat_gateway = true
      enable_vpn_gateway = true
    }
  }
}

locals {
  current_config = var.network_config[var.environment]
  public_subnets = [for s in local.current_config.subnets : s if s.type == "public"]
  private_subnets = [for s in local.current_config.subnets : s if s.type == "private"]
}

resource "aws_vpc" "main" {
  cidr_block = local.current_config.vpc_cidr

  tags = {
    Name        = "${var.environment}-vpc"
    Environment = var.environment
  }
}

resource "aws_subnet" "public" {
  count                   = length(local.public_subnets)
  vpc_id                  = aws_vpc.main.id
  cidr_block              = local.public_subnets[count.index].cidr
  availability_zone       = local.public_subnets[count.index].az
  map_public_ip_on_launch = true

  tags = {
    Name = "${var.environment}-public-subnet-${count.index + 1}"
    Type = "public"
  }
}

resource "aws_subnet" "private" {
  count             = length(local.private_subnets)
  vpc_id            = aws_vpc.main.id
  cidr_block        = local.private_subnets[count.index].cidr
  availability_zone = local.private_subnets[count.index].az

  tags = {
    Name = "${var.environment}-private-subnet-${count.index + 1}"
    Type = "private"
  }
}

resource "aws_nat_gateway" "main" {
  count         = local.current_config.enable_nat_gateway ? length(local.public_subnets) : 0
  allocation_id = aws_eip.nat[count.index].id
  subnet_id     = aws_subnet.public[count.index].id

  tags = {
    Name = "${var.environment}-nat-gateway-${count.index + 1}"
  }
}

resource "aws_eip" "nat" {
  count  = local.current_config.enable_nat_gateway ? length(local.public_subnets) : 0
  domain = "vpc"

  tags = {
    Name = "${var.environment}-nat-eip-${count.index + 1}"
  }
}

Conclusion

Dynamic blocks in Terraform are a game-changer when it comes to writing clean, maintainable, and flexible infrastructure code. They transform repetitive configurations into elegant, programmatic solutions that adapt to your needs. By mastering dynamic blocks, you’re not just writing better Terraform code – you’re building infrastructure that can evolve with your requirements without constant rewrites.

Remember, the key to success with dynamic blocks is finding the right balance between flexibility and readability. Start simple, test thoroughly, and gradually increase complexity as you become more comfortable with the patterns. Your future self (and your team) will thank you for writing infrastructure code that’s both powerful and maintainable.

FAQs

Can I use dynamic blocks with all Terraform resource types?

Not all resource arguments support dynamic blocks. Dynamic blocks can only be used for resource arguments that accept nested configuration blocks, such as ingress rules in security groups or setting blocks in various AWS resources. Simple attributes that accept single values or lists of strings cannot use dynamic blocks.

What’s the difference between count/for_each and dynamic blocks?

Count and for_each are used to create multiple instances of entire resources, while dynamic blocks create multiple nested blocks within a single resource. Use count/for_each when you need multiple separate resources, and use dynamic blocks when you need multiple configuration blocks within one resource.

Can I reference other dynamic blocks within the same resource?

No, you cannot directly reference one dynamic block from another within the same resource. Each dynamic block operates independently. If you need to share data between blocks, prepare it using local values before the resource definition.

How do I handle empty collections in dynamic blocks?

When the collection in for_each is empty, no blocks are generated. This is actually a feature – you can use conditional expressions like var.create_rules ? var.rules : [] to conditionally create blocks based on boolean flags.

Is there a performance impact when using many dynamic blocks?

Yes, complex dynamic blocks with large datasets can impact Terraform’s planning time. Each iteration needs to be evaluated during the plan phase. For better performance, pre-process data using locals and avoid deeply nested dynamic blocks when possible. Consider splitting very large configurations across multiple resources if performance becomes an issue.