Introduction
Have you ever found yourself writing the same Terraform configuration blocks over and over again, wishing there was a better way? If you’ve been working with Terraform for a while, you’ve probably encountered situations where you need to create multiple similar nested blocks based on a variable or conditional logic. That’s where dynamic blocks come to the rescue – they’re like the Swiss Army knife of Terraform configurations, making your infrastructure as code more flexible and maintainable.
What are dynamic blocks?
Think of dynamic blocks as your personal assistant that can generate configuration blocks on the fly. Instead of hardcoding multiple similar blocks, you can use a single dynamic block that iterates over a collection and creates the necessary configuration automatically. It’s like having a template that adapts to your needs – pretty neat, right?
In traditional Terraform configurations, when you need multiple similar blocks, you’d typically write them out one by one. This approach works, but it’s about as efficient as writing the same email to 20 different people manually instead of using a mail merge. Dynamic blocks solve this problem by allowing you to generate nested blocks programmatically, making your code DRY (Don’t Repeat Yourself) and much easier to maintain.
Understanding the anatomy of a dynamic block
Before we dive into the practical examples, let’s break down how a dynamic block actually works. The basic syntax might look a bit intimidating at first, but once you get the hang of it, it becomes second nature.
dynamic "block_type" {
for_each = var.some_collection
content {
# Configuration for each block
property = block_type.value
}
}The structure consists of three main components that work together like gears in a machine. First, you have the block_type, which specifies what kind of nested block you’re creating. Then there’s for_each, which takes a collection (list, set, or map) to iterate over. Finally, the content block defines what goes inside each generated block.
Common use cases for dynamic blocks
Security group rules in AWS
One of the most common scenarios where dynamic blocks shine is when configuring security group rules. Let’s say you need to allow traffic from multiple IP ranges to your EC2 instances. Instead of writing individual ingress rules, you can use a dynamic block:
variable "allowed_ips" {
type = list(object({
from_port = number
to_port = number
protocol = string
cidr_blocks = list(string)
}))
default = [
{
from_port = 80
to_port = 80
protocol = "tcp"
cidr_blocks = ["10.0.0.0/8"]
},
{
from_port = 443
to_port = 443
protocol = "tcp"
cidr_blocks = ["10.0.0.0/8", "192.168.0.0/16"]
}
]
}
resource "aws_security_group" "example" {
name = "example-sg"
description = "Example security group with dynamic rules"
dynamic "ingress" {
for_each = var.allowed_ips
content {
from_port = ingress.value.from_port
to_port = ingress.value.to_port
protocol = ingress.value.protocol
cidr_blocks = ingress.value.cidr_blocks
}
}
}This approach is much cleaner than writing multiple ingress blocks manually, wouldn’t you agree? Plus, if you need to add or remove rules, you just update the variable – no need to touch the resource configuration itself.
Container definitions in ECS
Another fantastic use case is when working with AWS ECS task definitions. You might need to define multiple containers with different configurations:
variable "containers" {
type = list(object({
name = string
image = string
memory = number
ports = list(number)
}))
}
resource "aws_ecs_task_definition" "app" {
family = "my-app"
requires_compatibilities = ["FARGATE"]
network_mode = "awsvpc"
cpu = "256"
memory = "512"
container_definitions = jsonencode([
for container in var.containers : {
name = container.name
image = container.image
memory = container.memory
essential = true
portMappings = [
for port in container.ports : {
containerPort = port
protocol = "tcp"
}
]
}
])
}Advanced techniques with dynamic blocks
Conditional dynamic blocks
Sometimes you want to create blocks only when certain conditions are met. You can achieve this by using conditional expressions with your for_each argument:
variable "enable_logging" {
type = bool
default = true
}
variable "log_buckets" {
type = list(string)
default = ["bucket1", "bucket2"]
}
resource "aws_s3_bucket" "example" {
bucket = "my-example-bucket"
dynamic "logging" {
for_each = var.enable_logging ? var.log_buckets : []
content {
target_bucket = logging.value
target_prefix = "logs/"
}
}
}This pattern is incredibly useful when you want to make certain features optional in your infrastructure. It’s like having a switch that can turn features on or off based on your environment or requirements.
Nested dynamic blocks
Here’s where things get really interesting – you can nest dynamic blocks within other dynamic blocks. It’s like inception, but for infrastructure code:
variable "load_balancer_rules" {
type = list(object({
priority = number
conditions = list(object({
field = string
values = list(string)
}))
target_group_arn = string
}))
}
resource "aws_lb_listener_rule" "example" {
listener_arn = aws_lb_listener.main.arn
dynamic "action" {
for_each = var.load_balancer_rules
content {
type = "forward"
target_group_arn = action.value.target_group_arn
}
}
dynamic "condition" {
for_each = var.load_balancer_rules
content {
dynamic "host_header" {
for_each = [for c in condition.value.conditions : c if c.field == "host-header"]
content {
values = host_header.value.values
}
}
}
}
}Best practices and common pitfalls
When working with dynamic blocks, there are several best practices you should keep in mind to avoid common mistakes that can make your code harder to understand or debug.
First, always use meaningful variable names. Instead of using generic names like item or element, use descriptive names that indicate what the block represents. This makes your code self-documenting and easier for your team members (or future you) to understand.
Second, be careful with the complexity of your dynamic blocks. While it’s tempting to create highly sophisticated nested structures, remember that readability should be your priority. If a dynamic block becomes too complex, consider breaking it down into smaller, more manageable pieces or using local values to prepare your data.
Here’s an example of how to improve readability with local values:
locals {
# Prepare and transform data before using in dynamic blocks
formatted_rules = [
for rule in var.security_rules : {
from_port = rule.port
to_port = rule.port
protocol = rule.protocol
cidr_blocks = rule.allow_from
description = "Allow ${rule.protocol} on port ${rule.port}"
}
]
}
resource "aws_security_group" "main" {
name = "main-sg"
dynamic "ingress" {
for_each = local.formatted_rules
content {
from_port = ingress.value.from_port
to_port = ingress.value.to_port
protocol = ingress.value.protocol
cidr_blocks = ingress.value.cidr_blocks
description = ingress.value.description
}
}
}Debugging dynamic blocks
Let’s face it – debugging dynamic blocks can sometimes feel like finding a needle in a haystack. But don’t worry, I’ve got some tricks up my sleeve that’ll make your life easier.
The terraform console command is your best friend when debugging dynamic blocks. You can test your expressions and see exactly what data is being generated:
terraform console
> var.allowed_ips
> [for rule in var.allowed_ips : rule.from_port]Another helpful technique is to use the terraform plan command with the -out flag to save your plan to a file, then inspect it carefully to see what resources and blocks are being created:
terraform plan -out=tfplan
terraform show -json tfplan | jq '.planned_values'Performance considerations
While dynamic blocks are powerful, they do come with some performance implications you should be aware of. Each time Terraform runs a plan or apply, it needs to evaluate all the dynamic blocks and their iterations. If you’re working with large datasets, this can slow down your Terraform operations.
To optimize performance, consider these strategies. First, use local values to pre-compute complex transformations instead of doing them inline within the dynamic block. Second, if you’re dealing with very large collections, consider whether all items really need to be in a single resource or if they could be split across multiple resources.
Real-world example: Building a flexible network architecture
Let’s put everything together with a real-world example that showcases the power of dynamic blocks. We’ll create a flexible network architecture that can adapt to different environments:
variable "environment" {
type = string
default = "dev"
}
variable "network_config" {
type = map(object({
vpc_cidr = string
subnets = list(object({
cidr = string
az = string
type = string # "public" or "private"
}))
enable_nat_gateway = bool
enable_vpn_gateway = bool
}))
default = {
dev = {
vpc_cidr = "10.0.0.0/16"
subnets = [
{ cidr = "10.0.1.0/24", az = "us-west-2a", type = "public" },
{ cidr = "10.0.2.0/24", az = "us-west-2b", type = "public" },
{ cidr = "10.0.10.0/24", az = "us-west-2a", type = "private" },
{ cidr = "10.0.11.0/24", az = "us-west-2b", type = "private" }
]
enable_nat_gateway = false
enable_vpn_gateway = false
}
prod = {
vpc_cidr = "10.1.0.0/16"
subnets = [
{ cidr = "10.1.1.0/24", az = "us-west-2a", type = "public" },
{ cidr = "10.1.2.0/24", az = "us-west-2b", type = "public" },
{ cidr = "10.1.3.0/24", az = "us-west-2c", type = "public" },
{ cidr = "10.1.10.0/24", az = "us-west-2a", type = "private" },
{ cidr = "10.1.11.0/24", az = "us-west-2b", type = "private" },
{ cidr = "10.1.12.0/24", az = "us-west-2c", type = "private" }
]
enable_nat_gateway = true
enable_vpn_gateway = true
}
}
}
locals {
current_config = var.network_config[var.environment]
public_subnets = [for s in local.current_config.subnets : s if s.type == "public"]
private_subnets = [for s in local.current_config.subnets : s if s.type == "private"]
}
resource "aws_vpc" "main" {
cidr_block = local.current_config.vpc_cidr
tags = {
Name = "${var.environment}-vpc"
Environment = var.environment
}
}
resource "aws_subnet" "public" {
count = length(local.public_subnets)
vpc_id = aws_vpc.main.id
cidr_block = local.public_subnets[count.index].cidr
availability_zone = local.public_subnets[count.index].az
map_public_ip_on_launch = true
tags = {
Name = "${var.environment}-public-subnet-${count.index + 1}"
Type = "public"
}
}
resource "aws_subnet" "private" {
count = length(local.private_subnets)
vpc_id = aws_vpc.main.id
cidr_block = local.private_subnets[count.index].cidr
availability_zone = local.private_subnets[count.index].az
tags = {
Name = "${var.environment}-private-subnet-${count.index + 1}"
Type = "private"
}
}
resource "aws_nat_gateway" "main" {
count = local.current_config.enable_nat_gateway ? length(local.public_subnets) : 0
allocation_id = aws_eip.nat[count.index].id
subnet_id = aws_subnet.public[count.index].id
tags = {
Name = "${var.environment}-nat-gateway-${count.index + 1}"
}
}
resource "aws_eip" "nat" {
count = local.current_config.enable_nat_gateway ? length(local.public_subnets) : 0
domain = "vpc"
tags = {
Name = "${var.environment}-nat-eip-${count.index + 1}"
}
}Conclusion
Dynamic blocks in Terraform are a game-changer when it comes to writing clean, maintainable, and flexible infrastructure code. They transform repetitive configurations into elegant, programmatic solutions that adapt to your needs. By mastering dynamic blocks, you’re not just writing better Terraform code – you’re building infrastructure that can evolve with your requirements without constant rewrites.
Remember, the key to success with dynamic blocks is finding the right balance between flexibility and readability. Start simple, test thoroughly, and gradually increase complexity as you become more comfortable with the patterns. Your future self (and your team) will thank you for writing infrastructure code that’s both powerful and maintainable.
FAQs
Can I use dynamic blocks with all Terraform resource types?
Not all resource arguments support dynamic blocks. Dynamic blocks can only be used for resource arguments that accept nested configuration blocks, such as ingress rules in security groups or setting blocks in various AWS resources. Simple attributes that accept single values or lists of strings cannot use dynamic blocks.
What’s the difference between count/for_each and dynamic blocks?
Count and for_each are used to create multiple instances of entire resources, while dynamic blocks create multiple nested blocks within a single resource. Use count/for_each when you need multiple separate resources, and use dynamic blocks when you need multiple configuration blocks within one resource.
Can I reference other dynamic blocks within the same resource?
No, you cannot directly reference one dynamic block from another within the same resource. Each dynamic block operates independently. If you need to share data between blocks, prepare it using local values before the resource definition.
How do I handle empty collections in dynamic blocks?
When the collection in for_each is empty, no blocks are generated. This is actually a feature – you can use conditional expressions like var.create_rules ? var.rules : [] to conditionally create blocks based on boolean flags.
Is there a performance impact when using many dynamic blocks?
Yes, complex dynamic blocks with large datasets can impact Terraform’s planning time. Each iteration needs to be evaluated during the plan phase. For better performance, pre-process data using locals and avoid deeply nested dynamic blocks when possible. Consider splitting very large configurations across multiple resources if performance becomes an issue.
