Mastering terraform module composition: building infrastructure like lego blocks

Have you ever watched a child build something amazing with Lego blocks? They start with simple pieces, combine them into larger structures, and before you know it, they’ve created an entire city. That’s exactly what terraform module composition is all about – taking small, reusable pieces of infrastructure code and combining them to build complex, scalable systems.

If you’re working with Terraform and finding yourself copying and pasting the same code over and over, or if your infrastructure files are becoming unwieldy monsters that nobody wants to touch, then you’re in the right place. Let’s dive into how module composition can transform your infrastructure-as-code game.

What is terraform module composition anyway?

Think of terraform modules as recipes in a cookbook. You wouldn’t write out the entire process of making bread every time you want to make a sandwich, right? You’d reference your bread recipe and focus on the sandwich assembly. That’s the beauty of module composition – it’s about creating reusable, self-contained units of infrastructure that you can mix and match like ingredients in your favorite dish.

Module composition in Terraform refers to the practice of building complex infrastructure by combining smaller, focused modules. Instead of having one gigantic configuration file that does everything (and probably gives you nightmares), you create small, purposeful modules that each handle specific aspects of your infrastructure. Then, you compose them together to create your complete infrastructure solution.

Why should you care about module composition?

Let me paint you a picture. You’re managing infrastructure for multiple environments – development, staging, and production. Without proper module composition, you might find yourself maintaining three separate, nearly identical configurations. When you need to make a change, you’re updating it in three places, praying you don’t miss something or introduce inconsistencies. Sound familiar?

Module composition solves this headache by allowing you to write once and use everywhere. It promotes the DRY principle (Don’t Repeat Yourself) while keeping your infrastructure code maintainable, testable, and most importantly, understandable by your future self and your teammates.

Core patterns for effective module composition

The building blocks pattern

Start with the smallest possible units of infrastructure. These are your atomic modules – think of them as the individual Lego blocks. For instance, you might have a module that creates a single S3 bucket with standard configurations, another that sets up a basic security group, or one that provisions an RDS instance.

Here’s what a simple S3 bucket module might look like:

# modules/s3-bucket/main.tf
resource "aws_s3_bucket" "this" {
  bucket = var.bucket_name
  
  tags = merge(
    var.common_tags,
    {
      Name = var.bucket_name
      Module = "s3-bucket"
    }
  )
}

resource "aws_s3_bucket_versioning" "this" {
  bucket = aws_s3_bucket.this.id
  
  versioning_configuration {
    status = var.enable_versioning ? "Enabled" : "Disabled"
  }
}

The composition layer pattern

Now comes the fun part – combining these atomic modules into something more meaningful. Your composition layer takes multiple building blocks and assembles them into a functional unit. For example, you might combine your S3 bucket module with an IAM role module and a Lambda function module to create a complete serverless application module.

# modules/serverless-app/main.tf
module "app_bucket" {
  source = "../s3-bucket"
  
  bucket_name = "${var.app_name}-data"
  enable_versioning = true
  common_tags = var.common_tags
}

module "lambda_function" {
  source = "../lambda"
  
  function_name = var.app_name
  s3_bucket_arn = module.app_bucket.bucket_arn
  runtime = var.runtime
  common_tags = var.common_tags
}

module "api_gateway" {
  source = "../api-gateway"
  
  api_name = "${var.app_name}-api"
  lambda_invoke_arn = module.lambda_function.invoke_arn
  common_tags = var.common_tags
}

The facade pattern

Sometimes you need to hide complexity behind a simple interface. That’s where the facade pattern shines. You create a high-level module that presents a simple API to consumers while orchestrating multiple complex modules under the hood. It’s like having a TV remote – you press one button, but behind the scenes, it’s coordinating power, input selection, and volume adjustments.

Best practices that will save your sanity

Keep modules focused and single-purpose

Each module should do one thing and do it well. If you find yourself creating a module called “everything-my-app-needs,” you’re probably trying to do too much. Break it down. Your future self will thank you when you need to debug or modify something.

Use consistent naming conventions

Establish naming patterns and stick to them religiously. If you call it vpc_id in one module, don’t call it network_id in another. Consistency reduces cognitive load and makes your modules more intuitive to use.

Version your modules like your life depends on it

Because sometimes, it does (at least your job might). Use semantic versioning and tag your module repositories. This way, different projects can use different versions of your modules, and you can evolve modules without breaking existing infrastructure.

module "networking" {
  source = "git::https://github.com/company/terraform-modules.git//networking?ref=v2.1.0"
  
  # ... configuration
}

Document with examples, not just descriptions

Instead of just describing what your module does, show how to use it. Include example configurations that demonstrate common use cases. Trust me, six months from now, you’ll appreciate having these examples when you’ve forgotten everything about the module you wrote.

Common pitfalls and how to dodge them

The over-abstraction trap

It’s tempting to create super-generic modules that can handle every possible scenario. Resist this urge! Over-abstracted modules become complex, hard to understand, and ironically, less reusable. It’s better to have three simple modules than one complicated monster that tries to do everything.

The tight coupling nightmare

When modules depend too heavily on each other’s internal implementation details, you create a house of cards. One small change can trigger a cascade of failures. Keep your module interfaces clean and well-defined. Pass data between modules using explicit inputs and outputs, not by reaching into another module’s guts.

The variable explosion syndrome

Ever seen a module with 50+ variables? It’s not pretty. If your module needs that many inputs, it’s probably doing too much. Consider breaking it into smaller modules or using sensible defaults for less critical configurations.

Real-world example: building a three-tier application

Let’s put it all together with a practical example. Imagine you’re building a typical three-tier web application with a load balancer, application servers, and a database. Here’s how you might compose it:

# environments/production/main.tf
module "network" {
  source = "../../modules/network"
  
  environment = "production"
  cidr_block = "10.0.0.0/16"
  availability_zones = ["us-west-2a", "us-west-2b"]
}

module "database" {
  source = "../../modules/rds-cluster"
  
  cluster_identifier = "myapp-prod"
  subnet_ids = module.network.database_subnet_ids
  vpc_id = module.network.vpc_id
  engine_version = "13.7"
}

module "application" {
  source = "../../modules/ecs-service"
  
  service_name = "myapp"
  vpc_id = module.network.vpc_id
  subnet_ids = module.network.application_subnet_ids
  database_endpoint = module.database.endpoint
  database_secret_arn = module.database.secret_arn
}

module "load_balancer" {
  source = "../../modules/alb"
  
  name = "myapp-prod-alb"
  vpc_id = module.network.vpc_id
  subnet_ids = module.network.public_subnet_ids
  target_group_arn = module.application.target_group_arn
}

See how clean that is? Each module has a specific responsibility, and they communicate through well-defined interfaces. You could easily reuse these same modules for your staging environment just by changing a few variables.

Testing your composed modules

Don’t forget about testing! Just because your modules are composed doesn’t mean you can skip validation. Use tools like Terratest or Kitchen-Terraform to create automated tests for your modules. Test individual modules in isolation, then test your compositions to ensure they work together correctly.

Start with simple validation tests – does the module create the expected resources? Then move on to integration tests – can the composed modules actually communicate with each other? Finally, consider end-to-end tests for critical infrastructure patterns.

The evolution of your module library

Your module library isn’t something you build once and forget about. It’s a living, breathing ecosystem that grows with your organization’s needs. Start small with the most common patterns you use, then gradually expand as you identify more opportunities for reuse.

Create a module registry or a well-organized repository structure. Document your modules thoroughly, including migration guides when you make breaking changes. Foster a culture where teams contribute back to the shared module library instead of creating their own variations.

Conclusion

Terraform module composition isn’t just a nice-to-have technique – it’s a fundamental practice that separates maintainable infrastructure code from spaghetti configurations that everyone’s afraid to touch. By breaking down your infrastructure into small, reusable modules and composing them thoughtfully, you create a system that’s easier to understand, test, and evolve.

Remember, the goal isn’t to create the perfect module structure from day one. Start simple, refactor as patterns emerge, and always keep the end user (which might be future you) in mind. Your infrastructure code should tell a story, and well-composed modules help make that story clear and compelling.

FAQs

How granular should my terraform modules be?

The sweet spot is usually at the level of logical infrastructure components. A module should be small enough to be reusable but large enough to be meaningful. For example, a module that creates just a single security group rule is probably too granular, while a module that sets up an entire VPC with security groups, NAT gateways, and routing tables is about right.

Should I use public modules from the terraform registry or write my own?

It depends on your needs! Public modules from the Terraform Registry can be great starting points, especially from verified publishers. However, most organizations end up writing custom modules to enforce their specific security policies, naming conventions, and architectural patterns. A hybrid approach often works best – use public modules for standard components and write custom modules for your unique requirements.

How do I handle module versioning when modules depend on each other?

Use semantic versioning religiously and maintain a compatibility matrix for your modules. When making breaking changes, bump the major version and provide clear migration guides. Consider using a monorepo approach for tightly coupled modules or implement integration tests that validate module compatibility across versions.

What’s the difference between using modules and using terragrunt?

Modules and Terragrunt serve different purposes and actually work great together! Modules are Terraform’s native way to create reusable infrastructure components, while Terragrunt is a thin wrapper that helps with keeping configurations DRY, managing remote state, and handling dependencies between configurations. Think of modules as your building blocks and Terragrunt as the construction manager coordinating the build process.

How do I refactor existing terraform code into modules without breaking everything?

Start with a gradual approach! First, identify logical boundaries in your existing code. Create modules for new infrastructure while keeping the existing code as-is. Then, during maintenance windows, gradually move existing resources into modules using terraform state mv commands. Always test in a non-production environment first, and keep detailed rollback plans. Consider using feature flags or running old and new configurations in parallel during the transition period.