How to Use Terraform's `for_each`, With Examples

Terraform is one of the most popular infrastructure as code (IaC) tools. With Terraform, you can write code that defines the infrastructure components you want and the configuration for them. You then execute that code, and Terraform will make sure that your infrastructure is set up the way you defined it. This means either creating new components or responding with "All of these are already created."

Sounds easy, right?

In reality, Terraform code can be quite complicated. That's especially likely when you want to do some advanced stuff, like iterating over a certain number of resources.

For example, let's say you want to create multiple virtual machines with the same configuration. For cases like that, in Terraform you can use `for_each`. In this post, you'll learn what that is as well as how and when to use it.

Infrastructure as Code With Terraform

If you're new to Terraform, before we move on, you need to understand what it actually is.

Modern environments are quite complex, and you have a few options when you want to add, change or destroy some components of your infrastructure. If you're on the cloud, you can go to your cloud provider web UI and execute any necessary action from there. You can also use CLI or even write some scripts yourself and call your cloud provider API.

There are, however, some limitations to all these options. Clicking in the web UI doesn't scale. And you don't want to create dozens of different services every time you need a new environment.

Using CLI or writing your own scripts is a step forward. But why not take two steps forward instead and use a dedicated infrastructure-as- code tool?

Terraform is one such tool. You write Terraform-specific code defining how you want your infrastructure to look. Then you execute Terraform and everything is taken care of for you. It's a highly efficient and scalable way of creating infrastructure.

Also, one of the biggest advantages of using Terraform is that it will  keep the state of your infrastructure saved. Therefore, it will always try to have your infrastructure in sync. So once you execute Terraform, it will only create, change or destroy resources that aren't in sync with the saved state.

Terraform Meta-Arguments

Before we dive into explaining how `for_each` works, let's briefly talk about what it actually is.

In Terraform, we mainly talk about resource blocks and module blocks. For example, when you want to create a virtual machine, you need to define a resource block with configuration details of that machine. Within the resource and module block, you can also use one of the five so-called meta-arguments. These are special instructions that aren't part of the resource configuration per se, but they instruct Terraform to do some action in relation to that resource. And one of these instructions is `for_each`.

As I already mentioned, the main purpose of the `for_each` meta-argument is to create multiple instances of a resource. So, as you can imagine, it's quite useful to know.

It's also worth mentioning that `for_each` has been added to Terraform in version 0.12. But I hope you've already upgraded to Terraform 1.x anyway.

Multiple Resources

To understand better what purpose `for_each` serves, let's see how you could achieve the same outcome in Terraform without using `for_each`.

The outcome we're talking about is deploying multiple copies of the same resource. So, let's take virtual machines, for example.

Normally, to deploy more than one virtual machine, you'd have to specify multiple resource blocks, like this:

resource "google_compute_instance" "vm1" {
name         = "vm1"
machine_type = "e2-small"
zone         = "us-central1-a"
(...)
}

resource "google_compute_instance" "vm2" {
name         = "vm2"
machine_type = "e2-medium"
zone         = "us-central1-a"
(...)
}

resource "google_compute_instance" "vm3" {
name         = "vm3"
machine_type = "f1-micro"
zone         = "us-central1-a"
(...)
}

Seems like a lot of duplicated code, doesn't it? That's exactly where `for_each` can help.

Instead of duplicating all that code for each virtual machine, you can define your resource once and provide a map or a set of strings to iterate over.

Take a look at the example. This is how achieving the same results as above would look with `for_each`:

resource "google_compute_instance" "vm" {
for_each = {
  "vm1" = "e2-small"
  "vm2" = "e2-medium"
  "vm3" = "f1-micro"
}
name = each.key
machine_type = each.value
zone = "us-central1-a"
(...)
}

As you can see, we defined the configuration parameters that differ per virtual machine as key-value pairs in the `for_each` block and left the parameters that are the same for each VM in the resource block. Then, we accessed the key-value pair by special keywords `each.key` and `each.value`.

What if you want to pass more than just two (key and value) parameters? For example, what if you want to also parameterize the zone in the above example? You can simply change the value to a map, as follows:

resource "google_compute_instance" "vm" {
for_each = {
  "vm1" = { vm_size = "e2-small", zone = "us-central1-a" }
  "vm2" = { vm_size = "e2-medium", zone = "us-central1-b" }
  "vm3" = { vm_size = "f1-micro", zone = "us-central1-c" }
}
name = each.key
machine_type = each.value.vm_size
zone = each.value.zone
(...)
}

You can pass as many parameters in the value as you want. Then in the actual resource configuration, you can reference them with `each.value.<parameter_key>`.

To keep your code clean and have the ability to reuse values for different resources, you can even extract the actual parameters into a variable:

locals {
virtual_machines = {
  "vm1" = { vm_size = "e2-small", zone = "us-central1-a" },
  "vm2" = { vm_size = "e2-medium", zone = "us-central1-b" },
  "vm3" = { vm_size = "f1-micro", zone = "us-central1-c" }
 }
}

resource "google_compute_instance" "vm" {
for_each = local.virtual_machines
name = each.key
machine_type = each.value.vm_size
zone = each.value.zone
(...)
}

`for_each` Versus `count`

If you're not new to Terraform, you may have used another meta-argument that seems like the same thing: `count`. And while `count` also lets you create multiple instances of the same resource, there's a difference between `count` and `for_each`. The latter isn't sensitive to changes in the order of resources.

A common issue with `count` is that once you delete any resource other than the last one, Terraform will try to force replace all resources that the index doesn't match.

You don't have that problem with `for_each` because it uses the key of a map as an index. You can't use both `count` and `for_each` on the same resources, but why would you anyway?

Are there any drawbacks to `for_each`? Yes.

Limitations of `for_each`

While `for_each` is pretty straightforward to use, there are some limitations you should be aware of. First of all, the keys in your `for_each` map block must have a known value. Therefore, for example, they can't be generated on the fly by functions (like `bcrypt` or `timestamp`). They also can't refer to resource-specific attributes that are provided by a cloud provider, like a cloud resource ID. Another limitation is the fact that you can't use sensitive values as arguments for `for_each`. Basically, when using `for_each`, you need to directly specify the values.

Summing up and Learning More

`for_each` is probably one of the most commonly used Terraform meta-arguments. Modern environments usually consist of multiple instances of resources for high-availability and scalability reasons. Using `for_each` is relatively easy, but you need a solid understanding of how it works to get the most benefits from it. It also has its own limitations.

In this article, you learned how `for_each` works and got some tips on how to use it efficiently. Now, you can try to play around with it yourself or look into other meta-arguments.

Terraform at Scale

Release’s Environment-as-a-Service, is an easy to use, highly scalable service that leverages Terraform to create snapshots of even the most complex environments and automatically manages their creation and teardown as part of your development lifecycle.