In my previous post I showed how to enable debug logs. Today I want to present how to improve terraform plan and terraform apply speed by configuring parallelism.

Terraform by default runs 10 concurrent operations. To reduce execution time on plan or apply operation we can increase this parameter.

By increasing parallelism you can hit the rate limit of your provider. Some cloud providers (like Cloudflare) inform about the number of API requests allowed in a period of time. Hitting the limit can impact your deployments.

TFE_PARALLELISM variable

The easiest way to increase parallelism in Terraform Cloud for Remote Execution is the TFE_PARALLELISM variable. It just requires a number. To set this you need to perform those steps:

• Select your workspace,
• Go to Variables tab,
• Add variable in Workspace variables panel and create TFE_PARALLELISM variable:

  Ensure you have selected Environment variable button

The change should be available on next execution.

Manage parallelism for each stage of execution

Terraform CLI allows configuring parallelism differently per command (terraform plan, terraform apply or terraform destroy). In Terraform Cloud we can also do this. In these cases, use TF_CLI_ARGS_plan="-parallelism=<N>" or TF_CLI_ARGS_apply="-parallelism=<N>" environment variables instead of TFE_PARALLELISM.

I prefer this way because it allows being more granular. I want to run plan fast because it makes a request about every resource.

To set TF_CLI_ARGS_plan="-parallelism=<N>" or TF_CLI_ARGS_apply="-parallelism=<N>" parameters perform same steps as in instruction written above for TFE_PARALLELISM.

Manage the variables in a single place

I showed how to configure a variable per workspace. Terraform Cloud allows configuring a Variable set which can be attached to each workspace, so we don’t need to repeat ourselves for each workspace.

To configure Variable set do:

• Go to your organization Settings
• Select Variable set tab and click button Create variable set
• In Variables panel you need to define your variables

What is left is to attach the variable set to your workspace, or you can enable this set for all workspaces in the organization.

Variables set has lower precedence than workspace variables. Definition of the same variable in workspace will be used in execution. Here you can read more.

Terraform Cloud is an application that helps teams use Terraform together. I am using it for side projects like my cloud infrastructure. Last time I had to see trace logs to find an issue with one of the managed resources.

Terraform has detailed logs which can be enabled by setting the TF_LOG environment variable to any value. This will cause detailed logs to appear on execution.

Enabling verbose logging in CLI

You can set TF_LOG to one of the log levels TRACE, DEBUG, INFO, WARN or ERROR to change the verbosity of the logs. You can set this variable in two ways. First option is to set variable for shell session:

$ export TF_LOG=TRACE
$ terraform plan

Second option is to change variable before command execution

$ TF_LOG=TRACE terraform plan

Enabling logging in Terraform Cloud

If this run is in Terraform Cloud or Terraform Enterprise with Remote Execution, perform these steps:

• Select your workspace,
• Go to Variables tab,
• Add variable in Workspace variables panel and create TF_LOG variable:

  Ensure you have selected Environment variable button

• Perform the run that you want to trace

Another option is to enable local execution

If you are working on your own project, it will be much more convenient to disable Remote Execution and execute the run locally. Go to workspace Settings and in Execution Mode panel switch the button to Local. Then you can run plan from your local machine.

Once the issue is resolved, unset the TF_LOG environment variable to disable the enhanced logging.