Terraform – State

Introduction

Terraform state is something which is usually tackled quite later in the game when folks have become quite comfortable with writing terraform code. But, we are going to do it much earlier :). You will see that it helps you visualise your code a lot better and have more interesting/innovative ideas.

In this blog entry, we discuss terraform state by breaking it down into smaller chunks.

You can find all the code for this blog here. Look for blog-5. So let’s start off….

What is terraform state?

Terraform state is a record of any infrastructure or configuration which has been provisioned by terraform. Simple as that. So, whenever you add/change any part of the terraform code and apply it, terraform will compare it with the state and apply only the required changes. It does not execute the whole code which makes it pretty performant if you have large infrastructures.

There are various ways to store the state. By default, it is stored locally in a file. But you can store it in an S3 bucket, Postgres database and many other ways. Different type of technologies can store a terraform state, more on this later.

Terraform state can be seen using the following command

Local terraform state

This is the default way of storing state in any terraform project. Local terraform state is stored as a JSON file. So yeah you could open it and take a peek into it. It’s not recommended to edit this file manually. Few things to remember

1. Terraform state stores unmasked passwords – yikees!!
2. The state is stored in the file called terraform.tfstate.
3. The state is only stored if the provisioning has been successful.
4. Caveating above, if the apply operation is partially successful it will store the partial state.

Screenshot of the terraform state from one of the previous blog is shown below

Remote terraform state

The local state is fine when you are playing with terraform on doing some small proof of concepts. But when the team expands and you need to work along with other people on terraform, local storage falls short. That is where terraform allows you to store the state remotely and also provide locking capabilities as well. There are many technologies which are supported by terraform to store its state. A list of available backends is on this link.

Locking

Sharing state between team members can lead to situations where both may end up updating the state at the same time. Leading to race conditions. One way would have been to store the state in some version control like git. But that is a bad idea and is not really a recommended way for a number of reasons starting from exposed passwords to developer mistakes etc. Well, this is where terraform steps in to provide tools to lock the state and not cause any collisions. During this blog entry, we will see how we can lock the state so as to avoid race conditions and related issues.

So the way locking works is that if someone is executing a code and has already acquired a lock on the terraform state, then your code would have to wait till the time the lock is released.

Terraform state backends

Being on the topic of remote state we need to talk about state backends or just backends. Terraform backends are really useful when you are working in teams and/or if you have large infrastructures to manage. Here are the two backends I have used recently. Found them to be quite easily integrated. There are two ways to provide backend information to the terraform code.

1. Hardwiring it in the code. Which is not a great idea
2. Provide it as a configuration file

AWS S3 storage

If you are using AWS(like I am) it is probably the first one to checkout. Terraform needs access to an S3 bucket and key information and it is game on. To use S3 bucket for remote state storage, your account needs to have permissions to list, get and put on the s3 bucket. For the purpose of this blog, we will extend the previous blog about variables and have the s3 backend configured like below by adding an additional file called backend.tf

Another way of configuring the backend is by storing all this in a configuration file and passing this to terraform when the state is applied using terraform apply. In this case, the backend.tf file looks like this

The configuration file cw-blog-5.ini will look like below

Finally, the terraform state is initialised like below

The output of the initialisation is below

To take this a step further we can introduce additional locking capability using AWS Dynamo DB. All terraform needs is a table to which it can access and has permissions to get, put and delete item.

So ahead and create a dynamo DB table. It has just one attribute called LockID which is of the type String.

An additional parameter is added to the config in case you are using configuration file to initialise.

In case of a configuration file fire the following command.

Below is the output using the configuration file.

The output of the initialisation is below

PostgreSql storage

Adding postgres as a backend is pretty simple as well. All you need is to provide a connection details of a postgres database. The backend.tf is shown below.

The database configuration file db.ini is shown below.

You can now initialise the terraform state and do an apply

As you can see terraform on initialisation creates a schema in Postgres called terraform_remote_state with a table called state.

Querying terraform state

Finally, once you have applied your configuration using terraform apply. You can always query your terraform state. Using the following commands

You can also filter the resource which have been created using id

There are a couple of more ways to do it and you can find a nice page in terraform documentation on this link.

Phew, this brings us to the end of this long post on terraform state. I hope you find it helpful and useful in your work. If you like it please do share it.