Terraform 9 of 10: Remote State using AWS Cloud9, Amazon S3, and Amazon DynamoDB

January 04, 2024

Background

This tutorial deals with Terraform remote state. AWS Cloud9 is the environment. Amazon S3 and DynmoDB are also used. Remote state is useful in cases where you might have multiple people making changes or if you have some sensitive state that you don't want available locally.

1 of 12. Open the Terraform documentation for "S3 backend"

Note: This is not the only piece of documentation consulted for this lab. Please see references for a few more links that may be helpful.

2 of 12. Open your AWS Cloud9 environment

Creating An Integrated Developer Environment (IDE) in the Cloud in Two Minutes!: AWS Cloud9(Step-by-Step)

3 of 12. Create a new directory and main.tf for a remote-state project.

Starting from your base "environment" folder, setup the folder for this project.


pwd
mkdir terraform-remote-state-example && cd terraform-remote-state-example
touch main.tf

4 of 12. Set up AWS provider and EC2 instance in "main.tf" file

Recall prior warnings about AMI being region-specific, before using this example.


provider "aws" {
  region = "us-east-1"
}

resource "aws_instance" "example" {
  ami           = "ami-079db87dc4c10ac91" #Amazon 2023 AMI
  instance_type = "t2.micro"
}

5 of 12. Initialize and apply

Not all output is shown here, but you should be familiar with these. The main thing here is that the backend (terraform.tfstate and the lock file) are still local.


terraform init
terraform plan
terraform apply

6 of 12. Create an S3 bucket and DynamoDB database.

The intent is store your Terraform state remotely. Note that you will need a "unique" S3 bucket name. The DynamoDB is used for the "lock" file, to prevent concurrent attempts at access to the state. Note: This is created outside of Terraform intentionally, to prevent a terraform destroy from taking it out.


aws s3api create-bucket --bucket tcb-devops-state-demo-remote-8675309-jenny --region us-east-1
aws dynamodb create-table \
  --table-name tcb-devops-state-lock-table \
  --attribute-definitions AttributeName=LockID,AttributeType=S \
  --key-schema AttributeName=LockID,KeyType=HASH \
  --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5 \
  --region us-east-1

7 of 12. Create and modify a "backend.tf" in the project folder.

This will have the parameters for an S3 backend, which includes your DynamoDB table.


pwd
touch backend.tf


terraform {
  backend "s3" {
    bucket         = "tcb-devops-state-demo-remote-8675309-jenny"
    key            = "terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true
    dynamodb_table = "tcb-devops-state-lock-table"
  }
}

8 of 12. after these changes, you do have to re-initialize.

You should get prompted about migrating from local state to remote state. Please confirm.

terraform init

9 of 12. Modify the infrastructure.

We want to change something, so we can be sure new changes are being written to our remote state. Add a "name" tag to your instance, in the main.tf


provider "aws" {
  region = "us-east-1"
}

resource "aws_instance" "example" {
  ami           = "ami-079db87dc4c10ac91" #Amazon 2023 AMI
  instance_type = "t2.micro"
  tags = {
    Name = "example-8675309-jenny"
  }
}

10 of 12. Plan and apply

Note that the "Name" tag is case-sensitive. "name" is different from "Name". The Camel-case version: "Name" successfully re-named the EC2.


terraform plan
terraform apply

11 of 12. Check the contents of the S3 bucket and DynamoDB table, and validate data is stored there.

Because the dynamoDB table is being used for the "lock", you can do a "terraform apply" or "terraform destroy"--- just don't respond to the prompt. Note that the local "terraform.tfstate" is EMPTY, because state is now being stored at the S3.

cat terraform.tfstate

12 of 12. Cleanup

Note: you did not create the bucket and DB via terraform, so you have to remove those too. Go back and double-check your AWS console, and make sure that the resources are gone. Note that if your bucket is not empty, "delete-bucket" will not work, and you should use "rb" instead.


terraform destroy
aws s3 rb s3://tcb-devops-state-demo-remote-8675309-jenny --force
aws dynamodb delete-table --table-name tcb-devops-state-lock-table