Migrate Your Terraform State to HCP Terraform

When you read the title to this blog post you might have gotten the idea that this post would be about the new tfmigrate tool from HashiCorp. This tool allows you to migrate many Terraform configurations from wherever you store the state into HCP Terraform.

However, can you do it in a simpler way without any additional tools? Turns out it is very easy to do it.

Set up an initial state

#

In this scenario I assume you have a state storage solution set up on Amazon S3.

My initial configuration looks like this:

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
    }
  }

  backend "s3" {
    bucket = "terraform-state-xxx-xxx"
    region = "eu-north-1"
    key    = "state/demo/terraform.tfstate"
  }
}

provider "aws" {
  region = "eu-west-1"
}

resource "aws_s3_bucket" "dummy" {
  bucket_prefix = "dummybucket"
}

This Terraform configuration targeting AWS creates an S3 bucket. Don’t be confused by the amount of S3 buckets involved here. One bucket is used for state storage (the one named terraform-state-xxx-xxx) and one is created using this Terraform configuration (the one with a name starting with dummybucket).

Initialize and apply this configuration to have an initial Terraform state file.

$ terraform init 
$ terraform apply

Commit the code to a git repository. I use GitHub because I have already connected GitHub to my HCP Terraform organization.

Create a HCP Terraform workspace

#

Now imagine that you want to create a workspace connected to this git repository and migrate the state file to HCP Terraform.

Your initial approach might be to create a VCS-connected workspace:

If you do that, and you connect it to the correct git repository, then this is what will happen in the first plan operation:

Oh no, one resource will be created! It seems that we can’t influence the terraform init command to use the -migrate-state flag.

What we need to do instead is to configure a CLI-driven workspace:

We can change the previous workspace we created to be a CLI-driven workspace.

Migrate the state file to HCP Terraform

#

With a CLI-driven workspace in place, comment out your current backend block and instead add a cloud block as shown in the following code:

terraform {
  required_providers {
    aws = {
      source = "hashicorp/aws"
    }
  }

  # backend "s3" {
  #   bucket = "terraform-state-xxx-xxx"
  #   region = "eu-north-1"
  #   key    = "state/demo/terraform.tfstate"
  # }

  # update the values to correspond to your environment
  cloud {
    organization = "mattias-fjellstrom" 

    workspaces {
      name = "demo-infrastructure"
    }
  }
}

Now run a terraform init operation:

$ terraform init
Initializing HCP Terraform...
Migrating from backend "s3" to HCP Terraform.
Do you wish to proceed?
  As part of migrating to HCP Terraform, Terraform can optionally copy
  your current workspace state to the configured HCP Terraform workspace.

  Answer "yes" to copy the latest state snapshot to the configured
  HCP Terraform workspace.

  Answer "no" to ignore the existing state and just activate the configured
  HCP Terraform workspace with its existing state, if any.

  Should Terraform migrate your existing state?

  Enter a value: yes

Acquiring state lock. This may take a few moments...
Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Using previously-installed hashicorp/aws v5.97.0

HCP Terraform has been successfully initialized!

Answer yes to the question about migrating the state to HCP Terraform. Once the operation is complete, verify that the state is indeed on HCP Terraform in your workspace:

This looks good!

Connect the workspace to GitHub

#

Now we are in a position where we can add the connection to GitHub again. Go to your workspace settings and select Version Control in the menu, then click on Connect to version control and follow the guide:

The next plan operation looks a lot better than the first one:

Migration complete.

If you need to migrate a large number of Terraform configurations you might be better off using the tfmigrate tool. See the documentation for how to install and use this tool.