Manage HCP Waypoint with Terraform

HCP Waypoint is an Internal Developer Platform (IDP) backed by no-code modules on HCP Terraform.

The backed by no-code modules on HCP Terraform part is in fact very important. HCP Waypoint is not a stand-alone service.

With the drawbacks out of the way, let’s move into the topic at hand: Manage HCP Waypoint with Terraform!

Waypoint concepts

#

Waypoint has a few concepts you need to be aware of. We will only encounter two of them in this post so I will limit myself to defining these two concepts:

• Template: a template is a metadata layer on top of a Terraform no-code module. It has a name and description. It specifies what values are required and allowed to be passed to the no-code module’s variables.
• Application: an instance of a template. There will be one workspace in HCP Terraform for each application in HCP Waypoint. Many applications can be created from the same template, but each application must have a unique name and create unique resources in the target platforms.

Demo

#

In the following demo I will show the Terraform code necessary to set up required resources on HCP Terraform, HCP, and Azure (my target platform for the demo).

Prerequisites

#

There are a number of prerequisites to follow along in this demo. There is no room to go through how to set each up, if you are interested in any specific details, reach out to me on LinkedIn and I will gladly discuss this with you!

At a high level you will need:

• An HCP Terraform organization with plus tier license
• An HCP organization with a HCP project where Waypoint is not yet enabled
• A GitHub organization added as a VCS provider to HCP Terraform
• An Azure subscription and access to the Azure tenant
• Admin credentials (or close to it) on all the above platforms configured in your terminal environment

For the HCP project prerequisit, you can configure a project using this code snippet:

resource "hcp_project" "default" {
  name = "hcp-waypoint-demo"
}

Then configure the HCP provider for the demo code below as follows:

provider "hcp" {
  project_id = "<project ID>"
}

Connect HCP Waypoint to HCP Terraform

#

You need to provide Waypoint a HCP Terraform token to allow it to read modules, projects, workspaces, and to provision and destroy workspaces on HCP Terraform.

You can give Waypoint access to your full organization, or you can limit what it can see and use.

I will create a dedicated project on HCP Terraform where I want HCP Waypoint to provision all workspaces:

resource "tfe_project" "waypoint" {
  name        = "hcp-waypoint-managed-workspaces"
  description = "Project for HCP Waypoint managed workspaces"
}

I will also create a dedicated team on HCP Terraform for Waypoint, give the team maintainer access to the project I just created, and finally create a team token that I can pass on to Waypoint:

resource "tfe_team" "waypoint" {
  name = "waypoint"
}

resource "tfe_team_project_access" "waypoint" {
  project_id = tfe_project.waypoint.id
  team_id    = tfe_team.waypoint.id
  access     = "maintain"
}

resource "tfe_team_token" "waypoint" {
  team_id = tfe_team.waypoint.id
}

You now have everything you need to connect HCP Waypoint to your HCP Terraform platform. The connection is done using the hcp_waypoint_tfc_config resource from the HCP provider:

resource "hcp_waypoint_tfc_config" "default" {
  tfc_org_name = "<your HCP Terraform organization name>"
  token        = tfe_team_token.hug.token
}

Create a no-code module on HCP Terraform

#

Before we look at the code: what is a no-code module?

It’s almost a normal Terraform module that you publish to your private Terraform registry. It differs in one important aspect: it includes configuration for the providers that the module uses.

A normal Terraform module should only specify what providers will be used (in the required_providers block), and possibly if there will be multiple instances (aliases) of the same provider. However, it should not configure the providers using provider blocks. The providers should be configured in the root module and passed into the module.

For no-code modules this is not true. Here you should configure the providers. A no-code module should stand on its own. This does not mean you must include provider blocks though! You can still configure many providers using environment variables.

Let’s assume we (or at least I) have the following simple Azure Terraform module published to a git repository accessible from my HCP Terraform organization:

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 4.27.0"
    }
  }
}

provider "azurerm" {
  features {}
}

variable "name_suffix" {
  type = string
}

variable "location" {
  type = string
}

resource "azurerm_resource_group" "default" {
  name     = "rg-${var.name_suffix}"
  location = var.location
}

It’s a trivial module creating an Azure resource group. The important part to remember is that it takes two inputs: name_suffix and location.

Create a module out from this repository in your private HCP Terraform registry:

# optionally set this up, or use a data source to reference an existing connection to github
resource "tfe_oauth_client" "github" {
  name                = "GitHub Organization"
  api_url             = "https://api.github.com"
  http_url            = "https://github.com"
  oauth_token         = "<your github token>"
  service_provider    = "github"
  organization_scoped = true
}

resource "tfe_registry_module" "resource_group" {
  name = "azurerm-resource-group"

  vcs_repo {
    display_identifier = "<github-organization>/<github-repo-name>"
    identifier         = "<github-organization>/<github-repo-name>"
    oauth_token_id     = tfe_oauth_client.github.oauth_token_id
    branch             = "main"
  }

  initial_version = "1.0.0"

  # add this because the name will not be reported back correctly causing it to be replaced!
  lifecycle {
    ignore_changes = [
      name,
    ]
  }
}

Creating the module is the first step. You must also make it into a no-code module. You do this using a different resource type (not as part of the tfe_registry_module resource type):

resource "tfe_no_code_module" "resource_group" {
  registry_module = tfe_registry_module.resource_group.id
  
  # what version of the registry module to pin to
  version_pin     = "1.0.0"

  # we add variable options for the location variable
  # however, no need to add it for the name_suffix (why will be explained later in the text)
  variable_options {
    name = "location"
    type = "string"
    options = [
      "westeurope",
      "northeurope",
      "swedencentral",
    ]
  }
}

Now everything on the HCP Terraform side is ready. We can now set up the corresponding part on HCP Waypoint. In this case we will create a template, but there are also add-ons.

A template is the base for an application on HCP Waypoint, and an add-on is extensions to an application. You must create an application (and thus a template) first to be able to install add-ons to your applications later.

locals {
  terraform_no_code_module_source = join("/", [
    tfe_registry_module.resource_group.registry_name,
    tfe_registry_module.resource_group.organization,
    tfe_registry_module.resource_group.name,
    tfe_registry_module.resource_group.module_provider,
  ])
}

resource "hcp_waypoint_template" "resource_group" {
  name                 = "azurerm-resource-group"
  summary              = "Create an Azure resource group"
  description          = "Create an Azure resource group in a given location with a given name suffix"
  terraform_project_id = tfe_project.waypoint.id
  labels               = ["azurerm"]

  # note the name convention from the local value above ...
  # the tfe_registry_module or tfe_no_code_module resources do not expose this attribute in a conveniant format
  terraform_no_code_module_source = local.terraform_no_code_module_source
  terraform_no_code_module_id     = tfe_no_code_module.resource_group.id

  variable_options = [
    {
      name          = "name_suffix"
      user_editable = true
      variable_type = "string"
      options       = [] # include this even if empty!
    },
    {
      name          = "location"
      user_editable = true
      variable_type = "string"

      # these options should correspond to the no-code module options above
      # if not, Terraform will report an error!
      options = [
        "westeurope",
        "northeurope",
        "swedencentral"
      ]
    }
  ]
}

Allow the HCP Terraform project to authenticate to the target platform(s)

#

As I mentioned above, no-code modules are different than normal modules. These modules must configure the providers that they use. However, you can still use things like environment variables for authentication.

The simplest approach is to allow all workspaces in the HCP Terraform project we created earlier to have access to authenticate to the target platforms. In this blog post the demo code needs to authenticate to Azure.

You could set up OIDC federation between HCP Terraform and Azure and set the required environment variables as a project variable set. You can read about how to do this in one of my earlier blog posts.

Again, I am happy to clarify how this is set up if you have any questions! Reach out to me on LinkedIn.

Provision infrastructure through HCP Waypoint

#

With everything set up you are ready to provision infrastructure using HCP Waypoint. This would normally be something your developers (or whoever you build the no-code modules for) would do.

Go to your HCP organization, select the correct HCP project and open the HCP Waypoint service page.

Click on Applications in the menu on the left to come to (the empty) list of deployed applications. To create a new application, click on Create an application to get started:

Select the template you want to base the application on. In this case there is only a single template available to select:

Configure the application by filling in the form. Notice how the location configuration is a dropdown with the allowed values you configured earlier:

Once the application is created you can see how the application was configured, as well as a few other details such as the status of the HCP Terraform workspace that this application is provisioned from:

If you go to the HCP Terraform workspace you can see information that this workspace is provisioned using a no-code module and that it has been created via HCP Waypoint:

You can go to Azure to verify that the resource group has been created if you want (I did!).

Before you tear down the demo infrastructure make sure to first delete the application from HCP Waypoint. Do this by selecting the Manage dropdown in the menu and selecting Delete application. If you don’t delete the application you will not be able to run terraform destroy on the HCP Terraform project.

Summary

#

Configuring HCP Waypoint together with HCP Terraform no-code modules takes a little bit of work. Luckily you can manage most of this using Terraform code. I say most because there are preview features of HCP Waypoint that you (currently) can’t configure using Terraform. These include creating agent groups and actions for these agent groups.

Once you have gotten everything to work I think HCP Waypoint is a nice IDP. I didn’t go through add-ons and actions in this post (maybe in a future post), they allow you to extend your applications with additional features (e.g. adding a database to a web application) and allow you to do day-2 type of operations (e.g. taking backups or upgrading your application version).