Terraform Registry

When you work with Terraform you will most likely use Terraform providers and external modules. These providers and modules are published to the public Terraform registry available at registry.terraform.io.

I’ve previously written about Terraform providers and modules in my Terraform associate course:

In this blog post you will learn about what the public Terraform registry is, how you use it in your Terraform configurations and what alternatives there are to the public registry.

What is the public Terraform registry?

#

The public Terraform registry is the official location for published provider versions and external modules.

Think of a registry as a storage location for provider binaries and Terraform module configurations. This is not an entirely accurate description because the registry itself does not store the provider binaries and module source code, it only stores references to these.

All official HashiCorp supported providers are published to the public Terraform registry. However, each provider is backed by its own git repository where the source code for the provider is located and the build and publish process for new versions take place. This is also true for Terraform modules published to the public registry. Each module has its own dedicated git repository.

The landing page for the public Terraform registry is available at registry.terraform.io.

• To see the available providers you go to registry.terraform.io/browse/providers
• To see the available modules you go to registry.terraform.io/browse/modules

The page for each provider and module contain thorough documentation to help you get started using that provider or module.

Using providers from the public Terraform registry

#

Referencing a provider that is published to the public Terraform registry is done by specifying a URL in the format <HOSTNAME>/<NAMESPACE>/<TYPE> in the required_providers block nested inside of a terraform block.

As an example, for the AWS provider:

terraform {
  required_providers {
    aws = {
      source  = "registry.terraform.io/hashicorp/aws"
      version = "5.58.0"
    }
  }
}

When you reference providers in the public Terraform registry you can skip the <HOSTNAME> part of the URL and use the shorthand <NAMESPACE>/<TYPE> instead.

The example above then becomes:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "5.58.0"
    }
  }
}

No matter how you have referenced your providers they will be downloaded once you run terraform init:

$ terraform init
Initializing the backend...
Initializing provider plugins...
- Finding hashicorp/aws versions matching "5.58.0"...
- Installing hashicorp/aws v5.58.0...
- Installed hashicorp/aws v5.58.0 (signed by HashiCorp)
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

After running the terraform init command you can see the downloaded providers in your working directory, more specifically in a new directory named .terraform:

$ tree -a .
.
├── .terraform
│   └── providers
│       └── registry.terraform.io
│           └── hashicorp
│               └── aws
│                   └── 5.58.0
│                       └── linux_amd64
│                           ├── LICENSE.txt
│                           └── terraform-provider-aws_v5.58.0_x5
├── .terraform.lock.hcl
└── main.tf

7 directories, 4 files

The exact directory structure will depend on your computer architecture and what providers you are referencing.

Using modules from the public Terraform registry

#

Referencing a module from the public registry is similar to referencing providers, you use a URL in the format <HOSTNAME>/<NAMESPACE>/<NAME>/<PROVIDER>:

module "network" {
  source  = "registry.terraform.io/terraform-aws-modules/vpc/aws"
  version = "5.16.0"
}

Just like with providers you can use the shorthand reference:

module "network" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "5.16.0"
}

External modules from the public registry are downloaded and placed in the .terraform directory when you run terraform init:

$ terraform init
Initializing the backend...
Initializing modules...
Downloading registry.terraform.io/terraform-aws-modules/vpc/aws 5.16.0 for vpc...
- vpc in .terraform/modules/vpc
Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Using previously-installed hashicorp/aws v5.58.0

Terraform has been successfully initialized!

You can see the downloaded module’s configuration files¹:

$ tree -L 4 -a .
.
├── .terraform
│   ├── modules
│   │   ├── modules.json
│   │   └── vpc
│   │       ├── .editorconfig
│   │       ├── .git
│   │       ├── .github
│   │       ├── .gitignore
│   │       ├── .pre-commit-config.yaml
│   │       ├── .releaserc.json
│   │       ├── CHANGELOG.md
│   │       ├── LICENSE
│   │       ├── README.md
│   │       ├── UPGRADE-3.0.md
│   │       ├── UPGRADE-4.0.md
│   │       ├── examples
│   │       ├── main.tf
│   │       ├── modules
│   │       ├── outputs.tf
│   │       ├── variables.tf
│   │       ├── versions.tf
│   │       └── vpc-flow-logs.tf
│   └── providers
│       └── registry.terraform.io
│           └── hashicorp
├── .terraform.lock.hcl
└── main.tf

10 directories, 17 files

As you can see, much of the backing git repository is downloaded.

As always, the .terraform directory should not be included in your own git repository.

Publishing providers and modules to the public Terraform registry

#

You can publish your own providers and modules to the public registry. The exact steps for this are outside the scope of this blog post.

For providers the high-level steps are:

1. Write your provider, suggestively using the Terraform Plugin Framework.
2. Document the provider in Markdown files.
3. Create a manifest file describing your provider.
4. Create a GitHub release.
5. Publish the release to the registry.

Find details in the documentation.

For modules the steps are much simpler:

1. Create a GitHub repository following the naming convention terraform-<PROVIDER>-<NAME>
2. Add the module source code. Use the standard module structure.
3. Add a repository description.
4. Create a release following semantic versioning x.y.z or optionally prefixed with a v as in vx.y.z.
5. Publish the module to the registry.

Find details in the documentation.

What else does the registry contain?

#

Apart from providers and modules there are also collections of Sentinel policies and HCP Terraform run tasks. Both are used together with HCP Terraform, but Sentinel policies can be used outside of HCP Terraform as well.

Sentinel is HashiCorp’s tool for policy-as-code.

Run tasks are custom integration points in the Terraform workflow in HCP Terraform. You can call your run tasks before and after plan and apply operations. This allows you to integrate with third-party systems to perform custom checks for your Terraform configurations and plan outputs.

Both these additional features are currently marked as beta.

Alternatives to the public Terraform registry

#

You do not have to publish your custom providers and modules to the public Terraform registry to be able to use them in your private Terraform configurations.

A common alternative is to set up your own private Terraform registry. This allows you to keep your own set of private modules and providers that only your organization can access. This is useful if you have strict rules around what modules and providers can be used and require full control.

If you are using HCP Terraform then you already have a private Terraform registry in your HCP Terraform organization.

You can read more about how you create your own private registry in the documentation for private registries and the registry HTTP API.

Conclusions

#

Even if you were not aware of the public Terraform registry from before, chances are you have been using it throughout your Terraform career.

The public Terraform registry is where all of the official Terraform providers are published, and a huge collection of external Terraform modules. Apart from providers and modules you can also find collections of Sentinel policies and HCP Terraform run tasks.

You can reference providers and modules from your Terraform configurations and they will be downloaded from the public registry when you run terraform init.

If you have a requirement to use a private registry you can. You can host your own private registry or use the private registry that comes with your HCP Terraform organization.