Hello World

As promised, before I dig into the exam objectives for becoming Vault Associate certified I will present a Hello World of HashiCorp Vault in this post. Going through the exam objectives part by part in order would be a somewhat difficult way of learning Vault unless you have first seen the whole picture at least once. This is also clear if you follow the official study guide provided by HashiCorp, they mix objectives left and right in each part of the guide. I believe it will be helpful to first go through this Hello World type of post where we touch on many of the concepts that we will later dive deeper into.

Installing Vault

#

Before we can do anything with Vault we need to install it. I prefer to install stuff with Homebrew, and conveniently enough HashiCorp allows me to do just that:

$ brew tap hashicorp/tap
$ brew install hashicorp/tap/vault

More detailed instructions for how to install Vault can be found in the official documentation.

After installing Vault I can verify that the installation was successful by running my first Vault CLI command:

$ vault -version

Vault v1.14.0 (13a649f860186dffe3f3a4459814d87191efc321), built 2023-06-19T11:40:23Z

I will be using Vault version 1.14.X for this series, unless I am taking too long to write this series so that 1.15.X arrives.

Starting the Vault server

#

In this post, and in most of the following posts in this series, I will be using the Vault development server. This is, as the name suggests, a server specifically for development and testing. It is not for production use. The development server simplifies the startup significantly compared to a production grade Vault cluster. In part nine of this series I will go through how to start Vault for real. We need to run a Vault server to be able to work with Vault, this is true no matter where you run Vault. So even if we use a hosted version of Vault in the HashiCorp Cloud Platform (HCP) we still have a server (but managed by HashiCorp).

Enough chit-chat, to start the development server I run a single command:

$ vault server -dev

(... output truncated ...)

==> Vault server started! Log data will stream in below:

(... output truncated ...)

WARNING! dev mode is enabled! In this mode, Vault runs entirely in-memory
and starts unsealed with a single unseal key. The root token is already
authenticated to the CLI, so you can immediately begin using Vault.

You may need to set the following environment variables:

    $ export VAULT_ADDR='http://127.0.0.1:8200'

The unseal key and root token are displayed below in case you want to
seal/unseal the Vault or re-authenticate.

Unseal Key: I+HW1DY/NHB8HS5Nc3uaK1phDLt0+zFTP1/W48JsF0I=
Root Token: hvs.4pr1W6DoLP7vjs6EIuxZHrjP

Development mode should NOT be used in production installations!

There are a few pieces to note in the output:

• We are recommended to set an environment variable by running export VAULT_ADDR='http://127.0.0.1:8200', this variable sets the address to Vault server. It runs on localhost (127.0.0.1) on port 8200. Note that the address uses HTTP and not HTTPS. You can run the development server with HTTPS by changing the start-up command to vault server -dev-tls, but I will not do that here.
• We are given an unseal key with the value I+HW1DY/NHB8HS5Nc3uaK1phDLt0+zFTP1/W48JsF0I=. We will not need to bother with unseal keys until part nine of this series, so more on those then.
• We are given a root token with a value of hvs.4pr1W6DoLP7vjs6EIuxZHrjP. This token will be used in all interactions with Vault.

Take a look at the contents of a file called .vault-token in your home directory:

$ cat ~/.vault-token

hvs.4pr1W6DoLP7vjs6EIuxZHrjP

This is the root token from the output above. This token will be used automatically when you run Vault CLI commands unless you explicitly set another token. Also, if you set an environment variable named VAULT_TOKEN with the value of a Vault token, then that environment variable will take precedence and be used first.

Apart from this we can see from the output clear warnings that the development server should not be used in production. We are ready to move on in our Hello World journey.

Working with secrets through secrets engines

#

What is a secrets engine?

Secrets engines are components which store, generate, or encrypt data. Secrets engines are incredibly flexible, so it is easiest to think about them in terms of their function. Secrets engines are provided some set of data, they take some action on that data, and they return a result.¹

The easiest secret engine to start working with is the Key/Value secrets engine, or kv secrets engine for short. The kv secrets engine actually exists in two versions, version 1 and 2. The main difference between them is that version 2 allows to have a history of versions for a given secret. When you start a development server Vault comes with the kv secrets engine version 2 enabled by default. However, I will enable the kv v1 secret engine:

$ vault secrets enable kv

Success! Enabled the kv secrets engine at: kv/

The output says that the kv secrets engine is enabled at kv/. This is a path in the Vault system. Everything in Vault exists at a path. If you remember only one thing from this post, let it be the previous sentence! We will see many examples of this throughout this blog series. I can verify what secrets engines I have with the following command:

$ vault secrets list

Path          Type         Accessor              Description
----          ----         --------              -----------
cubbyhole/    cubbyhole    cubbyhole_29554004    per-token private secret storage
identity/     identity     identity_07cbe533     identity store
kv/           kv           kv_7b18d0e0           n/a
secret/       kv           kv_f93f6e98           key/value secret storage
sys/          system       system_9cca3491       system endpoints used for control, policy and debugging

This list contains five secrets engines. The third one in the list is the one we just enabled. For now we will ignore the other secret engines listed, just remember that the Vault development server (and a regular Vault server) comes with a few secrets engines pre-enabled.

I would like to store a new secret using my kv secrets engine. I can do this with the following command:

$ vault kv put -mount=kv my-secret hello=world

Success! Data written to: kv/my-secret

At first the syntax is not at all intuitive. What the command is saying is that we want to put (or create) a new secret named my-secret using the kv secrets engine. The specific instance of the kv secrets engine is mounted at the path kv. Yes, this suggests that we can enable multiple instances of the same secrets engine at different paths. More on that in a future post. The content of the secret named my-secret is hello=world. Once I have a secret I can read that secret back from Vault:

$ vault kv get kv/my-secret

==== Data ====
Key      Value
---      -----
hello    world

This command says that we want to get (or read) a secret from the kv secrets engine, specifically the one available at the path kv/my-secret. Our data with Key equal to hello and Value equal to world is returned to us.

Authenticating using authentication methods

#

In the previous section we enabled a secrets engine, we wrote a secret, and finally read that secret back again. We did all this using the root token that was provided for us when we started the Vault development server. In essence we authorized our actions using this token. Is this how Vault is supposed to be used? Not at all!

The root token can do anything in Vault, so it should not be shared with more people than absolutely necessary. In fact, a best-practice is to only use the root token during initial server setup, and then create a new admin token with only the necessary permissions to administer the Vault server during its lifetime. The root token should be kept in a secure storage location of some sort.

So how are we supposed to use Vault? Users of Vault should authenticate to Vault using an authentication method. What is an authentication method, or auth method for short?

Auth methods are the components in Vault that perform authentication and are responsible for assigning identity and a set of policies to a user. In all cases, Vault will enforce authentication as part of the request processing. In most cases, Vault will delegate the authentication administration and decision to the relevant configured external auth method (e.g., Amazon Web Services, GitHub, Google Cloud Platform, Kubernetes, Microsoft Azure, Okta …).²

To put it another way, auth methods allow us to use a third-party (e.g. GitHub) to authenticate to Vault in order to get a token that allows us to perform actions in Vault. The quote above mentions identities and policies. A user of Vault can use any of the auth methods we enable on our server to prove that they are a specific identity on our server. This identity has one or more associated policies which are documents that specify what actions we can perform at specific paths in Vault. We will soon write a policy, but first let us enable an auth method.

I start by first listing what auth methods I currently have enabled:

$ vault auth list

Path      Type     Accessor               Description                Version
----      ----     --------               -----------                -------
token/    token    auth_token_c932f51c    token based credentials    n/a

It seems like I only have the token auth method enabled at the token/ path. This is a required auth method, because no matter what other auth method we use we end up with a token, which we then use to run Vault commands.

For this Hello World tutorial I will enable an auth method named userpass:

$ vault auth enable userpass

Success! Enabled userpass auth method at: userpass/

The userpass auth method is a basic username and password type of authentication mechanism. It is in fact not using a third-party system for authentication. In a production environment I would not recommend using it, but it is illustrative for the purpose of Hello World.

I will create a first username and password combination:

$ vault write auth/userpass/users/jane.doe password=S3cr3t policies=default

Success! Data written to: auth/userpass/users/jane.doe

I wrote data to auth/userpass/jane.doe, which basically means I created a user with username jane.doe. At this path I entered password=S3cr3t and policies=default. The data is self-explanatory but I will still explain it! I specified that the user with username jane.doe should have a password with the value of S3cr3t and that the user should have the policy named default attached to the token the user gets when she signs in. What is the default policy? It is a required policy that every user in Vault has, by default. You can edit the contents of the default policy if you wish but you can’t delete it. More on policies soon!

Now it is time for this jane.doe to sign in to Vault:

$ vault login -method=userpass username=jane.doe password=S3cr3t

Success! You are now authenticated.

Key                    Value
---                    -----
token                  hvs.CAESICC<truncated>bU9ZYnJxbVk
token_accessor         sGkEujtl9ZU9i4Cqk2iPcn57
token_duration         768h
token_renewable        true
token_policies         ["default"]
identity_policies      []
policies               ["default"]
token_meta_username    jane.doe

jane.doe successfully signed in and received a token in the response. This token can now be used to run other Vault commands as allowed by the attached policies - only the default policy in this case.

Restrictions using policies

#

In the previous section we authenticated using an auth method and we connected this with the default policy. As mentioned, the default policy is always provided to everyone who authenticates to the Vault server. We will now write our own policy and provide this policy to someone who authenticates to Vault. The purpose of a policy is to restrict what someone can do in Vault. In essence a policy states what actions can be performed on a given path in Vault - remember that everything in Vault is path based.

You write policies in Vault using the HashiCorp Configuration Language (HCL). A basic policy might look like this:

# jane.hcl

path "kv/jane.doe" {
    capabilities = ["read", "list", "create"]
}

This policy defines one path block for the path kv/jane.doe. This path corresponds to a secret in the kv secrets engine mounted at the kv/ path. The policy further says that at this path it allows the capabilities of read, list, and create. This is basically what policies are, one or more paths with associated capabilities.

I can create an actual policy object from the policy document:

$ vault policy write jane jane.hcl

Success! Uploaded policy: jane

With the policy created I update my user jane.doe to get this policy, apart from the default policy:

$ vault write auth/userpass/users/jane.doe password=S3cr3t policies=default,jane

Success! Data written to: auth/userpass/users/jane.doe

If I now sign in with the user I can see that the token I get is associated with the jane policy:

$ vault login -method=userpass username=jane.doe password=S3cr3t

Success! You are now authenticated.

Key                    Value
---                    -----
token                  hvs.CAESINa6<truncated>1U3ZZbjhwSlE
token_accessor         fhXC5AnQy2MiqI2hM32qbQub
token_duration         768h
token_renewable        true
token_policies         ["default" "jane"]
identity_policies      []
policies               ["default" "jane"]
token_meta_username    jane.doe

Using my new token, or jane.doe’s new token, to create the original secret we created earlier I am met by a 403 error:

$ vault kv put -mount=kv my-secret hello=world

Error making API request.

URL: GET http://127.0.0.1:8200/v1/sys/internal/ui/mounts/kv
Code: 403. Errors:

* preflight capability check returned 403, please ensure client's policies grant access to path "kv/"

This error happens because the policy (neither jane or default) allows me to create a secret at kv/my-secret. However, the jane policy does allow me to create a secret under kv/jane.doe:

$ vault kv put -mount=kv jane.doe hello=world

Success! Data written to: kv/jane.doe

Summary

#

There we have it! We have configured vault with secret engines and auth methods. We have created a custom policy and assigned it to a user. We have signed in as a user and created a secret. This has been my version of Hello World for Vault! In the next post in this series we will begin taking on the exam objectives and we start with the objective named: Compare authentication methods.