Hashicorp Vault is a handy tool for scalable secrets management in a distributed system or team-based project. Unfortunately, the only out-of-the-box way to configure it is through its API (or a UI), but most projects that need Vault will need to manage the configuration in source control.

There’s a workaround explained on the Hashicorp blog. It’s a neat hack, but here’s a quick note about why using Terraform’s Vault integration is a better idea for production use.

The Problem

The problem is that Hashicorp’s trick only works for positive changes. If you add something to your config, it’ll get added to the Vault server. If you remove it later, though, the automation won’t clean it up. The same problem occurs if you use other stateless tools like Ansible.

The Solution

Terraform solves this problem by sacrificing statelessness. It keeps track of everything it creates in a file stored on disk, or in one of its supported backends. By doing a three-way comparison between the config and the state file and the server, it can handle both positive and negative changes.

Going from a stateless config system to a stateful one is kind of a pain, but it does solve a lot of problems. In theory, the Vault config API is idempotent, so the state file is only needed during changes, and you can reconstruct it any time you have a config version that’s in sync with the server. On the other hand, Terraform is useful for managing other infrastructure, so it might be worth just accepting the Terraform way.

Things to be Aware of

• Everything that Terraform manages is stored unencrypted in the state file, so be careful about managing Vault secrets this way
• The state file is source control friendly, but there are advantages to using one of the external backends (like locking)
• The confusingly named vault_generic_secret in Terraform is for any generic value stored in Vault (including backend configuration), not just secrets

Example Code

Here’s a simple example config:

# Pointing to your vault server
provider "vault" {
  address = "https://vault.example.com:8200"
}

# Storing the state file in an AWS S3 bucket (for example)
terraform {
  backend "s3" {
    bucket         = "foo-organisation-vault"
    key            = "vault"
    region         = "us-east-1"
  }
}

# Adding a policy
source "vault_policy" "admin" {
  name = "admin"

  policy = <<EOF
path "*" {
        policy = "sudo"
}
EOF
}

Your Vault instance will need to be unsealed, then you can authenticate and use Terraform as normal.

$ export VAULT_ADDR=https://vault.example.com:8200
$ vault auth -method=userpass username=root password=hunter2
$ terraform plan -out /tmp/plan