Hashicorp Vault is a handy tool for scalable secrets management in a distributed system or team-based project. Unfortunately, the only out-of-the-box way to configure it is through its API (or a UI), but most projects that need Vault will need to manage the configuration in source control.
The problem is that Hashicorp’s trick only works for positive changes. If you add something to your config, it’ll get added to the Vault server. If you remove it later, though, the automation won’t clean it up. The same problem occurs if you use other stateless tools like Ansible.
Terraform solves this problem by sacrificing statelessness. It keeps track of everything it creates in a file stored on disk, or in one of its supported backends. By doing a three-way comparison between the config and the state file and the server, it can handle both positive and negative changes.
Going from a stateless config system to a stateful one is kind of a pain, but it does solve a lot of problems. In theory, the Vault config API is idempotent, so the state file is only needed during changes, and you can reconstruct it any time you have a config version that’s in sync with the server. On the other hand, Terraform is useful for managing other infrastructure, so it might be worth just accepting the Terraform way.
Things to be Aware of
- Everything that Terraform manages is stored unencrypted in the state file, so be careful about managing Vault secrets this way
- The state file is source control friendly, but there are advantages to using one of the external backends (like locking)
- The confusingly named
vault_generic_secretin Terraform is for any generic value stored in Vault (including backend configuration), not just secrets