How to migrate from ansible vault to hashicorp vault

How to migrate from ansible vault to hashicorp vault

2018-06-25 0 Par seuf

Ansible Vault

Today, all our project passwords are stored in ansible vault files, on file per environnement, stored in a « passwords » directory

They are loaded at the beginning of the main playbook :

- hosts: all:!localhost
  any_errors_fatal: true
  gather_facts: true
    - include_vars: password/password_{{ env }}.yml
      connection: local
      no_log: True
        - password
        - always

Usually, to add or modify a password, we have to fetch the ansible vault key from a remote server in the environment ; copy it into our workspace, then run the command

ansible-vault edit --vault-id passwords/password_dev.yml

Then we can edit the password, and re-commit / push the new vaulted password file to the git repository.

Hashicorp Vault

We are planning to migrate from ansible vault to hashicorp vault. This will simplify password management / re-generation with the great vault-ui.


To simplify the migration to Hashicorp Vault I’ve developped a tiny script to :

  • decrypt all ansible vault password files
  • convert each password file to json
  • import each password file into hashicorp vault
  • replace all passwords with the ansible lookup  hashivault into the yaml file.

The requirement to this scripts are :

  • jq : for json parsing
  • curl : for hashicorp vault api authentication
  • j2y : for converting yaml into json
  • vault binary : for password import into hashicorp vault
  • sed : for yaml replacement
  • All ansible vault keys are in current directory (named « .vault.<env>« )

here is the script :


export VAULT_ADDR=
export VAULT_ROLE_ID=0acee709-b16c-4abd-20a3-12159d6c8b36
export VAULT_SECRET_ID=fcf81d75-bd31-6e8f-239b-fc09c5dd744e

for i in $(find . -name 'password_*.yml'); do
  file=$(echo $i | cut -d '/' -f 2)
  env=$(echo $file | sed 's/password_//' | cut -d '.' -f 1)
  if [ -f ".vault.$env" ]; then
      cp $i $i.decrypt
      ansible-vault decrypt --vault-id .vault.$nna.$env $i.decrypt
      cp $i $i.decrypt
  # converting to json $i.decrypt
  j2y -r $i.decrypt > password_$env.json
  rm -f $i.decrypt

  # Authenticate to $VAULT_ADDR
  data="{\"role_id\": \"${VAULT_ROLE_ID}\", \"secret_id\": \"${VAULT_SECRET_ID}\" }"
  export VAULT_TOKEN=$(curl -s -k -X POST ${VAULT_ADDR}/v1/auth/approle/login -d "$data" | jq '.auth.client_token')

  vault write secret/ida/$nna/$env @password_$env.json

  # converting yml file with lookup hashi_vault
  cp $i.decrypt $
  sed -i "s!\([^:]*\):.*!\1: \"{{ lookup('hashi_vault', 'secret=secret/app/$env:\1') }}\"!" $
  dos2unix $

Now I just have to run the script and all my passwords are migrated.


To use the ansible lookup hashivault plugin the environment variables VAULT_ADDR and VAULT_TOKEN must be exported.

All our playbooks are launched from jenkins, So we just have to connect to hashicorp vault from Jenkins before ansible playbook.

To do that we use the Hashicorp Vault Jenkins Plugin that add a new Jenkins Hashicorp Vault credentialtials scoped to our application folder. Then before each build, the vault plugin will export the required variables.

Finally the ansible lookup plugin will fetch the passwords directly from hashicrop vault at runtime 🙂