Securely populating Azure Key Vault via Azure DevOps

Introduction

Update: This template is available in TheYAMLPipelineOne on GitHub

One the classic conundrums that occurs when properly managing Azure Key Vault is how to seed the values of Key Vault when the secrets cannot be populated via Infrastructure as Code (IaC). This may include scenarios when storing secrets to connect to either on prem or 3rd party resources.

Traditionally a common answer is to manually insert the secret to the Azure Key Vault. The flaw in this approach is first, the key vault must exist, the user must have proper access, and the organization must be comfortable giving the appropriate person access. Furthermore, there is an additional risk if not properly defined via IaC that the secret could be reverted.

Another approach, which also happens more than most would like to admit, is the secret itself is stored in source control. This is not a secure method and breaks a lot best practice.

I have also seen where the ADO Variables are passed in as override parameters to the ARM Deployment Task. This does securely pass the secrets; however, this requires the ARM/Bicep file to have the appropriate parameters mapped to receive the override parameters.

So then what is an alternative to securely populating Azure Key Vault via Azure DevOps? This post will talk and provide the code necessary to securely take a secret defined via an Azure Variable Group and securely insert it into an Azure Key Vault for additional services to consume.

Disclaimer

This is different than linking a Variable Group to an Azure Key Vault. That linkage is referring to a variable group having knowledge of a value already stored in Key Vault. This post will cover how to populate Key Vault secrets from an Azure DevOps Variable Group.

PreReqs

For this to work the service principal connecting Azure DevOps (ADO) and Azure should have at least an appropriate RBAC role to create/update a Key Vault secret.

To follow along with this walkthrough one should be leveraging YAML Multi Stage Templates. Feel free to check out a presentation I gave on leveraging YAML Deployment Pipelines. If you prefer reading here is another blog post covering an Introduction to YAML Pipelines.

Lastly another requirement is needing the ability to create a Variable Group in ADO and populate it with secret values as well as a Key Vault. This Key Vault may be standalone in Azure or created via IaC by a previous task. I will not go into how to create a Key Vault via IaC for sake of simplicity. If you are interested in creating an Azure Key Vault via IaC check out the Create an Azure Key Vault with RBAC and a secret azure quickstart.

Walkthrough

This process will involve the retrieval of a secret(s) from an ADO Variable Group, looping through the list of secrets via YAML Template and then using the AzurePowershell ADO task to populate an Azure Key Vault.

ADO Variable Group

The first requirement will be the creation of an ADO variable group which can be done by selecting Pipelines->Library and create a new group. I have created the below ‘Examples.VariableGroup.dev’

Take special note and thought when naming the variable group. Naming of deployment an Azure related resources is key. For example, and guidance check out my Intro: CI/CD Pipelines for Bicep post.

I would suggest naming the variables the same as you’d like them to appear in Key Vault. This will make thing simpler when we get to the Azure PowerShell task.

Another consideration is the calling pipeline needs to have access to the Variable Group. This can be confirmed by clicking “Pipeline permissions” and confirming either a.) the specified pipeline has access or b.) the variable group is open for all pipelines to use:

YAML Template

Since we want to be able to accommodate the need to populate 1 to x secrets it makes sense to account for this with a YAML template.

Something simple like:

variables: 
- group: 'Example.VariableGroup.dev' # variable group

parameters:
- name: secretList
  type: object
  default: ['testSecret1', 'testSecret2']
steps:     
- ${{ each secret in parameters.secretList }} :
  - template: ps_update_keyvault.yml
    parameters: 
      secretName: ${{ secret }}

For simplicity the variable group is hard coded, ideally this can be parametrized if your YAML is setup correctly. Can see how an environmentName parameter could make this even more flexible.

The secret list here does need to be seeded. This is an unfortunate limitation with how YAML populates the tasks required to run. There are ways to get all the variables in an ADO variable group; however, since these are secrets in order to do anything with them, they need to be loaded in the env of the PowerShell script. The most scalable approach will be to do read these in one at a time since we want to account for 1 to x variables.

Azure Powershell

Now this ps_update_keyvault.yml template will have a task to execute an Azure PowerShell task. Why Azure PowerShell vs PowerShell? Because we will use the Az.KeyVault Module to set the secret. The Azure PowerShell task has this module preloaded in addition to the functionality to authenticate to Azure via the Service Connection.

parameters:
    secretName: ''

steps:
- task: AzurePowerShell@5
  inputs:
    azureSubscription: 'Example - Dev'
    ScriptType: inlineScript
    azurePowerShellVersion: latestVersion
    inline: |
      $secretvalue = ConvertTo-SecureString $env:Mapped_Secret -AsPlainText -Force
      $secret = Set-AzKeyVaultSecret -VaultName "kv-adoSecret-test" -Name ${{ parameters.secretName}} -SecretValue $secretValue
  env:
    Mapped_Secret: $(${{ parameters.secretName }})

Couple of items to call out here. I mentioned the secerts in the Variable Group matching the name of the secrets in Key Vault. Can see here then we are reusing the secretName for both retrieval of the variable and population on the Key Vault. The other cool thing is the Mapped_Secret in the env section. It’s wrapped with $() as this will pass the parameter name in as a string; however, the variable at execution will grab it from the Variable Group.

Also be sure to pass in the Key Vault name as a parameter as this will again help with scaling this solution out as well as addressing how to pass the azureSubscription name with the appropriate Service Principle.

Can confirm that the secrets are now in the Key Vault and their contents were never written to any log or activity file

What about IaC to Configure Secrets

Wanted to talk about this section as depending on if you are using ARM/Bicep or Terraform the question may logically come up on what about the secrets you create via IaC. Not the value of the secret but the data about the secret, such values as active from and active to, etc..

ARM/Bicep if doing Incremental deployments this should be unimpacted the Azure PowerShell is only set the secret value.

If using Terraform the value for the secret can be ignored by leveraging the lifecycle and ignore_changes meta-argument to preserve the value of the secret and have it populated via ADO.

If you don’t have the secret set via IaC presently it will just create the vault in the Key Vault for you.

Conclusion

Hopefully this gives an idea of how to securely populate Azure Key Vault via Azure DevOps Variable Groups. This solution is secure, rerunnable, and scalable. If secret needs to be update one just has to update the appropriate Variable Group. Then re-run the last successful deployment. This will work since the Variable values are pulled by reference at execution time.