Efficient Configuration Management in AKS: Integrating Azure App Configuration for Seamless Loading of Key-Value Pairs into Config Maps

2023-06-13 8 min read Tutorial

Did you know that Azure App Configuration Service is a managed service that helps you centralize your application configuration? It provides a way to store all your app configs in one place and manage them centrally. It also provides a way to manage feature flags and control feature rollouts. I highlighted the feature management capabilities in a breakout session the Microsoft Build conference a few weeks ago. If you have not watched the session yet, you can find the recording here

Also, did you know that you can integrate Azure App Configuration Service with Azure Kubernetes Service (AKS) and use it as a source of truth for your app configs?

In this walk-through, I will show you how you can integrate Azure App Configuration Service with AKS and leverage the Azure App Configuration Kubernetes Provider to automatically sync key-value pairs with Kubernetes ConfigMaps and Secrets.

NOTE: The Azure App Configuration Kubernetes Provider is currently in preview

Prerequisites

Before you begin, you need to have the following installed on your machine:

Setup you Azure resources

First, let’s create an Azure App Configuration Service instance. You can create an instance using the Azure CLI or the Azure Portal. I will use the Azure CLI to create an instance. Run the following command to create an instance:

# set up some variables, change these to suit your needs
RG_NAME="appconfig-demo"
LOCATION="eastus"

# create a resource group
az group create \
  --name $RG_NAME \
  --location $LOCATION

# create an app config instance
APP_CONFIG_NAME=$(az appconfig create \
  --name appconfig-demo-$RANDOM \
  --resource-group $RG_NAME \
  --location $LOCATION \
  --query name \
  --output tsv)

Let’s add some key-value pairs to the Azure App Configuration Service instance. Run the following command to add a key-value pair:

az appconfig kv set \
  --name $APP_CONFIG_NAME \
  --key "settings.greeting" \
  --value "Hello" \
  --yes

az appconfig kv set \
  --name $APP_CONFIG_NAME \
  --key "settings.name" \
  --value "Paul" \
  --yes

We also want to demonstrate how to use Azure App Configuration Service to sync Kubernetes Secrets. Let’s create a Azure Key Vault instance, add a secret to it, then add a key vault reference to the Azure App Configuration Service instance. Run the following commands to create a Key Vault instance and add a secret to it:

# create a key vault instance
KEY_VAULT_NAME=$(az keyvault create \
  --name appconfig-demo-$RANDOM \
  --resource-group $RG_NAME \
  --location $LOCATION \
  --query name \
  --output tsv)

# add a secret to the key vault instance and get the secret uri
SECRET_URI=$(az keyvault secret set \
  --name SuperSecretPassword \
  --vault-name $KEY_VAULT_NAME \
  --value "123456" \
  --query id \
  --output tsv)

Now, let’s add a key vault reference to the Azure App Configuration Service instance by using the set-kevault subcommand. Run the following command to add a key vault reference:

az appconfig kv set-keyvault \
  --name $APP_CONFIG_NAME \
  --key "secrets.password" \
  --secret-identifier $SECRET_URI \
  --yes

Finally, we need to create an Azure Kubernetes Service (AKS) instance. Run the following command to create an AKS instance:

AKS_NAME=$(az aks create \
  --name appconfig-demo-$RANDOM \
  --resource-group $RG_NAME \
  --location $LOCATION \
  --node-count 1 \
  --generate-ssh-keys \
  --query name \
  --output tsv)

I Can Has… Azure permissions?

We need to grant the AKS node pool access to the Azure App Configuration Service instance and the Azure Key Vault instance. To do this, we’ll need to create a system assigned managed identity for the AKS node pool. Run the following command to create a system assigned managed identity:

# get the node resource group
NODE_RG_NAME=$(az aks show \
  --name $AKS_NAME \
  --resource-group $RG_NAME \
  --query nodeResourceGroup \
  --output tsv)

# get the node virtual machine scale set name
NODE_VMSS_NAME=$(az resource list \
  --resource-group $NODE_RG_NAME \
  --resource-type Microsoft.Compute/virtualMachineScaleSets \
  --query "[0].name" -o tsv)

# get the app config resource id
APP_CONFIG_ID=$(az appconfig show \
  --name $APP_CONFIG_NAME \
  --query id \
  --output tsv)

# create a system assigned managed identity for the node pool and give it access to read from the app config instance
az vmss identity assign \
  --role "App Configuration Data Reader" \
  --scope $APP_CONFIG_ID \
  --name $NODE_VMSS_NAME \
  --resource-group $NODE_RG_NAME

Now we need to allow the AKS node pool to access the Azure Key Vault instance. Run the following commands to retrieve the managed identity object id for the AKS node pool and give it access to read secrets from Azure Key Vault:

# get the managed identity object id
NODE_VMSS_MANAGED_OBJECT_ID=$(az vmss identity show \
  --name $NODE_VMSS_NAME \
  --resource-group $NODE_RG_NAME \
  --query principalId \
  --output tsv)

# give the managed identity read access to the key vault
az keyvault set-policy \
  --name $KEY_VAULT_NAME \
  --object-id $NODE_VMSS_MANAGED_OBJECT_ID \
  --secret-permissions get list

IMPORTANT! This is a critical step, if this is not done properly, the Azure App Configuration Kubernetes Provider will not be able to authenticate and sync the app configs with Kubernetes ConfigMaps and Secrets.

Install the Azure App Configuration Kubernetes Provider

Now that we have created all the Azure resources, let’s install the Azure App Configuration Kubernetes Provider. The Azure App Configuration Kubernetes Provider is a Helm chart that you can use to install the provider. Run the following command to install the provider:

# connect to the AKS cluster
az aks get-credentials \
  --name $AKS_NAME \
  --resource-group $RG_NAME

# use Helm to install the Azure App Configuration Kubernetes Provider
helm install azureappconfiguration.kubernetesprovider \
  oci://mcr.microsoft.com/azure-app-configuration/helmchart/kubernetes-provider \
  --version 1.0.0-preview \
  --namespace azappconfig-system \
  --create-namespace

The Helm installation will create a Kubernetes namespace called azappconfig-system and install the Azure App Configuration Kubernetes Provider in that namespace. You can verify that the provider is installed by running the following command:

kubectl get pods -n azappconfig-system

Along with the operator pod, you should see a new Custom Resource Definition (CRD) called AzureAppConfigurationProvider. The CRD is used to configure the Azure App Configuration Kubernetes Provider. You can view the CRD by running the following command:

kubectl get crd azureappconfigurationproviders.azconfig.io -o yaml

Configure the Azure App Configuration Kubernetes Provider

Now that we have installed the Azure App Configuration Kubernetes Provider, we need to configure it. The provider needs to know the Azure App Configuration Service instance name and the connection string. The provider will also need the client ID of the managed identity to authenticate against the key vault, so we’ll need to pull these values out and pass them into the CRD deployment. Run the following command to configure the provider:

# get the app config connection string
APP_CONFIG_ENDPOINT=$(az appconfig show \
  -n $APP_CONFIG_NAME \
  --query endpoint \
  --output tsv)

# get the VMSS managed identity's client id
NODE_VMSS_MANAGED_CLIENT_ID=$(az ad sp show \
  --id $NODE_VMSS_MANAGED_OBJECT_ID \
  --query appId \
  --output tsv)

# deploy the AzureAppConfigurationProvider CRD which maps the Azure App Configuration Service instance to desired Kubernetes ConfigMap and Secrets and uses the managed identity to authenticate against the key vault
kubectl apply -f - <<EOF
apiVersion: azconfig.io/v1beta1
kind: AzureAppConfigurationProvider
metadata:
  name: my-appconfig-provider
spec:
  endpoint: $APP_CONFIG_ENDPOINT
  target:
    configMapName: my-configmap
  keyValues: 
    selectors:
      - keyFilter: settings.*
      - keyFilter: secrets.*  
    keyVaults:
      target:
        secretName: my-secrets
      auth:
        managedIdentityClientId: $NODE_VMSS_MANAGED_CLIENT_ID
EOF

The Azure App Configuration Kubernetes Provider reference has more details on how to configure the provider. But a brief overview of the configuration is as follows:

spec.endpoint: The Azure App Configuration Service instance endpoint.
spec.target.configMapName: The Kubernetes ConfigMap name to sync the key-value pairs to.
spec.keyValues.selectors: The key-value pairs to sync to the Kubernetes ConfigMap (with selectors to filter for specific keys).
spec.keyVaults.target.secretName: The Kubernetes Secret name to sync the secrets to.
spec.keyVaults.auth.managedIdentityClientId: The authentication method to use to authenticate against the Azure Key Vault instance.

The provider will now start syncing the Azure App Configuration Service instance with the Kubernetes ConfigMap and Secret. You can verify that the provider is working by running the following commands:

kubectl logs deployment/az-appconfig-k8s-provider -n azappconfig-system

You should see the provider syncing the Azure App Configuration Service instance with the Kubernetes ConfigMap and Secret. You can also verify that the ConfigMap and Secret were created by running the following commands:

kubectl get configmap my-configmap -o jsonpath='{.data}'
kubectl get secret my-secrets -o jsonpath='{.data.secrets\.password}' | base64 --decode

Deploy the sample application

Now that we have the Azure App Configuration Kubernetes Provider configured, let’s deploy a sample application that will use the ConfigMap and Secret. Run the following command to deploy the sample application:

kubectl apply -f - <<EOF
apiVersion: v1
kind: Pod
metadata:
  name: my-app
spec:
  containers:
  - image: busybox
    name: mybusybox
    resources: {}
    args:
    - sleep
    - "3600"
    envFrom:
    - configMapRef:
        name: my-configmap
    env:
    - name: secrets.password
      valueFrom:
        secretKeyRef:
          name: my-secrets
          key: secrets.password
EOF

The sample application will use the ConfigMap and Secret to set environment variables. You can verify that the environment variables were set by running the following command:

kubectl exec my-app -- env

You should see the environment variables set from the ConfigMap and Secret.

Cleanup

When you are done testing, clean up the resources by running the following commands:

az group delete --name $RG_NAME --yes --no-wait

Conclusion

Azure App Configuration Kubernetes Provider offers a seamless solution for syncing key-value pairs and secrets from Azure App Configuration Service and Azure Key Vault with Kubernetes ConfigMaps and Secrets. By leveraging this integration, developers can simplify their configuration management process, eliminate the need for manual YAML editing, and enjoy a more efficient workflow. That is… unless you enjoy writing YAML manifests 😅

Next Steps

As stated at the top of this article, the Azure App Configuration Kubernetes Provider is currently in preview and I’d love to hear your thoughts, use cases, feedback on it, and whether or not this is useful to you. Feel free to reach out to me on Twitter @pauldotyu or on LinkedIn /in/yupaul.

If you want to learn more about the Azure App Configuration Kubernetes Provider, check out the official documentation in addition to the Kubernetes provider reference.

Lastly, if you want to see how I automated the end-to-end deployment of this demo using Terraform, checkout my GitHub repo. I’ll be posting more sample Terraform code in the future, so be sure to give the repo a ⭐️ and/or follow me on GitHub too.

Peace ✌️