wiki.getshifting.com

--- Sjoerd Hooft's InFormation Technology ---

User Tools

Site Tools


k8s-aksworkloadidentity

AKS with Workload Identity

Summary: Workloads deployed on an Azure Kubernetes Services (AKS) cluster require Microsoft Entra application credentials or managed identities to access Microsoft Entra protected resources, such as a Azure Key Vault. Microsoft Entra Workload ID integrates with the capabilities native to Kubernetes to federate with external identity providers. On this page I'll show you how to quickly deploy an AKS cluster with workload identity enabled.
Date: 26 January 2025

I'll show you the following steps:

  • Deploy an AKS cluster using the Azure CLI with the OpenID Connect issuer and a Microsoft Entra Workload ID.
  • Create a Microsoft Entra Workload ID and Kubernetes service account.
  • Configure the managed identity for token federation.
  • Deploy a test workload and verify authentication with the workload identity.
  • Grant a pod in the cluster access to secrets in an Azure key vault.

All command shown here are done with powershell in the Azure Cloud Shell.

Deploy an AKS cluster

First we'll define the variables we'll be using in the deployment:

$project = "akswi"
$rg = "rg-$project"
$loc = "westeurope"
$rgnodes = "rg-$project-nodes"
$aks = "aks-$project"
$id = "id-$project"
$aksSaName = "sa-$project"
$aksSaNamespace = "default"
$fc = "fc-$project"
$kv = "kv-$project"
$kvSecret = "secret-$project"

Then we'll create a resource group:

az group create `
  --name $rg `
  --location $loc

Next we'll create the AKS cluster with the workload identity enabled:

az aks create `
  --resource-group $rg `
  --name $aks `
  --node-count 1 `
  --node-resource-group $rgnodes `
  --enable-oidc-issuer `
  --enable-workload-identity `
  --generate-ssh-keys
 
# Get the oidc issuer url
$oidcIssuer = az aks show `
  --resource-group $rg `
  --name $aks `
  --query "oidcIssuerProfile.issuerUrl" `
  --output tsv
 
# Get cluster credentials
az aks get-credentials --resource-group rg-akswi --name aks-akswi --overwrite-existing
# Verify kubectl is working, the following command should return the default deployments in the cluster
kubectl get deployments --all-namespaces=true
By default, the issuer is set to use the base URL https://{region}.oic.prod-aks.azure.com/{tenant_id}/{uuid}, where the value for {region} matches the location to which the AKS cluster is deployed. The value {uuid} represents the OIDC key, which is a randomly generated guid for each cluster that is immutable.

Create a Microsoft Entra Workload ID and Kubernetes service account

First we'll create the Microsoft Entra Workload managed identity:

az identity create `
  --name $id `
  --resource-group $rg `
  --location $loc
 
# Get the managed identity client id (Client Id (Application Id) is used to authenticate workloads)
$idId = az identity show `
  --name $id `
  --resource-group $rg `
  --query clientId `
  --output tsv
 
# Get the managed identity object id (Object (Principal) Id is used to authorize workloads, aka, assign permissions in Azure)
$idPrincipalId = az identity show `
  --name $id `
  --resource-group $rg `
  --query principalId `
  --output tsv

Now we'll create a Kubernetes service account:

$saManifest = @"
apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    azure.workload.identity/client-id: $idId
  name: $aksSaName
  namespace: $aksSaNamespace
"@
$saManifest | Out-File -FilePath sa.yaml

We should now have a file called sa.yaml with the following content:

apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    azure.workload.identity/client-id: 154d6bab-f55b-439e-a331-230a318c819b
  name: sa-akswi
  namespace: default

We can now deploy the service account manifest to the cluster:

kubectl apply -f sa.yaml
# Verify the service account is created
kubectl get serviceaccounts -n default
NAME       SECRETS   AGE
default    0         29m
sa-akswi   0         34s

Configure the managed identity for token federation

Now we will create the federated identity credential between the managed identity, the service account issuer, and the subject:

az identity federated-credential create `
    --name $fc `
    --identity-name $id `
    --resource-group $rg `
    --issuer $oidcIssuer `
    --subject system:serviceaccount:$($aksSaNamespace):$($aksSaName) `
    --audience api://AzureADTokenExchange
Note there might be a delay in the federation process. If you get an error, wait a few minutes and try again.

In the azure portal you can verify the federated credential is created by going to the managed identity and selecting the federated credentials tab:

The federated credential for the managed identity

Deploy a test workload and verify authentication with the workload identity

$podManifest = @"
apiVersion: v1
kind: Pod
metadata:
  name: test-workload-identity
  namespace: $aksSaNamespace
  labels:
    azure.workload.identity/use: "true"
spec:
  serviceAccountName: $aksSaName
  containers:
    - name: webserver
      image: nginx:latest
"@
$podManifest | Out-File -FilePath pod.yaml
Ensure that the application pods using workload identity include the label azure.workload.identity/use: "true" in the pod spec.

We can now deploy the pod to the cluster:

kubectl apply -f pod.yaml
# Verify the pod is running
kubectl get pods
NAME                     READY   STATUS    RESTARTS   AGE
test-workload-identity   1/1     Running   0          12s
# log into the pod and verify the workload identity variables are loaded
kubectl exec -it test-workload-identity -- /bin/bash
# Note that the prompt changes to root@test-workload-identity:/#
root@test-workload-identity:/# env | grep -i azure
AZURE_TENANT_ID=4b9beca9-93dd-4927-ae18-6590a3480016
AZURE_FEDERATED_TOKEN_FILE=/var/run/secrets/azure/tokens/azure-identity-token
AZURE_AUTHORITY_HOST=https://login.microsoftonline.com/
AZURE_CLIENT_ID=154d6bab-f55b-439e-a331-230a318c819b

Grant a pod in the cluster access to secrets in an Azure key vault

To really test the workload identity, we'll grant the pod access to a secret in an Azure Key Vault. For that prupose we'll use a pod from the github.com/azure/azure-workload-identity project that outputs the secret to the log. Before we can grant that pod access to the key vault, we need to create the key vault and set permissions, as well as create a secret in the key vault:

az keyvault create `
  --name $kv `
  --resource-group $rg `
  --location $loc `
  --enable-purge-protection `
  --enable-rbac-authorization `
  --retention-days 7
 
# Get the keyvault id
$kvId = az keyvault show `
  --resource-group $rg `
  --name $kv `
  --query id `
  --output tsv
 
# Get the keyvault url
$kvUrl = az keyvault show `
  --resource-group $rg `
  --name $kv `
  --query properties.vaultUri `
  --output tsv
 
# Get my user id to set permissions
$userId = az account show `
  --query user.name `
  --output tsv
 
# Set permissions for the user
az role assignment create `
  --assignee $userId `
  --role "Key Vault Secrets Officer" `
  --scope $kvId
 
# Create a secret in the key vault
az keyvault secret set `
  --vault-name $kv `
  --name $kvSecret `
  --value "Hello123!"
 
# Set permissions for the workload identity
az role assignment create `
  --assignee-object-id $idPrincipalId `
  --role "Key Vault Secrets User" `
  --scope $kvId `
  --assignee-principal-type ServicePrincipal
Note that enabling purge protection is required for workload identity to work with Azure Key Vault. Because this is a test environment the retention days are set to 7.

Now we can deploy a pod that will access the secret in the key vault. The following deployment uses an image from the Github azure workload identity project, that outputs the secret to the log:

$podKvManifest = @"
apiVersion: v1
kind: Pod
metadata:
  name: sample-workload-identity-key-vault
  namespace: $aksSaNamespace
  labels:
    azure.workload.identity/use: "true"
spec:
  serviceAccountName: $aksSaName
  containers:
    - image: ghcr.io/azure/azure-workload-identity/msal-go
      name: oidc
      env:
      - name: KEYVAULT_URL
        value: $kvUrl
      - name: SECRET_NAME
        value: $kvSecret
  nodeSelector:
    kubernetes.io/os: linux
"@
$podKvManifest | Out-File -FilePath podkv.yaml
kubectl apply -f ./podkv.yaml
pod/sample-workload-identity-key-vault created
# Verify the pod is running
kubectl get pods
NAME                                 READY   STATUS    RESTARTS   AGE
sample-workload-identity-key-vault   1/1     Running   0          16s
test-workload-identity               1/1     Running   0          47m
# Verify the secret name is set correctly and can be accessed
kubectl describe pod sample-workload-identity-key-vault | grep "SECRET_NAME:"
      SECRET_NAME:                 secret-akswi
 
kubectl logs sample-workload-identity-key-vault
I0126 14:45:46.278443       1 main.go:63] "successfully got secret" secret="Hello123\!"

Useful Links

k8s-aksworkloadidentity.txt · Last modified: by 127.0.0.1