Azure DevOps CI/CD Pipeline to build and push Docker images to Azure Container Registry

Posted on by Martin Brandl

With Azure DevOps Pipelines, it is incredibly simple to build Docker images and have them automatically pushed into an Azure Container Registry. To create and push the images, you can use the pre-existing Docker@2 Azure Pipeline Task with an Azure Container Registry service connection.

However, there are environments where you cannot create an Azure Container Registry service connection because you do not have the required permissions. This is very common in enterprise projects where you often only have an existing Azure Resource Manager service connection in Azure DevOps available.

In this article, I’ll show you how to create a multi-stage Azure DevOps CI/CD pipeline using YAML to build and push Docker images to an Azure Container Registry using a simple Azure Resource Manager connection.

Azure DevOps

Prerequisites

Setup the environment (optional)

In our demo project, we will deploy a Docker image to a DEV and a QA environment. In Azure, we will create two Azure Resource Groups for this purpose. We will also create one Azure Container Registry per environment. This is how our setup will look like:

To create the environment, we will use the Azure CLI. In the following PowerShell script, we define the names for the Azure Resource Group and the Azure Container Registries. Then we create the resource groups with the az group create command and the container registry with az acr create:

# define names
$rgDevName = 'rg-ado-acr-sample-dev'
$acrNameDev = "acradosampledev"
$rgQaName = 'rg-ado-acr-sample-qa'
$acrNameQa = "acradosampleqa"
$location = 'germanywestcentral'

# create resource groups
$rgDev = (az group create --name $rgDevName --location $location --query 'id')
$rgQa = (az group create --name $rgQaName --location $location --query 'id')

# create ACRs
az acr create --name $acrNameDev --sku Basic --resource-group $rgDevName --location $location
az acr create --name $acrNameQa --sku Basic --resource-group $rgQaName --location $location

Once the script has been run, we have all the resources we need to create our CI/CD pipeline in Azure DevOps.

Implement the CI / CD pipeline

In this section I will show you how to create the CI/CD Azure DevOps pipeline to build a Docker container and push it to an Azure Container Registry. I will not go into the creation of the Azure Resource Manager connection and the creation of the pipeline itself. You can find a step-by-step guide from Microsoft on how to do that here:

Pipeline stages

Our pipeline consists of three stages. In the first stage, we create our Docker container and push it to the Azure Container Registry. We call this our “Build” stage.

In the next stage, we can deploy the container image to our favorite target platform like an Azure Container App. This is the “Deploy to Dev” stage.

In a software development project, there are usually other environments besides the development environment. In our demo, we also want to deploy our container to a quality assurance (QA) environment within the “Deploy to Qa” stage

Stage 1: Build

In the first stage, we authenticate to the Azure Container Registry, create our Docker container, and push the container image to an Azure Container Registry.

❌ How we cannot do it:

Usually we would use the @Docker2 Azure DevOps Task for this and specify a connection to a container registry. This would look something like this

- task: Docker@2
  displayName: Build and Push
  inputs:
    command: buildAndPush
    containerRegistry: dockerRegistryServiceConnection1
    repository: contosoRepository
    tags: |
      tag1
      tag2

In this article, however, we want to use an Azure Resource Manager Service Connection instead of a Container Registry Connection for authentication.

✅ How it works:

In the build stage, we use the AzureCLI@2 task twice and specify the corresponding Azure Service Connection. This allows us to execute Azure CLI commands in the context of our subscription.

In the first step, we use the az acr login command to log in to our development Container Registry. We also temporarily store the service principal id and key of the Azure endpoint in a variable so that we can reconnect to our container registry at a later stage. In the second step we build the Docker image and push it into the registry we logged into in the previous step:

stages:
- stage: Build
  pool:
    vmImage: ubuntu-20.04

  jobs:
  - job: build
    displayName: Build
    steps:
    - task: AzureCLI@2
      displayName: Login to DEV ACR
      name: setvar
      inputs:
        azureSubscription: dev-germanywestcentral-sc
        scriptType: pscore
        addSpnToEnvironment: true
        scriptLocation: inlineScript
        inlineScript: |
          az acr login --name $(containerRegistryNameDev)

          # store service principal id and key for later use
          echo "##vso[task.setvariable variable=SpId;isoutput=true; issecret=true]$env:servicePrincipalId"
          echo "##vso[task.setvariable variable=SpKey;isoutput=true; issecret=true]$env:servicePrincipalKey"

    - task: AzureCLI@2
      displayName: Build and Push to DEV ACR
      inputs:
        azureSubscription: dev-germanywestcentral-sc
        scriptType: pscore
        scriptLocation: inlineScript
        inlineScript: |
          docker build -t $(imageNameDev) .
          docker push $(imageNameDev)

Note that the script uses variables that I have configured at the top of the pipeline:

variables: 
  serviceConnectionDev: 'dev-germanywestcentral-sc'
  serviceConnectionQa: 'qa-germanywestcentral-sc'
  containerRegistryNameDev: 'acradosampledev'
  containerRegistryNameQa: 'acradosampleqa'
  imageBaseName: hello-world
  imageNameDev: "$(containerRegistryNameDev).azurecr.io/$(imageBaseName):$(Build.BuildId)"
  imageNameQa: "$(containerRegistryNameQa).azurecr.io/$(imageBaseName):$(Build.BuildId)"

Stage 2: Deploy to Dev

In the Deploy to Dev stage, we want to deploy our application to a service that can run a container-based workload using the image inside the (dev) container registry. We have already uploaded the image to our registry in the build stage and can use it for the deployment.

Here is the outline for the deployment stage. Of course, the implementation depends on the chosen target infrastructure:

- stage: DeployToDev
  dependsOn: Build
  condition: succeeded()
  displayName: Deploy to Dev
  pool:
    vmImage: ubuntu-20.04

  jobs:
  - job: deploy

    steps:
    - pwsh: Write-Host "Here you can deploy the container to your favorite target. For example, AKS, ACI, ACA or Azure Web App for Containers."
      displayName: Deploy container to target platform

Stage 3: Deploy to Qa

In order to deploy our application in this stage, we first need to get our container image into the qa container registry. What we don’t want to do is run the Docker build again since we have already built and potentially tested the container in the dev environment. Instead, we will copy the image from the dev container registry to the qa registry.

For that, we need to import the outputs (service principal credentials) from the build stage and expose them via the variable SpId and SpKey. After that, we authenticate against our qa registry with the previously used az acr login command. Then we can finally import the image using the az acr import command. Here we use the variables SpId and SpKey to authenticate to the source (dev) Container Registry:

- stage: DeployToQa
  dependsOn: 
    - Build
    - DeployToDev
  condition: succeeded()
  displayName: Deploy to Qa
  pool:
    vmImage: ubuntu-20.04

  jobs:
  - job: build
    displayName: Build
    variables:
      - name: SpId
        value: $[ stageDependencies.Build.build.outputs['setvar.SpId'] ]
      - name: SpKey
        value: $[ stageDependencies.Build.build.outputs['setvar.SpKey'] ]

    steps:
    - task: AzureCLI@2
      displayName: Import Image
      inputs:
        azureSubscription: qa-germanywestcentral-sc
        scriptType: pscore
        scriptLocation: inlineScript         
        inlineScript: |
          $imageName = "$(containerRegistryNameDev).azurecr.io/$(imageBaseName):$(Build.BuildId)"
          
          az acr login --name $(containerRegistryNameQa)

          az acr import `
          --name $(containerRegistryNameQa) `
          --username "$(SpId)" `
          --password "$(SpKey)" `
          --source $imageName

Summary

In a CI /CD pipeline with container, you want to make sure that the container is built only once but still is available in multiple registries (e.g. dev and qa). Here is the complete pipeline to achieve that. You can also find the pipeline and the scaffolding script on GitHub:

variables: 
  serviceConnectionDev: 'dev-germanywestcentral-sc'
  serviceConnectionQa: 'qa-germanywestcentral-sc'
  containerRegistryNameDev: 'acradosampledev'
  containerRegistryNameQa: 'acradosampleqa'
  imageBaseName: hello-world
  imageNameDev: "$(containerRegistryNameDev).azurecr.io/$(imageBaseName):$(Build.BuildId)"
  imageNameQa: "$(containerRegistryNameQa).azurecr.io/$(imageBaseName):$(Build.BuildId)"

trigger:
  branches:
    include:
    - main

stages:
- stage: Build
  pool:
    vmImage: ubuntu-20.04

  jobs:
  - job: build
    displayName: Build
    steps:
    - task: AzureCLI@2
      displayName: Login to DEV ACR
      name: setvar
      inputs:
        azureSubscription: dev-germanywestcentral-sc
        scriptType: pscore
        addSpnToEnvironment: true
        scriptLocation: inlineScript
        inlineScript: |
          az acr login --name $(containerRegistryNameDev)

          # store service principal id and key for later use
          echo "##vso[task.setvariable variable=SpId;isoutput=true; issecret=true]$env:servicePrincipalId"
          echo "##vso[task.setvariable variable=SpKey;isoutput=true; issecret=true]$env:servicePrincipalKey"

    - task: AzureCLI@2
      displayName: Build and Push to DEV ACR
      inputs:
        azureSubscription: dev-germanywestcentral-sc
        scriptType: pscore
        scriptLocation: inlineScript
        inlineScript: |
          docker build -t $(imageNameDev) .
          docker push $(imageNameDev)

- stage: DeployToDev
  dependsOn: Build
  condition: succeeded()
  displayName: Deploy to Dev
  pool:
    vmImage: ubuntu-20.04

  jobs:
  - job: deploy

    steps:
    - pwsh: Write-Host "Here you can deploy the container to your favorite target. For example, AKS, ACI, ACA or Azure Web App for Containers."
      displayName: Deploy container to target platform

- stage: DeployToQa
  dependsOn: 
    - Build
    - DeployToDev
  condition: succeeded()
  displayName: Deploy to Qa
  pool:
    vmImage: ubuntu-20.04

  jobs:
  - job: build
    displayName: Build
    variables:
      - name: SpId
        value: $[ stageDependencies.Build.build.outputs['setvar.SpId'] ]
      - name: SpKey
        value: $[ stageDependencies.Build.build.outputs['setvar.SpKey'] ]

    steps:
    - task: AzureCLI@2
      displayName: Import Image
      inputs:
        azureSubscription: qa-germanywestcentral-sc
        scriptType: pscore
        scriptLocation: inlineScript         
        inlineScript: |
          $imageName = "$(containerRegistryNameDev).azurecr.io/$(imageBaseName):$(Build.BuildId)"
          
          az acr login --name $(containerRegistryNameQa)

          az acr import `
          --name $(containerRegistryNameQa) `
          --username "$(SpId)" `
          --password "$(SpKey)" `
          --source $imageName

Published by Martin Brandl