Find outdated Azure ARM QuickStart Templates on GitHub

In my previous post Determine latest API version for a resource provider I showed you how to retrieve the latest API version for a specific resource provider using the Get-AzureRmResourceProviderLatestApiVersion cmdlet. In this post I will use the cmdlet to find any outdated resource provider within an ARM template. Also, we will analyze the Azure quickstart templates on GitHub!

To analyze an ARM template we have to convert the JavaScript Object Notation (JSON) to a custom PowerShell object that has a property for each field in the JSON.
We will use the following lean template for testing purpose (download):

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "resources": [
        {
            "type": "Microsoft.Storage/storageAccounts",
            "name": "myuniquestorageaccountname",
            "apiVersion": "2016-01-01",
            "location": "West Europe",
            "sku": {
                "name": "Standard_LRS"
            },
            "kind": "Storage"
        }
    ]
}

The first thing we have to do is to load the file using the Get-Content cmdlet and pipe the result to the ConvertFrom-Json cmdlet. To retrieve the resources we further pipe the result to the Select-Object cmdlet:


Get-Content leanArmTemplate.json |
        ConvertFrom-Json |
        Select-Object -ExpandProperty resources

This will give us all necessary information like the resource provider and the used API version:

type       : Microsoft.Storage/storageAccounts
name       : myuniquestorageaccountname
apiVersion : 2016-01-01
location   : West Europe
sku        : @{name=Standard_LRS}
kind       : Storage

Now finally to analyze the template, we have to iterate over each resource and retrieve the latest API version using the previous mentioned Get-AzureRmResourceProviderLatestApiVersion cmdlet. To get a handy output we create a new PsCustomObject containing the resource type, the used and latest API Version and a flag that specifies whether the used API version is the latest:

Get-Content leanArmTemplate.json |
        ConvertFrom-Json |
        Select-Object -ExpandProperty resources |
         ForEach-Object {
            $latestApiVersion = Get-AzureRmResourceProviderLatestApiVersion -Type $_.type
            [PsCustomObject]@{
                Type = $_.type
                UsedApiVersion = $_.apiVersion
                LatestVersion = $latestApiVersion
                Latest = $_.apiVersion -eq $latestApiVersion
            }
        }

This is the output:

Type                              UsedApiVersion LatestVersion Latest
----                              -------------- ------------- ------
Microsoft.Storage/storageAccounts 2016-01-01     2018-02-01     False

In this example, the used API for the resource provider Microsoft.Storage/storageAccounts is not up to date.

For the sake of convenience I wrapped the code into a Get-OutdatedResourceProvider cmdlet and added some special treatment to handle nested resources:

function Get-OutdatedResourceProvider
{
    [CmdletBinding()]
    [Alias()]
    [OutputType([string])]
    Param
    (
        [Parameter(Mandatory = $true, Position = 0)]
        [string]$Path,

        [switch]$IncludePreview
    )
    Get-Content $Path |
        ConvertFrom-Json |
        Select-Object -ExpandProperty resources |
        ForEach-Object {
        $latestApiVersion = Get-AzureRmResourceProviderLatestApiVersion -Type $_.type
        [PsCustomObject]@{
            Type = $_.type
            UsedApiVersion = $_.apiVersion
            LatestVersion = $latestApiVersion
            Latest = $_.apiVersion -eq $latestApiVersion
        }

        # a resource may include sub resources
        foreach ($subResource in $_.resources)
        {
            $latestApiVersion = Get-AzureRmResourceProviderLatestApiVersion -Type ('{0}/{1}' -f $_.type, $subResource.type)
            [PsCustomObject]@{
                Type = '{0}/{1}' -f $_.type, $subResource.type
                UsedApiVersion = $_.apiVersion
                LatestVersion = $_.apiVersion -eq $latestApiVersion
            }
        }
    }
}

🕵️‍♂️🕵️‍♂️🕵️‍♂️ Now it’s time to analyze some templates  🕵️‍♂️🕵️‍♂️🕵️‍♂️.

Lets clone the whole azure-quickstart-templates Git repository:

git clone https://github.com/Azure/azure-quickstart-templates.git

Then we can filter all ARM templates using the Get-ChildItem cmdlet:

$armTemplates = Get-ChildItem 'azure-quickstart-templates' -Filter 'azuredeploy.json' -Recurse

To analyze them we iterate over each template and pass it to the above created Get-OutdatedResourceProvider cmdlet. Because there might be some invalid templates, we try to parse the JSON first and filter all valid templates:

$invalidTemplates = @()
$analyzedTemplates = $armTemplates |
    ForEach-Object {
    $template = $_
    try
    {
        $null = $template | Get-Content | ConvertFrom-Json
        Get-OutdatedResourceProvider -Path $template.FullName
    }
    catch
    {
        $invalidTemplates += $template
    }
}

Finally we gather and output some metrics:

$validAnalyzes = $analyzedTemplates | Where-Object LatestVersion
$uptodateproviderCount = $validAnalyzes | Where-Object LatestVersion -eq TRUE | Measure-Object | Select-Object -ExpandProperty Count
$outdatedproviderCount = $validAnalyzes | Where-Object LatestVersion -ne TRUE | Measure-Object | Select-Object -ExpandProperty Count

Write-Host "Found $($armTemplates.Count) ARM templates in the Azure quickstart repository ($($invalidTemplates.Count) of them are invalid).
They are using $($analyzedTemplates.Count) resource providers. I was able to determine the latest version for $($validAnalyzes.count) resource provider. $uptodateproviderCount of them are using the latest API version whereas $outdatedproviderCount are using an outdated API." -ForegroundColor Cyan

Here is the output:

Found 675 ARM templates in the Azure quickstart repository (3 of them are invalid).
They are using 3873 resource providers. I was able to determine the latest version for 3288 resource provider. 18 of them are using the latest API version whereas 3270 are using an outdated API.

Note: There is no need to always use the latest API version. You should consider upgrading only if you encounter issues with the resource provider or if you want to use new features that are not part of the currently used API.

You can download the complete script and run it for yourself here.

Configure Azure Cloud Shell to use a profile hosted on GitHub

You may have noticed that you can run the Azure Cloud Shell without the portal as a separate component on https://shell.azure.com/

The shell is really handy since it can be used from everywhere. Today I want to show you how you can load a remote profile that is hosted on GitHub in the Azure Cloud Shell.

A PowerShell profile is used to add aliases, functions or variables to a session every time you start the shell.

The Azure Cloud Shell uses a fileshare stored on your storage account to persist files. This is also true for your profile. You can determine your profile path by entering:

$profile

You will see a path similar to this:

profile

This doesn’t mean that the profile exists, its just the path where Azure Cloud Shell tries to load your profile when you start it. You can determine whether the file actually exists using the Test-Path cmdlet:

Test-Path $profile

The cmdlet should return false if you didn’t already created a profile:

Capture

You could create a profile using the New-Item cmdlet and go to your file share to edit it. But you may like to have a history where you can compare the changes you made. You may also want to use the same profile for different accounts. So how can we connect a profile that is stored in a GitHub repository?

Lets start with adding the actual profile to our GitHub repository. My profile.ps1 contains a single function to print Hello World:

function Show-HelloWorld
{
    Write-Host "hello, world!"
}

Next we have to load the profile. For that purpose I have created another file called Set-Profile.ps1:

$profilePath = 'https://raw.githubusercontent.com/mjisaak/azure/master/profile.ps1'

$downloadString = '{0}?{1}' -f $profilePath, (New-Guid)
Invoke-Expression ((New-Object System.Net.WebClient).DownloadString($profilePath))

The $profilePath contains the URL to the previous created profile.ps1. I do append a query string to the path containg a random guid to prevent the web client from caching the file. This is particularly usefull when we update the profile.ps1 in the GitHub repository and want to load these changes without restarting the shell by dot sourcing the profile.
In line 4,  I download the profile.ps1 as a string and execute it using the Invoke-Expression cmdlet to load it to the runspace.

The last step we need to do is to set the content of the Set-Profile.ps1 to the actual PowerShell profile. We can do this by executing the following snippet in the Azure Cloud Shell:

(New-Object System.Net.WebClient).DownloadString('https://raw.githubusercontent.com/mjisaak/azure/master/Set-Profile.ps1') |
  Set-Content $profile -Force

The snippet is using the web client again but instead of executing the code,  it pipes the string to the Set-Content cmdlet to override the profile. I can verfiy that by retrieving the content of $profile. This should output the content of my Set-Profile.ps1:

Capture4

Finally to load the profile we can either restart the PowerShell session or dot source the profile as mentioned earlier:

. $profile

And now we can use all the aliases, variables and function we have defined in the profile that is stored on GitHub:

helloworld