ARM template is a great tool for Azure resources deployment. However, it's very tricky to use because:
- It's a JSON object with massive number of lines,
- Its JSON structure is quite complex so that it's not that easy to read at a glance,
- It's hard to validate if there is a typo or not, and
- We only know if the resource deployment is successful at runtime.
What if we can validate and/or test the ARM template without actually deploying it? Fortunately, Azure PowerShell provides a cmdlet, Test-AzureRmResourceGroupDeployment for testing purpose. It allows us to verify the ARM template without actually deploying resources. In this post, we're going to walk through how we can use the Test-AzureRmResourceGroupDeployment cmdlet with Pester, the PowerShell testing and mocking framework.
Using Test-AzureRmResourceGroupDeployment
Let's have a look at the ARM template below. It's a simple ARM template for Azure Logic Apps deployment.
| { | |
| "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#", | |
| "contentVersion": "1.0.0.0", | |
| "parameters": { | |
| "logicAppName1": { | |
| "type": "string", | |
| "metadata": { | |
| "description": "First part of the Logic App." | |
| } | |
| }, | |
| "logicAppName2": { | |
| "type": "string", | |
| "metadata": { | |
| "description": "Second part of the Logic App." | |
| } | |
| } | |
| }, | |
| "variables": { | |
| "logicApp": { | |
| "name": "[concat(parameters('logicAppName1'), '-', parameters('logicAppName2'))]" | |
| } | |
| }, | |
| "resources": [ | |
| { | |
| "name": "[variables('logicApp').name]", | |
| "type": "Microsoft.Logic/workflows", | |
| "location": "[resourceGroup().location]", | |
| "tags": { | |
| "displayName": "LogicApp" | |
| }, | |
| "apiVersion": "2016-06-01", | |
| "properties": { | |
| "definition": { | |
| "$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#", | |
| "contentVersion": "1.0.0.0", | |
| "actions": {}, | |
| "outputs": {}, | |
| "parameters": {}, | |
| "triggers": {} | |
| }, | |
| "parameters": {} | |
| } | |
| } | |
| ], | |
| "outputs": { | |
| "resourceId": { | |
| "type": "string", | |
| "value": "[resourceId('Microsoft.Logic/workflows', variables('logicApp').name)]" | |
| } | |
| } | |
| } |
This ARM template creates an empty Logic App instance. Deploying this with the New-AzureRmResourceGroupDeployment cmdlet is usually the way to validate/verify this ARM template. Using Test-AzureRmResourceGroupDeployment only returns messages when it has an error. Let's run the cmdlet:
| Test-AzureRmResourceGroupDeployment ` | |
| -ResourceGroupName my-resource-group ` | |
| -TemplateFile .\LogicApp.json ` | |
| -logicAppName1 my-new ` | |
| -logicAppName2 logic-app ` | |
| -ErrorAction Stop |
As the ARM template above is valid and test is successful, which returns NOTHING.
Problems
From this point, we only know that the ARM template deployment ITSELF will be successful. But, let's think about this. Even if the ARM template has been deployed successfully, we still can't guarantee that the deployed Azure resources are correctly configured or not. For example, with the ARM template above, the resource name is a concatenation of both parameters, logicAppName1 and logicAppName2. How can we ensure this concatenation has been successful? If there are more parameters involved in those naming and other configurations, our lives would be more complicated. The Test-AzureRmResourceGroupDeployment doesn't give us any indication to sort out this situation. In many cases, composing ARM templates need many template functions like concat(), parameters(), variables(), resourceId() and so forth. What makes the composition worse is those template functions are nested, which possibly results in missing some of opening or closing parentheses, and single quotation marks at some stage.
Therefore, merely running Test-AzureRmResourceGroupDeployment won't help much for testing.
Capturing Debug Messages
There's still a hope. If we use the debugging mode while running the cmdlet and capture those debugging messages, we might be able to get some clues. Let's change the PowerShell command:
| $DebugPreference = "Continue" | |
| Test-AzureRmResourceGroupDeployment ` | |
| -ResourceGroupName my-resource-group ` | |
| -TemplateFile .\LogicApp.json ` | |
| -logicAppName1 my-new ` | |
| -logicAppName2 logic-app ` | |
| -ErrorAction Stop | |
| $DebugPreference = "SilentlyContinue" |
The first command is to change the debugging mode. Its default value is SilentlyContinue, which suppresses all the debugging messages from the screen. Therefore, we need to expose all the debugging messages while the cmdlet is running, by changing its value to Continue. Let's run this and we will be able to see massive lines of debugging messages. If we want to store those debugging messages to a variable, append the output redirection operator, 5>&1 to the end like:
| $DebugPreference = "Continue" | |
| $output = Test-AzureRmResourceGroupDeployment ` | |
| -ResourceGroupName my-resource-group ` | |
| -TemplateFile .\LogicApp.json ` | |
| -logicAppName1 my-new ` | |
| -logicAppName2 logic-app ` | |
| -ErrorAction Stop ` | |
| 5>&1 | |
| $DebugPreference = "SilentlyContinue" |
Now, the $output has all lines of debugging messages. Print out each line, $output[INDEX], and we'll be able to find out the 33rd message like:
It starts with HTTP RESPONSE and its response body is actually a JSON string. Therefore, let's pickup the JSON string as an object.
| ($output[32] -split "Body:")[1] | ConvertFrom-Json |
OK. Now we have a proper output message. How can we use this for testing? Let's move on.
Testing with Pester
Pester is a testing and mocking framework for PowerShell. As it contains a BDD style test runner, it's easy to use.
If we are using Windows 10, it's already installed out-of-the-box. But, according to the official document, it's strongly recommended to update it before use.
Let's write a script to test ARM template deployment.
| Param( | |
| [string] [Parameter(Mandatory=$true)] $ResourceGroupName, | |
| [string] [Parameter(Mandatory=$true)] $TemplateFile, | |
| [hashtable] [Parameter(Mandatory=$true)] $Parameters | |
| ) | |
| Describe "Logic App Deployment Tests" { | |
| BeforeAll { | |
| $DebugPreference = "Continue" | |
| } | |
| AfterAll { | |
| $DebugPreference = "SilentlyContinue" | |
| } | |
| Context "When Logic App deployed without parameters" { | |
| try { | |
| $output = Test-AzureRmResourceGroupDeployment ` | |
| -ResourceGroupName $ResourceGroupName ` | |
| -TemplateFile $TemplateFile ` | |
| -logicAppName1 $null ` | |
| -logicAppName2 $null ` | |
| -ErrorAction Stop ` | |
| 5>&1 | |
| } | |
| catch { | |
| $ex = $_.Exception | Format-List -Force | |
| } | |
| It "Should throw exception" { | |
| $ex | Should -Not -Be $null | |
| $ex.Message | Should -Not -Be ([string]::Empty) | |
| } | |
| } | |
| Context "When Logic App deployed with parameters" { | |
| $output = Test-AzureRmResourceGroupDeployment ` | |
| -ResourceGroupName $ResourceGroupName ` | |
| -TemplateFile $TemplateFile ` | |
| -TemplateParameterObject $Parameters ` | |
| -ErrorAction Stop ` | |
| 5>&1 | |
| $result = (($output[32] -split "Body:")[1] | ConvertFrom-Json).properties | |
| It "Should be deployed successfully" { | |
| $result.provisioningState | Should -Be "Succeeded" | |
| } | |
| It "Should have name of" { | |
| $expected = $Parameters.LogicAppName1 + "-" + $Parameters.LogicAppName2 | |
| $resource = $result.validatedResources[0] | |
| $resource.name | Should -Be $expected | |
| } | |
| } | |
| } |
The first Context tests whether the cmdlet throws an exception or not, when parameters are not passed. The next one tests whether the Logic App instance name is correctly set or not. With Pester, we can run the test like:
| $params = @{ LogicAppName1 = "my-new"; LogicAppName2 = "logic-app" } | |
| $parameters = @{ ResourceGroupName = "my-resource-group"; TemplateFile = ".\LogicApp.json"; Parameters = $params } | |
| $script = @{ Path = ".\LogicApp.Tests.ps1"; Parameters = $parameters } | |
| Invoke-Pester -Script $script |
Now, we can confirm that all tests have been passed.
Integrating CI/CD Pipelines
If we're using VSTS, it's pretty straightforward – use Azure PowerShell Task. It would be even easier with the Pester extension. If we're not using VSTS, it's doubtful that it has such feature. In this case, we must login to Azure Resource Manager first, using a service principal. Then add the PowerShell script stated right above. Now, our CI/CD pipeline picks up the change and triggers the Azure PowerShell task to run this test. Too easy!
So far, we have walked through how to test ARM template deployment without actually deploying it, using Pester, which is an awesome tool for our PowerShell scripting. With this, we can test how ARM template deployment works, and what sort of results we can expect after the run. Therefore, we might be able to build more robust ARM templates.
ACKNOWLEDGEMENT: This post has originally been posted at Mexia blog.