Back in the day at one of my previous client engagements, I had a chance to have a quick chat about this:
Q: Can we sort an array within a Logic App workflow? A: As the Logic App is a workflow engine, only basic levels of data handling features are provided out-of-the-box. Therefore, that sorting should be done by an external app like an Azure Functions app. Q: Well, I'd like to pick up the latest one from the stored back-up files. How can I do this, with no code or less code? A: Good question! As our log files are stored at Azure Blob Storage and their filenames represent timestamp, we might be able to leverage the filenames.
As mentioned in the conversation above, Azure Logic App uses Workflow Definition Language (WDL) and provides full function references. However, there is no array sort related function, unfortunately. The user requirement seems pretty challenging with the limited native feature. You know, however, there's always a way as we always have. We can achieve the requirement within the Logic App with no code involved. In this post, I'm going to show how we can fetch the latest back-up file from Azure Blob Storage.
Assumptions
First of all, there are a couple of assumptions:
- Azure Blob Storage stores all the back-up files.
- The back-up files are stored daily and their filename has the format of
yyyyMMdd.json. For example, one of the files should be20191104.json.
Based on these assumptions, let's create actions within a Logic App.
List of Back-up Files
We need to fetch all the files from the Blob Storage. If we use the blob connector, it's pretty easy. Here is the screenshot and JSON definition.
| { | |
| ... | |
| "List_Backups": { | |
| "type": "ApiConnection", | |
| "runAfter": {}, | |
| "inputs": { | |
| "method": "get", | |
| "host": { | |
| "connection": { | |
| "name": "@parameters('$connections')['azureblob']['connectionId']" | |
| } | |
| }, | |
| "path": "/datasets/default/foldersV2/@{encodeURIComponent(encodeURIComponent('/backup'))}", | |
| "queries": { | |
| "nextPageMarker": "", | |
| "useFlatListing": false | |
| } | |
| } | |
| }, | |
| ... | |
| } |
Once this is done, the run result might look like this:
Conversion of Filenames to Integer Array
Like the result screen above, and based on our assumption, all the file names are in yyyyMMdd.json format. Therefore, let's take off the .json part, convert them into integer and put them into an array. The Select action can take care of those jobs. Here is the UI screen and JSON representation of the action.
| { | |
| ... | |
| "Select_Filenames_from_Backups": { | |
| "type": "Select", | |
| "runAfter": { | |
| "List_Backups": [ | |
| "Succeeded" | |
| ] | |
| }, | |
| "inputs": { | |
| "from": "@body('List_Backups')?['value']", | |
| "select": "@int(first(split(item()?['Name'], '.')))" | |
| } | |
| }, | |
| ... | |
| } |
After rerunning the workflow, the conversion result might look like:
Selection of the Latest Back-up File
Now, we've got all the file names converted into integer and put into the array. The Filter takes the latest file by comparing each list item to the maximum value of the array item. Here is the UI and JSON representation of the action.
| { | |
| ... | |
| "Take_Latest_Backup": { | |
| "type": "Query", | |
| "runAfter": { | |
| "Select_Filenames_from_Backups": [ | |
| "Succeeded" | |
| ] | |
| }, | |
| "inputs": { | |
| "from": "@body('List_Backups')?['value']", | |
| "where": "@equals(int(first(split(item()?['Name'], '.'))), max(body('Select_Filenames_from_Backups')))" | |
| } | |
| }, | |
| ... | |
| } |
This is the result of the filtering.
We've got the latest back-up file. The next action will be another Blob Storage connection to fetch the content of the file.
So far, instead of performing an array sort, we've used two actions to get the maximum value of an array. As the max() function used in the Filter action only applies to numbers, we converted the file name into numbers, which was not exactly the sort thing. Nonetheless, we got the result we required, which we can say as a workaround. Using min() or max() function might be expanding use cases to different ways.
In my next post, I'll show how to deal with the real-world use case using this approach.