Blog post header photo by Tom Wilson on Unsplash

Note: This blog post is a two part series and focuses on event-driven approach in Microsoft Azure

Introduction

If you are running applications in the cloud, you must have been using a variety of systems and services. Most of those systems are well interconnected to handle tasks/events initiated by the user. However, most of those tasks will most likely turn out to be boring and repetitive at a cretin point, and you will look at how to automate those workflows.

Automation can be easily achieved when the event is initiated by the user/admin in the real-time. But what if the tasks has to be run on different times. Or if the task is executed when another event occurs. A new automation approach has to be introduced here, an approach that is serverless in nature and only runs when an event happens. Enter event-driven approach.

As Microsoft describes it:

“An event-driven architecture uses events to communicate occurrences in system state changes, for example, to other applications or services. You can use filters to route specific events to different endpoints, multicast to multiple endpoints, and make sure your events are reliably delivered.”

This blog post series will focus on Microsoft Azure and their methods of taking the event-driven approach using Azure Event Grid and Azure Functions.

Solution Overview

Azure Event Grid was introduced back in 2017, It was complimentary to many modern application use cases that are built based on event-driven approach.

Simply put, Azure Event Grid is an event handler service that allows you to react to selected/filtered events within Azure services. It has 24-hour retry policy with exponential backoff to make sure events are delivered, and it is also supporting one-to-may fan out topology where many endpoints can be subscribed to the same event. It costs only $.60 per 1 million operation.

Azure Function on the other hand is a “serverless” compute instance on-demand that you can trigger by events occur in Azure service or externally on premises or third party service.

There are many examples out there on how you can combine the two services, in this blog we will work on simple task as shown below:

1. Azure VM is being created in a specific Resource Group
2. The VM creation event will be captured by Azure Event Grid.
3. Event Grid will trigger an Azure Function (1) written in PowerShell (You can choose your scripting language of choice).
4. The script will then modify the VM. In this example since I care about following Pure Cloud Block Store best practices. The script will configure the machine with iSCSI and MultiPath configuration.
5. Another Function (2) will be triggered by the completion of Function (1) and it will pass all the VM information.
6. Finally Function (2) will provision Cloud Block Store volume via in-guest iSCSI to the created Azure VM.

The blog post will be using PowerShell for both automating the setup and scripting the code, so get your terminal ready :)

Step 1: Prerequisite

1. Install Az module and Connect to Azure, for more information to Microsoft docs here.

Connect-AzAccount

2. Create a Resource Group, Ideally this you will separate once for where VMs reside and the rest of Azure App function resources will be created, one is good for testing easier to clean-up later.

$location = "eastus"
$resourceGroup = "Azure-events-blog"
New-AzResourceGroup -Name $resourceGroup -Location $location

3. Create Azure Storage Account

$storageName = "azureeventblogblob"
New-AzStorageAccount -ResourceGroupName $resourceGroup -Name $storageName -SkuName Standard_LRS -Location $location 

4. Check if Azure Event Grid resource is enable in your Subscription

if ((Get-AzResourceProvider -ProviderNamespace Microsoft.Eventgrid).RegistrationState -eq "NotRegistered"){
    Register-AzResourceProvider –ProviderNamespace Microsoft.Eventgrid
}

Step 2: Create Azure Function

Let’s first create function app, and configure its triggers and output.

$functionName = "function-app-blog"
New-AzFunctionApp -Name $functionName -ResourceGroupName $resourceGroup -StorageAccount $storageName -Runtime PowerShell -FunctionsVersion 4 -Location $location 

In order to create a function, I choose to work with VS Code in order to write the code locally then deploy it to the created app. There is a couple of requirement you would need to follow to prep your local development machine, check it out here.

Alternatively, you can just jump to Azure portal, then select Function App from the recourse group.

From the left-side panel, select Function then +Create.

You can now go ahead and click on it once its created, then select Integration. Since we choose the Http Trigger template, Let’s modify our trigger by deleting http and add Event Grid Trigger instead. I choose the parameter as EventData.

Good, so now we move to Code. I just want to log out the event when it occurs, so select the run.ps1, lets delete the http template function and copy/paste the below in order to capture an event. Then click save

param($EventData,$TriggerMetadata)

Write-Host "New event occurred ..."

$EventData| ConvertTo-Json | Write-Host

This is an important step since once you capture an event, you would understand what are the properties and values are being sent from event grid so we then can build the function logic based on it.

Next let’s configure Event Grid …

Step 3: Create Azure Event Grid

Since the aim is to capture Virtual Machine creation events in specific Resource Group. We start by navigating to a resource group, then click on Events > Azure Function.

Fill the naming for both the Subscripts and Topic, oh yeah I haven’t mention the Azure Event Topic, It is where the events capturer by Event Grid subscription will be published to a Topic, then event handler aka Azure Function in this scenario can consume the event from the topic. If you are familiar with Apache Kafka it is similar concept.

The other thing that we select ws the EventType: ResourcesWriteSuccess. Since we need to test capturing only when objects/resource are created successfully on the resource group.

Note: When you select the Azure Function created previously as an endpoint and it is not showing up, make sure that you have configured the Input Trigger to Event Grid on the function.

Let’s try create an event …

Step 4: Create an Event

Creating a VM for testing event creation is my aim here, so I wrote a quick PowerShell lines rather than clicking through the console. I could’ve used Terraform though :P

$vmName = "Event-VM"
$UserName = "user"
$Password = ConvertTo-SecureString "Password" -AsPlainText -Force
$psCred = New-Object System.Management.Automation.PSCredential($UserName, $Password)
New-AzVm `
    -ResourceGroupName $resourceGroup `
    -Name $vmName `
    -Location $location `
    -Credential $psCred
    -VirtualNetworkName 'Event-Vnet' `
    -SubnetName 'Event-Subnet' `
    -SecurityGroupName 'Event-NetworkSecurityGroup' `

Once the VM is created successfully, let’s move back to Azure Function to monitor the creation event.

Step 5: Monitor Azure Function

The reason behind going into monitoring the captured events is to understand the event schema for:

• Building the logic into the Azure Function Code.
• Filtering for specific events once they captured in Event Grid. You don’t want to to filter in the Azure Function since you are paying for every function Invocation.

From the function console, we can use the Code+Test to monitor for log stream at the bottom of the page. But I think wha is better is the Monitor section where you can see both function invocations and the logs.

From the invocation traces I noticed two things. First from the warning, The JSON formate event got truncated, so I will increase the depth of the Write-Host in the function code.

$eventGridEvent | ConvertTo-JSON -Depth 4 |  Write-Host

Second thing, let’s get a look on how the eventData JSON schema.

There are three ways of filtering the Event Grid schema. First lets navigate back to Event Grid Subscription, and select Filters. 1 - Event types: This is what we have used when we created the Subscription. For capturing only a success creation use Resource Write Success.

2 - Subject Filter: From the subject attribute in JSON data:

"subject": "/subscriptions/xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx/resourcegroups/<ResourceGroupName>/providers/Microsoft.Compute/virtualMachines/"

We can use it to apply Subject Begins With filter. See the screenshot below:

3 - Advanced Filtering: I sort of lean into using Advanced filters, because with subject filter you can only specify the subject key. With Advance, you can select any key attribute and you can have multiple values with different operator matching. See the example below where I used the data.resourceProvider, data.operationName

And with this we made sure we filtered the events, and made sure only VM creation events in the scoped Resource Group will invoke the Azure Function.

Closing

That’s enough for one blog I think, let’s continue in Part 2 where we will use the current setup with an example of how to modify Virtual Machine or maybe Virtual Machine Scale Set creation and configure it to be ready to provision storage volumes from Pure Cloud Block Store.

• ← Previous Post
• Next Post →