Deploying Azure Infrastructure Resources with Azure Devops – Part 2

In part one we setup a project in Azure Devops, create an Azure repo, added an ARM template to the repo and created a build pipeline.  In this post we’ll create a release pipeline to actually deploy our resources in Azure.

Within Azure Devops open our project and select ‘Releases’ and select ‘New Pipeline’

1

On the next page choose under select template choose ‘Start with an empty job’

2

By default the stage is called Stage 1 as per the above screenshot, I’ve renamed mine to Deployment

3.png

The point of stages are the different releases to environments you might have such as dev, stage, production.  You can add these stages in and release to each individually.  Understanding this was key to me in understanding why tools such as Azure Devops are so very useful.  In this example I’m going to stick with the one stage though.

I now select Save and accept the default save location when prompted.  I then select ‘Add an artifact’

4.png

I select my Source, this is the build pipeline we created in part one. I leave everything else at default and select Add and then hit save.

In the deployment stage I click on 1 job, 0 task

5

6

I click + in Agent job and search for and select ‘Azure Resource Group Deployment’

In the Resource Group Deployment Task I set information such as the subscription to deploy to and the resource group.  In part 1 we said we’d deploy to a resource group called RG_Network which was created by the build pipeline for us.

7.png

Leave Template Location as Linked Artifact and click the three dots by Template Location to browse to the artifact which was created for us by the build pipeline

8.png

Hit Save

In the top right of the page select the down arrow by ‘Release’ and select Create A Release

9.png

 

Leave everything except for ‘Select the version for the artifact sources for this release’  Hit the drop down arrow and select your source artifact.  Click create and our release is created

A message appears saying the release has been created

10.png

In my case it was called Release-1 and I can click on that to see the release details.  In my case the deployment happened very quickly as the ARM template is very basic and just deploys a vnet but we can see it said the deployment was successful.

11.png

To see if it really was successful I can look in my RG_Network resource group to see if the vnet was created

12.png

 

And it was.  Using Azure Devops I’ve deployed an ARM template which created a vnet in Azure.

Earlier in this post I made a comment about stages and how getting started with Azure Devops has helped me understand why tools like this are so useful.  The deployment stage I deployed to could easily have been called Dev or Test and then I can have another stage called Production.  The key bit here is it’s the same ARM template being deployed in each stage so consistency of what is deployed is guaranteed across the stages.

Obviously there may be necessary changes between environments like test and production, such as IP addressing.  This can be handled with variables in the various stages to override the parameters of the ARM template.  I’ll cover those and some other features in future blog post.  Obviously what has been done here is fairly simple, just a single ARM template deploying one vnet.  As I learn more about using Azure Devops I’ll write it up 🙂

 

Deploying Azure Infrastructure Resources with Azure Devops – Part 1

I’ve recently had the opportunity to start playing with Azure Devops and as I’m historically and primarily an infrastructure guy I wondered how Azure Devops can help me with such things as Infrastructure as Code.

What I’ll show in this post is how to deploy ARM infrastructure resources such as VMs and vnets using ARM templates and Azure Devops.  I’m not going to go into how you can purchase Azure Devops, there are many ways to gain access to it https://azure.microsoft.com/en-gb/pricing/details/devops/azure-devops-services/

so I’m going to dive straight into how I deploy ARM templates with Azure Devops.

First an overview of what I’m going to be using:

Azure Devops (obviously)

Azure Repos (Azure repos are a git repo)

Build Pipeline

Release Pipeline

And my aim here is to deploy an Azure virtual network using Azure Devops

Your Devops URL is https://dev.azure.com/yourorgname

Step 1 – Create a Project

When first going to Azure Devops this is the page I’m greeted with:

1

I create a project called ‘Azure Infrastructrure’ and leave the visibility set as Private.  When the projected is created I’m shown a ‘Welcome to the project’ screen

2

Down the left hand side you can see items such as Boards, Repos, Pipeleines, test Plans and Artifacts.  I’ll only be dealing with Repos, and Pipelines.

Firstly I choose Repos and I choose ‘or initialze with a Readme’ to create a new repo and click intiialize.  Note you also have options to clone the repo to your computer or push an existing repo.  For simplicity I’ll just be initializing a new repo.

3.png

An empty repo is created:

4

I’m going to create a new folder called ARM-Vnet by selecting New – Folder.  Note that you cannot create an empty directory so you have to create a placeholder file when you create the directory:

5

I’m also going to create a folder called ARM-VMs

6

I upload some existing ARM templates that I have for creating a vnet and virtual machines into the relevant folders.

7.png

Build Pipeline

The next step is to create a build pipeline.  You can read more about pipelines here

Select Pipelines – Build – New Pipeline

In the new pipeline window select ‘Use the visual designer to create a pipeline without Yaml’  It doesn’t look like a selectable item but ‘Use the visual designer’ section is.

8

Select your source repository.  As mentioned earlier I’m using Azure Repos

9.png

Select Continue

On the select template page choose ‘Start with an empty job’ and the build pipeline is created and immediately opens up for you to start customising the pipeline.  Note that by default it names the pipeline after your project and appends -CI to it.

11

Click on the + sign in Agent job 1.

Search for azure

Select Azure Resource Group Deployment and click Add to add Azure Resource Group Deployment to the agent job

12.png

Fillout the following details in the task:

Change the name of the deployment if you wish

Select the Azure subscription.  A note on selecting the Azure subscription, in my example my account is in the same Azure AD Tenant as where I’m going to target the deployment of my resources, so I can select authorize.  If this is not the case you need to create service connection between Azure Devops and the Azure AD Tenant to enable the deployment of resources from Azure Devops.  More on this at https://docs.microsoft.com/en-us/azure/devops/pipelines/library/connect-to-azure?view=vsts

13

Action – Create or Update Resource Group

Select an existing resource group or set a name for a new resource group

Set the location

Browse and select the to ARM template located in our repo

Browse and select the ARM template parameters if required

Deployment Mode – set to Validation Only

14-new

15

Click + on the agent job again and add a ‘Copy Files’ task.  Give the task a name and set the options as follows:

Set the source folder to the folder holding the relevant ARM template.  In my case this is my ARM-Vnet folder, under which is the ARM template for my virtual network.

16.png

Click + on Agent job 1 to add a new task and choose ‘Publish Build Artifact’

All the options can be left at their defaults for this task

16

Save the pipeline by clicking the arrow next to ‘Save and Queue’ and select Save.  We have now finished our build pipeline.  What does this do exactly?  It runs a validation against our ARM template, which is equivalent to Test-AzureRMResourceGroupDeployment cmdlet or the Azure CLI az group deployment validate command.  It checks that our template is syntactically correct and then copies it to the agent build directory (a directory on a VM essentially that is used to run the pipeline is my understanding) and then publishes that template as an artifact.

When defining the Azure Resource Group Deployment I specified a resource group called RG_Network, if we look at my current resource groups in my subscription we’ll see that no such resource group exists yet:

18

In the build pipeline if I now select ‘Queue’

19

20

Leave all the options and click Queue.  A message near the top of the page reports that the request has been queued and shows you the build number.  You can click that message to see the Build

21

Once the build has completed successfully the resource group RG_Network has been created in my Azure subscription but there are not resources in the resource group.

 

The resources will be deployed via our release pipeline in part 2.

 

 

 

Azure Learning

Recently announced is the new Microsoft learning site for all things Azure

 

 

“Introducing a new approach to learning

The skills required to advance your career and earn your spot at the top do not come easily. Now there’s a more rewarding approach to hands-on learning that helps you achieve your goals faster. Earn points, levels, and achieve more!

 

https://docs.microsoft.com/en-us/learn/

 

At first glance these new training and learning options seem very good.  I’ll be trying them out soon enough

Introducing Role-based Microsoft & Azure Certification Shakeup — Build Azure

Last July, Microsoft Learning announced some upcoming changes to the Microsoft Azure Certifications to make them more role-based. Recently, surrounding the Microsoft Ignite 2018 conference, they announced and released further information about these changes to transform the Azure certification tracks. This news includes more than just announcement of new Azure certification exams, but also the…

via Introducing Role-based Microsoft & Azure Certification Shakeup — Build Azure

vNet Peering PowerShell

We have a hub and spoke design in Azure for our vNets and needed to peer the vNets together.

This can be done in an ARM template and we could have deployed all three networks in one go and peered them as part of the ARM template deployment.  For various reasons that approach didn’t really work for us, 1 reason was the customer is very sensitive to change control and having all 3 vNets being controlled by one ARM deployment didn’t sit very well with them and made them nervous.

So PowerShell was the obvious answer, and it’s very simple in PowerShell anyway.  So we deploy all 3 vNets using separate ARM templates and then peer them together with powershell.  Code also over on github

#Peering for hub and spoke network design
#Variables Section
$hubVnetResourceGroup = "RG_hubVnet"
$hubVnetName = "hubprodVnet"
$spoke1VnetResourceGroup = "RG_spoke1vnet"
$spoke2VnetResourceGroup = "RG_spoke2Vnet"
$spoke1VnetName = "spoke1prodVnet"
$spoke2VnetName = "spoke2prodVnet"
$hubVnet = Get-AzureRmVirtualNetwork -Name $hubVnetName -ResourceGroupName $hubVnetResourceGroup 
$spoke1Vnet = Get-AzureRmVirtualNetwork -Name $spoke1VnetName -ResourceGroupName $spoke1VnetResourceGroup
$spoke2Vnet = Get-AzureRmVirtualNetwork -Name $spoke2VnetName -ResourceGroupName $spoke2VnetResourceGroup
#End Variables

#Add Hub to spoke1 peer and allow gateway transit through hub1
Add-AzureRmVirtualNetworkPeering -Name 'hubtospoke1peer' -VirtualNetwork $hubvnet -RemoteVirtualNetworkId $spoke1vnet.id -AllowForwardedTraffic  -AllowGatewayTransit 

#Add spoke 1 to hub and use hub 1 gateways
Add-AzureRmVirtualNetworkPeering -Name 'spoke1tohubpeer' -VirtualNetwork $spoke1vnet -RemoteVirtualNetworkId $hubVnet.id -AllowForwardedTraffic  -UseRemoteGateways 

#Add hub to spoke2 peer and allow gateway transit through hub
Add-AzureRmVirtualNetworkPeering -Name 'hubtospoke2peer' -VirtualNetwork $hubvnet -RemoteVirtualNetworkId $spoke2vnet.id -AllowForwardedTraffic  -AllowGatewayTransit

#Add spoke 2 to hub and use hub 1 gateways
Add-AzureRmVirtualNetworkPeering -Name 'spoke2tohubpeer' -VirtualNetwork $spoke2vnet -RemoteVirtualNetworkId $hubVnet.id -AllowForwardedTraffic  -UseRemoteGateways 





Playing with Azure Firewall

What is Azure Firewall  – A fully stateful firewall as a service.

Before you can deploy Azure Firewall you need to register the provider in your subscription : https://docs.microsoft.com/en-us/azure/firewall/public-preview

Register-AzureRmProviderFeature -FeatureName AllowRegionalGatewayManagerForSecureGateway -ProviderNamespace Microsoft.Network

Register-AzureRmProviderFeature -FeatureName AllowAzureFirewall -ProviderNamespace Microsoft.Network

It can take up to 30 minutes for the feature registration to complete

The easy way to get going and play with Azure Firewall is to use the quickstart template https://github.com/Azure/azure-quickstart-templates/tree/master/101-azurefirewall-sandbox

I’ve used the above template to get up and running with Azure Firewall quickly, easily and so I don’t have to click around in the portal.

First up is Network Rules – the template deploys adds an example rule in which I have deleted so I can start from scratch.  At the moment I have no network rules in my firewall

netrule1

If I now try to telnet to another server of mine that has RDP open to the internet we can see the connection is not successful

telnet1

I now add a rule to my firewall:

netrule1

And now the telnet connection to 3389 to the target is successful:

telnet2

Next up is Application Rules.  Application Rules allow you to control what FQDNs can be accessed and, somewhat obviously, these rules are http and https based. Again the deployment I used created a single Application Rule which I have deleted to give me a clean slate from which to start:

apprules1

Now if I try to browse the web from my server in Azure I’m blocked.  Interestingly the message I get in the web browser depends on whether I’ve gone to a https site or http:

web1

web2

I now add a rule for http and https for *.microsoft.com

apprules2

I can now browse to Azure.microsoft.com

web3.PNG

 

To get the abilty to filter http/https traffic like this you’d have to deploy a Network Virtual Applicance (NVA) and to control what can route to where in your Azure infrastructure you’d require an NVA or something like a Linux IaaS server running iptables.

Azure firewall looks like a good solution to filtering internet traffic and for controlling routing between servers in Azure.  Compared to some NVA devices or iptables it may be more basic (at the moment) but it certainly does offer what I see a lot of people asking for.

VM Deployment with ARM Template

This ARM template will deploy x number of virtual machines (takes the number you need as a parameter), a storage account in the target resource group for boot diagnostics, deploys the VMs in an availability group and deploys x number of data disks per VM.  Both the number of data disks (per VM) and size are parameters.  It attaches the VMs to an existing vnet/subnet, both of which are parameters.  it also sets the Locale to UK on Windows servers using a custom script extension as per https://www.lewisroberts.com/2017/03/01/set-language-culture-timezone-using-powershell/

Template over on my Github page https://github.com/pagyP/AzARM/tree/master/VMs

This template will be updated over time to include additional functionality.

OMS Deployment with ARM Template

An ARM template to deploy a basic OMS workspace in Azure.  It deploys a workspace and two OMS solutions: Anti-Malware assessment and Updates.

Another template for OMS deployment, which is identical to the above, but links the workspace to an existing automation account.  Note it does not create the automation account, the automation account must already exist.

These templates are over on my Github page