Azure on Wesley Camargo

How to Import an Existing Azure Resource into Your Terraform State with Azure Storage Backend

Thu, 15 Feb 2024 15:41:21 +0000

Have you ever for any reason didn’t had your Azure resource under Terraform and want to starting managing it by the HashCorp Tool? In this step-by-step guide, we’ll walk you through the process of importing an existing Azure resource into your Terraform state file, where your backend is configured to use an Azure Storage Account.

Preparing the Backend: Creating the Storage Account for State Management

Before we start, we need to set up an Azure Storage Account, which will be the backend for your Terraform state files. This ensures that your state is kept in a secure, centralized location, making it easier to manage, especially when working within a team.

The Azure CLI script below will create a Resource Group and a Storage Account with the container, where the state files will be created later:

1
2
3


az group create -l uksouth -n rg-tfstate
az storage account create -n sttfstateimport -g rg-tfstate
az storage container create -n tfstate --account-name sttfstateimport

Preparation of the Terraform File

Now that our Azure backend ready, lets configure the Terraform files to use it.

We will first add the azurerm backend, to do so, we will need to provide the the details from the storage account created previously, it will be required the storage account name, container name, the resource group, and the key of our workload. After that, we will create a simple resource group:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


provider "azurerm" {
 features {}
}

terraform {
 backend "azurerm" {
 container_name = "tfstate"
 key = "myworkload"
 resource_group_name = "rg-tfstate"
 storage_account_name = "sttfstateimport"
 }
}

# Create a resource group
resource "azurerm_resource_group" "example" {
 name = "rg-myresourcegroup"
 location = "UKSouth"
}

Initialize the Terraform workspace and apply the changes:

1
2


terraform init;
terraform apply;

Terraform State file

After the creation of this resource, lets check the state file hosted on our storage account. Note that we only have the resource group being managed by Terraform.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31


{
 "version": 4,
 "terraform_version": "1.6.5",
 "serial": 5,
 "lineage": "f5c71b49-ac3a-e0ec-2529-b017b3c3965f",
 "outputs": {},
 "resources": [
 {
 "mode": "managed",
 "type": "azurerm_resource_group",
 "name": "example",
 "provider": "provider[\"registry.terraform.io/hashicorp/azurerm\"]",
 "instances": [
 {
 "schema_version": 0,
 "attributes": {
 "id": "/subscriptions/337ba254-3aa0-4551-ba8e-89debefaa373/resourceGroups/rg-myresourcegroup",
 "location": "uksouth",
 "managed_by": "",
 "name": "rg-myresourcegroup",
 "tags": {},
 "timeouts": null
 },
 "sensitive_attributes": [],
 "private": "xxxx"
 }
 ]
 }
 ],
 "check_results": null
}

Creation of Resource Outside Terraform Using Azure CLI

Sometimes, resources are created outside of Terraform, maybe manually or through scripts, or they might even be there before we start to adopt an Infra as Code strategy. In our case we will simulate it using Azure CLI. Create the Azure resource you want to manage with Terraform. For example, you can create a storage account, VM, or any other resource.

1

az storage account create -n sttoimport -g rg-myresourcegroup

After the resource creation, note the id of the resource. It will be needed later for the import process.

The Import Process on Terraform

Now that we have our resource and the Terraform backend is ready, we can import the existing resource into the Terraform state

Write a Terraform resource block in your configuration file that corresponds to the resource you created. Make sure the name and type match the Azure resource.

1
2
3
4
5
6
7


resource "azurerm_storage_account" "example" {
 name = "sttoimport"
 resource_group_name = azurerm_resource_group.example.name
 location = azurerm_resource_group.example.location
 account_tier = "Standard"
 account_replication_type = "LRS"
}

If you try to apply, an error message will appear, stating that the resource already existis and must be imported

The simplest way to achieve this is by adding the import block of code into your tf file. You will need to specify the id of the resource (that one we noted before) and to which resource it should be imported.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


import {
 to = azurerm_storage_account.example
 id = "/subscriptions/<subscription id>/resourceGroups/myResourceGroup/providers/Microsoft.Storage/storageAccounts/stportalcreated"
}

# # Create a storage account
resource "azurerm_storage_account" "example" {
 name = "sttoimport"
 resource_group_name = azurerm_resource_group.example.name
 location = azurerm_resource_group.example.location
 account_tier = "Standard"
 account_replication_type = "LRS"
}

After adding it, run terraform apply once again.

Terraform State file

Now we can check once again our Terraform state file. We can note that it’s much more extense, and has not only the resource group as in the first deployment, but the storage account as well.

 1
 2
 3
 4
 5
 6
 7
 8
 9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100


{
 "version": 4,
 "terraform_version": "1.6.5",
 "serial": 6,
 "lineage": "f5c71b49-ac3a-e0ec-2529-b017b3c3965f",
 "outputs": {},
 "resources": [
 {
 "mode": "managed",
 "type": "azurerm_resource_group",
 "name": "example",
 "provider": "provider[\"registry.terraform.io/hashicorp/azurerm\"]",
 "instances": [
 {
 "schema_version": 0,
 "attributes": {
 "id": "/subscriptions/337ba254-3aa0-4551-ba8e-89debefaa373/resourceGroups/rg-myresourcegroup",
 "location": "uksouth",
 "managed_by": "",
 "name": "rg-myresourcegroup",
 "tags": {},
 "timeouts": null
 },
 "sensitive_attributes": [],
 "private": "xxxx"
 }
 ]
 },
 {
 "mode": "managed",
 "type": "azurerm_storage_account",
 "name": "example",
 "provider": "provider[\"registry.terraform.io/hashicorp/azurerm\"]",
 "instances": [
 {
 "schema_version": 4,
 "attributes": {
 "access_tier": "Hot",
 "account_kind": "StorageV2",
 "account_replication_type": "LRS",
 "account_tier": "Standard",
 "allow_nested_items_to_be_public": true,
 "allowed_copy_scope": "",
 "azure_files_authentication": [],
 "blob_properties": [
 {
 "change_feed_enabled": false,
 "change_feed_retention_in_days": 0,
 "container_delete_retention_policy": [],
 "cors_rule": [],
 "default_service_version": "",
 "delete_retention_policy": [],
 "last_access_time_enabled": false,
 "restore_policy": [],
 "versioning_enabled": false
 }
 ],
 "cross_tenant_replication_enabled": true,
 "custom_domain": [],
 "customer_managed_key": [],
 "default_to_oauth_authentication": false,
 "edge_zone": "",
 "enable_https_traffic_only": true,
 "id": "/subscriptions/337ba254-3aa0-4551-ba8e-89debefaa373/resourceGroups/rg-myresourcegroup/providers/Microsoft.Storage/storageAccounts/sttoimport",
 "identity": [],
 "immutability_policy": [],
 "infrastructure_encryption_enabled": false,
 "is_hns_enabled": false,
 "large_file_share_enabled": null,
 "local_user_enabled": true,
 "location": "uksouth",
 "min_tls_version": "TLS1_2",
 "name": "sttoimport",
 "network_rules": [
 {
 "bypass": [
 "AzureServices"
 ],
 "default_action": "Allow",
 "ip_rules": [],
 "private_link_access": [],
 "virtual_network_subnet_ids": []
 }
 ],
 "nfsv3_enabled": false,
 // ... additional attributes truncated for brevity
 "tags": {},
 "timeouts": null
 },
 "sensitive_attributes": [],
 "private": "xxxx",
 "dependencies": [
 "azurerm_resource_group.example"
 ]
 }
 ]
 }
 ],
 "check_results": null
}

Conclusion

Integrating existing Azure resources into Terraform doesn’t have to be complex. By following the steps described in this guide, you can quickly and efficiently bring your Azure infrastructure under Terraform’s infrastructure-as-code management. Happy coding!

Don’t run out of credits on Azure!

Wed, 06 Dec 2023 19:01:04 +0100

Don’t run out of credits on Azure! How to shutdown Azure Virtual Machines automatically with Terraform

If you are distracted and forgetful like me, you probably already forgot an Azure Virtual Machine running without need and just figured it out when your Azure credits were gone. To avoid this, check a simple tip on how to solve this problem with Terraform.

Photo by Tristan Gassert on Unsplash

In my previous post, I have already shown how to create an Azure Virtual Machine using Terraform, now I’m adding this feature, which, for me, is very important and helps me to not spend all my Azure Credits in a forgotten VM 🙃.

What is Azure VM Auto-Shutdown?

Azure VM Auto-Shutdown is like setting a timer for your Azure VM. Auto-shutdown allows you to set a specific time so your Virtual Machine will turn off. This is a handful when you create VMs for tests, or even in development/QA environments, where you don’t need your VMs running at night when your team is not working.

Terraform resource for auto-shutdown of an Azure VM

To make it work, we will use the Terraform resource azurerm_dev_test_global_vm_shutdown_schedule. This is part of the Azure Terraform provider. The usage is quite simple, as we can see below:

The most important information required are the VM id, daily_recurrence_time, and timezone. Based on this information your VM will be shut down at the time you have defined.

Complete Terraform to create an Azure VM with auto-shutdown

To facilitate the understanding of how to integrate it in a real-life scenario, I have implemented the shutdown process in the following complete Terraform file:

Conclusion

Below it is the link to my GitHub repository where I am creating a library to have all references related to Terraform. I hope that this post despite being simple might be helpful! wesleycamargo/terraform (github.com)

👋 If you find this helpful, please click the clap 👏 button below a few times to show your support for the author 👇

🚀Join FAUN Developer Community & Get Similar Stories in your Inbox Each Week

Getting Started with Azure: What are Azure Virtual Machines and how to create it using Terraform

Tue, 21 Nov 2023 18:45:11 +0100

Getting Started with Azure: What are Azure Virtual Machines and how to create it using Terraform

Azure VMs are like your own computer on the cloud. You can choose between different Operational Systems, such as Linux and Windows, and they can be used for different goals, like hosting a website for example.

Photo by imgix on Unsplash

Virtual Machines are part of a category called Infrastructure as Code, and they are still relevant in many scenarios, even with the advent of Platform as Code, where you are in control of many aspects, such as security, identity and access, high availability, and others.

What do I need to create a Virtual Machine on Azure?

To create the VM there are a few Azure resources that are necessary to be created before, as we can see in the image below:

Resources that are Pre-requisites

There are some resources that are not necessarily part of the same life cycle of your VM, and you might have already created them on your Azure environment. On the script that I am providing below, they are being created, but you might decide to use a resource that already exists.

**Resource Group: **A resource group works similarly to a folder, where you can group the resources that have something in common. It can be resources of the same type, such as VMs, or the most recommended, resources that share the same life cycle.

**Virtual Network: **In an Azure VNet, you are able to create a private space, where your resources can communicate securely. It works similarly to the network that you are connected to right now to read this post, but it sits on Azure.

Subnet: They are subdivisions of a Virtual Network, helping to keep organized and segmented by different types of resources, like Databases, Front and Backend applications.

Resources that are part of the Virtual Machine life cycle

**Disks: **Disks are used to store operating systems, applications, or data. Each VM created on Azure has at least one OS Disk, but it can have multiple.

Network Interface Card: Also known as NIC, a Network Interface Card is responsible for connecting the VM with the network that we saw earlier. Similar to disks, each VM must have at least one, but also multiple NICs are supported.

Public IP: A Public IP is the communication between the VM and the world. This resource is not mandatory, and in most cases inside a business context, it should be created very cautiously, to avoid any security risk. However, as it is a simple test and I don’t want to complicate it, we need the Public IP to be able to connect to our VM. It is connected to the Network Interface.

Terraform Script

Pre-requisites

Below is the section of Terraform that creates the pre-requisites. As I mentioned before, it is possible to use existing resources, however, you need to adapt the script to use them.

In this script, we are creating the resource group, virtual network, and subnet.

Virtual Machine

Finally, this is the script to create the VM resources. Here we create first a Public IP, the Network Interface Card, and associate the PIP on it, and a VM. Note that the Terraform VM resource already abstracts the disk internally. It’s also interesting to highlight that the VM associates the NIC created before, creating a dependency, which Terraform will identify and create in the correct order.

Complete Script

Below is the complete script, including provider, variables, and outputs:

Conclusion

👋 If you find this helpful, please click the clap 👏 button below a few times to show your support for the author 👇

🚀Join FAUN Developer Community & Get Similar Stories in your Inbox Each Week

From Terraform to Azure Bicep: What You Need to Know about syntax

Fri, 17 Feb 2023 18:31:38 +0100

From Terraform to Azure Bicep: What You Need to Know about syntax

Photo by Botond Czapp: https://www.pexels.com/photo/birds-flying-on-blue-sky-7255793/

Hey there Terraform experts! I am confident that you know the power of infrastructure as Code in managing and provisioning IT infrastructure and have been applying it with the famous HashiCorp tool. But have you heard about Azure Bicep, the newer Microsoft IaC language for managing Azure resources? If you’re curious about the differences between Terraform and Bicep, and how Bicep can help you better manage your Azure resources, you’ve come to the right place. Let’s dive in and explore the world of Infra as Code with Azure Bicep! In this first post, I’m going to break down the basic differences between Terraform and Bicep file structures. These infrastructure-as-code tools have their own unique syntax and components, so it’s important to know what sets them apart. By the end of this post, you’ll have a solid grasp of what makes them tick.

Introducing Terraform and Bicep

Terraform and Azure Bicep are two popular Infra as Code (IaC) tools that allow developers and operations teams to manage and provision cloud infrastructure using development practices.

Terraform is a tool developed by HashiCorp that supports multiple cloud platforms, including Azure, while Azure Bicep is a newer tool developed by Microsoft specifically for managing Azure resources. Both tools have their own syntax and language for defining Infrastructure as Code and offer benefits such as consistency, scalability, and version control. However, there are also some differences between the two, such as how they handle state management and the availability of community-built providers. In the following sessions, we’ll explore the similarities and differences between Terraform and Azure Bicep, and help you decide which tool is best for your infrastructure management needs.

Language and Syntax

Terraform and Bicep has a lot in common. Both use their own domain-specific languages, with a declarative approach, support modularization, and are designed to be human-friendly(I am looking at you, ARM Templates 🙃). Terraform uses HashiCorp Configuration Language, also known as HCL, developed by HashiCorp and has several providers, supporting not only Azure, but AWS, GCP, and even VMWare. Bicep is developed by Microsoft with the intention to abstract the usage of ARM Templates and was developed specifically for Azure, which gave a great integration.

The structure of both languages is similar, within the same building blocks, with the main components being resources, inputs, and outputs, and as I said above, quite easy to read. Below we can see a comparison of the creation of an Azure Storage Account:

Image prepared by the author

Now let’s analyze the main differences:

Provider

The first and biggest difference that note is that in the Terraform example, we have a section called “provider”. As Terraform supports not only Azure but also other providers, it is required to declare the provider that we want to use.

As Bicep is designed specifically for Azure, the provider statement is not required.

Input

Another difference is that the input parameters have different reserved words. In Terraform it’s called variable, the default values are declared with a default statement and its syntax require curly brackets.

Bicep, in turn, declares the inputs by the reserved word param , the attribution of a default value can be done with the equals sign and it’s not required any brackets. Bicep also has a variable reserved world, but it’s used not for inputs, below we can see more about it.

Local variables

Local variables in Terraform have a special section called locals. All necessary variables must be declared in this section and must be accessed by the syntax local.<variable name> and can be used to create combinations and complex operations.

locals { resource_group_name = “myTerraformResourceGroup” location = “West Europe” }On Bicep, it is not necessary to have a specific session, being possible to declare anywhere in the file (although is recommended to do in the beginning to make it more readable). It also makes the syntax a little simpler than in terraform var location = ‘westeurope’

Resources

Finally something in common! For both, Terraform and Bicep, the resource is declared by the reserved word… resource! And there is more in common, in both cases, when declaring the resource you need to declare which resource is desired and you also need to create an alias to this resource, so it will be possible to access information from it, even in the case that you have several resources of the same type. The internal properties will be different depending on each resource type, but in general, you need to provide the resource name, SKU, location, and so on.

Although the concepts are quite the same, the syntax is slightly different as we can see below:

Terraform resource:

resource “azurerm_storage_account” “storage” { name = var.storage_account_name resource_group_name = local.resource_group_name location = local.location account_tier = “Standard” account_replication_type = “LRS” }Bicep resource:

resource storage 'Microsoft.Storage/storageAccounts@2022-09-01' = { kind: 'StorageV2' location: location name: storageAccountName sku: { name: 'Standard_LRS' } }n

Output

The outputs follow a similar logic as the inputs. Terraform requires curly brackets, Bicep no. In terraform, it is necessary to specify the resource type first, then the alias, and finally the property. It is also important to highlight that is not possible to assign a type to an output in Terraform.

output “storage_account_name” { value = azurerm_storage_account.storage.name }Bicep is again, slightly simple. No brackets and you can reference by the alias directly, without having to declare the resource type. In Bicep is also possible to assign a type in the output, which is useful in some cases.

output storageName string = storage.namen

Terraform storage account example

Bicep storage account example

Conclusion

In conclusion, while Terraform and Bicep share some similarities in their purpose and functionality, they have significant differences in syntax, which can affect the user experience and overall efficiency in deploying infrastructure as code. Understanding these syntax differences is crucial for developers to effectively utilize the benefits of each tool and to determine which one is the best fit for a particular project or organization.

In the upcoming posts, we will dive deeper into other differences between both tools, exploring their impact on various aspects of infrastructure deployment and management, and offering practical tips and solutions to common challenges. Stay tuned to learn more about how to optimize your infrastructure as code with Terraform and Bicep!

👋 If you find this helpful, please click the clap 👏 button below a few times to show your support for the author 👇

🚀Join FAUN Developer Community & Get Similar Stories in your Inbox Each Week

Optimize Your Cloud Environment with Azure Platform Landing Zones

Sun, 12 Feb 2023 20:36:00 +0100

Optimize Your Cloud Environment with Azure Platform Landing Zones

Photo by Pascal Meier on Unsplash

What are Azure Landing Zones and why are they important?

Imagine that you are building a house, it is necessary to prepare the ground and have a solid foundation to grow your walls. Moreover, it is also necessary to prepare essential services, for instance, plumbing, heating and insulation, and electric cables in this foundation, otherwise, it will be more complicated and expensive (although possible) to do it later.

An Azure Landing Zone is the same as the house foundation, with essential services to support your platform or applications.

We can say that Azure Landing Zone is a standardized and repeatable framework for deploying and managing resources in Microsoft Azure. It provides a set of best practices and policies, that helps to improve the cloud deployment process and ensure that resources are deployed in a consistent, scalable, and secure manner.

Azure Platform Landing Zones

The set of essential services that are shared by multiple applications is also called Platform Landing Zone. These services, such as Networking and Identity management, are typically managed and centralized by dedicated teams, allowing application teams to concentrate on their core business objectives.

Azure Platform Landing Zone

Microsoft divided the Platform Landing Zones into 8 design areas. These design areas are topics that must be carefully planned to ensure the security and performance of the platform:

**Billing and Active Directory **—Billing is the process of tracking and paying for the resources used in an Azure environment. The Azure billing and cost management platform provide the ability to monitor usage and costs, set budgets and alerts, and optimize spending. Azure Active Directory is a cloud-based identity and access management service that provides centralized identity management and robust security features.
**Identity and access management **— IAM controls access to Azure resources. Azure Active Directory is the main service used for IAM and helps manage user identities, groups, and access to cloud-based apps.
Network topology and connectivity — It deals with the design and communication between virtual network resources. A good design ensures secure and efficient communication and access to the internet and external resources.
Resource organization — It involves grouping and managing resources within an Azure environment. It provides structure and scalability, making it easier to manage and control resource access. Resource groups, subscriptions, and policies to improve efficiency are used to reduce complexity, making it easy to adminstrate and ensure policy compliance in the platform.
Security — This is one of the most important design areas when planning the landing zone. It involves protecting resources with a multi-layered approach, including access control, network security, encryption, and incident response planning.
**Management — **Management in Azure Landing Zones covers processes and tools used to manage and maintain resources. A well-managed landing zone needs clear roles, a change management process, and a structured approach to resource management.
**Governance **— This is about setting the rules to keep your Azure environment organized and secure. Azure has tools like Policy and Role-Based Access Control to enforce your policies. To make things easy, make sure roles are clear and have a plan for making changes.
Platform automation and DevOps — As I am a DevOps guy for almost 10 years, I am suspicious to talk about this topic :D. It basically ensures that tools and processes make the deployment and management of applications and infrastructure easier and faster.

Conclusion

Azure Platform Landing Zones are the perfect solution for businesses looking to streamline their cloud deployment and management processes. With its centralized services, specialized teams, and powerful tools for automation and DevOps, it makes it easier than ever to bring new applications and services to market. Not to mention, the added security and governance features give peace of mind when it comes to managing resources in the cloud. So, whether you’re just starting your cloud journey or looking to optimize your existing operations, Azure Landing Zones are the way to go! So why wait? Get on board and start experiencing the benefits for yourself!

👋 If you find this helpful, please click the clap 👏 button below a few times to show your support for the author 👇

🚀Join FAUN Developer Community & Get Similar Stories in your Inbox Each Week

Don’t repeat yourself! Creating Azure Bicep modules 💪

Mon, 28 Nov 2022 18:34:16 +0100

Don’t repeat yourself! Creating Azure Bicep modules 💪

Since our Infrastructure is Code now, why don’t we leverage the best software development practices? Check out this post on achieving reusability on your IaC templates using Bicep Modules!

Photo by Prateek Katyal on Unsplash

What are Azure Bicep Modules?

Bicep Modules are the application of the **Don’t Repeat Yourself **principle, where a Bicep template calls another Bicep template. As a result, this practice enables the reusability of existing code and can speed up the creation of future resources.

Imagine, for example, that you need to control how infrastructure resources are created. To accomplish this, you can implement rules in the bicep templates that, for example, restrict the size of virtual machines or the regions in which they can be created.

Since the definition has already been created and the rules, will probably remain the same in the next project, it is not necessary to create a new definition. We can create a module that allows us to save valuable time and money by reusing these definitions.

How to create an Azure Bicep Module?

If you are already familiar with Bicep Templates, creating a Bicep Module is effortless, as a module has exactly the same structure and it is not necessary to add any special configuration. It is possible, inclusive, to reuse existing templates as modules.

How to consume an Azure Bicep Module?

Declaring a Bicep Module

When working with Bicep Modules, the syntax changes compared to resources. Instead of declaring resourcethe declaration is now module**. **Additionally, it is necessary to reference the module path, which can be local or remote.

Image prepared by the author

It is also mandatory to add a symbolic name to the resource. The symbolic name can be used to recover an output from the module to be used in another part of the template.

Image prepared by the author

The property name** **is mandatory. It will become the nested template name in the final template.

Image prepared by the author

Passing parameters to Bicep modules

The parameters on modules follow the same syntax and contain the same features as regular templates. However, to send the parameters to a module it must be under the params node. It must match those declared in the module, including the validations.

Image prepared by the author

How to deploy an Azure Bicep Module?

Deployment of Bicep Modules works exactly like Bicep Templates. It is possible to use Azure CLI, Azure PowerShell, or even through Azure API.

I explain in more detail in this post how to deploy using Azure CLI. Below we can see the script used to deploy the templates and modules created above.

After running it is possible to see in the deployments of Azure Resource Group the main.bicep deployment:

Image prepared by the author

And if we click on the nested storage deployment we can observe the resource storageAccount being deployed :

Image prepared by the author

Conclusion

Reusability is one of the main advantages that we can leverage from modularization in any programming language and with Infrastructure as Code templates, it wouldn’t be different. Another benefit is that it also will ensure that your environment is more stable as your templates will be already tested by others, and most importantly, will also save implementation time.

I hope this helps you, and see you at the next one!

👋 If you find this helpful, please click the clap 👏 button below a few times to show your support for the author 👇

🚀Join FAUN Developer Community & Get Similar Stories in your Inbox Each Week

Stop breaking your environment: How to get started with Azure DevOps Pipelines

Mon, 07 Nov 2022 15:17:27 +0100

Stop breaking your environment: How to get started with Azure DevOps Pipelines

Have you ever had problems with manual deployments? It is not about if but when an error will happen. I even saw entire sites being replaced wrongly! It is scary, isn’t it? Azure DevOps Pipelines will help you to avoid such mistakes, so check on this post how to deploy your application in minutes, taking advantage of all benefits that automated pipelines can bring to you!

Photo by Louis Hansel on Unsplash

What is Azure DevOps Pipelines?

A Deployment Pipeline is an automated process that combines Continuous Integration (CI) and Continuous Delivery (CD), deploying your application in a given environment. In other words, it automatically builds, tests, and deploys your code to make them available to other people.

Prerequisites

Before we start creating the pipelines, there are configurations that we need to prepare in advance. My friend Jev Suchoi and I have already created a series of posts covering all these topics:

How to create an Azure DevOps organization
How to configure your Azure DevOps project
[Basic git commands you need to know](http://Git Basic commands in a nutshell)
Pushing code to git
Connecting Azure DevOps with Azure
Creating a variable group on Azure DevOps

How to create an Azure DevOps pipeline

Azure DevOps Pipelines are an implementation of deployment pipelines that supports multiple programming languages and clouds. Basically, everything can be deployed using this tool.

So let’s create our first pipeline!

First of all, we need our application under a Version Control System. In this post, I showed how to create a local repository for a dotnet core application, and **here **I show how to push the code to a remote repository on Azure Repos.

With the repo in place, go to Pipelines on the left menu, and then click on New pipeline:

Image by Author

In the new window, choose your repository. In this example we are using the Azure Repo that we created on the post mentioned above:

Image by Author

Choose the **Starter pipeline. **If you already have your own YAML, you can choose the Existing option.

Image by Author

Azure DevOps Pipeline anatomy

Azure DevOps will create a skeleton of a pipeline. It is quite simple, but enough to start. Let’s see what we have there:

**Trigger — **This is the configuration about which branch(or branches) our pipeline will be triggered if we have new commits. Normally this is tied to your branch strategy, by default it is created to the default branch in the chosen repository.
**Pool **— The pools are a set of machines that run our build inside. Azure DevOps has two pools when it’s created. The **Default **is a pool in which you can add your own agents. **Azure Pipelines **on the other hand is a SaaS agent that Microsoft provisions.
**Steps **— Finally it starts to be fun, we have the steps! They are instructions that we send to agents in order to complete our pipeline. In some cases, they can be simple command-line scripts, but we can also make use of “packaged scripts” in the Microsoft marketplace, also known as Tasks.

Image by Author

To prepare the pipeline for dotnet core, we will change the pool to windows-latest, as I am using a Windows machine, I want to keep the same.

And we will also use a dotnet core task, with the publish command. It will prepare all the artifacts necessary to deploy our application.

In the next step, we will use an Azure Web App task, that will deploy in deploy in App Service previously created in Azure. Pay attention that a service connection is required here, so make sure that the name matches yours.

Below there is all you need to create this pipeline. Simply delete the content created automatically and replace it.

How to create an Azure DevOps pipeline with stages

Now let’s make it a bit more complex :). The pipeline above will cover some simple scenarios, mostly for testing and learning. However, if you need something more robust, you will need to organize it in a better way.

There are two resources on Azure Pipelines that will enable it: Stages and jobs.

What are Jobs in Azure DevOps?

In a few words, **Jobs **are sets of **Steps **that run together. Every pipeline has at least one job, even if you do not specify one, by default your steps will be encapsulated in a job.

There are multiple scenarios where the use of multiple jobs is beneficial, as per separate responsibilities, running different jobs in parallel, and so on.

There is one special type of job for deployments. It is recommended to use it as brings some benefits as Deployment history and the possibility to use pre-configured strategies, such as canary, blue-green, etc.

To specify a job, you can simply add this to your pipeline:

What are Stages in Azure DevOps?

As we saw above that Jobs are sets of Steps, so guess what? Stages are sets of… Jobs!

To determine the boundaries of your deployment, pausing and validating before moving on, Stages must be used.

I recommend structuring your pipelines in the following manner:

One stage to gather all artifacts that will be used during the deployment, also known as a “build stage” (in some cases, you don’t need a real build generating binaries, as some languages don’t produce them, as IaC languages or even PHP for example).
Different deployment stages for different environments, for example, Development, UAT, and Production. It will allow you to validate the quality of your code during the journey toward production.

The stage structure is similar to the job syntax, and contains one or more jobs:

Below we can see the conversion of the simple pipeline to a complete pipeline using **Jobs **and Stages:

Key takeaways

Azure Pipelines is a powerful and very flexible tool that can technically support all programming languages, clouds, and technologies. It is part of Azure DevOps ecosystem, having native integration with with the other tools, do not require external integrations, however it also supports if required, and it makes quite easy to configure and increase productivity from the first moment you start to use it.

I hope this would help you, and see you in the next post!

👋 If you find this helpful, please click the clap 👏 button below a few times to show your support for the author 👇

🚀Join FAUN Developer Community & Get Similar Stories in your Inbox Each Week

How to deploy Management Groups with Azure Bicep and Azure DevOps

Mon, 26 Sep 2022 15:04:40 +0200

How to deploy Management Groups with Azure Bicep and Azure DevOps

If you have several Subscriptions on your Azure Tenant, Management Groups can be very handy to organize them. Check in this post, on how to deploy Management Groups using Azure Bicep.

Photo by Ian Battaglia on Unsplash

What are Azure Scopes?

According to official Microsoft documentation “Scope is the set of resources that access applies to.” This is used to have granularity when assigning permission in your Azure resources. The majority of Azure resources are deployed into the Resource Group scope, and when we use Azure bicep, this is the default. But there are four levels of scopes in Azure as we can see in the image below:

Azure Scope levels — Image from Microsoft Docs

With that said, when we are deploying a resource different than those deployed at the resource level, we need to specify against which scope we are running it. For Management groups, the scope must be the **tenant. **In the following sessions, we will see how to set it up.

Bicep scope to deploy Azure Management Groups

As said before, the default scope in an Azure Bicep script is the resource group. For most traditional resources such as App Services, or Storage Accounts, it is not necessary to specify it, but to deploy management groups it is necessary to specify the tenant as the target scope:

Image prepared by Author

Below it’s possible to see the code necessary for the most basic creation of a Management Group:

How to call an Azure Bicep template at the Management Group scope with Azure CLI and Azure DevOps YAML pipelines

To run deployments against tenant scope it is also necessary to specify it in Azure CLI

Below there is an Azure DevOps YAML pipeline with the task configured to deploy the bicep file created above:

SPN permissions to deploy Azure Management Groups

By default, the Service Principal Name does not have permission to deploy tenant resources. You need to grant it at the root scope “/” to make it work.

In this case, the error below will show up:

ash AuthorizationFailed: The client with object id does not have authorization to perform action 'Microsoft.Resources/deployments/validate/action' over scope '/providers/Microsoft.Resources/deployments/main' or the scope is invalid.

How to elevate user permissions as Azure AD Global Administrator

First, you need to elevate your permissions as user Global Administrator into Azure AD:

How to grant Service Principal Name permissions to deploy Azure Management Groups

After setting up your permissions as Global Administrator, you are able to set your SPN with the correct permissions:

The best Azure Management Groups naming convention

It is also crucial to properly name your Management Groups, making them easy to maintain, especially if you are adopting Infrastructure as Code with automated pipelines. Also, if you have multiple directories, you also need to efficiently identify which directory a particular management group belongs to.

My friend @DevJevNL has an excellent proposal to tackle the naming convention in a series of posts, here is his suggestion for Management Groups.

Conclusion

Although it is a very simple process, there are some tricks to deploying management groups. In this post, I tried to clarify all the necessary steps to deploy it. Below it is possible to visualize the management group deployed in our Azure Tenant.

Image prepared by Author

References

https://learn.microsoft.com/en-us/azure/role-based-access-control/elevate-access-global-admin Enterprise-Scale/EnterpriseScale-Setup-azure.md at main · Azure/Enterprise-Scale *This article will guide you through the process of configuring permissions in your Azure environment to enable ARM…*github.com

Creating Infrastructure as Code for Azure with Azure Bicep step by step

Wed, 23 Mar 2022 10:54:57 +0100

Creating Infrastructure as Code for Azure with Azure Bicep step by step

Photo by Arisa Chattasa on Unsplash

One of the natural steps towards DevOps and Cloud Adoption is to create your Infrastructure using Code. For Azure, the most convenient language for this is Azure Bicep which abstracts several aspects of this complex process, and supports more resources available in Azure than any other tool, making it one of the most powerful Infrastructure as Code tools for Azure.

What is Azure Bicep?

According to Microsoft Documentation, this is the definition of Bicep:

Bicep is a domain-specific language (DSL) that uses declarative syntax to deploy Azure resources. In a Bicep file, you define the infrastructure you want to deploy to Azure, and then use that file throughout the development lifecycle to repeatedly deploy your infrastructure.

In summary, this is an abstraction over ARM templates which simplifies A LOT the development of Infrastructure as Code. Its syntax is much simpler and less verbose than ARM but still has all its capabilities.

As bicep is a DSL, we can consider ARM as an Intermediate Language and we can “build” bicep into ARM to perform our deployments. A curiosity is that the name bicep is a play on words for ARM (Azure Resource Manager) 😄.

How to create an Azure Resource Group with Azure Bicep?

Usually, one of the firsts resources on Azure that we need to create is a Resource Group, as it will group all other resources, and make it easier to administrate.

Create an Azure Resource Group using Azure Bicep it is quite simple, you need just the name of the resource group and the location that it will be created. Additionally, it is necessary to define the *targeScope *as a subscription, it will be explained in more detail below.

It is possible to deploy it in different ways, using Azure PowerShell, Azure CLI, or even Azure API’s. In this example, we will use Azure CLI to execute the deployment.

How to execute an Azure Resource Group deployment with Azure Bicep and Azure CLI?

On the latest versions of Azure CLI, there are different commands to run deployments against different scopes on Azure. To create Azure Resource Groups, it is necessary to use the *subscription *scope, which was also defined on the bicep template above.

The command used is az deployment sub create :

After running the script, we can check the resource group with the command az group show --name rg-bicepdemo :

Image by author

How to create an Azure Storage Account with Azure Bicep and Azure CLI?

When we will create a “common” resource, like Storage Account, Resource Group scope must be used. This is the default behavior of bicep, so it is not necessary to specify like it was for Resource Group creation.

To create resources on Resource Group scope, the Azure CLI command used must be az deployment group create :

To check the Storage Account creation run the command az storage account show --name stbicepdemo

Image by author

As they are in different scopes, it is not possible to run both bicep files with one command, however, I prepared a PowerShell script to run them in sequence:

Conclusion

Azure Bicep is one of the most promising Infra as Code tools for Azure, and it is quite simple to create new templates for it. Its syntax is very simple and easy to learn, so why not give it a chance? :)

See you in the next post! [

](https://faun.to/bP1m5)

If this post was helpful, please click the clap 👏 button below a few times to show your support for the author 👇

First steps with Azure: Creating an Azure Resource Group with Azure CLI

Mon, 07 Mar 2022 18:25:56 +0100

First steps with Azure: Creating an Azure Resource Group with Azure CLI

An Azure Resource Group is a container that holds related resources for an Azure solution. This is an important resource in Azure and in this post, I will continue the series of Basic commands for Azure CLI and show how it is possible to create Azure Resource Groups using Azure CLI.

Photo by Simon Infanger on Unsplash

What is Az CLI?

Az CLI or Azure CLI is a command line interface with Azure. It can be used in any command line prompt, including PowerShell and Bash, and it allows the creation of resources and resource groups, executing actions, and much more possibilities. In the previous post, I showed the basic commands to connect and configure your Azure Subscription using Azure CLI, if you don’t know how to configure your Azure CLI local environment you can check it here.

What is an Azure Resource Group

According to Microsoft documentation, this is the definition of an Azure Resource Group:

A resource group is a container that holds related resources for an Azure solution. The resource group can include all the resources for the solution, or only those resources that you want to manage as a group. You decide how you want to allocate resources to resource groups based on what makes the most sense for your organization. Generally, add resources that share the same lifecycle to the same resource group so you can easily deploy, update, and delete them as a group.

In practice, we can consider it as a “Directory” on Azure where you can group your resources. There are many ways to organize it, the most common is to make them small with resources of the same context.

For more information, you can check the official documentation here.

Az CLI commands for Azure Resource Group

Azure Resource Group - Summary of commands

Below there is a summary of all commands used in this tutorial. You can copy it and replace the values between <> to your own values :)

How to use Az CLI Help?

It is also possible to check all commands related to resource groups with the command az --help . For more details about how to use the help check this post where I also show how to login into Azure Subscription with Azure CLI.

az group help — Image by author

How to list existing Azure Resource Groups with Az CLI

The command used to list Azure Resource Groups is az group list . It will return a JSON by default. To have a more human understandable output you can add --output tablein front of the command or any other output of your choice. The final command would beaz group list --output table .

az group list — Image by author

How to create an Azure Resource Group with Az CLI

To create a new Azure Resource Group, it is necessary to provide two required information:

Location: The Azure Region where your resource group will be created.
**Name: **The name of your Resource Group

The final command is az group create --location <location of your resource group> --name <name of your resource group> . Replace the content between <> with your own values .

az group create — Image by author

To check additional options use the command az group --help .

After creation, run again the command az group list --output tableto check if the resource group is shown.

az group list — Image by author

How to show details of an Azure Resource Group with Az CLI

To check the details of a specific Resource Group, you can run the command az group show --name <your resource group name> . This command returns basically the same information after the creation command.

az group show — Image by author

How to delete an Azure Resource Group with Az CLI

The delete command will prompt if you are sure about executing the delete operation. This is because all resources created under this resource group will also be deleted, so run this command only when you want to delete them as well.

After the conclusion of the command, after running the list command the created resource group is not listed anymore.

Conclusion

The Resource Groups are an important part of Azure and automating its creation is a fundamental part of a DevOps process.

See you in the next post! [

](https://faun.to/bP1m5)

If this post was helpful, please click the clap 👏 button below a few times to show your support for the author 👇

The entry gate to Azure: Configuring your Azure subscription with Azure CLI

Fri, 25 Feb 2022 16:51:35 +0100

The entry gate to Azure: Configuring your Azure subscription with Azure CLI

Azure CLI is a command line interface with Azure. It can be used in any command line prompt, including PowerShell and Bash, and it allows the creation of resources and resource groups, executing actions like an Azure Data Factory pipeline, and much more possibilities. In this post, I will show the basic commands to connect and configure your Azure Subscription.

Photo by Jan Tinneberg on Unsplash

How to set an Azure Subscription with Azure CLI

Below is a summary of all commands used

Now let’s see the details of execution for each command.

How to know which commands to use in Azure CLI

At least in the beginning until you get familiar with the commands a very useful tip is to make use of the help command. It’s quite simple and you just need to add --help or -h after the command. The most basic would be to run it after az like az --help . It will show all subgroups and commands available for the tool.

Image by author

The easiest way to login into Azure is using the interactive mode. It will open a new tab on your browser with the authentication page.

To do it run the command az login and choose your account:

Image by author

By default, after choosing your account it will prompt a list of available subscriptions on that account. These subscriptions can also be listed with the command az account list, described below.

How to set a subscription with Azure CLI

First, we need to see which subscriptions are available. To do it run the command az account list , this command will display all subscriptions available in your logged accounts.

Image by author

Once you have your subscriptions id, you can connect to it through the command az account set --subscription . After this command, you can confirm if the subscription was settled by running the command az account show

Ìmage by author

Conclusion

Although the commands are simple, they are very important, as they are an entry gate to interact with Azure. [

](https://faun.to/bP1m5)

If this post was helpful, please click the clap 👏 button below a few times to show your support for the author 👇

Criando Azure Resource Group com Azure CLI

Thu, 24 Feb 2022 18:26:54 +0100

Criando Azure Resource Group com Azure CLI

O Azure CLI é uma interface de linha de comando para o Azure. Ele pode ser usado com PowerShell ou em bash, com ele é possível criar recursos, resource groups, executar ações no Azure entre outras coisas. Aqui vou mostrar os comandos básicos para conectar ao Azure utilizando o Azure CLI e como criar um resource group.

Photo by Milad Fakurian on Unsplash

No vídeo abaixo é possível acompanhar todo o processe de conexão ao Azure e criação do Resource Group usando Azure CLI. Após o vídeo estão os comandos utilizados e um breve resumo sobre seu uso.

Como conectar ao Azure com Azure CLI

Azure CLI - comandos para Account

Abaixo estão todos os comandos utilizados para realizar login e configurar a Azure Subscription para ser utilizada

Agora vamos ver os detalhes de execuçao dos comandos:

az -h — Lista os comandos disponíveis no Azure CLI

az login- Abre a página de login no browser para selecionar a conta que será conectada

az account list - Lista as subscription disponíveis az account set -subscription - Seleciona a subscription que será usada az account show - Exibe as informações da subscription atual

Como criar um Azure Resource Group com Azure CLI

Azure CLI — comandos para Resource Group

az group -h - Lista os comandos disponíveis para Resource Groups az group create -h - Lista os argumentos necessários para criar um Resource Group

az group create --location -–name - Cria um Resource Group

az group list --output table - Lista Resource Groups da subscription em formato de tabela

az group delete --name- Apaga o Resource Group

Creating and publishing PowerShell Modules to Azure Artifacts with Azure DevOps YAML Pipelines

Thu, 03 Feb 2022 00:02:46 +0100

Creating and publishing PowerShell Modules to Azure Artifacts with Azure DevOps YAML Pipelines

PowerShell Modules are very useful and can save a lot of time if well designed and created. Check on this post how to create and pack a basic PowerShell Module, and publish it into Azure Artifact using Azure DevOps YAML Pipelines.

Photo by Trnava University on Unsplash

Creating PowerShell Module and PowerShell Module Manifest

Creating a PowerShell Module - psm1

A PowerShell Module is a way to organize and pack a set of PowerShell components to be reused or shared. The most common components to be shared are functions.

In this example, we will create the module Example.Module with the extension psm1 , and include a PowerShell function.

Creating a PowerShell Module Manifest - psd1

To create the Module Manifest, it is possible to run the cmdlet New-ModuleManifest . It will generate the psd1 manifest file with default configurations.

The most important variables are NestedModules and RootModule which must contain the name of the psm1 file.

Creating a nuspec file for the PowerShell Module

To create the nuspec file you need to run the command nuget spec Example.Module . It will generate a base file that it is possible to replace with your values.

Making the PowerShell Module version dynamic

In both psm1(1) and nuspec (2) files there is a variable $(version) . This variable will be replaced by the version defined in the deployment pipeline. It uses the Azure DevOps macro syntax to consume the variables.

Image by author

Creating an Azure Artifacts Feed

For instructions on how to create a feed, you can check this post where I also show how to push a dotnet Core package to Azure Artifacts:

How to send .Net Core NuGet packages to Azure Artifacts | by Wesley Camargo | Medium

Deploying PowerShell Module with Azure DevOps YAML Pipeline into Azure Artifact

Version Number to update PowerShell Module

To simplify this example, we will provide the version number of the module by Azure DevOps parameter(1). Then we populate a variable with the value of the parameter using template expression syntax(2). In a future post, I will how to bump this number automatically.

Image by author

Replacing version number in nuspec and module manifest

The next step is to replace the variable $(version) mentioned above with the version number. To do it will be used the replace token task. Note that we specify the extensions of the manifest and nuspec files.

Image by author

Packing PowerShell Module with NuGet

To pack the module is used the NuGetCommand task, and specified the nuspec file. After generating the NuGet package it will be published as a pipeline artifact - be careful not to confuse with Azure Artifacts, we are almost there but not yet :) - which will be consumed in the next stage: deployment.

Image by author

Pushing PowerShell Module to Azure Artifacts

Finally, in the deployment stage, we will push our package into our Azure Artifacts. It will also use the NuGet Command task to push it, sending the nupkg generate in the build stage. It is also necessary to provide the name of the Azure Artifacts Feed.

Image by author

Complete azure-pipelines.yml to publish PowerShell Modules to Azure Artifacts

[

](https://faun.to/bP1m5)

If this post was helpful, please click the clap 👏 button below a few times to show your support for the author 👇

Delivery as Code with Azure DevOps Release Engine

Fri, 28 Jan 2022 19:05:03 +0100

Delivery as Code with Azure DevOps Release Engine

Delivery as Code is the concept of having all necessary configurations of your delivery defined in a file, which can be kept under source control and have all good practices applied to application development over it. It ensures that your deliveries are consistent and reliable.

Photo by Jackson Hendry on Unsplash

Everything as Code

In the past years, everything is becoming “as Code”. It started with Infrastructure as Code, aka IaC, and today there are many languages supporting it, since the AWS Cloud Formation, passing through Terraform and Pulumi, that are Cloud agnostics, until the Azure ARM Templates and his newborn son Azure Bicep.

But there are also other uses of as Code, such as Database as Code, Pipelines as Code, Monitoring as Code, Security as Code, and so forth, this list is big and still growing.

Why have my definitions as code?

One of the reasons behind this is if you have the instructions to create your environment versioned, you can easily reproduce the steps to create it. Imagine that for some reason you lose your infrastructure environment, in some minutes you can spin up a new environment based on the versioned definitions.

Another reason is that as you can easily create a new environment, you can apply development good practices on it. For example, you can deploy a new environment using the versioned scripts, and run Unit Tests against it. With frameworks like Pester this is easy to implement.

What is Delivery as Code?

The advantages of having definitions under source control are clear, so why do not apply them in the Delivery of the software?

With that in mind and my passion to automate everything that I can, I started to develop an easy way to configure the releases and make them as simple as possible, based on my experience working in the DevOps field.

To configure a new delivery, instead of configuring manually or having scripts of how to implement it, you just need to inform what you need and all the complexity of the implementation will be abstracted. This is the declarative approach very keen on Infrastructure as Code, which allows you have indepotent deployments.

What is Azure DevOps Release Engine

Azure DevOps Release Engine is an approach developed to support Delivery as Code. It was built on top of Azure DevOps YAML Templates and can be used to deploy any kind of technology, with any kind of tool anywhere.

It means that you can deploy languages such as:

dotnet Core
nodeJS
Java

Using any tool like:

Azure Bicep
Terraform
Cloud Formation
Pulumi

In any platform:

Azure
AWS
VMWare
OnPremisses data centers

How does Azure DevOps Release Engine work?

Azure DevOps Release Engine was created using Azure DevOps YAML Pipelines as a support tool. It consists of a centralized repository where they centralize deployment definitions - the instructions of how to deploy - are versioned and Client/Application repositories which must have the declaration of what should be deployed.

The client application repository must have a YAML pipeline that extends from the main YAML in the centralized repository - here there is a post explaining how the extended pipelines work. The main file is responsible to parse the instructions provided and deciding which definition must be used.

Centralized Release definitions

The definitions in this repository can be reused, saving time and allowing all deployments of the same kind of component to be deployed in the same way, which leads to more consistent and less problematic releases, as the deployment instructions were well tested.

Client/Application repositories

The client repositories are the repos where your applications are versioned. It can have any sort of resource to be deployed and to consume the Release Engine you just need to create a YAML file following the Azure DevOps Release Engine schema.

How to set up Azure DevOps Release Engine?

Importing the repo into Azure DevOps

In the Azure DevOps Release Engine repository at GitHub copy the link to the repository following the sequence below:

Copy the repository URL — Image by author

Click in Import repository — Image by author

Paste the link and import — Image by author

After it you should have the repository in your Azure DevOps repos, with the structure below:

Image by author

How to set up the application to consume the Azure DevOps Release engine?

Repository structure

In a dotnet Core application repository add the file azure-pipelines.yml in the root folder, and one file corresponding to each environment that will be deployed in a “variables” directory. I recommend having the source code under a different directory to keep the repository cleaner and tidy, in my case it’s under the srcdirectory.

Image by author

Creating the azure-pipelines.yml file

This file contains all instructions on what resources need to be deployed. I will add the complete file in the end so it will be possible to just copy it ;).

“Importing” a repository in Azure DevOps — Image by author

The beginning of the file contains pipeline configurations that will be the same in most of the pipelines. The declaration of Release Engine as a repository resource is very important, as the templates will be consumed from this resource that will be “imported”.

Azure DevOps Release Engine settings

With Release Engine imported, it is necessary now to extend from the main.yml file (1). This file is basically a parser that will read the information provided as parameters and redirect to the correct file. I will deep dive into this file in future posts.

Image by author

The most important section is the parameters. This section contains all information that will be provided to the engine. As mentioned above this information will be parsed by the main file, and according to the resource types provided. call the right definition to deploy it.

**Release settings **Inside parameters, the session **settings (2) **contain the configuration of your release.

**Azure settings **In the current version, it is possible to enable or disable the deployment of your infrastructure, application, and build (3). Here it is also necessary to provide the directory of your variable files. The variables will be explained later. It is possible to provide the values straight on this file,

**Environments **It is also necessary to provide information about your Azure environment (4), such as service connection, resource group where the resources will be deployed, and so on. If the parameter **new **under resource group is provided, it will also create the resource group.

Last but not least, you must declare which environments will be deployed. The way that it was built will create exactly the same configuration for all environments, which ensures that the implementation in your production environment will follow the same steps as in the others, increasing the reliability of the delivery, as was mentioned at the beginning of this post.

Configuring resources to be deployed with Azure DevOps Release Engine

The latest section in the azure-pipelines.yml is the resource. This is a list of resources that will be deployed in this release and its configurations. In this example, we are going to deploy a dotnet Core application into an Azure Web App. Let’s go through the main items in this section:

Image by author

**name **- This is the name of the web app that will be created into Azure. This is provided through a parameter that is stored in the respective variable file of the environment that is running. It means that you can deploy in different environments like dev, uat and prd with unique names, avoiding conflicts.

**type **- The type is parsed inside the main.yml file and calls the right deployment definition for the type provided. This example is a dotnet Core application but it is possible to deploy any resource with the deployment definition already implemented on Release Engine.

runName - Run name is the name of the Azure DevOps job. This is necessary to make the run unique and have the possibility to link dependent resources.

enabled - It is possible to easily disable the deployment of this resource.

deploy.type - Inform which type of deployment it is. In this example, the dotnet Core application will be deployed into an Azure Web App.

deploy.infrastructure.servicePlanName - Similar to the name, we are informing the name of the service plan that will be used by our Azure Web App.

Configuring variables for Azure DevOps Release Engine

As mentioned before, it is necessary one variable file per environment. The variable file allows having resources with unique names in the environments.

This file will be loaded automatically by the engine, so the only action necessary is to add the file following the patter {env-name}-vars.yml, where env-name is the name defined at azure-pipelines.yml

After being loaded by the engine, the variables will be available like any other variable in Azure DevOps. In azure-pipelines the variables are accessed using the macro syntax $(variable-name).

Complete template

Below there is the complete template for dotnet Core and Azure Web Apps. You can also check the repository with more examples and the implementation of the engine on Azure DevOps Release Engine repository at GitHub.

Takeaways

The Azure DevOps Release Engine simplifies the release configuration, abstracting the release definition providing an “as Code” approach for your Delivery.

This is a work in progress but built on top of a very mature process based on real-life experience, solving real-life problems that we face in our day-by-day activities.

I hope it can help, and see you in future posts! [

](https://faun.to/bP1m5)

If this post was helpful, please click the clap 👏 button below a few times to show your support for the author 👇

How to extend an Azure DevOps YAML Pipeline Template

Thu, 30 Dec 2021 17:28:36 +0100

How to extend an Azure DevOps YAML Pipeline Template

If you need to control what is allowed during your deployment, extending the YAML Templates is the easiest way to know what is being done on your deployments.

Photo by Zonduurzaam Deventer on Unsplash

Azure DevOps YAML Templates

YAML Templates are the smartest way to create your pipelines. It allows you to break down the pipelines into small pieces reusing your logic, which leads to some best practices as Don’t Repeat Yourself and Dependency Inversion.

It’s also possible to use the *Declarative *instead of the Imperative approach, therefore who will consume your pipelines does not need to know the details of how to deploy an application, but just say what is necessary to deploy.

With these principles being correctly applied, it is possible to build a reliable and robust way to deploy new applications, but who will consume it doesn’t need to be a DevOps expert.

Creating an Azure DevOps YAML Templates

Parameters on Azure DevOps YAML Templates

On top of the template, the first section is parameters . As our goal is to make the templates reusable, makes complete sense to have a way to change values according to which application is consuming the template. To keep it simple in this example, it was parametrized the application name.

`ash parameters:

name: applicationName type: string `

Stages on Azure DevOps YAML Templates

To have an extendible template, it must be created on the level of the stages. This example has two of them, one for build and one more for development environment. Below you can check the full template script:

Extending Azure DevOps YAML Templates

Configuring triggers on Azure DevOps YAML Templates

On top of the consumer file, in our case azure-pipelines.yml , we need to declare the triggers of the pipeline. It is mandatory to include at least one value for branch(it can be * if needed), or none . You can also set the path where your files are, and your pipeline will run only when those files change.

ash trigger: branches: include: - main - develop paths: include: - src/*

Referencing Azure DevOps YAML Templates

To extend a template, you need to provide the relative path for your template file. In this example, there is a templates directory with the file template.yml so on the consumer file, it’s necessary to provide the template path in the format ./templates/template.yml .

Image prepared by author

We also need to provide the parameter defined in our template. To this, you just need to add the parameters section right below the template, the name of the parameter, and of course, the value for it. Below there is the complete “consumer” file.

Conclusion

In this post, I introduced the basic concepts of how to extend Azure DevOps YAML Pipelines. This is easy to implement but can yield good fruits in future implementations.

In future articles, I will deep dive into this subject and show the full potential of this approach :).

How to suppress warning and error messages in Azure CLI

Mon, 20 Dec 2021 09:02:01 +0100

How to suppress warning and error messages in Azure CLI

In Azure CLI some commands can return errors and warning messages, which can cause troubles on your script, so check out how to suppress it

Photo by Muhammad Daudy on Unsplash

How to suppress warnings on Azure CLI

Command group is in preview and under development

In some cases the CLI used can be under development and show the message: Command group is in preview and under development. Reference and support levels: [https://aka.ms/CLI_refstatus](https://aka.ms/CLI_refstatus)

At the moment that I am writing this post, the az devops CLI variable-group for example is always returning the message above, which have been causing troubles and breaking my pester tests :)

Photo prepared by author

Quiet mode on Azure CLI

For Azure CLI, the simplest way to make the commands “quiet” is by adding the global parameter --only-show-errors in your command. **It will suppress only the warnings **and keep the message in case you have any errors during the execution.

So in our example for az pipelines variable-group our command will be like this:

ash az pipelines variable-group list --org $organization -p $project --only-show-errors

How to suppress errors on Azure CLI

In some cases, you may need to also suppress the errors during your script execution and there is one more option for it. I would recommend this approach only if you really do not care if something went wrong during your script execution.

Redirecting the console output STDERR

A common scenario to redirect streams is when the symbol > is used to write the output of the command in a file. It redirects the standard output - STDOUT - to the file added as a parameter.

In the situation that is necessary to suppress the errors, we need to redirect the output of the error stream — STDERR.

To do it you simply need to add 2>nul after the CLI command, which means that the second stream (STDERR) will be redirected to null .

So in our example for az pipelines variable-group the command will be:

ash az pipelines variable-group list --org $organization -p $project 2>nul

Conclusion

Although this topic looks simple, sometimes it’s not so easy to find this information in the documentation, especially regarding the error suppressing topic, and also the comparison of both approaches is not documented.

I grouped both, how to suppress warnings and errors in one place, so would be to choose which option suits better in each scenario.

References

Redirecting error from Command Prompt — Visual C++ | Microsoft Docs

https://github.com/Azure/azure-cli/issues/13384

Creating Azure Databricks with Bicep and Azure DevOps YAML Pipelines

Fri, 20 Aug 2021 19:45:12 +0200

Creating Azure Databricks with Bicep and Azure DevOps YAML Pipelines

Continuing the DataOps Automation series, in this post I will demonstrate how to create your Databricks workspace using Infrastructure as Code with Bicep and Azure DevOps YAML Pipelines to deploy it in Azure. Let’s check it out!

Photo by Benjamin Jopen on Unsplash

In the previous articles, you can check how to create an Azure Data Factory with git integration using Bicep and also the easiest way to create a CI/CD pipeline for Azure Data Factory

At the end of the post, you can check a complete YAML pipeline read to be used!

What is a Bicep template?

If you are not familiar with Bicep, this is a DSL for the ARM Templates, that has a cleaner syntax, better support to modules, and other great features. It also has a nice Visual Studio Code extension that helps a lot in building templates from scratch.

Nice extension for Visual Studio Code! — Image from Microsoft Docs

When you run a deployment using a bicep template, it transpile the file and generates a native ARM Template, similar to what happens between Typescript and Javascript.

If you want to learn more about it, check the official documentation here.

Creating the Infrastructure as Code for Azure Databricks

Creating a Resource Group with Azure CLI

To deploy our Databricks workspace we need to have a Resource Group created in your Azure Subscription. The easiest way to do that is through Azure CLI. To do that you need to log in with the command az login and then run the following command:

az group create -name RG-Databricks -location northeuropeReplace the parameters name and location for those that best suit your needs.

Creating Bicep templates for Azure Databricks

The Bicep template for Azure Databricks it’s quite simple and has 4 main sections:

Parameters:

Workspace Name: If you do not provide a value, it uses a uniqueString function, which ensures that it will always be the same, as this is based on the resource group id.
Location: By default, this is using the resource group location
SKU: The default value is trial, so be sure to replace it in your real-life scenario ;).

**Variables: **The template also has the variable managedResourceGroupName . This resource group will be managed by Azure Databricks to create your clusters later on.

**Resources: **Here we have only one resource that is the Databrics Workspace. This is consuming the API version 2018–04–01 .

**Output: **This section contains the variables that you need to expose. It means that they can be accessed by other processes later on in your deployment process.

Deploying Bicep with Azure DevOps YAML Pipelines

Building Bicep Template

As I said before, it is possible to “build” the Bicep template transpiling it into an ARM Template. So to ensure that our template is valid, let’s do it!

To build it, you will also use the Azure CLI. The command is also quite simple:

az bicep build –file –outdir To create an Azure DevOps YAML Pipeline, we will run the command above in an AzureCLI task:

After transpile the Bicep into ARM Template, it will deploy as a Deployment Artefact read to be consumed:

Image prepared by author

Build your Bicep is a good practice, as you can validate that there is no error in it and gather all necessary files to be deployed.

Deploying Bicep with YAML Pipelines

With the artifacts generated in the build step, now we are ready to deploy them in our environment. For that it is necessary to add one more stage into the build YAML, with the instruction to run the ARM Template:

It consumes the task AzureResourceManagerTemplateDeployment@3 that is responsible to run the ARM Template into Azure Subscription. You can have more information about this task here.

This stage represents the development stage. To deploy it in other environments, you just need to copy it and replace the values according to your environments :)

Real life example

To check these examples in more realistic situations, I have been working in a GitHub repository with examples of pipelines configured and working. You can check that in the DevOps Nights GitHub here: devopsnights/azuredevops-yaml-quickstart-templates (github.com)

Databricks YAML Pipeline

As promised below you can check a full Azure DevOps YAML Pipeline configured and working:

Conclusion

Bicep has become a very relevant tool for Azure and has a good potential to be the default tool for Infrastructure as Code in Azure. So start to be familiar with it now, will be very relevant in the future :)

I hope this post could help you, and see you in the next post!

Azure Data Factory CI-CD made simple: Building and deploying ARM templates with Azure DevOps YAML…

Fri, 13 Aug 2021 18:24:55 +0200

Azure Data Factory CI-CD made simple: Building and deploying ARM templates with Azure DevOps YAML Pipelines

The easiest way to publish/deploy Azure Data Factory artifacts

How to Create Azure Data Factory using IaC

Infrastructure as Code

There are tons of reasons to use IaC in your projects, as you can check here. One of them is that Infrastructure as Code is the easiest and fast way to implement your environment. So, as we want to keep our ADF deployment simple, why not use it :)?

In this example, I will use Bicep templates to deploy our Data Factory. If Bicep is new to you, it is basically a DSL (Domain Specific Language) to make things easier for ARM Templates users.

Bicep is has a good extension for VS Code— Image from Microsoft Docs

In this post, you can check how to create the Bicep file for Data Factory with git integration that will be used to deploy the ADF.

It will create a link between your Azure Data Factory and your git repo (it works on Azure DevOps and GitHub), so when you create a pipeline in your Data Factory, it will also be versioned into the git repo.

Git Repository

Repository Structure

The repo structure will depend on each project. One of the best practices is to keep all necessary code to deploy a project in the same repo. Due to that, I like to create a structure where I have the name of the component, and always an “src” folder underneath. In this case, we will make the src folder as “Root folder” in the git integration process.

Repository Structure — Image by author

Required files to build ARM Templates

Some files are necessary to have in our repo to generate the templates. These files need to be added to the src folder, and they will be referenced during the build stage.

In the src folder create the file package.json. It contains the metadata of the package that will be used to build the ADF Artifacts.

In the same folder also create the file publish_config.json with the content below. It will not impact the generation of the ARM Templates, but it’s necessary to run the build:

The last file is arm-template-parameters-definition.json. This contains the definitions of your ARM Parameters. I won’t go into details, as it requires a dedicated post for it. For the initial version, you can just create the content below:

After create git integrate and all necessary files this is how your repo will look like:

Repository structure for Azure Data Factory — Image by author

How to create Build YAML Pipelines for Azure Data Factory

No doubt that pipeline as code will be the future of pipeline definitions, with the capacity to be under version control and reusability.

Variables

The first configuration that is necessary is the variables. They will be used both for build and release later on.

The most important variables during the build stage are:

workingDir - This is the src directory. There must be the required files mentioned above.
dataFactoryResourceId - Fill this with the resource Id of your ADF. It’s a good idea to make it parametrizable to work in different environments

Build and Deployment Variables — Image by author

Building Data Factory

As mentioned above, we need to consume the ADFUtilities NPM package in the build process.

In the first two tasks, NodeJS is configured in the build agent. Pay attention to workingDir variable that we mentioned in the Variables section.

In the latest two tasks, we are calling the NPM package to Validate and “Build” our Data Factory and Generate the ARM Templates. In the last task “artifacts” is the relative output directory. It means that it will be the output directory to the ARM Templates.

Build tasks for Azure Data Factory — Image by author

In the next tasks, we are copying the ARM Templates from the output to the staging directory, and “building“ the bicep files into ARM Templates. These ones will be used to create the Azure Data Factory Workspace.

Build tasks for Azure Data Factory — Image by author

After running the build pipeline, you will have the artifacts that will be consumed during the deployment stage:

Azure Data Factory artifacts — Image by author

Azure Data Factory ARM Templates — Image by author

How to create Release YAML Pipelines for Azure Data Factory

To deploy Data Factory we are using the run Once strategy. It will consume the artifacts created on the build stage

Development

When the git integration is enabled in development environment, as the code is produced in the workspace, there is no need to publish in this environment. It will deploy only the environment using the Infrastructure as Code templates. It also contains a dependency for the build stage.

Development environment — Image by author

UAT and Production

In UAT we have a dependency on the development environment. In Production, the dependency is on UAT. As in these stages, we need to deploy both, infrastructure and code, we will use a preDeploy and deploy job:

preDeploy stage — Image by author

deploy stage — Image by Author

When run it will first create the Infrastructure using the Bicep files in the preDeploy Stage, and then deploy the artifacts generated on the build stage. And this is the final result:

Deployment Pipeline — Image by Author

Key Takeaways

There is more than one strategy to deploy Azure Data Factory, I have tried most of them and they work very well, but this one from my point of view is the simplest and clean way to achieve fully automated deployments.

Below you can check the complete YAML used to deploy it. You can also check other Azure DevOps YAML Templates examples in my GitHub repo: devopsnights/azuredevops-yaml-quickstart-templates (github.com)

I hope it can be useful!

DataOps Automation — Creating Azure Data Factory with git integration using Bicep

Sun, 25 Jul 2021 00:48:52 +0200

DataOps Automation — Creating Azure Data Factory with git integration using Bicep

An important feature available in Azure Data Factory is the git integration, which allows us to keep Azure Data Factory artifacts under Source Control. This is a mandatory step to achieve Continuous Integration and Delivery later on, so why not configure this using Infrastructure as Code with Bicep in a fully automated way?

Image prepared by author

In the official Microsoft documentation there is a good topic explaining how to integrate Azure Data Factory with git, but through the Azure Portal. In this post, I will explain how to do that using Bicep, the new IaC language for Azure, to make it possible to include this step into your CI/CD process, and also how to deploy only on needed environments.

Azure Data Factory Git integration

Okay, but why do I need to use git on my ADF? To explain this, I’ll list some advantages that the git integration offers, taken from Microsoft’s documentation:

Source control

As your data factory workloads become crucial, you would want to integrate your factory with Git to leverage several source control benefits like the following:

Ability to track/audit changes.
Ability to revert changes that introduced bugs.

Partial saves

When authoring against the data factory service, you can’t save changes as a draft and all publish must pass data factory validation. Whether your pipelines are not finished or you simply don’t want to lose changes if your computer crashes, git integration allows for incremental changes of data factory resources regardless of what state they are in. Configuring a git repository allows you to save changes, letting you only publish when you have tested your changes to your satisfaction.

Collaboration and control

If you have multiple team members contributing to the same factory, you may want to let your teammates collaborate with each other via a code review process. You can also set up your factory such that not every contributor has equal permissions. Some team members may only be allowed to make changes via Git and only certain people in the team are allowed to publish the changes to the factory.

Better CI/CD

If you are deploying to multiple environments with a continuous delivery process, git integration makes certain actions easier. Some of these actions include:

Configure your release pipeline to trigger automatically as soon as there are any changes made to your ‘dev’ factory.
Customize the properties in your factory that are available as parameters in the Resource Manager template. It can be useful to keep only the required set of properties as parameters and have everything else hardcoded.

Better Performance

An average factory with git integration loads 10 times faster than one authoring against the data factory service. This performance improvement is because resources are downloaded via Git.

Git Integration is only for Development Environment

There are two flows recommended by Microsoft to perform CI/CD on ADF, (you can check it on this post), in both cases the development happens in the Development Data Factory workspace only, and the artifacts are promoted through CI/CD deployments. It means that we should not create this integration for non-dev environments like UAT or Production. I will also cover how to identify the environment and create the integration only for your development environment :)

Creating Bicep templates for Azure Data Factory step by step

To keep the explanation simple, I’ll show each stage of the bicep file. Each stage is a deployable file that you can execute individually with the command

*az deployment group create -f .bicep -g *In the end, we will have the complete file ready to use.

Creating Data Factory Workspace

In the first stage, we are creating only the workspace with the most basic configurations. You can see that for the location we are using the location of the Resource Group where we are executing this deployment.

Linking GitHub with Azure Data Factory

In this stage we achieve the first goal of this post: create the **git integration. **I include the parameters necessary to connect with a GitHub repo, and then a section into “properties”. Now lets check the parameters:

**accountName - **This is your GitHub/Azure DevOps account name.
**repositoryName **- Name of your repository, pay attention that you don’t need your account name at the beginning.
**collaborationBranc **- The branch that will be used to integrate your feature branches. It will depend on your branch stratety. For git flow, it should be developed branch, for GitHub Flow, master/main.
**rootFolder **- The folder in your repo where the databricks artifacts will be versioned.
projectName - This parameter is only used for Azure DevOps. It’s the Team Project name of your repo.
**type **- This parameter defines if your repo is hosted on GitHub or Azure DevOps. For GitHub, the value must be:

FactoryGitHubConfigurationFor Azure Devops the value is:

ash FactoryVSTSConfiguration

Image prepared by author

Checking the environments

As was explained above, we must create git integration in the development environment only. If you are using only GitHub or only Azure DevOps, this stage should be enough for your scenario. To check that, we need the following steps:

An environment parameter.
Extract the content of repoConfiguration to an external variable.
Use a ternary if to check if the environment provided is development. If it is, then use the variable, if not, use an empty group.

Image prepared by author

The bicep file of this stage:

Linking GitHub or Azure DevOps to Azure Data Factory

In case that in your scenario you have both Azure DevOps and GitHub, you can prepare your template to support that. This template is more flexible and can be used in more situations. Let’s check the differences from the previous one:

Included a “User Friendly” parameter to indicate the type of the repo.
Check the type parameter and create a variable with the expected value.
Create two configurations, one for GitHub and one for Azure DevOps.
Include another ternary if nested into the first one

Image prepared by author

Below you can check the final bicep template:

Real life example

To check these examples in more realistic situations, I have been working in a repository in GitHub with examples of pipelines configured and working. You can check that in the DevOps Nights GitHub here: devopsnights/azuredevops-yaml-quickstart-templates (github.com)

In this post, I am showing how to consume this Bicep Template into an Azure DevOps Pipeline.

I hope this post can help you to create your ADF in a more automated way, and see you in the next post!

References

Source control — Azure Data Factory | Microsoft Docs

Factories — Configure Factory Repo — REST API (Azure Data Factory) | Microsoft Docs

Microsoft.DataFactory/factories 2018–06–01 — ARM template reference | Microsoft Docs

DataOps Automation — Deploying Databricks notebooks with Azure DevOps YAML Pipelines

Tue, 15 Jun 2021 18:54:07 +0200

DataOps Automation — Deploying Databricks notebooks with Azure DevOps YAML Pipelines

In this post, I will show an easy way how to deploy your Databricks notebooks using Azure DevOps and YAML pipelines.

This will be the first of a series of posts, showing how to deploy code and infrastructure of Data Platform tools. I created GitHub repo to keep the examples, so if you want you can check that here: wesleycamargo/DataOps (github.com) 🙂

Image prepared by author

Project structure

The organization of this repo is based on components, so I’ll keep everything necessary to deploy some kind of component together. If you have a different structure, remember to update the yaml templates with your paths.

For databricks we have a /databricks and a /src folder, in the future it will be important to segregate from IaC code. You can have your project folders on this level, in my example, I have two notebooks inside a calculator folder`.

Project structure — Image prepared by author

Preparing artifacts

In the build stage, we will create a copy of our notebooks into a staging folder, and then publish them as build artifacts to be consumed later in the process.

We will copy all folders under /src folder, so when we deploy that into databricks workspace all folders will be created into the workspace as well.

If you run the pipeline now you already can see the artifacts and the structure:

Image prepared by author

Setting variables

In the deployment stage, we will need to consume some sensitive information like your databricks Personal Access Token. To avoid hard coding this information and to keep this example simple, let’s create some variables in the pipeline.

Image prepared by author

Deploy

To deploy we will use a special kind of job called deployment 🙂, there are some advantages to use this instead of the regular job, but I’ll do not explore them now.

In the first step, we are setting up the python version to 3.x, to that we are using a task to abstract this activity.

In the second one, we are setting app our databricks workspace. Basically, we are creating a .databrickscfg file with your token and databricks URL. To populate this file we need to consume the variables created before. So be sure that you have databricks.host and databricks.token create. We are also installing the databricks CLI to run on the next step.

And finally, in the last step we are importing the artifacts generated before using databricks CLI.

Below you can see the complete YAML template.

Hope that post can help you and see you in the next post! 😃

Azure Reports: Exporting Azure resources with Powershell and Azure CLI

Wed, 28 Apr 2021 17:31:36 +0200

Azure Reports: Exporting Azure resources with Powershell and Azure CLI

A common scenario when we work with DevOps and cloud is the need to export information about your resources. In this post, I’ll show the simplest way to extract that with the command line.

The idea of this post is to show the mechanism behind the extraction, I’ll run a simple command to recover the resources and choose some columns to export to a CSV file. You can use this as a base of more complex reports to handle properties, cross with other information, etc.

The first step is to connect in our Azure Account using Azure CLI (you can also do it using Azure Powershell), and recover all resources in our subscription:

Then we create an Array List to populate, iterate on the resources recovered above, and add the needed fields on the array:

Now we create a file name based on date and time to export the populated array. In my case I’m using the Delimiter “;”, as in my favorite regionalization this is the default pattern for CSV files. If this is not your case, you can remove that parameter.

This is the complete script, including the outputPath parameter:

And this is the file exported:

This is the link for my GitHub repository, and the intention is to fill it with more complex reports :)

https://github.com/wesleycamargo/AzureReports

See you in the next post!

Deploying a DotNet Core app in Azure Web App by command line

Thu, 07 Jan 2021 09:01:16 +0100

Deploying a DotNet Core app in Azure Web App by command line

If you need to fast deploy your dotnet core App in Azure, but don’t want to configure an entire CI/CD Pipeline, check in this post on how to deploy it!

To deploy the web app we will use a simple DotNet Core MVC application.

First of all, let´s create some parameters to prepare our resources. We are using a random id to avoid create the resources with some existing name:

To create the app, we will use an MVC Sample in the dotnet core templates. If you already have an existing project to deploy, skip the first line. We will prepare the publishing package with the dotnet core publish command and then zip it in a file:

In Azure, we will create a resource group and an app service plan, that are pre reqs to run a WebApp.

And finally, execute the deployment in the WebApp created above.

Save Money with Azure: Handling your resources with Azure Functions

Sat, 01 Aug 2020 22:30:15 +0200

ONE OF THE INCESSANT THINGS WHEN WE USE CLOUD PROVIDERS, IT’S HOW TO HAVE THE BIGGEST EFFICIENCY WITH OUR RESOURCES. IF YOU HAVE AZURE VIRTUAL MACHINES, FOR EXAMPLE, REDUCE THE TIME OF NON-PRODUCTION ENVIRONMENTS STAYS TURNED ON TO 12 HOURS AT WEEKDAYS CAN REDUCE YOUR BILL BY 67%.

Save Money with Azure: Handling your resources with Azure Functions

In fact, when a VM is created, there is an option for shutdown within the scheduled time, but it’s a bit limited, due to nonexistence of an option to startup the VMs, nor the option to do this for dependent VMs as a Domain Controller that should be online when the other VMs start.

A good, simple, and cheaper solution for this, is to use an Azure Function to startup and shutdown the Virtual Machines. So let’s see how we can do it!

What you will need:

Azure Account
Visual Studio Code
VS Code — Azure Functions Plugin
An Azure Virtual Machine

Preparing the Virtual Machine

Now we need to create a beacon in the VMs that we need to shutdown/startup. For this function, we’ll use a tag shutdownDaily with the value true. Later we will filter the VMs with this tag and shut it down in the function script.

Creating the shutdown function.

Open the Visual Studio Code, and create a new Powershell Function:

Select the Timer Function option and give a name to the function:

Now we need to define how often the function will be executed. Here we will execute every day at 7 PM.

The Azure Functions use cron expression to define the frequency. (You can have more information here)

It will create a basic function based on a template. We just need to keep the parameter $Timer.

For ease, you can replace all code with the block below:

Publishing the function on Azure

Now let’s publish the function on Azure Account.

First Sign in to Azure. It will redirect you to a Microsoft Login Screen in your browser. Enter your credentials and go back to Visual Studio Code.

Click in the Upload Icon, and select the desired subscription to deploy:

Now create a new function, give a name, and select the region. It will create all resources with this name. If you need to change something, use the advanced mode. After creation it will deploy the code automatically:

Given permissions to the function

After creating the function, we need to authorize it to manipulate Azure Resources. On function Settings, open Identity menu and change the status to On and save this.

It will enable Azure role assignments button, touch this, and Add role assignment. Here I’ll create permission as a contributor int the subscription, but use the best approach to your environment:

Testing the function

To test the function open it and run:

Expand the Logs menu, and check if everything is ok :)

Creating a startup function.

For the startup function, create a new function and in the step to replace the auto-generated code, use the code below:

Conclusion

Thi’s a simple use with one Virtual Machine, but you can expand this, with more complex environments as dependent resources and uses for other resources.

I hope that it can be helpful to you!

Make your deployment faster parallelizing your jobs in Azure DevOps - Classic Editor

Fri, 05 Jun 2020 22:27:53 +0200

Make your deployment faster parallelizing your jobs in Azure DevOps - Classic Editor

One of the most regular works in clients or companies that I faced, is to improve your existing CI/CD pipelines, reducing errors or the time run. In this post, I’ll show how to configure your jobs using multi-configuration.

When do I have to use this?

I’ll use as a didactic example an application with two projects: an API and a Web Site. This maybe can be a simple example, but you can find a better use for this approach in your day-to-day activities, ideally with tasks that take a long time to execute, as creating resources like Virtual Machines, Web Apps, or any other operations. The greater advantage is that if you have one job or ten, it will take just the time for one execution.

Below you can see a representation of the process. Notice that we have two Web App deployments serialized. However maybe you can have many processes, and if you have to wait for all of them, it can have a long time-consuming.

Otherwise, create a parallel activity approach, as you can see in the next image, you can save a lot of time just doing a simple configuration.

Configuring the CD pipeline

I’ll use an existing build with two outputs: an API and a WebApp. The names used in the artifacts are important, we will use this later to define our application name. Here, I’ll show you how to create an application and send it to Azure DevOps.

Create a classic CD pipeline for WebApp:

After creation, open the variables tab and create a new var named application.** **In the value enter the name of your artifacts split by a comma:

Go back to the Tasks tab and open the agent options. In item 3, notice that it has the same variable that we created before. Item 4 it’s how many parallel jobs you would like to run.

To use the multi-configuration, your WebApps names need some pattern. Here, I created a suffix and the end of the name will be the artifact name (or the split variable):

Now let’s configure the WebApp options. This is easy, just configure according to your pattern using the variable name:

Now, check if you have enough parallel jobs available in your account. The default’s it’s one parallel job for self-hosted and one Microsoft hosted agents, but if you have users with MSDN in their accounts, each one increases plus one parallel job. If you don’t have, you need to buy new parallel agents.

Running and testing :)

After running our pipeline, we can see it’s working running in two parallel agents:

After all, processes finished, we can see that’s done!

Pushing your code to GitHub and Azure Repos at the same time

Fri, 22 May 2020 06:33:08 +0200

Pushing your code to GitHub and Azure Repos at the same time

If you have an existing repository hosted by an Azure Repos or in GitHub, you can add new remotes easily. Let’s see how we can do it!

Creating and versioning a local project

Let’s create our project. In an empty directory type the CLI commands below:

It will create a new .net core web application, with a gitignore file and versioned in a local git.

Creating an Azure Repos repository and pushing your code

In your Azure DevOps account, follow these steps:

In the next screen disable the ‘Add a README’ option and enter your repo name:

In the empty repository created you have some options to push your code, we’ll push from an existing repository, so copy the command as the figure:

In your terminal paste the command, remember to check if you’re in your project directory. The highlighted log will confirm if your push was successful:

Refresh your Azure Repos page and your code was pushed!

Maratona Azure DevOps -Aprendendo na prática

Thu, 21 May 2020 19:26:41 +0200

Maratona Azure DevOps -Aprendendo na prática

Quer aprender como utilizar o Azure DevOps na prática? Então assista a Maratona Azure DevOps com mais de 12 horas de vídeos GRATUITOS.

Ontem eu e a Jaqueline Ramos finalizamos a segunda edição da Maratona Azure DevOps, foram 3 vídeos com aproximadamente 2 horas de conteúdo cada onde abordamos conceitos de DevOps e como aplicar na prática utilizando a ferramenta.

A idéia da segunda edição foi de complementar o conteúdo da primeira, onde mostramos novidades como Pipelines YAML, importações de repositório e itens que não tivemos tempo de exibir na primeira.

Abaixo estão os links para os vídeos da primeira e segunda edição, vou adicionar os vídeos da segunda edição na frente pois estão mais atualizados, espero que gostem!

Segunda Edição

Primeiro episódio:

No primeiro dia da segunda edição da Maratona Azure DevOps tivemos uma abordagem mais conceitual falando sobre:

Introdução a DevOps: Cultura e Práticas relacionadas
Carreira em DevOps
Ferramentas de colaboração

Segundo episódio:

No segundo dia da segunda edição da maratona apresentamos alguns itens que não foram cobertos na primeira edição em:

Azure Boards - Como gerenciar times, criar Dashboards, alterar templates de processos
Azure Repos-Importando repositórios, utilizando Az CLI para alterações nos repositórios, criando um repositório local e enviando para o Azure DevOps, boas práticas para criação de repositórios
Azure Pipelines - Criando Pipelines YAML
Azure Artifacts - Criando um feed e enviando pacotes NuGet para o Azure Artifacts

Terceiro episódio:

No terceiro dia da maratona apresentamos:

Azure Test Plans - Conheça como você pode gerenciar seus testes com o Azure Devops através de Work Items e automações de teste
Azure DevOps e Integrações -Já utiliza Jenkins, SonarCloud, AWS ou outras ferramentas em seus projetos? Saiba como integra-las ao Azure DevOps e obter o melhor para sua empresa
Preços - Entenda como são feitas as cobranças do Azure DevOps, e COMO UTILIZAR OS RECURSOS GRATUÍTOS!

Primeira Edição

Primeiro episódio:

Azure DevOps: Introdução (Como começar, administração, custos, etc) Azure Boards: Trabalhando com WorkItens e Processo de desenvolvimento

Segundo episódio:

Utilizando Azure Repos: Controle de versão

Terceiro episódio:

Azure Pipelines: Criando seu primeiro pipeline de entrega

Quarto episódio:

Azure Test Plans: Testes integrados ao seu pipeline Azure Boards: Dashboards

How to send .Net Core NuGet packages to Azure Artifacts

Sat, 09 May 2020 04:49:34 +0200

How to send .Net Core NuGet packages to Azure Artifacts

Learn how you can send .net core NuGet packages to Azure Artifacts.

To increase reusability in our company, an interesting thing that we can consider is to share our packages. Using .net, both core or standard, you can consider NuGet as a package manager. The Azure DevOps has an implementation for a NuGet as a Service named Azure Artifacts.

In this post, I’ll show you how to deploy manually a NuGet package step by step.

Below are all commands used in this tutorial, but I’ll explain each one.

Let’s get the hands dirty!

Creating and configuring an application

To start, we need a .net core class library application. To do this, type on terminal:

dotnet new classlibIt will create a basic project with a .csproj and one class. Rename the class to Math and put the code below on this:

To send new packages, first, we need to configure a connection between our app and the feed that will be created. For now, we’ll add a nuget.config file in our project. On terminal type:

dotnet new nugetconfigA file like this will be created:

This is the standard NuGet feed, after create ours we will update this file.

Now let’s package the project in a NuGet package using this command:

dotnet packIt will create a nupkg file in the bin directory. We’ll use this later.

The project it’s done, now let’s create the feed.

Creating a new artifact feed with Azure Artifacts

Access your Azure DevOps account and follow the steps:

On the next screen click on Connect to feed:

Here is the connection information necessary to connect our app with the feed. Copy the Project setup content and update the value of the nuget.config created before.

Connecting to feed

To make the connection, we need to download and install a connection provider. Save the content below in a file and run:

After installing the credential provider, we can authenticate with Azure DevOps. In the previous section in Publish packages, it’s given to you command with your connection information, add the parameter interactive. It will be used to start a new browser session and do the authentication. Without this parameter, the command won’t work.

dotnet nuget push .\bin\Debug\NugetPackage.1.0.0.nupkg –source “MeuFeed” –api-key teste –interactiveIt will be provided a URL and code, open the URL in the browser, and paste the code there.

Enviando pacotes Nuget .Net Core para o Azure Artifacts

Fri, 08 May 2020 00:40:29 +0200

Enviando pacotes Nuget .Net Core para o Azure Artifacts

Aprenda como publicar packages nuget no Azure Artifacts!

Para podermos ter maior reusabilidade dentro de nossa empresa, é interessante a utilização de pacotes compartilhados. No caso do .net, tanto core como standart utilizam o Nuget como Gerenciador de pacotes.

Nesse post irei ensinar passo a passo para realizar a publicação manual de um pacote nuget.

Ao longo do post irei detalhar cada um deles, mas abaixo estão todos os comandos executados no terminal:

Agora vamos botar a mão na massa!

Criando e configurando a aplicação

Vamos iniciar criando uma aplicação do tipo class library no .net core. Para isso digite no terminal (se você não tem familiaridade com o .net core nesse post ensino como criar uma aplicação web no VS Code):

dotnet new classlibO projeto criado é bem básico, possuindo apenas um .csproj e uma classe. Renomeie a classe para Math e adicione o código abaixo:

Vamos empacotar o projeto em um nuget package executando o comando:

dotnet packSerá criado um arquivo nupkg na pasta bin. Usaremos ele mais tarde.

Para enviar novos pacotes precisamos realizar a conexão da nossa aplicação com o feed que vamos criar. Vamos então adicionar um arquivo nuget.config em nosso projeto. No terminal digite:

dotnet new nugetconfigSerá criado um arquivo em nosso projeto como este:

O feed configurado é o padrão, para adicionar as configurações do nosso feed, vamos criá-lo e obter as conexões.

Criando o feed de artefatos no Azure Artifacts

Acesse sua conta do Azure DevOps e siga os passos abaixo:

Na tela que irá aparecer clique em Connect to feed:

Aqui estão as informações de conexão que iremos utilizar. Copie o conteúdo de Project setup e substitua o valor do arquivo nuget.config que criamos.

Conectando ao feed

Para realizar a conexão, precisamos baixar um provider. Para isso abre uma nova janela no VS Code e copie o conteúdo do arquivo abaixo salvando como CredentialProvider.ps1 e execute:

Agora que foi instalado o Credential Provider vamos conseguir autenticar no Azure DevOps. Na seção anterior em Publish packages é disponibilizado um comando copie ele e não se esqueça de adicionar o parâmetro interactive. Ele serve para iniciar uma sessão no browser e realizar a autenticação. Sem ele o comando não irá funcionar.

dotnet nuget push .\bin\Debug\NugetPackage.1.0.0.nupkg –source “MeuFeed” –api-key teste –interactiveCopie e abra no browser a URL que será fornecida e copie o código:

Como criar uma conta no Azure DevOps

Mon, 27 Jan 2020 14:00:32 +0100

Como criar uma conta no Azure DevOps

Olá pessoal, irei ensinar a vocês como criar uma conta gratuita no Azure DevOps, vamos ao passo a passo:

Acesse o site https://dev.azure.com e clique no botão Start free:

Existem 3 opções principais:

Acessar como uma conta Microsoft existente (email @outlook, @hotmail, etc)
Criar uma nova conta
Acessar com uma conta do GitHub

Vamos criar uma nova conta, mas caso você já possua uma conta no Github também poderá utilizar.

Se você possuir uma conta Microsoft (outlook, hotmail, msn) poderá inserir aqui, caso não possua clique na opção para criar uma nova conta de email:

Insira um novo email que esteja disponível:

Insira sua senha:

Em região escolha Brazil:

Nesse momento sua conta será criada, aguarde alguns instantes enquanto esse processo é realizado:

Sua conta já está criada!

Agora precisamos criar um Team Project. Ele pode ser o nome da sua equipe, seu produto, ou o que fizer mais sentido para o seu projeto. Aqui podemos nomea-lo como “Projetos”

Agora seu Azure DevOps está pronto para ser utilizado!

Esse é o setup básico, como disse acima existem outras opções, mas ao final todas irão criar sua organização no Azure DevOps!

Gerenciando transformações de configuração com Azure Key Vault e VSTS

Thu, 15 Feb 2018 16:21:22 +0100

Gerenciando transformações de configuração com Azure Key Vault e VSTS

Quem nunca teve dificuldade para fazer a gestão de arquivos de configuração que atire a primeira pedra!

Com a automação dos processos de build e deploy, uma etapa muito importante do processo é alteração das chaves dos arquivos de configuração, pois em cada um de nossos ambientes, teremos também valores diferentes, como por exemplo a connection string de um banco de dados.

Irei abordar uma maneira inteligente de realizar isso, através da transformação no momento do deploy utilizando o Azure Key Vault no momento de realização da Release no Visual Studio Team Services, então mãos a obra!

Para demonstrar, irei utilizar uma aplicação web simples, que será compilada no VSTS e publicada em um Azure Web App (em outro post irei explicar como criar e configurar um :)).

O que é o Azure Key Vault?

O Azure Key Vault é um recurso do Azure onde são armazenadas chaves e segredos para realizar a comunicação entre aplicações. Com ele podemos armazenar dados sensíveis(Segredos), Certificados .PFX, Key encription Keys (KEK) e também é possível utilizar sua API para realizar criptografia. Para mais informações à respeito, segue o link da documentação https://azure.microsoft.com/en-us/services/key-vault/

Entre as vantagens de utilização destacam-se:

Gestão de forma centralizada
Maior segurança na administração dos dados sensíveis
Maior facilidade na administração, pois pode ser feita no Azure Portal

Como implementar?

O primeiro passo é criar um Azure Key Vault no Azure Portal:

Selecione Add e na sequência preencha nome, Resource Group e Location. O restante das informações podem ser mantidas. Clique em Create:

Após a criação, selecione “Secrets” e “Generate/Import”:

Na próxima etapa iremos realizar a criação da chave/secret. Em Upload Options selecione Manual, e adicione o nome da variável e o valor. Note que por ser um secret aparecem asteriscos quando o valor é digitado:

Agora que temos nossa chave criada, vamos ao VSTS realizar a configuração de um Release Definition utilizando o Key Vault.

Para isso iremos utilizar um Variable Group e nele iremos realizar o link com o Key Vault.

Selecione a opção de Build/Release e em seguida Library. Após isso crie um novo Variable Group:

Agora devemos nomear o Variable Group e selecionar Link Secrets from an Azure Key Vault. Então selecione uma subscription e o Key Vault criado anteriormente. Após selecionar o Key Vault, será solicitada uma autorização para utilização, pode ser necessária uma permissão mais elevada no VSTS para relizar este passo:

Selecione as variáveis desejadas no grid que será habilitado e salve o Variable Group:

Para utilizar na Release Definition, vá em Variables e selecione Variable Groups. Clique em Link variable group:

Agora selecione o Variable Group criado, e clique em Link.

Pronto, agora as variáveis do Azure Key Vault podem ser utilizadas no processo de Release. Para utilizar, é o mesmo processo de utilização de uma variável criada durante a release, utilizando $(nomedavariavel) em algum script ou parâmetro.

Espero que este artigo seja útil, ajudando no processo de transformações de arquivos tornando mais prático e seguro!

Abraço!