Last Updated
*This article was last updated on July 13th, 2020.
Hello again to all my loyal readers!
Communication Square is back again with a comprehensive understanding of Azure Virtual Machines.
I promise you all that when you leave this page today, you’ll be an Azure VM expert!
So, before we start, I want to know how many of you are acquainted with the idea of Azure and data science virtual machines?
For my beginner fellows, Azure is basically a cloud computing platform using which you launch services, servers on Azure and Virtual Machines (VMs) for Linux and Windows. It’s the freedom to build, manage, and deploy applications on a massive, global network using your favorite tools and frameworks.
So, let's go ahead and see what we are going to learn in today’s blog.
First, I will start off with basic and then we also discuss about pricing;What is Microsoft Azure? How does it work?
It provides a computing platform, and acts as a online portal for users to access and manage resources and services provided by Microsoft. The users can access services and resources like Virtual Machines, storage and so on, as long as they have an Microsoft Azure account and a working internet connection.
What is an Azure VM?
In computing, a virtual machine is an emulation of a computer system. Virtual machines are based on computer architectures and provide the functionality of a physical computer. Their implementations may involve specialized hardware, software or both.
VM is basically a raw server that you get from your cloud provider. It's like your own personal computer, rather than purchasing it, you are renting it out on the cloud. Right? When you are renting it out on the cloud, it is the same as if you are on your own computer. It's just a fresh piece of operating system, you can install as many software as you want, you can make it be a web server, you can configure it to be anything.
If you compare it with AWS, which is also a popular cloud computing service; with AWS, this same service is called EC2, in Azure its called VM.
What is an Azure Virtual Network?
An Azure virtual network (VNet) is the representation of your own network in the cloud. It's a logical isolation of the Azure cloud dedicated to your subscription.
A virtual network is just like a representation of your network but on the cloud. For example, we launch the server on the cloud, and we connect to our virtual network. When we connect to our virtual network that is on Azure, our network will include that server as if it's on our own network. Your server will be on the cloud, but your company or your company’s network will feel as if the server is on your own network. And like I said it's a logical isolation of the Azure cloud dedicated to your subscription, i/e. Whatever resources you have launched on the cloud, it logically isolates it from the rest of the resources that are there on the Azure, it could be your own resources or someone’s else's resources as well. It logically isolates them and insulates it from other resources.What are best features of Microsoft Azure?
1. Isolation
Each of these networks work independently when creating a virtual network, you can divide them into segments. You can configure the virtual network to use your own DNA servers.
2. Internet Communication
By default, when you are launching any instances in the VM it can access the Internet
As and when you need you can enable inbound access to specific resources.
3. Azure Resource Communication
Resources which fall under Azure virtual network can communicate with each other using private IP addresses, irrespective of the resources are from different subnets.
They provide default routing between subnets, on-premise networks so you don't have to configure and manage routes.
4. Virtual Network Connectivity
It can be connected to each other, enabling resources in any virtual network to communicate with resources in any other virtual network.
5. On-Premise Connectivity
A virtual network can be connected to an on-premises network, enabling resources to communicate with each other
6. Traffic Filtering
Network traffic can be filtered from resources in a virtual network by source IP address and port
7. Routing
Azure’s routing can be optionally overridden by default through configuration with your own routes or by propagating BGP (Border Gateway Protocol) routes through network gateway.
Azure Virtual Network: DHCP (Dynamic Host Configuration)
Azure uses Dynamic Host Configuration Protocol services to allocate Ip addresses from the ranges you assign to virtual network. Each IP address lease has an infinite duration.
Components
Subnet
Each visual network can be divided into subparts, called subnets. Subnets are further configured using Network security groups.
When you divide a virtual network into subparts, it is called an Azure subnet. These subnets can be given unique properties.
For example, in this same virtual network you have three virtual machines; one VM is in one subnet and the other VM is in another subnet.
You can configure each of this subnet to be unique. For example, we want to create a public subnet. A public subnet is a subnet which has internet access.
We can configure the properties in a way that the VM in one subnet has the internet access while the VM in another subnet does not have the internet access. A subnet with no internet access is called a private subnet. But when you see it from a broader prospect, all these subnets are on the same network. But because of the different configuration of subnets, they reflect different properties. So being on the same network, different virtual machines can have different properties using subnets.
Network Security Groups
NSGs are just like firewalls, which filter the traffic coming in and going out of your VMs. These settings can be done easily using Azure.
Say I want to connect to a web server, I need to go through an HTTP protocol. All of this is configured in a firewall, that firewall is called Network Security Groups.
Network Security Groups are just firewalls in which you put in the properties that you want the subnet to reflect. For example, I want to connect to my web server, so I have to allow all HTTP traffic on that subnet and for that I'll have to attach that particular network security group to that subnet.
Let's Suppose, we have this virtual network; In the virtual network we have subnets and a subnet is attached to an NSG. This NSG is where you'll be configuring all the protocols, all the properties you want your machine to reflect. This firewall will be attached to the subnet, in which you'll be deploying your VM. and this subnet is included in your virtual network. One virtual network can have as many subnets as you want, and each subnet will have different properties because of its NSG.Why do We Need a Virtual Network?
Virtual networks act as a communication channel between resources launched in the cloud. Why virtual? Because there are no physical wires involved in connecting these resources.
If you want to launch two virtual machines and you want both of them to communicate with each other; for example, you want to deploy a web server and a database server and you want the web server to talk to the database server.. For this communication we need a channel and hence we need a network. But why virtual networks? If you guys have noticed that in cloud there are no wires there are no switches there are no routers, so your VM is connected using a logical connection. The two servers are logically isolated and that is why we call it virtual.
Difference between a Classic and Latest Virtual Machine on Azure
There are two kinds of virtual machines available; latest and classic. The major difference between the classic VM and latest VM is in the way these VMs are deployed. Classic VMs are available only at the older version of Azure but are still supported in the latest azure portal.
When Azure started, service management API or SM API was being used, but with time Microsoft came up with a new API called Azure resource manager API or ARM. So all the latest virtual machines that you deploy, use the ARM API, whereas the classic VMs use SM API. The difference between these APIs are more about access control and permission. I’ll recommend you create the latest version as it has more access control than the classic ones. Classic VM’s are obsolete now.
Azure Virtual Machine: Workloads
When you are deciding to move to Azure there are two ways in which you can carry out.
Suitable Workloads
It is nothing but which is highly available service workloads, it could be commercial online stores. It can also be for periodic workloads such as:
Marketing campaigns which are seasonal on a website of an organization.
Annual Sales during festive holidays.
For unpredicted workloads like startups where they are unaware of their growth.
It can also be utilized by organization who simply want to offload their infrastructure to the cloud.
Unsuitable Workloads
If you want to move your applications to the cloud in which you are not seeing the cost difference it is not suitable. There are some regulations from the authorities or the local government who decides that you cannot move to the cloud based on their regulations.
How to do Virtual Machine Configuration
There are a couple of ways to get a VM up and running on the cloud.
First, we are going to look at cloud - first provisioning of VMs.
Cloud - First Provisioning of VMs
What this means is that we build and configure a machine in the cloud. It does not exist as a separate physical machine anywhere beforehand.
There are three ways to do this.
- You can build it through clicking at the Azure management portal.
- You can write a script to do it either on windows or other operating systems.
- You can write a program called an Azure REST API.
Once you have made your choice, you will need to select an image and VM size to start from.
This newly created disk will be stored in Azure storage service and your machine will boot.
Azure Virtual Machines gives you the flexibility of virtualization for a wide range of computing solutions with support for Linux, Windows Server, SQL Server, Oracle, IBM, SAP, and more.Temp Drive
Temp-drive is a non-persistent place to store data on disk and it is actually stored on the host-local disk.
Do not store any data on the temp-drive. This is a free disk. A lot of people will be tempted to put data on here but if your VM is rebooted or it’s moved to a different host, the temp drive will be gone.
It is used for:
- Guest OS swap/paging file
- SQL Server TempDB (optional configuration)
Basic A and standard A uses HDD temp drive. All other VMs uses SSD temp-drive.
VM Series
People often get lost in the Azure world while looking at the virtual machines. There are so many variations, people just get blinded with all the options. But don’t you worry. I have got it all simplified for you.
There are different VM series in Azure. Each series is based on the set of traits.
Each series is named after a letter. These letters mean something. Within a series there can be generations. e.g.
- D series
- D v2 series
Each series is further broken into sizes.
Azure VM Specialization
There are also some special letters that tell about a VM and its capabilities.
S= supports SSD/premium storage
M= higher than normal memory
R= additional RDMA NIC
If I see a VM with letter S it means that it supports SSD. Here is the cool thing, if I deploy a DS2 VM instead of a D2 VM, it's the same cost. What changes is the cost of the disk which is not included in the cost of the VM.
Similarly, A2m_v2 has four times more memory than A2_v2.
H16r has an extra RDMA(remote direct memory access) NIC.
The most common VM series are the A series, the D series and the F series.
DESC. | BASIC A | STANDARD A | D | F |
|---|---|---|---|---|
How To Remember It? | Very Low Specs | A Is the Starting Point | D is for disk | F-series |
CPU | Limited Processor | Limited Processor | XEON | XEON |
Unique Trait | Limited Azure Features | Nothing Special | IOPS/Throughput | CPU/RAM Balance |
Typical Roles | Test/Low-End DCs | Low Spec Machines | Database | Application/Web |
*ACU Azure Compute Unit allows us to measure the efficacy of each VM depending on its computing power.
VM Family
The A Series
Entry-level economical VMs for dev/test
A-series VMs have CPU performance and memory configurations best suited for entry level workloads like development and test. They are economical and provide a low-cost option to get started with Azure. Av2 Standard is the latest generation of A-series VMs with similar CPU performance but more RAM per vCPU and faster disks.
Example use cases include development and test servers, low traffic web servers, small to medium databases, servers for proof-of-concepts, and code repositories.
B-Series
Economical burstable VMs
Bs-series are economical virtual machines that provide a low-cost option for workloads that typically run at a low to moderate baseline CPU utilization, but sometimes need to burst to significantly higher CPU utilization when the demand rises.
Example use cases include development and test servers, low-traffic web servers, small databases, micro services, servers for proof-of-concepts, build servers.
Premium Storage:
Supported
Premium Storage Caching:
Not Supported
- The B-series burstable VMs are ideal for workloads that do not need the full performance of the CPU continuously, like web servers, small databases and development and test environments. These workloads typically have burstable performance requirements.
- The B-Series provides these customers the ability to purchase a VM size with a price conscious baseline performance that allows the VM instance to build up credits when the VM is utilizing less than its base performance.
D Series
General purpose compute
The D-series Azure Virtual Machines offer a combination of vCPUs, memory, and temporary storage able to meet the requirements associated with most production workloads.
The D v3 virtual machines are hyper-threaded general-purpose VMs based on the 2.3 GHz Intel® XEON ® E5-2673 v4 (Broadwell) processor. They can achieve 3.5 GHz with Intel Turbo Boost Technology 2.0.
The D v4 and Dd v4 virtual machines are based on a custom Intel® Xeon® Platinum 8272CL processor, which runs at a base speed of 2.5Ghz and can achieve up to 3.4Ghz all core turbo frequency. The Dd v4 virtual machine sizes feature fast, large local SSD storage (up to 2,400 GiB) and are well suited for applications that benefit from low latency, high-speed local storage. The D v4 virtual machine sizes do not have any temporary storage.
The Da v4 and Das v4 Azure VM-series provide up to 96 vCPUs, 384 GiBs of RAM and 2,400 GiBs of SSD-based temporary storage and feature the AMD EPYC™ 7452 processor.
The Ds-series, Dds-series, and the Das-series VMs both support Azure Premium SSDs and Ultra Disk storage depending on regional availability.
Example use cases include many enterprise-grade applications, relational databases, in-memory caching, and analytics. The latest generations are ideal for applications that demand faster CPUs, better local disk performance or higher memories.
DC-series
Protect data in use
DC-series virtual machines are a new family of VMs to protect the confidentiality and integrity of your data and code while it's processed in Azure through the use of secure enclaves. This is in addition to the existing built-in encryption capabilities that protect data in Azure while it's at rest and in transit.
These VMs are backed by the latest generation of Intel Xeon E-2176G 3.7GHz Processor with SGX technology. With Intel Turbo Boost Technology this processor can reach up to 4.7GHz.
Example use cases include confidential querying in databases, creation of scalable confidential consortium networks and secure multiparty machine learning algorithms. The DC-series VMs are ideal to build secure enclave-based applications to protect customers code and data while it's in use.
E-Series
Optimized for in-memory hyper-threaded applications
The E-series Azure virtual machines are optimized for heavy in-memory applications such as SAP HANA. These VMs are configured with high memory-to-core ratios, which makes them well-suited for relational database servers, with medium to large caches, and in-memory analytics. The Ev3-series VMs range from 2 to 64 vCPUs and 16-432 GiB of RAM, respectively.
The E v4 and Ed v4 virtual machines are based on a custom Intel® Xeon® Platinum 8272CL processor, which runs at a base speed of 2.5Ghz and can achieve up to 3.4Ghz all core turbo frequency. The E and Ed v4 virtual machine sizes feature up to 504 GiB of RAM and are ideal for memory-intensive enterprise applications. The Ed v4 also include a fast and large local SSD storage (up to 2,400 GiB) making them well suited for applications that benefit from low latency, high-speed local storage. The E v4 virtual machine sizes do not have any temporary storage.
The Ea v4 and Eas v4 Azure VM-series feature the AMD EPYC™ 7452 processor and provide up to 96 vCPUs, 672 GiBs of RAM and 2,400 GiBs of SSD-based temporary storage.
The Es, Eds and the Eas VM series both support Azure Premium SSDs and Ultra Disk storage depending on regional availability.
Example use cases include SAP HANA (E64s_v3 only), SAP S/4 HANA application layer, SAP NetWeaver application layer, SQL Hekaton and other large in-memory business critical workloads.
F-Series
Compute optimized virtual machines
F-series VMs feature a higher CPU-to-memory ratio. They are equipped with 2 GB RAM and 16 GB of local solid state drive (SSD) per CPU core, and are optimized for compute intensive workloads. The Fsv2-series features 2 GiB RAM and 8 GB of local temporary storage (SSD) per vCPU. The Fsv2-series is hyper-threaded and based on the 2.7 GHz Intel Xeon® Platinum 8168 (SkyLake) processor, which can achieve clock speeds as high as 3.7 GHz with the Intel Turbo Boost Technology 2.0.
Example use cases include batch processing, web servers, analytics, and gaming.
G-Series
Memory and storage optimized virtual machines
G-series VMs feature the Intel® Xeon® processor E5 v3 family, two times more memory, and four times more Solid State Drive storage (SSDs) than the General Purpose D-series. G-series features up to ½ TB of RAM and 32 CPU cores, and provide unparalleled computational performance, memory, and local SSD storage for your most demanding applications.
Example use cases include large SQL and NoSQL databases, ERP, SAP, and data warehousing solutions.
H-Series
High Performance Computing virtual machines
The HB-series VMs are optimized for HPC applications driven by memory bandwidth, such as fluid dynamics, explicit finite element analysis, and weather modeling. HB VMs feature 60 AMD EPYC 7551 processor cores, 4 GB of RAM per CPU core, no hyperthreading, and up to 4 Managed Disks. The AMD EPYC platform provides more than 260 GB/sec of memory bandwidth.
The HC-series VMs are optimized for HPC applications driven by intensive computation, such as implicit finite element analysis, reservoir simulation, and computational chemistry. HC VMs feature 44 Intel Xeon Platinum 8168 processor cores, 8 GB of RAM per CPU core, no hyperthreading, and up to 4 Managed Disks. The Intel Xeon Platinum platform supports Intel’s rich ecosystem of software tools and features an all-cores clock speed of 3.4 GHz for most workloads.
Example use cases include fluid dynamics, finite element analysis, seismic processing, reservoir simulation, risk analysis, electronic design automation, rendering, Spark, weather modeling, quantum simulation, computational chemistry, heat transfer simulation.Ls-Series
Storage optimized virtual machines
The latest Lsv2-series features high throughput, low latency, directly mapped local NVMe storage. The Lsv2 VMs run on the AMD EPYC™ 7551 processor with an all core boost of 2.55GHZ up to a 3.0GHz single core boost. The Lsv2 series VMs offer up to 80 vCPUs in a hyper-threaded configuration, with 8 GiB of memory per vCPU and up to 19.2TB (10x1.92TB) available directly to the VM.
The Ls-series VMs are storage optimized. These are ideal for applications requiring low latency, high throughput, and large local disk storage. These VMs are built on Intel Haswell processor technology, specifically E5 Xeon v3 processors with 4, 8, 16, and 32 core VM sizes. Ls-series VMs support up to 6 TB of local SSD and offer unmatched storage I/O performance.
Example use cases include NoSQL databases such as Cassandra, MongoDB, Cloudera, and Redis. Data warehousing applications and large transactional databases are great use cases as well.
M-Series
Memory optimized virtual machines
The M-series family of Azure virtual machines are memory optimized and are ideal for heavy in-memory workloads such as SAP HANA. The M-Series offer up to 4 TB of RAM on a single VM. In addition, these VMs offer a virtual CPU count of up to 128 vCPUs on a single VM to enable high performance parallel processing.
Example use cases include SAP HANA, SAP S/4 HANA, SQL Hekaton and other large in-memory business critical workloads requiring massive parallel compute power.
Mv2-Series
Largest memory optimized virtual machines
The Azure Mv2-series virtual machines are hyper-threaded and feature Intel® Xeon® Platinum 8180M 2.5GHz (Skylake) processors, offering up to 416 vCPU on a single VM and offer 3TB, 6 TB and 12 TB memory configurations. This is by far the largest-memory virtual machine offered on Azure and provide unparalleled computational performance to support large in-memory databases.
Example use cases include SAP HANA, SAP S/4 HANA, SQL Hekaton and other large in-memory business critical workloads requiring massive parallel compute power.
N-Series
GPU enabled virtual machines
The N-series is a family of Azure Virtual Machines with GPU capabilities. GPUs are ideal for compute and graphics-intensive workloads, helping customers to fuel innovation through scenarios like high-end remote visualization, deep learning, and predictive analytics.
The N-series has three different offerings aimed at specific workloads:
The NC-series is focused on high-performance computing and machine learning workloads. The latest version—NCsv3—features NVIDIA’s Tesla V100 GPU.
The NDs-series is focused on training and inference scenarios for deep learning. It uses the NVIDIA Tesla P40 GPUs. The latest version - NDv2 - features the NVIDIA Tesla V100 GPUs.
The NV-series enables powerful remote visualization workloads and other graphics-intensive applications backed by the NVIDIA Tesla M60 GPU.
NCsv3, NCsv2, NC and NDs VMs offer optional InfiniBand interconnect to enable scale-up performance.
Example use cases include simulation, deep learning, graphics rendering, video editing, gaming and remote visualization.
Azure: VM Sizing
In Azure you select from a range of predefined configuration options that correspond to different VM sizes.
The VM size determines characteristic such as:
- The Number and Speed of The Processors
- Amount of Memory
- Maximum Number of Network Adapters
- Data Disks You Can Attach to It
- Maximum Size of a Temporary Disk
General Purpose Virtual Machine Sizes
General purpose VM sizes provide balanced CPU-to-memory ratio. Ideal for testing and development, small to medium databases, and low to medium traffic web servers.
Compute Optimized Virtual Machine Sizes
Compute optimized VM sizes have a high CPU-to-memory ratio and are good for medium traffic web servers, network appliances, batch processes, and application servers.
Memory Optimized Virtual Machine Sizes
Memory optimized VM sizes offer a high memory-to-CPU ratio that are great for relational database servers, medium to large caches, and in-memory analytics.
Storage Optimized Virtual Machine Sizes
Storage optimized VM sizes offer high disk throughput and IO, and are ideal for Big Data, SQL, and NoSQL databases.
GPU Optimized Virtual Machine Sizes
GPU optimized VM sizes are specialized virtual machines available with single or multiple NVIDIA GPUs. These sizes are designed for compute-intensive, graphics-intensive, and visualization workloads.
High Performance Compute VM Sizes
Azure H-series virtual machines are the latest in high performance computing VMs aimed at high end computational needs, like molecular modeling, and computational fluid dynamics.
I hope you are not lost anymore in the world of Azure virtual machines.
This is all about Azure VM series for now. If you still have any queries or you want any help in deciding which VM series to select, feel free to contact us, we would be pleased to offer any help.