What is esx cluster?

Using VMware ESXi on a host server maximizes that hardware because you can run multiple applications on isolated virtual machines (VMs). However, most users need more VMs, which in most cases cannot fit on a single physical server. This is where VMware clusters come in.

VMware clusters group multiple physical servers and treat them as a single machine by aggregating and managing their combined resources as a unit. When used in server virtualization, VMware clusters unlock vSphere High Availability (HA), load balancing and VMware vSAN features.

A vSphere cluster is a set of ESXi hosts configured to share resources such as processor, memory, network and storage. In vSphere environments, each cluster can accommodate a maximum of 32 ESXi hosts, with each host supporting up to 1024 VMs.

Using vSphere clusters allows IT administrators to aggregate and organize virtualization resources in a VMware environment and tie them back to the underlying physical resources. Suppose a cluster contains three physical servers, each with four dual-core processors running at 8 GHz and having 16 GB of memory.

The total computing power for such a cluster becomes 192 GHz, while the memory available is 48 GB. With this setup, you don’t need to worry about the underlying cluster resources’ physical composition. All you need is to set up resource pool policies based on the aggregate available resources via the vCenter Server.

The VMware infrastructure assigns the resources to the VMs within the boundaries of those policies automatically.

For example, the marketing department reserves 128 GHz and 32 GB from a cluster of 192 GHz and 48 GB, leaving 64 GHz and 16 GB for the sales department. The fact that you’ve allocated 64 GHz of computing power and 16 GB of memory to the sales department doesn’t mean you cannot resize them on the fly.

If the sales department’s workload increases, you can dynamically raise its computing power from 64 GHz to 92 GHz. You can also increase memory consumption from 16 GB to 20 GB without shutting down the associated VM. Reserving resources for the VM doesn’t mean they leave the marketing department immediately.

When you take away 32 GHz reserved for the marketing department, it’s assigned to the sales department only when the marketing department is idle. When the marketing department increases its capacity, it takes away its 32 GHz automatically. Using resources in this manner does not lead to wastages even though you’ve reserved them for different pools.

High Availability (HA) is a system characteristic that describes its ability to operate continuously without downtime. Availability is usually expressed as a ratio between uptime (total time the system is available) over downtime (the total time the system is not available) in a given year.

For example, the popular metric “five-nines”, or 99.999% availability, translates to roughly 5.26 minutes or less of total downtime in a year. Organizations can achieve HA in different ways, including redundant network interface cards (NICs), HA applications, and server clusters.

VMware uses a license feature known as vSphere HA to provide broad-based and cost-effective high availability at the virtualization layer. When enabled, vSphere HA automatically restarts the failed VMs on other ESXi hosts that have spare capacities. This minimizes service disruptions and downtime while eliminating the need for costly dedicated hardware and additional software.

While the terms vSphere HA and vSphere Fault Tolerance (FT) are often used interchangeably, they mean different things. vSphere HA focuses on achieving the least possible downtime where high performance is the top priority. vSphere FT also ensures minimal downtime but doesn’t focus on delivering high performance during a system failure event.

vSphere HA leverages a High Availability cluster (a logical grouping of ESXi hosts pooled on the same network) to protect against ESXi hosts, VMs and application failure. Restarting VMs on different ESXi hosts is possible because the HA cluster has shared storage that maintains virtual machine disk (VMDK) files accessible to all the hosts within the cluster.

vSphere HA uses a feature known as Fault Domain Manager (FDM), an agent to monitor physical servers’ availability. When you set up a VMware cluster, the vCenter Server places the FDM agent on each cluster’s ESXi hosts. One of the ESXi hosts in the cluster becomes the master, while others are slaves. The master host monitors the signals from slaves in the cluster and communicates with the vCenter Server.

If the master host fails to detect a signal from any host or virtual machine in the vSphere environment, it instructs vSphere HA to undertake remedial steps. If the entire host has failed, all VMs on that hardware get restarted on other servers in the cluster with spare capacities. If the VM has failed, vSphere HA restarts it on other hosts within the cluster.

Like vSphere HA, VMware Distributed Resource Scheduler (DRS) is a licensable feature that you can add to the VMware cluster. When you enable VMware DRS, vCenter Server uses its system algorithms coupled with your own defined rules to manage and optimize the cluster resources.

VMware DRS treats the amalgamated CPU, memory and storage resources as global resource pools that all virtual machines in the cluster can access. VMware DRS intelligently monitors the workload of running VMs and their resource consumptions on ESXi hosts against resource assignment policies within the cluster.

In case a particular workload violates the set policies, or there is potential for improvement, VMware DRS leverages VMware vMotion to reassign the VMs to different ESXi hosts within the cluster dynamically.

When you create a new VM, you don’t have to specify the host if you have enabled the DRS feature. VMware DRS automatically collects hosts’ details and the new VM’s resource consumption details in the cluster and generates recommendations for placements.

This way, VMware DRS provides load balancing and quality of service (QoS) functionalities. By migrating VMs to different ESXi hosts automatically, VMware DRS enhances the performance within vSphere environments. For this reason, most organizations leverage VMware DRS with vSphere HA to achieve failover and load balancing.

In the case of failover, vSphere HA restarts VMs on other ESXi hosts automatically, while DRS intelligently checks the available computing resources to recommend VM placements within the cluster.

An organization requires an effective load balancer—whether in on-premises datacenters or public clouds—to maintain application availability to its customers, partners and end users. Besides ensuring optimal response times and high service availability to mission-critical applications, an effective load balancer allows an enterprise to scale up and accommodate any surge in traffic.

Parallels® Remote Application Server (RAS), an all-in-one virtual desktop infrastructure (VDI) solution, provides effective load balancing via its High Availability Load Balancing (HALB) feature. Parallels HALB eliminates the challenges associated with multi-gateway environments by ensuring that traffic gets redirected only to healthy gateways.

The process of deploying Parallels HALB is straightforward. Just install and configure the HALB appliance, then add it to Parallels RAS from the Console, and you’re good to go. Most importantly, you can run multiple HALB appliances in Parallels RAS simultaneously, further minimizing downtime to guarantee application availability.

This article is part of the VMware vSphere 7.x - VCP-DCV Study Guide. Check out this page first for an introduction, disclaimer, and updates on the guide. The page also includes a collection of articles matching each objective of the official VCP-DCV.

Objective 1.6 is a parent of multiple key topics discussing vSphere clustering. Therefore, this article is a quick introduction and an overview of all the essential concepts to understanding ESXi clusters.

In VMware vSphere, there is no logic to talk about clusters without mentioning Distributed Resource Scheduler (DRS) and High Availability (HA); these are the two essential services that support the management of VMware clusters. However, we will discuss these concepts in the following topics.

When I began studying VMware a couple of years ago, I had the same question that many in the virtualization realms had when the technology was just rising. What if a single server fails? Why put all your eggs (VMs) in one basket (Server)? It wouldn't be worth the benefits of saving computing resources by placing a couple of virtual servers into a physical one, as if this fails, you could lose all services.

Due to this conception, for many, it was hard to flip the mental switch about the advantages of virtualization. People were still uncertain about virtualization, even though VMware included similar and even better clustering technologies than physical machines. These new virtual services delivered capabilities that previously required complex or expensive solutions to implement.

These services also provide significantly higher hardware utilization and better alignment of IT resources with business goals and priorities. In the past, companies have had to assemble a patchwork of the operating system or software application-specific solutions to obtain the same benefits.

While using virtualization, even a powerful server can be overloaded with multiple computing tasks. To ensure better performance and reliability, we can use clustering technologies.

Let's go back to the basics.

A Host represents the aggregate computing and memory resources of a physical server. For example, suppose the physical server has two 12-core CPUs running at 3.0 GHz each and 256 GB of system memory. In that case, the Host will have 72 GHz of computing power and 256 GBs of memory available for running virtual machines assigned to it.

A Cluster represents the aggregate computing and memory resources of a group of physical servers sharing the same network and storage arrays. For example, if the group contains four servers, each has two 12-core CPUs running at 3.0 GHz and 256 GB of memory. The Cluster will then have 288 GHz of computing power and 1,024 GB of memory available for the running virtual machines assigned to it.

Admins (the virtual resource owners) do not need to be concerned with the physical composition (number servers, quantity, and type of CPUs) of the underlying cluster to provision resources. Admins set up the resource provisioning policies based on the aggregate available resource. vSphere will automatically assign the appropriate resources dynamically to the virtual machines within the boundaries of those policies.

Resource pools simplify control over the resources. They provide a flexible and dynamic way to divide and organize computing and memory resources from a Host or a Cluster. vSphere virtualizes and aggregates industry-standard servers (processors, memory, their attached network, and storage capacity) into logical resource pools (from a single ESX Server host or a VMware cluster) that can be allocated to virtual machines on demand. Any Resource Pools can be partitioned into smaller Resource Pools at a fine-grain level to divide further and assign resources to different groups or for various purposes.

In general, the two essential services that support the management of VMware clusters are VMware DRS and HA.

VMware DRS provides automatic initial virtual machine placement and makes automatic resource relocation and optimization decisions as hosts are added or removed from the cluster or the load on individual virtual machines changes.

Now, suppose an ESXi host goes offline. It will take down all virtual machines running on that single host. VMs won't come back online until you restore the server or restore them to another server using a backup and restore process. These processes can be very time‑consuming. VMware HA allows these virtual machines running on the host to be switched over to use other host resources in the cluster in the case of host machine failures.

More about these topics and concepts in following vSphere 7 objectives.

Understanding Clones in VMware vSphere 7

Horizon 7 Architecture Planning

The topic reviewed in this article is part of the VMware vSphere 7.x Exam (2V0-21.20), which leads to the VMware Certified Professional – Data Center Virtualization 2021 certification.

Section 1 - Architectures and Technologies.

Objective 1.6 – Describe ESXi cluster concepts

See the full exam preparation guide and all exam sections from VMware.

To create a cluster after VMware vCenter deployment, you need:

Hardware requirements to use ESXi clustering features:

Let’s take a look at the steps to create a cluster once we have connected to the Web UI of our vCenter Server in VMware vSphere Client.

You have 2 approaches to configuring the vCenter inventory and creating an ESXi cluster:

In this post, we use the first approach.

Right-click the datacenter in vSphere Client and click Add Host in the context menu.

The Add Host wizard is launched with 7 mandatory steps.

You can see two ESXi hosts added to the datacenter in the VMware vCenter inventory in the screenshot below.

Once the datacenter is created and ESXi hosts are added, you can choose to create a new vSphere cluster.

Now you can configure High Availability (HA) and set up Distributed Resource Scheduler (DRS) in your vSphere cluster. After configuring HA, you can configure Fault Tolerance (note that fault tolerance differs from HA).

To unlock the true potential of a VMware ESXi hypervisor environment, utilizing the VMware vSphere cluster functionality is key. Running VMware ESXi in the cluster configuration with VMware vCenter Server allows you to utilize the enterprise datacenter VMware vSphere technologies, including HA (high availability) and DRS (distributed resource scheduler).

With a VMware ESX Cluster, you define 2 or more physical machines that will provide resources for the hosts (or resource pools) that are assigned to that cluster. By using ESX clusters, you can achieve high availability (VM HA) and load balancing of virtual machines (called VMware DRS, discussed below)

VMware terminology differs from other IT terminologies. It is specifically focused on VMware virtualization and has completely different meaning than terminologies for other IT vendors and systems. VMware beginners should not be discouraged to learn this technology. You don't have to manage thousands cloud-scale enterprise infrastructure to learn the basics of VMware vSphere architecture which you would then be able to implement in small or medium-size business.  So today's post is another part of my new “Definitions” category – What is VMware Cluster?

Basically, a cluster is a group of servers which can share resources such as memory, CPU and storage. Clusters provide flexible and dynamic ways to organize the aggregated computing and memory resources in the virtual environment and link them back to the underlying physical resources because as you know, the virtual workloads are running in memory but they use real resources. Real physical memory, real storage or real CPU.

While a VMware cluster is a “generic” name for this group of hosts, it does not specify exactly what functions, or which features have this cluster activated. We can very easily create a VMware cluster but if we do not activate any functions, then the cluster is just kind of doing nothing. That's why we are talking about:

A single host represents the aggregate computing and memory resources of a single physical x86 server. There will be several VMs running on the top and sharing resources the host can offer. A single host is installed with VMware ESXi which can be installed on compatible hardware as VMware ESXi hypervisor (Check How-to Apply a free license) or as ESXi with paid licensing (standard, advanced, enterprise).

A cluster acts and can be managed as a single entity. It represents the aggregate computing and memory resources of a group of physical x86 servers sharing the same network and storage arrays. For example, if the group contains eight servers with four dual-core CPUs each running at 4GHz and 32GB of memory, the cluster has an aggregate 256GHz of computing power and 256GB of memory available for running virtual machines.

The central management is assured via a separate component called vCenter Server. Admin manages the whole infrastructure via single console. While vCenter server is necessary for cluster configuration and management, it is not necessary to assure VMware vSphere High Availability (HA) function. It means that you can have a vCenter failure, but the cluster continues to provide HA for your VMs and in case there is a hardware failure, the VMs which were running on a failed host, are restarted automatically on remaining hosts within VMware HA enabled cluster.

TIP: How to configure VMware HA Cluster.

1 vCenter can manage several data centers, so you can create several datacenter objects within the UI to reflect your geographical situation.

Datacenter Objects – Each datacenter object that you create within your vSphere environment can have several clusters.

Cluster Objects – each cluster has then (or not) Resource pools and VMs.

The cluster settings can be modified by Selecting a particular cluster object > Configure > Edit Settings

And there you can enable VMware HA, VMware DRS or VMware VSAN….. after making sure that all the requirements are met of course… -:)

Activate VMware DRS with Fully automated, partially automated or manual configuration….

Activate VMware HA and Proactive HA (if your hardware has HA provider enabled)

Shared storage is a storage which can be “seen” by all servers which participate in VMware cluster. The goal is to have a shared data store connected to every host and as such, all the hosts can see, manage and run the VMs which are “stored” on that shared datastore. Each host, however, can take ownership of the particular VM, only once. It means that one VM can be executed at the same time only by a single host.

Shared storage can be a single device (NAS, SAN) or also a VMware VSAN which is using local disks and Flash devices in each server, to create a shared storage pool. But this is a particular case (even if very popular). The “traditional” shared storage is usually represented in many small enterprises (SMBs) by single SAN or NAS.

You cannot enable HA on a cluster which does not have any kind of shared storage. Also, vMotion needs a shared storage to move VMs from one host to another. And who says vMotion, says  VMware Distributed Resource Scheduler (DRS) as DRS is checking actually the performance of your VMs and VMs are actually moved to other hosts automatically, via vMotion.

Stay tuned through RSS, and social media channels (Twitter, FB, YouTube).

More posts from ESX Virtualization: