Containers are massively growing in adoption across most enterprise organizations. Many enterprises leverage cloud environments like Microsoft Azure to run their containerized workloads. In Microsoft Azure, Azure Kubernetes Service (AKS) allows running Kubernetes-orchestrated containers using a SaaS service. Due to the ease of provisioning new Kubernetes clusters using AKS, organizations may quickly have many different AKS clusters, leading to challenges as the number of environments increases. Recently, Microsoft announced the public preview of Azure Kubernetes Fleet Manager, which helps to solve the challenges of managing multiple AKS clusters in Azure.
Challenges of managing multiple Azure Kubernetes Service (AKS) clusters
Back in the day, when virtualization took off, virtual machines were provisioned right and left. It led to a new term called “VM sprawl.” Similarly, the same problem can develop with the proliferation of Kubernetes clusters in the cloud across different regions, availability zones, etc. As organizations using Microsoft Azure and Azure Kubernetes Services (AKS) continue to provision AKS clusters, these numbers can grow and expand.
Businesses that are starting to scale up the number of AKS clusters may see the following challenges:
- AKS clusters that exist in multiple subscriptions, including Dev, Staging, QA, Testing, and Production
- Clusters existing across different regions
- Lifecycle management across multiple clusters in different environments
- Configuration management across AKS clusters
- Consistent containerized app scheduling
- Exposing the same app across multiple clusters and regions
- Monitoring the health of apps across multiple clusters
- Monitoring the health of the AKS clusters themselves
- Without a centralized management solution to manage different AKS clusters across different environments and regions, it is not easy to meet the challenges and business objectives mentioned above.
What is Azure Kubernetes Fleet Manager Preview?
The new Azure Kubernetes Fleet Manager Preview is a new solution that helps to address the challenges of multiple AKS clusters across various environments and regions. It is meant to solve the at-scale and multi-cluster problems across the Azure Kubernetes Service (AKS) environment. The Azure Kubernetes Fleet Manager resource creates a Fleet Resource cluster that manages all other member clusters.
The Azure Kubernetes Fleet Manager Preview works by creating a Fleet Resource cluster that takes care of the management activities of the other member clusters in the AKS Fleet. So, the Fleet Manager is a Kubernetes cluster that provides the Hub Kubernetes API for orchestrating operations through your Fleet of Kubernetes clusters. You can export services from member clusters to the Fleet resource, synced to the Hub. Once the resource is synced, it can be consumed by any other member clusters in the Fleet.
Architectural overview of Azure Kubernetes Fleet Manager
Types of clusters you can join and limitations
Note the following AKS clusters you can provision as member clusters:
- AKS clusters across the same or different resource groups within the same subscription
- AKS clusters across different subscriptions of the same Azure AD tenant
- AKS clusters from different regions but within the same tenant
As a point to note, the solution in its Preview form currently limits the number of AKS clusters you can join to the same fleet resource at 20 member clusters.
Azure Kubernetes Fleet Manager Benefits
Note the following capabilities and benefits of the new Azure Kubernetes Fleet Manager Preview:
- Centralized Management of all clusters at scale
- Managed hub cluster
- Kubernetes configuration propagation
- Multi-cluster load balancing
Centralized Management of all clusters at scale
In addition, it helps organize and manage your AKS clusters from a centralized management and control plane. You can join your existing Azure Kubernetes Service (AKS) clusters as member servers to the cluster Fleet. When you join your existing AKS clusters to the Fleet, it opens the door to using the cluster metadata for scenarios where you can benefit from multi-cluster scenarios, including resource propagation and multi-cluster load balancing, which we will describe below.
In addition, Azure Kubernetes Fleet Manager supports joining your AKS clusters from various sources. These include different regions, subscriptions, and resource groups. The only requirement is the AKS resources need to be part of the same Azure AD tenant.
The Azure Kubernetes Fleet Manager is the single-pane-of-glass interface allowing you to manage all your clusters from one location. In addition, using the Kubernetes Fleet Manager enables treating all the clusters as a single logical cluster by sharing metadata between them and exposing fleet operations.
Managed Hub Cluster
As mentioned earlier, the Azure Kubernetes Fleet Manager creates a Managed Hub Cluster. This cluster serves as a management cluster for the AKS member clusters that are part of the Azure Kubernetes Fleet resource. In addition, since the Managed Hub Cluster is a Kubernetes cluster itself, Kubernetes information can be shared between the Hub cluster and the member clusters.
The Hub Cluster exposes a single Hub cluster API that allows orchestrating Kubernetes tasks through the management cluster for all the member clusters as part of the AKS Fleet. Instead of managing and carrying out tasks directly for each cluster, it allows a much more streamlined management interface for operational tasks.
Kubernetes Configuration Propagation
One of the challenges that existed before Azure Kubernetes Fleet for AKS admins is Kubernetes resource propagation. Before the AKS Fleet solution, AKS admins would have to create Kubernetes namespaces once application teams detailed these requirements manually. They would also have to figure out which clusters would meet the namespace requirements and even create new clusters to configure new namespaces.
With Azure Kubernetes Fleet Manager, AKS admins now have something called ClusterResourcePlacement. This construct controls how cluster-scoped resources are propagated to member clusters. It includes two components:
- Resource selection – AKS admins can granularly pick which resources get propagated between Kubernetes cluster members in the Fleet based on metadata from each member cluster
2. Target cluster selection – AKS admins can also choose which member clusters they want to propagate resources. They can select all clusters or a subset of member clusters based on defined characteristics as part of a policy.
Azure Kubernetes Fleet Manager resource propagation
Multi-cluster load balancing
One of the driving reasons for deploying resources to cloud environments is the ultra-resiliency and availability offered by cloud environments. Azure Kubernetes Service (AKS) provides resiliency and availability backed by robust Azure infrastructure resources.
Building on those capabilities, organizations deploying large-scale AKS deployments want to have the ability to load-balance incoming traffic across multiple AKS clusters, often located in different regions. This configuration helps to provide resiliency in the event of a cluster failure or if an entire region’s resources go down.
Azure Kubernetes Fleet Manager provides the ability to configure Layer 4 multi-cluster load-balancing for incoming traffic across multiple clusters for workloads and services.
Multi-cluster load balancing
More features coming
Keeping in mind the Azure Kubernetes Fleet Manager is still in Preview, Microsoft has many more features coming to the offering soon. Note some of those upcoming features and capabilities:
- Arc-enabled Kubernetes member clusters – Microsoft plans to support Arc-enabled Kubernetes resource types in the future. This feature will open the door to Kubernetes clusters that exist anywhere that are Arc-enabled, including on-premises and in other clouds.
- Multi-cluster networking – The current layer 4 north-south load balancing will be extended to provide the equivalent Layer 7 North-South HTTPS load balancing with Gateway API, including east-west communication and multi-cluster service mesh.
- Lifecycle management of member clusters – Microsoft will soon be bringing lifecycle management of member clusters to the solution, allowing you to have an orchestrated experience for upgrading all member clusters of the Fleet.
- Fully-managed hub cluster – Microsoft will be extending the resiliency of the fully-managed hub cluster with a region failover capability for business continuity and disaster recovery
Wrapping up
The new Azure Kubernetes Fleet Manager is an excellent step in the right direction for centralized management of Azure Kubernetes (AKS) clusters. Its Preview form already offers excellent capabilities, including centralized management, Kubernetes resource propagation, and multi-cluster load balancing. However, it looks like the new features coming down the pipeline for Azure Kubernetes Fleet Manager will take the solution to the next level by adding Azure Arc-enabled Kubernetes to the possibilities of member servers and other great features.
Learn more about Azure Kubernetes Fleet Manager Preview here: Azure Kubernetes Fleet Manager – Cluster Management | Microsoft Azure
