Managed Clusters

This section contains information specific to configuring managed (AKS) Kubernetes clusters using Cluster API Provider for Azure (CAPZ). See self-managed clusters for information specific to provisioning these clusters.

Documents under the Managed Clusters area:

CAPZ's Managed Cluster versus CAPZ's ASOManaged Cluster

There are two APIs which can create AKS clusters and it's important to understand the differences.

ManagedCluster - is the original API to provision AKS clusters introduced with the 0.4.4 release. This more closely matches the CAPI style for YAML code for other providers. The original code was based on directly using the Azure Go SDK, but in the 1.11.0 release was switched to utilize Azure Service Operator (ASO) as a dependency for provisioning. This supports the preview API, but does not natively support all AKS features available. The ManagedCluster API will eventually be deprecated, see the roadmap for more information.

ASOManagedCluster - was created in the 1.15 release of CAPZ and creates a CAPI-compliant wrapper around the existing ASO definitions for AKS clusters. It has 100% API coverage for the preview and current AKS APIs via the ASO AKS CRDs. This is the long-term plan to support provisioning AKS clusters using CAPZ. See the roadmap for more information

Why is ASOManagedCluster the future?

The biggest challenge with ManagedCluster is attempting to keep up with the velocity of feature changes to the frequently changing AKS API. This model requires every single feature to be added into code directly. Even with the simplification of code to utilize ASO as a dependency, there still is quite a bit of time required to keep up with these features. Ultimately, this is an unsustainable path long-term. The asoManaged*Patches feature enables patching/adding various AKS fields via ASO to help try to fill some of these gaps, but this comes at the cost of unknown full testing.

ASOManagedCluster enables 100% API coverage natively and is easy to keep up with since it is primarily a dependency update to CAPZ. Furthermore it has the advantage of making it easier to import existing deployed clusters which have no existing CAPZ or ASO YAML code defined.

For the complete history of this transition, see the Managing Azure Resources with Azure Service Operator and Automate AKS Features available in CAPZ design proposal documents.

ASO's AKS versus CAPZ's ASOManaged AKS

It is possible to not utilize CAPZ at all and simply utilize ASO to provision an AKS cluster definition directly. The advantages that CAPZ brings over this approach are the following:

  • Robust Testing - CAPZ is utilized to test Kubernetes and AKS using this code with numerous end-to-end tests. ASO has no AKS-specific testing.
  • Simplification of Infrastructure as Code (IaC) definitions - With ASO you have to figure out how to put together every field and there are some small examples. CAPZ provides kustomize template samples connected to clusterctl generate template as well as a helm chart.
  • Management scale - CAPZ enables use of ClusterClass so you can have a smaller chunk of code to manage numerous clusters with the same configuration.
  • Heterogeneous Kubernetes management - it is possible with CAPZ to manage self-managed (not possible at all with ASO) and managed clusters with a single management control plane and similar IaC definitions.
  • Multi-cloud IaC consistency - even though it's still wrapping ASO, there still is some consistency in the code contract to provision Kubernetes clusters with the ~30 other CAPI infrastructure providers.

General AKS Best Practices

A set of best practices for managing AKS clusters is documented here: https://learn.microsoft.com/azure/aks/best-practices