Configuring the Kubernetes Cloud Provider for Azure
The Azure cloud provider has a number of configuration options driven by a file on cluster nodes. This file canonically lives on a node at /etc/kubernetes/azure.json. The Azure cloud provider documentation details the configuration options exposed by this file.
CAPZ automatically generates this file based on user-provided values in AzureMachineTemplate and AzureMachine. All AzureMachines in the same MachineDeployment or control plane will all share a single cloud provider secret, while AzureMachines created inidividually will have their own secret.
For AzureMachineTemplate and standalone AzureMachines, the generated secret will have the name “${RESOURCE}-azure-json”, where “${RESOURCE}” is the name of either the AzureMachineTemplate or AzureMachine. The secret will have two data fields: control-plane-azure.json
and worker-node-azure.json
, with the raw content for that file containing the control plane and worker node data respectively. When the secret ${RESOURCE}-azure-json
already exists in the same namespace as an AzureCluster and does not have the label "${CLUSTER_NAME}": "owned"
, CAPZ will not generate the default described above. Instead it will directly use whatever the user provides in that secret.
Overriding Cloud Provider Config
While many of the cloud provider config values are inferred from the capz infrastructure spec, there are other configuration parameters that cannot be inferred, and hence default to the values set by the azure cloud provider. In order to provider custom values to such configuration options through capz, you must use the spec.cloudProviderConfigOverrides
in AzureCluster
. The following example overrides the load balancer rate limit configuration:
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
kind: AzureCluster
metadata:
name: ${CLUSTER_NAME}
namespace: default
spec:
location: eastus
networkSpec:
vnet:
name: ${CLUSTER_NAME}-vnet
resourceGroup: cherry
subscriptionID: ${AZURE_SUBSCRIPTION_ID}
cloudProviderConfigOverrides:
rateLimits:
- name: "defaultRateLimit"
config:
cloudProviderRateLimit: true
cloudProviderRateLimitBucket: 1
cloudProviderRateLimitBucketWrite: 1
cloudProviderRateLimitQPS: 1,
cloudProviderRateLimitQPSWrite: 1,
- name: "loadBalancerRateLimit"
config:
cloudProviderRateLimit: true
cloudProviderRateLimitBucket: 2,
CloudProviderRateLimitBucketWrite: 2,
cloudProviderRateLimitQPS: 0,
CloudProviderRateLimitQPSWrite: 0
External Cloud Provider
To deploy a cluster using external cloud provider, create a cluster configuration with the external cloud provider template.
After the cluster has provisioned, install the cloud-provider-azure
components using the official helm chart:
helm install --repo https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/helm/repo cloud-provider-azure --generate-name --set infra.clusterName=${CLUSTER_NAME}
The Helm chart will pick the right version of cloud-controller-manager
and cloud-node-manager
to work with the version of Kubernetes your cluster is running.
After running helm install
, you should eventually see a set of pods like these in a Running
state:
kube-system cloud-controller-manager 1/1 Running 0 41s
kube-system cloud-node-manager-5pklx 1/1 Running 0 26s
kube-system cloud-node-manager-hbbqt 1/1 Running 0 30s
kube-system cloud-node-manager-mfsdg 1/1 Running 0 39s
kube-system cloud-node-manager-qrz74 1/1 Running 0 24s
For more information see the official cloud-provider-azure
helm chart documentation.
If you’re not familiar with using Helm to manage Kubernetes applications as packages, there’s lots of good Helm documentation on the official website.
Storage Drivers
Azure File CSI Driver
To install the Azure File CSI driver please refer to the installation guide
Repository: https://github.com/kubernetes-sigs/azurefile-csi-driver
Azure Disk CSI Driver
To install the Azure Disk CSI driver please refer to the installation guide
Repository: https://github.com/kubernetes-sigs/azuredisk-csi-driver