This document will help you get a CAPZ Kubernetes cluster up and running with your custom image.
An image defines the operating system and Kubernetes components that will populate the disk of each node in your cluster.
By default, images offered by “capi” in the Azure Marketplace are used. You can list these reference images with this command:
az vm image list --publisher cncf-upstream --offer capi --all -o table
Note: These images are not updated for security fixes and it is recommended to always use the latest patch version for the Kubernetes version you wish to run. For production-like environments, and for more control over your nodes, it is highly recommended to build and use your own custom images.
For your custom image to work with Cluster API, it must meet the operating system requirements of the bootstrap provider. For example, the default
kubeadm bootstrap provider has a set of
preflight checks that a VM is expected to pass before it can join the cluster.
The reference images are each built to support a specific version of Kubernetes. When using your custom images based on them, take care to match the image to the
version: field of the
MachineDeployment in the YAML template for your workload cluster.
To upgrade to a new Kubernetes release with custom images requires this preparation:
- create a new custom image which supports the Kubernetes release version
- copy the existing
AzureMachineTemplateand change its
image:section to reference the new custom image
- create the new
AzureMachineTemplateon the management cluster
- modify the existing
MachineDeploymentto reference the new
AzureMachineTemplateand update the
version:field to match
See Upgrading workload clusters for more details.
To use a custom image, it needs to be referenced in an
image: section of your
AzureMachineTemplate. See below for more specific examples.
To use an image from the Shared Image Gallery, fill in the
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 kind: AzureMachineTemplate metadata: name: capz-shared-gallery-example spec: template: spec: image: sharedGallery: resourceGroup: "cluster-api-images" name: "capi-1234567890" subscriptionID: "01234567-89ab-cdef-0123-4567890abcde" gallery: "ClusterAPI" version: "0.3.1234567890"
If you build Azure CAPI images with the
make targets in Image Builder, these required values are printed after a successful build. For example:
$ make -C images/capi/ build-azure-sig-ubuntu-1804 # many minutes later... ==> sig-ubuntu-1804: Build 'sig-ubuntu-1804' finished. ==> Builds finished. The artifacts of successful builds are: --> sig-ubuntu-1804: Azure.ResourceManagement.VMImage: OSType: Linux ManagedImageResourceGroupName: cluster-api-images ManagedImageName: capi-1234567890 ManagedImageId: /subscriptions/01234567-89ab-cdef-0123-4567890abcde/resourceGroups/cluster-api-images/providers/Microsoft.Compute/images/capi-1234567890 ManagedImageLocation: southcentralus ManagedImageSharedImageGalleryId: /subscriptions/01234567-89ab-cdef-0123-4567890abcde/resourceGroups/cluster-api-images/providers/Microsoft.Compute/galleries/ClusterAPI/images/capi-ubuntu-1804/versions/0.3.1234567890
Please also see the replication recommendations for the Shared Image Gallery.
If the image you want to use is based on an image released by a third party publisher such as for example
Flatcar Linux by
Kinvolk, then you need to specify the
sku fields as well:
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 kind: AzureMachineTemplate metadata: name: capz-shared-gallery-example spec: template: spec: image: sharedGallery: resourceGroup: "cluster-api-images" name: "capi-1234567890" subscriptionID: "01234567-89ab-cdef-0123-4567890abcde" gallery: "ClusterAPI" version: "0.3.1234567890" publisher: "kinvolk" offer: "flatcar-container-linux-free" sku: "stable"
This will make API calls to create Virtual Machines or Virtual Machine Scale Sets to have the
Plan correctly set.
To use a managed image resource by ID, only the
id field must be set:
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 kind: AzureMachineTemplate metadata: name: capz-image-id-example spec: template: spec: image: id: "/subscriptions/01234567-89ab-cdef-0123-4567890abcde/resourceGroups/myResourceGroup/providers/Microsoft.Compute/images/myImage"
A managed image resource can be created from a Virtual Machine. Please refer to Azure documentation on creating a managed image for more detail.
Managed images support only 20 simultaneous deployments, so for most use cases Shared Image Gallery is recommended.
To use an image from Azure Marketplace, populate the
version fields and, if this image is published by a third party publisher, set the
thirdPartyImage flag to
true so an image Plan can be generated for it. In the case of a third party image, you must accept the license terms with the Azure CLI before consuming it.
apiVersion: infrastructure.cluster.x-k8s.io/v1beta1 kind: AzureMachineTemplate metadata: name: capz-marketplace-example spec: template: spec: image: marketplace: publisher: "example-publisher" offer: "example-offer" sku: "k8s-1dot18dot8-ubuntu-1804" version: "2020-07-25" thirdPartyImage: true