Skip to content
samperrin.com, home
Development

Cluster API Workload Cluster (vSphere)

Deploy your first workload cluster with Cluster API

Sam Perrin
3 min read

Prerequisites

  • PhotonOS Kubernetes template within vSphere
  • HAProxy template within vSphere
  • Working Management Cluster (follow these steps Cluster API Setup Steps (vSphere))
  • Access to the Management Cluster with kubectl
  • Access to a machine with clusterctl

Create clusterctl.yaml to store configuration values. Replace the values where required. Take particular note of values between < and >.

Shell
mkdir -p $HOME/.cluster-apitee $HOME/.cluster-api/clusterctl.yaml >/dev/null <<EOF## -- Controller settings -- ##VSPHERE_USERNAME: "administrator@vsphere.local"            # The username used to access the remote vSphere endpointVSPHERE_PASSWORD: "vmware1"                                # The password used to access the remote vSphere endpoint ## -- Required workload cluster default settings -- ##VSPHERE_SERVER: "vcenter1.domain.com"                                 # The vCenter server IP or FQDNVSPHERE_DATACENTER: "Datacenter"                                      # The vSphere datacenter to deploy the management cluster onVSPHERE_DATASTORE: "Datastore"                                        # The vSphere datastore to deploy the management cluster onVSPHERE_NETWORK: "VM Network"                                         # The VM network to deploy the management cluster onVSPHERE_RESOURCE_POOL: "<ClusterName>/Resources/<ResourcePoolName>"   # The vSphere resource pool for your VMsVSPHERE_FOLDER: "vm/<FolderName>/<ChildFolderName>"                   # The VM folder for your VMs. Set to "" to use the root vSphere folderVSPHERE_TEMPLATE: "photon-3-kube-v1.18.2"                             # The VM template to use for your management cluster.VSPHERE_HAPROXY_TEMPLATE: "capv-haproxy-v0.6.4"                       # The VM template to use for the HAProxy load balancerVSPHERE_SSH_AUTHORIZED_KEY: "<ssh key>"                               # The public ssh authorized key on all machines in this cluster. Set to "" if you don't want to enable SSH, or are using another solution.EOF

Change the value of WORKLOAD_CLUSTER_NAME to be the name of your new workload cluster.

Shell
export WORKLOAD_CLUSTER_NAME="wlc01"mkdir -p $HOME/$WORKLOAD_CLUSTER_NAMEclusterctl config cluster $WORKLOAD_CLUSTER_NAME --infrastructure vsphere --kubernetes-version v1.18.2 --control-plane-machine-count 1 --worker-machine-count 3 > $HOME/$WORKLOAD_CLUSTER_NAME/cluster.yaml

Review the cluster config file and make any required changes. Some suggestions to review are CPU, Memory, Storage and Pods CIDR range.

You can use sed to do a find and replace within the file, the below examples are for changing the Storage and Memory on all virtual machines as part of this deployment and the Pods CIDR.

  • Pod CIDR from 192.168.0.0/16 to 192.168.200.0/24
  • Memory from 8192 to 4096
  • Storage from 25 to 20
Shell
sed -i "s/192.168.0.0\/16/192.168.200.0\/24/g" $HOME/$WORKLOAD_CLUSTER_NAME/cluster.yamlsed -i "s/memoryMiB: 8192/memoryMiB: 4096/g" $HOME/$WORKLOAD_CLUSTER_NAME/cluster.yamlsed -i "s/diskGiB: 25/diskGiB: 20/g" $HOME/$WORKLOAD_CLUSTER_NAME/cluster.yaml

Deploy the cluster components. Monitor the deployment of virtual machines within vSphere.

Shell
kubectl apply -f $HOME/$WORKLOAD_CLUSTER_NAME/cluster.yaml

Use kubectl to see cluster progress. The control planes won’t be Ready until we install a CNI in a later step.

Shell
kubectl get cluster --all-namespaceskubectl get kubeadmcontrolplane --all-namespaces

Retrieve the kubeconfig file for the workload cluster, and set it as KUBECONFIG environment variable.

Shell
kubectl get secret $WORKLOAD_CLUSTER_NAME-kubeconfig -o=jsonpath='{.data.value}' | { base64 -d 2>/dev/null || base64 -D; } > $HOME/$WORKLOAD_CLUSTER_NAME/kubeconfigexport KUBECONFIG="$HOME/$WORKLOAD_CLUSTER_NAME/kubeconfig"

Deploy Calico CNI to $WORKLOAD_CLUSTER_NAME. (or any other CNI)

Because we have set KUBECONFIG in the above step our kubectl commands will run against the local-control-plane cluster. If you want to be certain, add --kubeconfig="$HOME/$WORKLOAD_CLUSTER_NAME/kubeconfig" to the commands.

Shell
kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml#Example with kubeconfig specified#kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml --kubeconfig="$HOME/$WORKLOAD_CLUSTER_NAME/kubeconfig"

Reset your KUBECONFIG back to the management cluster

unset KUBECONFIGexport KUBECONFIG=$HOME/local-control-plane/kubeconfig

You will now be able to continue performing cluster management operations. Some examples below.

Shell
#Retrieve related vSphere VMskubectl get vspheremachine#Retrieve related machineskubectl get machine#Retrieve additional machine detailskubectl describe machine <machine-name>#Retrieve additional cluster informationkubectl describe cluster local-control-plane#Get machinedeployment (replicaset equivalent for machines)kubectl get machinedeployment#Scale worker nodes machinedeploymentkubectl scale machinedeployment $WORKLOAD_CLUSTER_NAME-md-0 --replicas=4#Get various machine resources at oncekubectl get cluster,machine,machinesets,machinedeployment,vspheremachine#Get most/all Cluster API resourceskubectl get clusters,machinedeployments,machinehealthchecks,machines,machinesets,providers,kubeadmcontrolplanes,machinepools,haproxyloadbalancers,vsphereclusters,vspheremachines,vspheremachinetemplates,vspherevms

The next article will cover deploying an application to your new workload cluster.

Previous Article in this Series: Cluster API Setup Steps (vSphere)

DevelopmentKubernetesVMware