Multi-cloud Kubernetes vs. multi-cluster Kubernetes

Multi-cloud Kubernetes is an architectural strategy that involves running a single Kubernetes cluster whose nodes exist in multiple clouds. In a multi-cloud Kubernetes setup, some nodes might operate in AWS, while others run in Azure, for example. All the nodes belong to the same cluster and are managed via the same control plane.

Multi-cloud Kubernetes is distinct from multi-cluster Kubernetes.

Multi-cluster Kubernetes uses multiple Kubernetes clusters at the same time, but this doesn't necessarily mean IT teams run Kubernetes across multiple clouds. They could set up two or more clusters within the same cloud to create a multi-cluster architecture.

In addition, multi-cluster Kubernetes results in two or more separate Kubernetes control planes, although a central management layer often unites them. This differs from multi-cloud Kubernetes, which has a single cluster and one control plane that manages resources spread across multiple clouds.

Benefits of multi-cloud Kubernetes clusters

Multi-cloud Kubernetes is not the most common way to deploy Kubernetes. Most people set up a cluster on a single cloud or infrastructure platform. However, building a multi-cloud cluster offers several potential advantages, such as the following:

• Reliability. If IT teams spread nodes across multiple clouds, workloads have a higher chance of remaining available if one of the clouds fails.
• Backup and disaster recovery. Nodes in multiple clouds provide a fast and easy way to move pods from one cloud to another in case one cloud goes down.
• Performance. Nodes in multiple clouds can improve workload performance in situations where teams need to serve users located in different regions. Teams can deploy pods on nodes that are close to each set of users, which reduces network latency.
• Cost management. The ability to create nodes based on VM instances from multiple cloud providers can lower overall costs. With a longer list of VM instances to choose from, teams can find the best cost-performance balance for each node.

How to set up a multi-cloud Kubernetes cluster

The process for setting up a multi-cloud Kubernetes cluster is relatively straightforward. Here are the basic steps:

• Deploy VM instances in multiple clouds. You need a minimum of one VM in each cloud.
• Install Kubernetes on all the VMs so they can operate as nodes within the Kubernetes cluster.
• Configure at least one node to serve as the control plane node. For high availability setups, you can configure multiple control plane nodes.
• Join additional nodes to the cluster as worker nodes.

Here's an overview of how to implement this setup using nodes based in AWS and Azure.

Step 1. Create AWS and Azure VMs

First, create your VMs.

In AWS, you can create a VM in the console or use a command like the one below. Note: Substitute a valid Amazon Machine Image ID for ami-xxxxxxxx.

aws ec2 run-instances --image-id ami-xxxxxxxx --count 1 --instance-type t2.micro

On Azure, launch a VM instance with the following code:

az vm create \
  --resource-group $RESOURCE_GROUP_NAME \
  --name $VM_NAME \
  --image $VM_IMAGE \
  --admin-username $ADMIN_USERNAME \
  --generate-ssh-keys \
  --public-ip-sku Standard

Ensure the VMs are provisioned with IP addresses that are routable from the internet, as nodes based in different clouds need to connect to each other over the internet. Alternatively, if you don't want to connect your nodes to the internet, set up a private connectivity tunnel between your two clouds.

Step 2: Install Kubernetes on all VMs

Next, connect to each VM via SSH, and install Kubernetes with the following command:

sudo apt-get install -y kubelet kubeadm kubectl

This command works on modern Ubuntu-based VMs. If you provision VMs with a different Linux-based OS, refer to the Kubernetes documentation for installation instructions.

This command installs the essential Kubernetes components, including kubeadm, which we use in the steps below to configure VMs to serve as either control plane or worker nodes.

Step 3. Configure a control plane node

Now, you're ready to set up a control plane node. Log in to the VM you want to use as the control plane node. Then, use kubeadm to set it up, as seen here:

sudo kubeadm init --control-plane-endpoint=$IPADDR --pod-network-cidr=$POD_CIDR --node-name $NODENAME --apiserver-cert-extra-sans=$IPADDR --ignore-preflight-errors Swap

Step 4. Install Kubernetes on worker nodes

You can now SSH into the other VM instances and set them up as worker nodes joined to the cluster. This cluster is managed by the control plane node you configured in the previous step.

To perform this step, first log in to the control plane node you configured previously, and run the following command:

kubeadm token create --print-join-command

This command outputs a string, which contains the information you need to join other nodes to the cluster, such as the control plane's IP address, port and generated token. Copy this output. Then, log in to the other VM instances, and run the following command:

sudo kubeadm join 1.2.3.4:6443 --token j4eice.33vgvgyf5cxw4u8i \
     --discovery-token-ca-cert-hash
sha256:a2ea4c87e83eab70edc4f39c2e7077389c3dd010c20cadfb9c58d7278cc3deec

Be sure to specify the proper IP address for the control plane node (1.2.3.4 in the example above). Also, modify the other variables as appropriate based on the output of the kubeadm command you ran on the control plane node.

At this point, you have a cluster that includes nodes running in both AWS and Azure. You can use these nodes to deploy workloads that span multiple clouds but are managed as part of a single Kubernetes cluster.