Kubernetes Setup Guide on Suble.io

Introduction

Welcome to our detailed guide on setting up Kubernetes on Suble.io. As you've decided to embark on this journey, you probably already know the immense potential Kubernetes holds for managing containerized applications at scale. If you're still wrapping your head around the basics, we're here to shed some light before diving into the setup process.

Kubernetes, also known as K8s, is an open-source platform designed to automate deploying, scaling, and managing containerized applications. Containers allow you to bundle your software with all its dependencies, leading to efficient, reliable, and fast deployments. Kubernetes takes it a step further by managing a cluster of machines and orchestrating containers across them. From handling failover for your applications to providing a consistent environment for deployment, Kubernetes offers a range of advantages that help organizations streamline their operations.

Now, let's talk about our platform — Suble.io. We provide a virtualized environment to host your applications, whether they are small-scale projects or large enterprise solutions. For this guide, we recommend setting up Kubernetes on at least three VMs under the Mega Package (4GB each) to ensure sufficient resources for the cluster to function effectively. We also suggest assigning one floating IP address for seamless network access and flexibility.

This guide provides step-by-step instructions and useful tips to help you configure Kubernetes on Suble.io. Whether you're an experienced developer or a newcomer to container orchestration, this guide aims to get your Kubernetes cluster up and running efficiently.

Terminology and Notation

• local$ <command>: Run this command on your local computer.
• all$ <command>: Run this command on all servers as root.
• master$ <command>: Run this command on the master server as root.
• worker$ <command>: Run this command on all worker servers as root.

Step 1 - Preparing Resources

1. Create 3 VM instances, each with at least 4GB of memory (Mega Package). Use Ubuntu 20.04 as the operating system.
2. Order 1 Floating IP (optional but highly recommended). This will be used for the Load Balancer later.

Step 2 - Install containerd and Kubernetes Packages

Installing containerd

1. Download the containerd service file:

   all$ wget https://raw.githubusercontent.com/containerd/containerd/main/containerd.service
   all$ mv containerd.service /usr/lib/systemd/system/
   

2. Reload systemd unit files:

   all$ systemctl daemon-reload
   

3. Install containerd:

   all$ wget https://github.com/containerd/containerd/releases/download/v1.6.2/containerd-1.6.2-linux-amd64.tar.gz
   all$ tar Czxvf /usr/local containerd-1.6.2-linux-amd64.tar.gz
   all$ systemctl enable --now containerd
   

Installing runc

1. Download and install runc:

   all$ wget https://github.com/opencontainers/runc/releases/download/v1.1.6/runc.amd64
   all$ install -m 755 runc.amd64 /usr/local/sbin/runc
   

Setting Up CNI Plugins

1. Install CNI plugins:

   all$ wget https://github.com/containernetworking/plugins/releases/download/v1.2.0/cni-plugins-linux-amd64-v1.2.0.tgz
   all$ mkdir -p /opt/cni/bin
   all$ tar Czxvf /opt/cni/bin cni-plugins-linux-amd64-v1.2.0.tgz
   

Configuring containerd

1. Generate the default configuration:

   all$ mkdir -p /etc/containerd/
   all$ containerd config default | sudo tee /etc/containerd/config.toml
   

2. Edit /etc/containerd/config.toml:

   [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
       SystemdCgroup = true
   

3. Restart containerd:

   all$ systemctl restart containerd
   

Installing Kubernetes Packages

1. Add the Kubernetes apt repository:

   all$ curl -fsSL https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
   all$ cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
   deb http://packages.cloud.google.com/apt/ kubernetes-xenial main
   EOF
   all$ apt-get update
   all$ apt-get install kubeadm kubectl kubelet
   

Configuring Sysctl Settings

1. Load required kernel modules:

   all$ cat <<EOF | tee /etc/modules-load.d/k8s.conf
   overlay
   br_netfilter
   EOF
   all$ modprobe overlay
   all$ modprobe br_netfilter
   

2. Apply sysctl settings:

   all$ cat <<EOF | tee /etc/sysctl.d/k8s.conf
   net.bridge.bridge-nf-call-iptables = 1
   net.ipv4.ip_forward                = 1
   net.ipv6.conf.default.forwarding   = 1
   EOF
   all$ sysctl --system
   

Step 4 - Deploying and Configuring the Control Plane

1. On the master node, pull Kubernetes images:

   master$ kubeadm config images pull
   

2. Initialize the control plane:

   master$ kubeadm init        --pod-network-cidr=10.244.0.0/16        --kubernetes-version=v1.27.1        --ignore-preflight-errors=NumCPU        --upload-certs        --apiserver-cert-extra-sans 10.0.0.1
   

3. Configure kubectl for the root user:

   master$ mkdir -p /root/.kube
   master$ cp -i /etc/kubernetes/admin.conf /root/.kube/config
   

4. Deploy the Flannel CNI:

   master$ kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
   

Step 5 - Adding Worker Nodes

1. Generate the join command on the master node:

   master$ kubeadm token create --print-join-command
   

2. Execute the join command on each worker node:

   worker$ kubeadm join ...
   

3. Verify nodes:

   local$ kubectl get nodes
   

Step 6 - Setting Up External Connectivity

1. Install MetalLB:

   local$ kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.13.9/config/manifests/metallb-native.yaml
   

2. Configure MetalLB IP Pool:

   apiVersion: metallb.io/v1beta1
   kind: IPAddressPool
   metadata:
     name: first-pool
     namespace: metallb-system
   spec:
     addresses:
     - 89.23.86.53-89.23.86.53
   

   
   Apply the configuration:

   local$ kubectl -n metallb-system apply -f ip-pool.yml
   

3. Deploy the NGINX Ingress Controller:

   local$ helm pull oci://ghcr.io/nginxinc/charts/nginx-ingress --untar --version 0.17.1
   local$ cd nginx-ingress
   local$ helm install nginx-ingress .
   

4. Deploy a test web application:

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: web-app
   spec:
     replicas: 1
     ...
   

   local$ kubectl -n default apply -f web-app-deployment.yml
   

Congratulations! You now have a fully functional Kubernetes cluster on Suble.io with external connectivity.