Maximizing Resource Efficiency: Cost Optimization in Multi-Tenant vClusters

Cost can be a significant concern for large organizations operating Kubernetes clusters at scale, especially when managing clusters with bigger teams. The provisioning and maintenance of these clusters often pose challenges. In this blog, we will dive into the concept of the multi-tenant approach and draw a comparison with the multi-cluster approach. Finally, we will explore the vcluster and its potential in addressing these challenges.

What is Multi-Tenancy?

Multi-tenancy is an approach where a single infrastructure resource, such as a server or a cluster, can be utilized by multiple tenants simultaneously.

A tenant refers to a separate entity or user group that requires its own isolated environment with dedicated resources.

Scenario

Imagine an organization with 10 ongoing projects, each with its dedicated team. In this scenario, the applications for these projects are deployed using Kubernetes, and each team would require its own development cluster, test cluster, and production cluster.

The Multi-Cluster Approach

It becomes evident that we could potentially have nearly 30 running clusters (multiple clusters), and this situation poses significant challenges in terms of management and resource utilization. The need to physically maintain each cluster presents a logistical burden, and the associated costs escalate due to increased resource consumption by master nodes and the implementation of high-availability features.

Managing Multi-Cluster

There are tools available, such as Rancher, that can help you manage multiple Kubernetes clusters running on-premises and in the cloud as a single pane of glass. However, even with these tools, running multiple Kubernetes clusters can be expensive.

Why Multi-Cluster is expensive

• The cost of the hardware and software required to run each cluster.
• The cost of the staff required to manage and maintain each cluster.
• The cost of the networking infrastructure required to connect the clusters.

Namespace isolation

One way to achieve multi-tenancy in Kubernetes is to use namespace isolation. The organization gets only one cluster and a dedicated team to manage the cluster they are also responsible for providing namespaces to teams.

Namespaces provide a way to logically isolate resources within a single Kubernetes cluster. This means that each tenant can have their own namespace, and their resources will be isolated from the resources of other tenants.

Namespace isolation is a simple and effective way to achieve multi-tenancy in Kubernetes. However, it does not provide complete isolation between tenants. For example, tenants can still see each other’s pods and services.”

Scenario

Let’s say you have two tenants, Team A and Team B. You want to create a namespace for each tenant so that their resources are isolated from each other.

To do this, you would create two namespaces, one for Team A and one for Team B. They will then deploy their applications to their appropriate namespace.

Challenges with Namespace Multi-Tenancy:

1. Providing Access to Teams and Developers: One challenge is determining how to grant access to teams and developers in a shared cluster.
2. Restricting Access: In addition to granting access, it is crucial to restrict access appropriately.
3. Resource Restriction: Controlling and limiting the allocation of cluster resources is another challenge. It is important to prevent any single team or project from monopolizing resources, ensuring fairness and efficient utilization across all projects within the shared environment.
4. Application and Project Isolation: Achieving isolation between applications or projects within the shared cluster is a challenge. This separation is particularly vital in development environments where rigorous testing takes place and production environments where the live application runs. Each application or project must operate independently without interfering with or affecting others.

vCluster the Ultimate Multi-Tenant Approach

Virtual clusters are Kubernetes clusters that exist within another Kubernetes clusters. They reuse the worker nodes and networking of the host cluster, but they have their own control plane. This means that virtual clusters are completely isolated from each other, even though they share the same underlying infrastructure.

Virtual clusters are similar to virtual machines in that they partition a single physical cluster into multiple separate ones. However, virtual clusters are more lightweight than virtual machines, and they do not require any additional hardware.

How to install vcluster

First Download the vcluster Cli by running the following in macOS you can check here for other versions.

curl -L -o vcluster "https://github.com/loft-sh/vcluster/releases/latest/download/vcluster-darwin-amd64" && sudo install -c -m 0755 vcluster /usr/local/bin && rm -f vcluster

To confirm if installed check the version of vcluster

vcluster --version

Creating vclusters

In order to create a vcluster using the vcluster cli just run the following

vcluster create team-a
vcluster create team-b

This will take a few minutes to be ready let's dive first into the architecture to understand what is happening under the hood.

When you run that command the following things happen:

1. Creating namespace vcluster-team-a: A new namespace called "vcluster-team-a" is being created.
2. Detected local Kubernetes cluster minikube...: The system has detected that the local Kubernetes cluster being used is "minikube."
3. Deploy vcluster with a NodePort & sync real nodes: The virtual cluster "team-a" will be deployed using a NodePort service type.
4. Vcluster by default comes as a helm chart but you can also use the kubectl apply

What happened in the host namespace?

When you deploy vcluster it created a new namespace and the cluster is deployed in the namespace

kubectl get ns

we can view the pods and see what is running in the vcluster:

kubectl get pods -n vcluster-team-a

By default, virtual clusters (vclusters) are deployed as a single pod, managed by a StatefulSet, which consists of two containers:

It also deploys a coredns pod which will be used for networking inside the new virtual cluster

vCluster components

Control Plane:

This container encompasses essential components such as the API server, controller manager, and a connection or mounting point to the data store.

By default, vclusters utilize sqlite as the data store and run the API server and controller manager provided by k3s.

k3s is a certified Kubernetes distribution and a project supported by CNCF sandbox.

Syncer:

The distinctive feature of a vcluster lies in its virtual nature, meaning it does not possess actual worker nodes or a dedicated network. Instead, a syncer is employed.

The syncer’s role is to replicate or copy the pods created within the vcluster onto the underlying host cluster. Once replicated, the host cluster takes charge of scheduling and running these pods.

The vcluster continuously ensures synchronization between the vcluster pod and the corresponding host cluster pod to maintain consistency.

vcluster Control Plane

1. Kubernetes API server: This is the endpoint to which you direct your kubectl requests specifically for the vcluster. It handles communication and manages the vcluster’s resources.
2. Data store: The API server stores all the resources related to the vcluster in a data store. In real clusters, the commonly used data store is etcd, which stores the state of the cluster.
3. Controller Manager: The Controller Manager is responsible for maintaining the desired state of various objects within the vcluster. It ensures that the number of replicas defined in ReplicaSets and other controllers match the actual state in the data store by creating or deleting pod objects as needed.
4. Scheduler (Optional) : The Scheduler is responsible for distributing and assigning workloads (pods) to specific nodes within the vcluster. It considers various factors, such as resource availability and constraints, to make optimal scheduling decisions.

Vcluster Syncer

The vcluster utilizes a syncer mechanism that duplicates the pods created within the vcluster to the underlying host cluster. This ensures that the pods are also available and scheduled within the host cluster. The syncer keeps both the vcluster pods and the host cluster pods synchronized, ensuring consistency between the two.

Kubernetes Resources

At a high level, the vcluster operates purely in a virtual environment. When you create Kubernetes resource objects using the vcluster API server, they are stored exclusively in the vcluster’s data store, which is typically sqlite (although other external data store options are available).

This applies specifically to higher-level Kubernetes resources like:

1. Deployments,
2. StatefulSets,
3. Custom Resource Definitions (CRDs).

These objects exist solely within the virtual cluster and do not propagate to the API server or data store (etcd) of the underlying host cluster.

However, to actually launch containers and make them functional, the vcluster synchronizes certain low-level resources such as Pods and ConfigMaps mounted within Pods to the corresponding namespaces in the underlying host cluster.

This synchronization allows the scheduler of the underlying host cluster to schedule these pods effectively. Essentially, the vcluster bridges the gap between the purely virtual environment and the underlying host cluster by synchronizing specific resources required for container execution.

Networking in vclusters

Pod-To-Pod Traffic:

The syncer component of the vcluster ensures that pods run within the host namespace of the underlying cluster. Consequently, these pods possess standard cluster-internal IP addresses, enabling communication between them via IP-based networking.

Pod-To-Service Traffic:

By default, the vcluster also synchronizes Services (while removing unnecessary information) to facilitate pod-to-service communication. However, instead of relying on the DNS names of services within the host cluster, the vcluster employs its own DNS service. This DNS service enables vcluster pods to utilize intuitive DNS mappings, similar to those in a typical cluster.

Running applications in vclusters

Let us deploy our first application in vcluster, We will be using the stefan podinfo application to demonstrate this.

deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: podinfo
spec:
  minReadySeconds: 3
  revisionHistoryLimit: 5
  progressDeadlineSeconds: 60
  strategy:
    rollingUpdate:
      maxUnavailable: 0
    type: RollingUpdate
  selector:
    matchLabels:
      app: podinfo
  template:
    metadata:
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "9797"
      labels:
        app: podinfo
    spec:
      containers:
      - name: podinfod
        image: ghcr.io/stefanprodan/podinfo:6.4.0
        imagePullPolicy: IfNotPresent
        ports:
        - name: http
          containerPort: 9898
          protocol: TCP
        - name: http-metrics
          containerPort: 9797
          protocol: TCP
        - name: grpc
          containerPort: 9999
          protocol: TCP
        command:
        - ./podinfo
        - --port=9898
        - --port-metrics=9797
        - --grpc-port=9999
        - --grpc-service-name=podinfo
        - --level=info
        - --random-delay=false
        - --random-error=false
        env:
        - name: PODINFO_UI_COLOR
          value: "#34577c"
        livenessProbe:
          exec:
            command:
            - podcli
            - check
            - http
            - localhost:9898/healthz
          initialDelaySeconds: 5
          timeoutSeconds: 5
        readinessProbe:
          exec:
            command:
            - podcli
            - check
            - http
            - localhost:9898/readyz
          initialDelaySeconds: 5
          timeoutSeconds: 5
        resources:
          limits:
            cpu: 2000m
            memory: 512Mi
          requests:
            cpu: 100m
            memory: 64Mi

Services.yaml

apiVersion: v1
kind: Service
metadata:
  name: podinfo
spec:
  type: ClusterIP
  selector:
    app: podinfo
  ports:
    - name: http
      port: 9898
      protocol: TCP
      targetPort: http
    - port: 9999
      targetPort: grpc
      protocol: TCP
      name: grpc

Apply the configurations above and let's check what was deployed in our application

As we can see the application was created with all its components.

Let's now switch back to the Main Cluster

vcluster disconnect

First, let's view what is going on in the namespaces.

kubectl get ns

Here we can see that we have a namespace created for Team-a which contains the vcluster.

Next, let's view what is running in the namespace.

kubectl get all -n vcluster-team-a

Although we deployed a deployment for the pod info we can only see the pods that are running since the high-level resources are only running in the virtual machine while the low level resources like the pod, services, and container are being synced in the main cluster.

This now brings the real benefits of a multi-tenant approach and isolation of different applications and teams.

In this blog, we have explored the benefits of utilizing vclusters, which include cost reduction and enhanced isolation for teams and applications. However, managing multiple vclusters can pose a new challenge. This is where Loft Labs enters the scene. Loft Labs is the developer of the open-source vcluster project and offers an Enterprise version with a plethora of features tailored for larger teams.

To dive deeper into Loft Labs and its offerings, stay tuned for my upcoming blog post. If you have any inquiries or would like to connect, feel free to reach out via email at onai.rotich@gmail.com or through Twitter or LinkedIn. Together, let’s simplify Kubernetes management and make it more accessible.