Multi-tenancy on K8s- vCluster - Rafay Product Documentation
Overview
This system template enables the deployment of a virtual cluster (vCluster) on a host Kubernetes cluster, providing a lightweight, isolated environment for multi-tenant workloads or use cases requiring Kubernetes-level abstraction.
The template provisions a vCluster within a custom namespace, complete with preconfigured resource quotas to ensure controlled and efficient resource allocation. The vCluster operates independently within the host Kubernetes cluster, offering logical isolation while utilizing shared infrastructure.
As part of the output, users receive a kubeconfig file with the necessary privileges, enabling secure access to the vCluster.
For detailed steps to get started, refer to the vCluster System Template Get Started Guide.
Initial Setup
The platform team is responsible for performing the initial configuration and setup of the vCluster template. The sequence diagram below outlines the high-level steps. In this process, the platform team will configure and share the template from the system catalog to the project they manage and then share the template downstream with the end user.
End User ProjectSystem CatalogPlatform AdminEnd User ProjectSystem CatalogPlatform AdminSelects vCluster Template from System CatalogShares Template with Predefined ControlsTemplate Available in End User's Project
End User Flow
The end user launches a shared template, provides required input values, and deploys the virtual cluster.
Host Kubernetes ClusterRafay ProjectEnd UserHost Kubernetes ClusterRafay ProjectEnd UserLaunches Shared Template for vClusterProvides Required Input Values (API Key, Namespace, Blueprint, Agent)Clicks "Deploy"Provisions a vCluster within a Custom NamespacevCluster Deployed SuccessfullyProvides Kubeconfig File as Output
The templates are designed to support both:
- Day 0 operations: Initial setup
- Day 2 operations: Ongoing management
Resources
A virtual Kubernetes cluster running inside the custom namespace, operating independently while sharing the host cluster infrastructure
Pre-Requisites
- Host Cluster: Ensure that a Kubernetes host cluster is available and ready for vCluster deployment
- Rafay Configuration: At template launch, supply the following configuration values
- API Key
- REST Endpoint
- Agent Configuration: A configured agent must be available in the project where the template will be used, and it should be in a healthy state. Follow these instructions to deploy a new agent. Existing agents within the project can be reused if available
Tenant Isolation
To ensure complete isolation for vClusters within a Kubernetes environment, several Kubernetes features and best practices can be leveraged:
Namespace Isolation: Each vCluster operates within its own namespace, providing a level of separation from other namespaces. This is achieved using standard Kubernetes namespace isolation, ensuring that resources are scoped to the specific vCluster.
Network Policy: A network policy can be implemented to isolate the vCluster namespace from other namespaces, controlling the flow of network traffic between them. This ensures that only authorized traffic can reach the vCluster, further enhancing security.
Kata Runtime Class: The Kata runtime class can be used to force the vCluster and its deployed pods to run in a dedicated runtime class, providing an additional layer of isolation by using lightweight virtual machines (VMs) instead of traditional containers.
OPA Gatekeeper for Admission Control: Open Policy Agent (OPA) Gatekeeper can be used to enforce policies for the vCluster during the admission process, ensuring compliance and preventing the deployment of unauthorized configurations.
Configuration
At template launch, provide the required configuration values as exposed by the Platform Admin. This may include:
- Credentials:
- API_KEY: Rafay controller API key
- vCluster Configuration:
- vCluster Name: Specify the name of the virtual cluster
- Host Cluster Name: Select the host Kubernetes cluster for the vCluster
- Namespace: Provide the namespace where the vCluster will be deployed
After entering the required information, click Deploy to initiate the vCluster provisioning.
Input Variables for vCluster System Template
General Configuration
| Name | Value Type | Description |
|---|---|---|
| Distribution | Text | Kubernetes distribution used for the vCluster |
| Blueprint | Text | Name of the blueprint applied to the vCluster |
| Blueprint Version | Text | Version of the blueprint applied to the vCluster |
| Host Cluster Name | Text | Name of the host Kubernetes cluster where the vCluster runs |
| Host Project | Text | Project where the host cluster resides |
| Namespace | Text | Namespace where the vCluster is deployed |
| Project | Text | Project where the vCluster is managed |
| vCluster Name | Text | Name of the virtual cluster |
| vCluster Store Size | Text | Storage size allocated for the vCluster |
| vCluster Version | Text | Version of the vCluster distribution |
| Username | Text | Username used for accessing the vCluster |
Namespace Configuration
| Name | Value Type | Description |
|---|---|---|
| Namespace Annotations | JSON | Annotations applied to the vCluster namespace |
| Namespace Labels | JSON | Labels applied to the vCluster namespace |
| Namespace Quotas | JSON | Resource quotas applied to the vCluster namespace |
| Namespace Quota Size | Text | Predefined quota size for the vCluster namespace |
Access Configuration
| Name | Value Type | Description |
|---|---|---|
| Allowed Namespaces | JSON | List of namespaces the vCluster can access |
| Enable Network Policy | Text | Enables or disables network policies for the vCluster |
| Enable Kata Runtime | Text | Enables or disables Kata Containers runtime in the vCluster |
| Enable Kube Config | Text | Enables or disables generation of the kubeconfig |
| Tolerations | JSON | Tolerations applied to the vCluster pods |
Launch Time
The estimated time to launch a vCluster using this template is approximately 4 minutes.