# Shared clusters ## What Will You Do In this part of the self-paced exercise, we will focus on the following use case: _Clusters are shared between teams. Platform teams want to enforce governance controls but not at the cost of slowing down application teams._ For this example, we will consider two teams, **alpha** and **beta** sharing a cluster, **shared-cluster**. 1. Step 1: Create a "central" project for infrastructure management and to enforce governance (through blueprints etc.) 2. Step 2: Create projects for the two teams (e.g. alpha & beta) 3. Step 3: Import the shared cluster to the "central" project by applying the bootstrap 4. Step 4: Share the imported cluster with projects, alpha and beta. Apply resource quotas 5. Step 5: Add labels to existing namespaces in the imported cluster to align with the teams that own and operate them 6. Step 6: Create a blueprint with "namespace sync on" and apply it to the cluster 7. Step 7: Verify that the namespaces are imported to the right projects and are healthy 8. Step 8: Create "project specific roles" for users in alpha/beta teams and verify access Declarative specifications are also provided for this exercise. ## Step 1: Create a central project for the platform team - Login into your Org as an Org Admin - Click on **New Project** - Provide the name for the project (e.g. central) and a suitable description - Click **Create** ## Step 2: Create projects for the two teams - Click on **New Project** - Provide names for the projects (e.g. alpha and beta) and a suitable description - Click **Create** ## Step 3: Import the shared cluster to the "central" project - Navigate to project **central** - Click on **New Cluster** - Select **Import Existing Kubernetes Cluster** - Click **Continue** - Select **Environment** and **Kubernetes Distribution** as appropriate - Provide a cluster name (e.g. shared-cluster-alpha-beta) and description as needed - Click **Continue** - We will keep the Blueprint selection as minimal for now, click **Continue** - Download the cryptographically unique bootstrap YAML required to register the cluster - Apply bootstrap to the clusters to initiate the import process Use kubectl to apply the "cluster bootstrap" file on your K8s cluster. ```bash kubectl apply -f [path to file]/shared-cluster-alpha-beta-bootstrap.yaml ``` This will create a namespace for the k8s mgmt operator, download the container images, and register with the controller. This one time import process can take ~2 minutes. ```bash namespace/rafay-system created serviceaccount/system-sa created clusterrole.rbac.authorization.k8s.io/rafay:manager created clusterrolebinding.rbac.authorization.k8s.io/rafay:rafay-system:manager-rolebinding created clusterrole.rbac.authorization.k8s.io/rafay:proxy-role created clusterrolebinding.rbac.authorization.k8s.io/rafay:rafay-system:proxy-rolebinding created priorityclass.scheduling.k8s.io/rafay-cluster-critical created role.rbac.authorization.k8s.io/rafay:leader-election-role created rolebinding.rbac.authorization.k8s.io/rafay:leader-election-rolebinding created customresourcedefinition.apiextensions.k8s.io/namespaces.cluster.rafay.dev created customresourcedefinition.apiextensions.k8s.io/tasklets.cluster.rafay.dev created customresourcedefinition.apiextensions.k8s.io/tasks.cluster.rafay.dev created service/controller-manager-metrics-service-v3 created deployment.apps/controller-manager-v3 created configmap/connector-config-v3 created configmap/proxy-config-v3 created deployment.apps/rafay-connector-v3 created service/rafay-drift-v3 created validatingwebhookconfiguration.admissionregistration.k8s.io/rafay-drift-validate-v3 created ``` Once the k8s operator is operational, it will "establish and maintain a heartbeat" with the controller. ## Step 4: Share the imported cluster and apply resource quotas ### Share the imported cluster - Navigate to **central** project - Click on the gear icon next to the shared cluster (e.g. shared-cluster-alpha-beta) and select the **Manage Cluster Sharing** option from the dropdown - Select the **Specific Projects** options for Sharing - Select **alpha** and **beta** from the list - Click **Save** ### Apply resource quotas - Click on the gear icon next to project **alpha** in the listing page - Click on the **Resource Quotas** tab - Configure Project Limit (per cluster limit) and Namespace Limit as appropriate Important: Project Resource quotas ensure that the cluster resource utilization within that specific project context never exceeds the configured limit. This makes sure that teams cannot consume resources (more than what is assigned) and impact other teams with whom the cluster is being shared. Repeat the same exercise for project **beta**. ## Step 5: Add labels to existing namespaces In this specific example, we have two existing namespaces (alpha1, alpha2) in the shared cluster that belong to the alpha team and two existing namespaces (beta1, beta2) that belong to the beta team. There is also an "infra" namespace where common services such as security tooling etc. have been installed by the platform. ```bash kubectl get ns ``` ```bash NAME STATUS AGE alpha1 Active 89s alpha2 Active 86s beta1 Active 82s beta2 Active 79s default Active 62m infra Active 66s kube-node-lease Active 62m kube-public Active 62m kube-system Active 62m rafay-system Active 25m ``` The platform offers a **namespace reconciliation** feature that can synchronize existing namespaces to the platform and convert them into **managed namespaces**. As part of this process, we would want to make sure that namespaces alpha1/alpha2 get synchronized to alpha project and namespaces beta1/beta2 and beta project. In order to do this, add **k8smgmt.io/project** label with the project name to each of the namespaces. ```bash kubectl label ns alpha1 k8smgmt.io/project=alpha kubectl label ns alpha2 k8smgmt.io/project=alpha kubectl label ns beta1 k8smgmt.io/project=beta kubectl label ns beta2 k8smgmt.io/project=beta ``` If the label is not added, the namespace gets synchronized to the parent project (which is "central" project in this case). We will not add a label to "infra" namespace, it will be synchronized to the central project. ## Step 6: Apply a blueprint with "namespace sync on" - Navigate to the **central** project - Click on **New Blueprint** - Provide a name (e.g. central-blueprint) and an appropriate description - Click **Save** - Provide a version name (e.g. v1), select **minimal** as the base blueprint - Turn the **Enable Namespace Sync** on - Click **Save Changes** - Navigate to the shared cluster (e.g. shared-cluster-alpha-beta) - Click on the gear icon next to the cluster and select the **Update Blueprint** option from the dropdown - Select the blueprint and the version (e.g. central-blueprint, v1) - Click **Save and Publish** - Wait for the Blueprint sync process to complete ## Step 7: Verify that namespaces are imported to the right projects Navigate to the namespaces page in the **central** project. You should be able to see the "infra" namespace listed within this project context. You should be able to similarly see alpha1/alpha2 namespaces in alpha project and beta1/beta2 namespaces in beta project. ## Step 8: Create "project specific roles" for users - Login into your Org as an Org Admin - Navigate to System -> Users - Click on **New User** - Input appropriate details (do not add any groups as yet) - Click **Save & Exit** - You will see a pop-up stating that the user has not been assigned to any groups. Select **Yes** to continue - Select the user that was just created - Click on the Projects tab - Select **Assign User To Project** - Select **alpha** project from the dropdown on the left - Select the **Workspace Admin** role - Click **Save & Exit** The workspace admin role allows the user to: - Create a new namespace - Update resource quotas on existing namespaces - Delete an existing namespace All of these actions listed above can be performed within their specific project context (i.e. they cannot delete namespaces belong to other teams in the shared cluster etc.) and is also constrained by the governance controls put in place by the platform teams (e.g. creation of a new namespace is blocked if the project resource quota is exceeded). ## Recap Congratulations! At this point, you have successfully leveraged "Projects" and "Workspace as a service" capabilities to achieve isolation between two teams that share a cluster.