Architecture

The Kube API server is at the center of all operations within Kubernetes. Admins interact with it through the CLI or by accessing the API directly. Almost any operation on the cluster can be performed using these access patterns. As a result, securing the Kube API server SHOULD be the first line of defense for any organization.

KubeCTL is the preferred CLI for running commands against Kubernetes clusters. Cluster administrators use this CLI to manage their Kubernetes clusters. KubeCTL operations are sensitive to network latency. As a result, organizations follow the deployment patterns described below to enable remote operations on Kubernetes clusters.

Common Access Patterns

Publicly Accessible API Server

Often administrators publicly expose the API Server to enable remote management of the cluster using KubeCTL. There are several issues with this approach:

• It is grossly insecure and at minimum, the API Server is a target for DoS attacks
• Locking down access using IP/CIDR based access rules are operationally impractical

Important

Deploying k8s clusters with publicly accessible API servers is an extremely POOR security practice and is NOT recommended at all.

VPN or Bastion Hosts

An administrator may deploy a bastion host to secure access to the k8s API Server. The KubeCTL commands are performed from the bastion host. There are several issues with this approach:

• The bastion host becomes a single point of failure
• The security blast radius with a bastion host is significant i.e. the user has access to everything that the bastion can access
• The geo-location of the bastion host becomes critical when there are multiple clusters deployed globally
• Deploying and managing multiple bastions in multiple locations is operationally expensive and cumbersome
• Suboptimal placement of the bastion will introduce extra latency for user access and significantly impact their productivity and user experience

Kube API Access Proxy

Goals

The goals of the Kube API Access Proxy are

Great User Experience

• Low friction for users
• Users should be able to continue using "vanilla" KubeCTL. They should not be required to install any special client software
• Any existing scripts/tools using KubeCTL should continue to work

Great Admin Experience

• Cluster admins should not be required to install, configure and maintain anything
• The capability should be instantly available once the cluster is visible in the Web Console
• Should work across both imported and provisioned clusters

Centralized Security Enforcement

• Ability to define and manage a centralized "access policy" across a fleet of globally distributed clusters across multiple environments, providers
• No changes to firewall rules on remote clusters
• Support group membership-based policies
• Support time bound access policies
• Ability to perform "Instant Access Revocation" across an entire fleet of clusters for a group/user
• RBAC/ABAC policy enforcement at the network edge
• Ability to limit KubeCTL access only via browser based virtual terminal
• Ability to disable/enable KubeCTL access when required
• Implement a break glass process for KubeCTL access to sensitive production grade environments
• Ability to require "active authenticated session" with Web Console before using Kubeconfig file

Centralized Auditing

• Full auditing of KubeCTL activities with central access to audit logs for all user activity for security and governance

Great Performance

• The target k8s cluster can be deployed anywhere even behind a firewall
• Provide built-in high availability and no single point failure network path
• Provide a heavily optimized, low latency and highly reliable network path between KubeCTL and the target k8s cluster

Access Patterns

With the "Kube API Access Proxy", organizations can enable their users and CI/CD systems to securely interact with remote clusters without taking on unnecessary security risks. The image above describes some of the common access patterns possible

• User accesses cluster using the built-in, browser based KubeCTL shell in the Web Console
• User/System accesses cluster using their KubeCTL CLI with a downloaded Kubeconfig
• User/System accesses cluster using their Helm3 CLI which uses KubeCTL underneath
• User/System accesses cluster using the Kube APIs

Workflow (Access through console)

kube-API server (Cluster) Relay agent (Cluster) Controller User kube-API server (Cluster) Relay agent (Cluster) Controller User Login request through browser Determines Type of user (Local or IDP) Local or IDP user authentication Performs user authentication and authorization Assigns role(s) to the user for the session Initiates kubectl access through console Service account creation with configured inactive/idle period Service account created with appropriate role/role binding User executes kubectl commands Relay agent forwards user requests to kube-api server Check if service account needs to be removed based on configured idle period Initiation of service account, role, role binding removal Service account, role, role binding get removed

Workflow (Access through CLI)

kube-API server (Cluster) Relay agent (Cluster) Controller User kube-API server (Cluster) Relay agent (Cluster) Controller User Downloads kubeconfig from the controller Initiates a kubectl command (TLS connection, client authentication) Validates CN from the client certificate in kubectl request to identify user Performs user authentication and authorization Service account creation with configured inactive/idle period Service account created with appropriate role/role binding Relay agent forwards user requests to kube-api server Check if service account needs to be removed based on configured idle period Initiation of service account, role, role binding removal Service account, role, role binding get removed