Kubernetes Component Configuration - Rafay Product Documentation

Kubernetes Component Configuration

Kubelet Arguments

Kubelet settings can be customized to optimize configurations based on workload and operational requirements. Kubelet parameters are exposed in both cluster and node settings, allowing configuration at a global or node-specific level. In addition to key-value Kubelet arguments, structured KubeletConfiguration YAML is supported to configure any valid upstream Kubelet parameter.

Cluster-level settings are applied to all nodes. When a node defines its own configuration, the node-level settings take precedence for matching fields. For fields that do not exist at the node level, the cluster-level configuration continues to apply. Each node receives a merged configuration using this priority model.

Types of Kubelet Configuration

Kubelet configuration is supported through two mechanisms, depending on the level of control required:

1. Kubelet Extra Args (Key–Value Arguments)

Kubelet Extra Args allow configuring command-line style Kubelet parameters using simple key-value pairs. These fields are suitable for flat arguments such as --max-pods, timeouts, feature gates, and other non-nested flags. Multiple arguments can be added in the Add/Edit Kubelet Extra Args dialog, and the settings are applied at the cluster or node level.

This method is intended for straightforward tuning where parameters can be expressed as single key-value entries.

2. KubeletConfiguration Overrides (YAML-Based)

KubeletConfiguration Overrides provide support for the full upstream KubeletConfiguration API and allow defining structured YAML, including nested fields, boolean policies, durations, and advanced configuration blocks. This method enables configuring any valid Kubelet parameter, including complex fields that cannot be expressed through simple key-value arguments.

Overrides can be applied at both the cluster and node level, with node-level settings taking precedence when both levels define the same fields.

YAML Reference Example

A built-in reference panel is available to help construct valid KubeletConfiguration YAML. Selecting the help icon in the editor opens a read-only example based on the upstream KubeletConfiguration API. This example includes commonly used fields such as authentication, authorization, cgroup settings, cluster DNS configuration, and other standard parameters defined by Kubernetes.

The reference panel is intended as a guide for understanding the structure and formatting of the KubeletConfiguration specification. Values can then be added or modified in the editable YAML editor on the right, where cluster-level or node-level overrides are entered.

This reference is provided to ensure that KubeletConfiguration Overrides follow the correct structure, field types, and formatting as defined by Kubernetes.

The following are the default Kubelet arguments configured by Rafay. These arguments cannot be overridden, and attempting to override them will result in deployment failure. However, additional arguments can be added if needed.

KubeletConfiguration Examples

The following examples illustrate commonly used KubeletConfiguration fields that can be applied at either the cluster level or node level.

Example 1: Enable singleProcessOOMKill for selected nodes

singleProcessOOMKill: true

This configuration can be added as a node-level override so that the setting applies only to the selected nodes.

Example 2: Configure imageMaximumGCAge

imageMaximumGCAge: "1h"

Example 3: Combined advanced parameters

maxPods: 500
imageMinimumGCAge: "2m2s"
imageGCHighThresholdPercent: 80

Click Save after updating the configuration.

Kubelet Arguments can be added, edited, or deleted on Day 2. Kubelet Arguments cannot be modified at both the cluster level and node level simultaneously during Day 2 updates; changes can be applied either at the cluster level or in bulk for selected nodes. KubeletConfiguration Overrides are also supported on Day 2 and can be updated at the cluster level or node level as required.

Control Plane Overrides

Control Plane Overrides enable advanced customization of Kubernetes control plane components in upstream clusters. This feature allows modification of control plane arguments, volumes, and volume mounts without exposing or editing the full static pod manifest files.

This capability is supported only for control plane nodes and does not apply to worker nodes. There is no node-level configuration for this feature. All overrides are applied at the cluster level.

Purpose

Control Plane Overrides provide a controlled mechanism to customize Kubernetes control plane behavior while preserving required defaults and overall cluster stability.

This feature can be used to:

The complete YAML manifest for control plane components is not exposed. Only specific override sections are configurable.

What Can Be Overridden

Only the following sections of the control plane manifests can be modified:

  1. Extra args (key-value pairs)
  2. Extra volumes
  3. Extra volume mounts

Other sections such as container image, probes, resource settings, security context, networking configuration, and priority settings are not configurable.

Only the fields that require customization need to be provided. Existing values remain unchanged unless explicitly overridden.

Scope and Application

Editing Control Plane Overrides

Selecting the Edit option under Control Plane Overrides opens a tabbed configuration panel.

Separate tabs are available for:

Each tab provides:

Configuration entered in one tab applies only to that component.

Extra Args

Extra args allow configuration of control plane arguments using key-value pairs.

Each entry consists of:

Behavior

Example 1: Simple Argument

UI entry:

Key Value
profiling false

Result

The existing profiling setting is updated to false for the selected component across all control plane nodes.

Example 2: Comma-Separated Argument

Certain arguments accept comma-separated values, such as:

When modifying such arguments:

Example UI entry:

Key Value
enable-admission-plugins NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,Priority,ResourceQuota,NodeRestriction,EventRateLimit,PodSecurity

Result

The existing list is fully replaced with the provided value, preserving required entries and appending the new plugin.

If only PodSecurity is provided, all previously configured plugins are removed and cluster functionality may break.

This rule applies to all comma-separated arguments.

For supported parameters and default values, refer to the official Kubernetes documentation.

Extra Volumes

The Extra volumes section accepts YAML input in string format.

Example:

- name: safe-dir
  emptyDir: {}

Mandatory Rules

If formatting is incorrect or the hyphen is omitted, the control plane manifest may become corrupted and functionality may be affected.

Matching Behavior

The name field determines whether the configuration overrides or adds a volume.

Extra Volume Mounts

The Extra volume mounts section also accepts YAML input in string format.

Example:

- name: safe-dir
  mountPath: /tmp/safe-apiserver
  readOnly: false

Rules

Volumes and volume mounts must use consistent name values.

Cluster Specification Example

Control plane overrides are defined in the cluster specification under controlPlaneOverrides

Below is an example spec:

apiVersion: infra.k8smgmt.io/v3
kind: Cluster
metadata:
  name: test-pfug-2
  project: defaultproject
spec:
  blueprint:
    name: minimal
  config:
    autoApproveNodes: true
    kubernetesVersion: v1.33.5
    installerTtl: 365
    platformVersion: v0.1.0
    kubeletExtraArgs:
      max-pods: "600"
    controlPlaneOverrides:
      kubeApiserver:
        extraArgs:
          profiling: "false"
          v: "4"
        extraVolumeMounts: |
          - name: safe-dir
            mountPath: /tmp/safe-apiserver
            readOnly: false
        extraVolumes: |
          - name: safe-dir
            emptyDir: {}
      kubeControllerManager:
        extraArgs:
          profiling: "false"
          v: "2"
        extraVolumeMounts: |
          - name: safe-dir
            mountPath: /tmp/safe-controller
            readOnly: false
        extraVolumes: |
          - name: safe-dir
            emptyDir: {}
      kubeScheduler:
        extraArgs:
          profiling: "false"
          v: "2"
        extraVolumeMounts: |
          - name: safe-dir
            mountPath: /tmp/safe-scheduler
            readOnly: false
        extraVolumes: |
          - name: safe-dir
            emptyDir: {}
  network:
    cni:
      name: Calico
      version: 3.28.1
    podSubnet: 10.244.0.0/16
    serviceSubnet: 10.96.0.0/12
  nodes:
    - arch: amd64
      hostname: vih-12
      operatingSystem: Ubuntu22.04
      privateip: 10.0.0.128
      roles:
        - Master
        - Worker
        - Storage
      ssh:
        ipAddress: 129.146.39.135
        port: "22"
        privateKeyPath: /Users/vihari/.ssh/vihari_oci_ssh
        username: ubuntu

type: mks

Day 2 Operations

Control Plane Overrides can be updated after cluster provisioning.

Go to Clusters → Select Cluster → Configuration → Control Plane Overrides → Edit.

Viewing Applied Configuration

The applied control plane configuration can be viewed from the Nodes tab.

Go to Clusters → Select Cluster → Nodes → Node Actions → View Control Plane Config.

This opens a read-only view of the Kube API Server, Controller Manager, and Scheduler configuration for that node. Editing is not supported from this screen.

A read-only view of the full control plane YAML is available for each control plane node. This allows verification that overrides have been applied correctly without accessing the node directly.

Important Considerations