Aug

v2.8 - Self Hosted

23 Aug, 2024

This release introduces:

• Fresh Controller Installation:
  • A new controller installation is available for EKS, GKE, and Airgapped Controller environments.
• Upgrade Path for 2.7 to 2.8:
  • An upgrade path is provided from version 2.7 to 2.8 for GKE, EKS, and Airgapped controllers.
• Config Builder Tool:
  • The 2.8 controller package includes a convenient config builder tool.
  • This command-line tool simplifies configuration by prompting users for input and generating YAML files for Baremetal, EKS, and GKE environments.
  • Usage:

    ./configbuilder baremetal
    

• Rcollect Enhancements The Rcollect tool has been enhanced with the following improvements:
  • Background Execution: Rcollect can now be run in the background as a terminal process.

    ./rcollect kuberesources --background --timeout 10m --since 10h
    

  • Minion and Master Version Collection: Rcollect now collects minion and master versions as part of the log collection process.
  • Timeout Flag: A new timeout flag allows you to specify a duration for Rcollect to run.

Amazon EKS

Enabling secret encryption on an existing cluster

Currently, secret encryption with a KMS key can only be configured during cluster creation (Day 0) using the 'save and customize' option. Support is now added to enable secret encryption on existing clusters (Day 2) as well.

For more information on this feature, please refer here.

Day 0 Configuration

For new clusters, you can find this configuration in the cluster settings. Based on valid cloud credentials, you will see the option to enable secret encryption during Day 0 configuration.

Day 2 Configuration

For existing clusters, you can navigate to the cluster view, then go to Configuration where you will find an option to enable Secret Encryption.

RCTL Cluster Configuration with Secret Encryption

kind: Cluster
metadata:
  name: cluster-config
  project: defaultproject
spec:
  blueprint: minimal
  blueprintversion: 2.7.0
  cloudprovider: eks-cloud
  cniprovider: aws-cni
  proxyconfig: {}
  type: eks
---
addons:
- name: coredns
  version: v1.10.1-eksbuild.4
- name: vpc-cni
  version: v1.15.1-eksbuild.1
- name: kube-proxy
  version: v1.28.2-eksbuild.2
- name: aws-ebs-csi-driver
  version: latest
apiVersion: rafay.io/v1alpha5
kind: ClusterConfig
managedNodeGroups:
- amiFamily: AmazonLinux2
  desiredCapacity: 2
  iam:
    withAddonPolicies:
      autoScaler: true
  instanceTypes:
  - t3.xlarge
  maxSize: 2
  minSize: 0
  name: ng-b476082a
  version: "1.28"
  volumeSize: 80
  volumeType: gp3
metadata:
  name: cluster-config
  region: us-west-2
  tags:
    email: demo@rafay.co
    env: dev
  version: "1.28"
secretsEncryption:
  keyARN: arn:aws:kms:us-west-2:xxxxxxxxxxxxx:key/xxxxxxxxxxxxxxx
vpc:
  cidr: 192.168.0.0/16
  clusterEndpoints:
    privateAccess: true
    publicAccess: false
  nat:
    gateway: Single

IAM Permissions Required

When enabling secret encryption with a KMS key on EKS clusters, ensure that the IAM roles or users performing these actions have the following permissions assigned:

• For key listing: kms:ListKeys
• For secret encryption: kms:DescribeKey, kms:CreateGrant
• For Encryption on Existing EKS Cluster : eks:AssociateEncryptionConfig

These permissions are necessary for listing KMS keys, managing encryption grants, and associating an encryption configuration to an existing cluster that don't already have encryption enabled.

Irreversible Action: Secrets Encryption

Once enabled, secrets encryption cannot be disabled. This action is irreversible.

Amazon Linux 2023 Support

This release introduces support for AmazonLinux2023 in node groups. Users can now leverage the benefits of AL2023 by creating node groups to use this Amazon Linux version.

Node Group Configuration with AL2023

Example: Creating an EKS Node Group with AL2023 based Custom AMI

managedNodeGroups:
- ami: <custom_ami>
  desiredCapacity: 1
  iam:
    withAddonPolicies:
      autoScaler: true
  instanceTypes:
  - t3.xlarge
  maxSize: 1
  minSize: 1
  name: al2023_customami
  overrideBootstrapCommand: |
    [AmazonLinux2023]
  ssh:
    allow: true
    publicKeyName: <awskey>
  volumeSize: 80
  volumeType: gp3

Amazon EKS Cluster with AL2023 node

Amazon EKS and Azure AKS

Toleration Support for FleetOps Pre-Hook/Post-Hook Configuration

In this release, we are adding toleration support in the FleetOps pre-hook and post-hook configuration.

Supported Interfaces

• UI
• RCTL
• SystemSync
• Swagger API's

Note: Support for Terraform will be delivered in a subsequent release. Additionally, Node Selectors and Tolerations are applicable only when the Runner type is set to Cluster. For more information on this feature, please refer to Fleet Tolerations here

GKE Clusters

Network Policy and Dataplane V2 for GKE Clusters

This enhancement provides users with advanced networking capabilities ensuring improved security and performance for their applications. Dataplane V2 provides enhanced observability, scalability, and resilience, enabling seamless traffic management across clusters. Additionally, Network Policy support allows fine-grained control over network traffic, ensuring that only authorized communications occur between services.

Upstream Kubernetes for Bare Metal and VMs

GitOps SystemSync with Write Back to Git

With this feature, users will be able to configure the platform to continuously sync cluster specifications for upstream Kubernetes clusters with a Git repository. Changes can be made in a bidirectional manner.

For more information on this, please refer here.

• If the cluster specification is updated in the Git repository, the platform will update the corresponding upstream Kubernetes cluster to bring it to the desired state
• If the upstream Kubernetes cluster's state is modified by an authorized user using the UI or CLI, the changes are automatically written back to the configured Git repository
• To enable GitOps SystemSync for Upstream Kubernetes (MKS) on Bare Metal and VMs, users must create cloud credentials. These credentials are necessary for GitOps Agent to facilitate the GitOps synchronization.

GitOps Agent Update Required

To use GitOps for upstream cluster types, you must update the GitOps agent to version r2.8.0+.

Cloud Credential Support for Upstream Kubernetes Cluster(MKS)

This release introduces support for managing cloud credentials for upstream Kubernetes clusters (MKS) within the platform. These credentials are essential for enabling GitOps SystemSync functionality and have been integrated into the UI, RCTL, and SystemSync interfaces.

SystemComponentsPlacement Support

SystemComponentsPlacement allows configuring the scheduling of system components on dedicated nodes. This release introduces support for systemComponentsPlacement as part of the new extended configuration schema.

Using systemComponentsPlacement in RCTL

To utilize systemComponentsPlacement in RCTL, you need to pass the --v3 flag when applying your cluster configuration. Here's an example:

./rctl apply -f <cluster configuration> --v3

apiVersion: infra.k8smgmt.io/v3
kind: Cluster
metadata:
  name: demo-mks
  project: defaultproject
spec:
  blueprint:
    name: minimal
    version: latest
  cloudCredentials: demo-mks-creds
  config:
    autoApproveNodes: true
    dedicatedControlPlane: true
    highAvailability: true
    kubernetesVersion: v1.28.9
    location: sanjose-us
    network:
      cni:
        name: Calico
        version: 3.26.1
      podSubnet: 10.244.0.0/16
      serviceSubnet: 10.96.0.0/12
    nodes:
    - arch: amd64
      hostname: demo-mks-scale-w-tb-2
      operatingSystem: Ubuntu20.04
      privateIP: 10.12.105.227
      roles:
      - Worker
    - arch: amd64
      hostname: demo-mks-scale-w-tb-7
      labels:
        app: infra
      operatingSystem: Ubuntu20.04
      privateIP: 10.12.110.50
      roles:
      - Worker
      taints:
      - effect: NoSchedule
        key: app
        value: infra
    - arch: amd64
      hostname: demo-mks-scale-w-tb-3
      operatingSystem: Ubuntu20.04
      privateIP: 10.12.29.164
      roles:
      - Worker
    - arch: amd64
      hostname: demo-mks-scale-w-tb-6
      labels:
        app: infra
      operatingSystem: Ubuntu20.04
      privateIP: 10.12.101.223
      roles:
      - Worker
      taints:
      - effect: NoSchedule
        key: app
        value: infra
    - arch: amd64
      hostname: demo-mks-scale-c-tb-3
      operatingSystem: Ubuntu20.04
      privateIP: 10.12.118.110
      roles:
      - ControlPlane
  systemComponentsPlacement:
    nodeSelector:
      app: infra
    tolerations:
    - effect: NoSchedule
      key: app
      operator: Equal
      value: infra
    - effect: NoSchedule
      key: infra
      operator: Equal
      value: rafay
  type: mks

Kubernetes Certificate Rotation

Support is now added to rotate Kubernetes certificates for upstream Kubernetes clusters via UI and API. This can be done either manually or automatically based on certificate expiry.

For more information on this feature, please refer here.

CentOS 7 EOL

Due to CentOS 7 reaching its end-of-life (EOL) on June 30, 2024, this release no longer supports creating new clusters using CentOS 7.

We recommend transitioning to alternative supported operating systems like Rocky Linux, AlmaLinux, or RHEL as replacements for CentOS 7.

Terraform Provider

Caching

There are certain scenarios where diff is shown even when there are no changes. A caching mechanism has been implemented for TF schema to alleviate this issue for the following resources:

• rafay_eks_cluster
• rafay_aks_cluster

Important

Limited Access - This capability is selectively enabled for customer orgs. Please reach out to support if you want to get this enabled for your organization.

Deprecation

Cluster Templates

The Cluster Templates feature set is deprecated with this release and support for this feature will be removed in a future release.

vSphere LCM

The vSphere Cluster Lifecycle Management capability is deprecated with this release, and support for this feature will be removed in a future release.

Environment Manager

RCTL support

With the introduction of this capability, it is possible to execute Environment Manager workflows using the RCTL Command Line interface utility.

Support for HCP Terraform Provider

Platform teams will be able to seamlessly integrate and leverage existing investments in HCP Terraform with the introduction of this provider option.

Support for OpenTofu Provider

Support for OpenTofu provider option is now added as part of the resource template configuration.

Clusters

Resources page

Sorting support is now available for all columns.

Secrets Management

Vault Integration

Vault Agent Injector version that is deployed to the clusters has been updated to v1.17.2 with this release.

2.8 Release Bug Fixes