# Aug

## v1.1.34 - Terraform Provider

**13 August, 2024**

An updated version of the [Terraform provider](https://registry.terraform.io/providers/RafaySystems/rafay/latest) is now available.

This release includes enhancements around resources listed below:

**Existing Resource**

- **rafay_eks_cluster** : Added Day 2 support for secret encryption for EKS resource.
- **rafay_environment** : Added support for environment variable and file override in this resource.
- **rafay_resource_template** : Added `hcpterraform` provider support including HCP terraform specific options in this resource.

There are certain scenarios where diff is shown even when there are no changes. A caching mechanism has been implemented for TF schema to alleviate this issue for the following resources:

- rafay_eks_cluster
- rafay_aks_cluster

### v1.1.34 Terraform Provider Bug Fixes

| Bug ID | Description |
| --- | --- |
| RC-36073 | Getting wrong kubeconfig while downloading from terraform |
| RC-35832 | Unable to Fetch V3 Workload created via terraform code on the Rafay UI while Pipeline Creation for Workload Deployment |
| RC-32044 | Maxpodspernode field set to 0 in cluster config of EKS (Managed nodegroup) created using terraform |
| RC-35649 | Unable to add timeouts in resource templates using Terraform |
| RC-35400 | Terraform plan/apply detecting redundant diff in resource template's provider block |

## v2.8 - SaaS

**09 Aug, 2024**

Note

With this release, Environment Manager and Fleet Plan capabilities are available to all Orgs and are no longer feature flagged.

### Amazon EKS

#### Enabling secret encryption on an existing cluster

Currently, secret encryption with a KMS key can only be configured during cluster creation (Day 0) using the 'save and customize' option. Support is now added to enable secret encryption on existing clusters (Day 2) as well, along with the option to configure it during cluster creation (Day 0) using the cluster form.

> For more information on this feature, please refer [here](https://docs.rafay.co/clusters/eks/secret_encryption/).

**Day 0 Configuration**

For new clusters, you can find this configuration in the cluster settings. Based on valid cloud credentials, you will see the option to enable secret encryption during Day 0 configuration.

**Day 2 Configuration**

For existing clusters, you can navigate to the cluster view, then go to Configuration where you will find an option to enable Secret Encryption.

**RCTL Cluster Configuration with Secret Encryption**

```yaml
kind: Cluster
metadata:
  name: cluster-config
  project: defaultproject
spec:
  blueprint: minimal
  blueprintversion: 2.7.0
  cloudprovider: eks-cloud
  cniprovider: aws-cni
  proxyconfig: {}
  type: eks
---
addons:
- name: coredns
  version: v1.10.1-eksbuild.4
- name: vpc-cni
  version: v1.15.1-eksbuild.1
- name: kube-proxy
  version: v1.28.2-eksbuild.2
- name: aws-ebs-csi-driver
  version: latest
apiVersion: rafay.io/v1alpha5
kind: ClusterConfig
managedNodeGroups:
- amiFamily: AmazonLinux2
  desiredCapacity: 2
  iam:
    withAddonPolicies:
      autoScaler: true
  instanceTypes:
  - t3.xlarge
  maxSize: 2
  minSize: 0
  name: ng-b476082a
  version: "1.28"
  volumeSize: 80
  volumeType: gp3
metadata:
  name: cluster-config
  region: us-west-2
  tags:
    email: demo@rafay.co
    env: dev
  version: "1.28"
secretsEncryption:
  keyARN: arn:aws:kms:us-west-2:xxxxxxxxxxxxx:key/xxxxxxxxxxxxxxx
vpc:
  cidr: 192.168.0.0/16
  clusterEndpoints:
    privateAccess: true
    publicAccess: false
  nat:
    gateway: Single
```

IAM Permissions Required

When enabling secret encryption with a KMS key on EKS clusters, ensure that the IAM roles or users performing these actions have the following permissions assigned:

- **For key listing**: `kms:ListKeys`
- **For secret encryption**: `kms:DescribeKey`, `kms:CreateGrant`
- **For Encryption on Existing EKS Cluster** : `eks:AssociateEncryptionConfig`

These permissions are necessary for listing KMS keys, managing encryption grants, and associating an encryption configuration to an existing cluster that don't already have encryption enabled.

Irreversible Action: Secrets Encryption

Once enabled, secrets encryption cannot be disabled. This action is irreversible.

#### Amazon Linux 2023 Support

This release introduces support for **AmazonLinux2023** in node groups. Users can now leverage the benefits of AL2023 by creating node groups to use this Amazon Linux version.

**Node Group Configuration with AL2023**

**Example: Creating an EKS Node Group with AL2023 based Custom AMI**

```yaml
managedNodeGroups:
- ami: <custom_ami>
  desiredCapacity: 1
  iam:
    withAddonPolicies:
      autoScaler: true
  instanceTypes:
  - t3.xlarge
  maxSize: 1
  minSize: 1
  name: al2023_customami
  overrideBootstrapCommand: |
    [AmazonLinux2023]
  ssh:
    allow: true
    publicKeyName: <awskey>
  volumeSize: 80
  volumeType: gp3
```

**Amazon EKS Cluster with AL2023 node**

### GKE Clusters

#### Network Policy and Dataplane V2 for GKE Clusters

This enhancement provides users with advanced networking capabilities ensuring improved security and performance for their applications. [Dataplane V2](https://docs.rafay.co/clusters/gke/provisioning.md) leverages the power of eBPF, providing enhanced observability, scalability, and resilience, enabling seamless traffic management across clusters. Additionally, Network Policy support allows fine-grained control over network traffic, ensuring that only authorized communications occur between services.

> For more information on this feature, please refer [here](https://docs.rafay.co/clusters/gke/provisioning/).

### Upstream Kubernetes for Bare Metal and VMs

#### GitOps SystemSync with Write Back to Git

With this feature, users will be able to configure the platform to continuously sync cluster specifications for upstream Kubernetes clusters with a Git repository. Changes can be made in a bidirectional manner.

> For more information on this, please refer [here](https://docs.rafay.co/clusters/upstream/baremetal_vm/system_sync/).

- If the cluster specification is updated in the Git repository, the platform will update the corresponding upstream Kubernetes cluster to bring it to the desired state
- If the upstream Kubernetes cluster's state is modified by an authorized user using the UI or CLI, the changes are automatically written back to the configured Git repository
- To enable GitOps SystemSync for Upstream Kubernetes (MKS) on Bare Metal and VMs, users must create cloud credentials. These credentials are necessary for GitOps Agent to facilitate the GitOps synchronization.

GitOps Agent Update Required

To use GitOps for upstream cluster types, you must update the GitOps agent to version r2.8.0+.

#### Cloud Credential Support for Upstream Kubernetes Cluster(MKS)

This release introduces support for managing cloud credentials for upstream Kubernetes clusters (MKS) within the platform. These credentials are essential for **enabling GitOps SystemSync functionality** and have been integrated into the UI, RCTL, and SystemSync interfaces.

#### SystemComponentsPlacement Support

SystemComponentsPlacement allows configuring the scheduling of system components on dedicated nodes. This release introduces support for `systemComponentsPlacement` as part of the new extended configuration schema. This functionality is currently supported in RCTL, V3 APIs and GitOps System Sync.

> For more information on this, please refer [here](https://docs.rafay.co/clusters/upstream/cli/).

**Using `systemComponentsPlacement` in RCTL**

To utilize `systemComponentsPlacement` in RCTL, you need to pass the `--v3` flag when applying your cluster configuration. Here's an example:

```
./rctl apply -f <cluster configuration> --v3
```

```yaml
apiVersion: infra.k8smgmt.io/v3
kind: Cluster
metadata:
  name: demo-mks
  project: defaultproject
spec:
  blueprint:
    name: minimal
    version: latest
  cloudCredentials: demo-mks-creds
  config:
    autoApproveNodes: true
    dedicatedControlPlane: true
    highAvailability: true
    kubernetesVersion: v1.28.9
    location: sanjose-us
    network:
      cni:
        name: Calico
        version: 3.26.1
      podSubnet: 10.244.0.0/16
      serviceSubnet: 10.96.0.0/12
    nodes:
    - arch: amd64
      hostname: demo-mks-scale-w-tb-2
      operatingSystem: Ubuntu20.04
      privateIP: 10.12.105.227
      roles:
      - Worker
    - arch: amd64
      hostname: demo-mks-scale-w-tb-7
      labels:
        app: infra
      operatingSystem: Ubuntu20.04
      privateIP: 10.12.110.50
      roles:
      - Worker
      taints:
      - effect: NoSchedule
        key: app
        value: infra
    - arch: amd64
      hostname: demo-mks-scale-w-tb-3
      operatingSystem: Ubuntu20.04
      privateIP: 10.12.29.164
      roles:
      - Worker
    - arch: amd64
      hostname: demo-mks-scale-w-tb-6
      labels:
        app: infra
      operatingSystem: Ubuntu20.04
      privateIP: 10.12.101.223
      roles:
      - Worker
      taints:
      - effect: NoSchedule
        key: app
        value: infra
    - arch: amd64
      hostname: demo-mks-scale-c-tb-3
      operatingSystem: Ubuntu20.04
      privateIP: 10.12.118.110
      roles:
      - ControlPlane
  systemComponentsPlacement:
    nodeSelector:
      app: infra
    tolerations:
    - effect: NoSchedule
      key: app
      operator: Equal
      value: infra
    - effect: NoSchedule
      key: infra
      operator: Equal
      value: rafay
  type: mks
```

#### Kubernetes Certificate Rotation

Support is now added to rotate Kubernetes certificates for upstream Kubernetes clusters via UI and API. This can be done either manually or automatically based on certificate expiry. This capability streamlines the entire certificate rotation process.

> For more information on this feature, please refer [here](https://docs.rafay.co/clusters/upstream/baremetal_vm/cert_rotation/).

Note

Please note that Kubernetes certificate rotation feature described above is not supported in Windows-based upstream clusters.

### Terraform Provider

#### Caching

There are certain scenarios where **diff** is shown even when there are no changes. A caching mechanism has been implemented for TF schema to alleviate this issue for the following resources:

- [rafay_eks_cluster](https://registry.terraform.io/providers/RafaySystems/rafay/latest/docs/resources/eks_cluster)
- [rafay_aks_cluster](https://registry.terraform.io/providers/RafaySystems/rafay/latest/docs/resources/aks_cluster)

Important

Limited Access - This capability is selectively enabled for customer orgs. Please reach out to support if you want to get this enabled for your organization. Support for more resources will be added with future releases.

### Deprecation

#### Cluster Templates

The Cluster Templates feature set is deprecated with this release and support for this feature will be removed in a future release. This means that no enhancements or bug fixes will be added to this feature. Users can leverage Environment Manager to achieve the same use case.

#### vSphere LCM

The vSphere Cluster Lifecycle Management capability is deprecated with this release, and support for this feature will be removed in a future release. Users are encouraged to migrate to the Upstream Kubernetes for Bare Metal and VMs (MKS) offering. The infrastructure for Upstream Kubernetes on vSphere environment can be deployed leveraging Environment Manager.

### Environment Manager

#### RCTL support

With the introduction of this capability, it is possible to execute Environment Manager workflows using the RCTL Command Line interface utility. RCTL CLI can be embedded into the preferred workflow automation (e.g. CI/CD pipeline) to perform operations such as creation of templates, deployment/shutting down environments.

#### Support for HCP Terraform Provider

Platform teams will be able to seamlessly integrate and leverage existing investments in HCP Terraform with the introduction of this provider option. With this integration, Platform teams can define templates and enable self-service for developers/data scientists within Rafay while the infrastructure provisioning & management of state files can be handled by HCP Terraform.

> For more information on this, please refer [here](https://docs.rafay.co/env_manager/templates/resource_templates/).
>
> **Note:** This provider option is only available to HashiCorp licensed customers.

#### Support for OpenTofu Provider

Support for OpenTofu provider option is now added as part of the resource template configuration. This provides customers the flexibility to choose the provider of choice for infrastructure provisioning and state management.

### Clusters

#### Resources page

Sorting support is now available for all columns.

### Cost Management

#### Cost Explorer UI improvements

A number of improvements have been implemented with this release including:

- Tool tips that detail how the efficiency scores are calculated
- Additional columns (Resource Efficiency - CPU, Resource Efficiency - Memory, Utilized Cost, Idle Cost) on export of data
- Availability of secondary filters based on primary filter selection
- Sorting of data in Cost Explorer table based on Efficiency Score

### Secrets Management

#### Vault Integration

Vault Agent Injector version that is deployed to the clusters has been updated to v1.17.2 with this release.

### 2.8 Release Bug Fixes

| Bug ID | Description |
| --- | --- |
| RC-35542 | MKS: Containerd version is not being updated on all nodes during K8s upgrade |
| RC-34562 | OpenCost agent is not deployed with a priorityClassName |
| RC-34492 | AKS: Unable to update cloud credentials associated with a cluster using RCTL v3 spec |
| RC-32012 | GKE: Cluster creation fails when the name includes rafay |
| RC-36073 | Incorrect kubeconfig when downloaded through Terraform |
| RC-33947 | Unintuitive error message on a blueprint sync failure when the add-on name is the same as the Rafay system add-on |
| RC-36557 | UI: Network policies do not persist when creating a new blueprint version |