KOP EKS Clusters - ARC Zonal Shift - Rafay Product Documentation

ARC Zonal Shift

Overview

ARC Zonal Shift enables traffic to move away from an Availability Zone (AZ) to improve application resilience during zonal events. The feature supports both manual zonal shift operations and automated zonal autoshift behavior.

Zonal Shift is controlled centrally through cluster configuration and managed from the Console after cluster provisioning.

Note
ARC Zonal Shift configuration is currently supported via RCTL, Terraform, API, and System Sync. UI support will be added in a future release.

IAM Permissions

In addition to the permissions defined in the full IAM policy, the following permissions are required for ARC Zonal Shift to function correctly.

Minimum Required Permissions

The following permissions are mandatory for the supported zonal shift operations:

{
    "Effect": "Allow",
    "Action": [
        "arc-zonal-shift:StartZonalShift",
        "arc-zonal-shift:CancelZonalShift",
        "arc-zonal-shift:ListZonalShifts",
        "arc-zonal-shift:UpdateZonalAutoshiftConfiguration",
        "arc-zonal-shift:CreatePracticeRunConfiguration",
        "arc-zonal-shift:UpdatePracticeRunConfiguration",
        "arc-zonal-shift:DeletePracticeRunConfiguration"
    ],
    "Resource": "*"
},
{
    "Effect": "Allow",
    "Action": [
        "cloudwatch:DescribeAlarms"
    ],
    "Resource": "*"
}

Full Permissions

The following permissions provide complete access to all ARC Zonal Shift capabilities:

{
    "Effect": "Allow",
    "Action": [
        "arc-zonal-shift:CancelPracticeRun",
        "arc-zonal-shift:CancelZonalShift",
        "arc-zonal-shift:CreatePracticeRunConfiguration",
        "arc-zonal-shift:DeletePracticeRunConfiguration",
        "arc-zonal-shift:GetAutoshiftObserverNotificationStatus",
        "arc-zonal-shift:GetManagedResource",
        "arc-zonal-shift:ListAutoshifts",
        "arc-zonal-shift:ListManagedResources",
        "arc-zonal-shift:ListZonalShifts",
        "arc-zonal-shift:StartPracticeRun",
        "arc-zonal-shift:StartZonalShift",
        "arc-zonal-shift:UpdateAutoshiftObserverNotificationStatus",
        "arc-zonal-shift:UpdatePracticeRunConfiguration",
        "arc-zonal-shift:UpdateZonalAutoshiftConfiguration",
        "arc-zonal-shift:UpdateZonalShift"
    ],
    "Resource": "*"
},
{
    "Effect": "Allow",
    "Action": [
        "cloudwatch:DescribeAlarms"
    ],
    "Resource": "*"
}

Day-0 Configuration (Cluster Specification)

ARC Zonal Shift is configured during cluster creation by updating the cluster specification on Day 0. The following example shows a cluster specification with both Zonal Shift and Auto Zonal Shift configuration.

kind: Cluster
metadata:
  name: gopimallela-zs-ui
  project: defaultproject
spec:
  blueprint: minimal
  blueprintversion: 4.1.0
  cloudprovider: uday-qa-acc
  cniprovider: aws-cni
  proxyconfig: {}
  type: eks
---
accessConfig:
  authenticationMode: CONFIG_MAP
addons:
- name: coredns
  version: v1.12.1-eksbuild.2
- name: vpc-cni
  version: v1.20.4-eksbuild.2
- name: kube-proxy
  version: v1.33.3-eksbuild.4
- name: aws-ebs-csi-driver
  version: latest
addonsConfig: {}
apiVersion: rafay.io/v1alpha5
autoModeConfig: {}
autoZonalShiftConfig:
  allowedWindows:
  - Mon:00:00-Mon:01:00
  enabled: true
  outcomeAlarms:
  - arn:aws:cloudwatch:us-west-2:211125364662:alarm:rafay-core-dev-RDSinstanceCPUUtilization
kind: ClusterConfig
managedNodeGroups:
- amiFamily: AmazonLinux2023
  desiredCapacity: 2
  iam:
    withAddonPolicies:
      autoScaler: true
  instanceTypes:
  - t3.xlarge
  maxSize: 2
  minSize: 2
  name: ng-17e9c131
  version: "1.33"
  volumeSize: 80
  volumeType: gp3
- amiFamily: AmazonLinux2023
  desiredCapacity: 2
  iam:
    withAddonPolicies:
      autoScaler: true
  instanceTypes:
  - t3.xlarge
  maxSize: 2
  minSize: 2
  name: ng-2
  nodeRepairConfig:
    enabled: true
    maxParallelNodesRepairedCount: 2
    maxUnhealthyNodeThresholdCount: 1
    nodeRepairConfigOverrides:
    - minRepairWaitTimeMins: 20
      nodeMonitoringCondition: NetworkingReady
      nodeUnhealthyReason: ContainerRuntimeFailed
      repairAction: Replace
    - minRepairWaitTimeMins: 10
      nodeMonitoringCondition: Ready
      nodeUnhealthyReason: ContainerRuntimeFailed
      repairAction: Replace
  version: "1.33"
  volumeSize: 80
  volumeType: gp3
metadata:
  name: gopimallela-zs-ui
  region: us-west-2
  tags:
    email: gopikrishna@rafay.co
    env: dev
  version: "1.33"
vpc:
  cidr: 192.168.0.0/16
  clusterEndpoints:
    privateAccess: true
    publicAccess: false
  nat:
    gateway: Single
zonalShiftConfig:
  enabled: true

Zonal Shift Parameters

zonalShiftConfig

zonalShiftConfig:
  enabled: true

Parameter	Description
`enabled`	Enables ARC Zonal Shift for the cluster. Acts as the central switch required for both manual and automatic zonal shift operations. Can be configured during Day-0 via cluster specification and updated later as a Day-2 operation. Once enabled, zonal shift operations are available from the Console.

autoZonalShiftConfig

autoZonalShiftConfig:
  enabled: true
  outcomeAlarms:
    - arn:aws:cloudwatch:...
  allowedWindows:
    - Mon:00:00-Mon:01:00

Parameter	Description
`enabled`	Enables automatic zonal autoshift. When enabled, traffic can be automatically shifted during zonal events and practice runs.
`outcomeAlarms`	Mandatory CloudWatch alarm ARNs used to evaluate practice run results. If an alarm enters ALARM state, the practice run is marked as failed.
`allowedWindows`	Defines time windows when practice runs are allowed. Format: `Day:HH:MM-Day:HH:MM` (UTC).

Day-0 vs Day-2 Operations

Day-0 (Cluster Provisioning)

Allowed:

Configure zonalShiftConfig.enabled
Establish centralized Zonal Shift capability for the cluster.

❗ Important: UI-based zonal shift operations are not supported during Day-0 configuration.

Day-2 (Post-Provision Management)

Available from the Console:

Enable or disable Zonal Shift
Start manual zonal shift
View history
Configure Auto Zonal Shift
Add outcome alarms
Configure practice run windows
Manage autoshift settings

Managing Zonal Shift in the Console

Coming Soon

UI-based management for ARC Zonal Shift will be available in a future release. The screenshots below represent how the experience will look in the Console.

After cluster creation:

Navigate to the cluster and open Configuration.
Locate EKS Zonal Shift and click on Manage.

Settings

Displays and controls centralized Zonal Shift enablement.

Shows current state.
Enable or disable Zonal Shift for the cluster and click Save

History

The History tab provides a consolidated view of all zonal shift activities performed on the cluster, including manual shifts, automatic shifts, and practice runs.

Users can filter entries based on:

Status — Active, Expired, or Canceled
Type — Zonal Shift, Practice Run, or Zonal Autoshift

This tab helps users:

Track past and current zonal shift operations.
Verify whether a shift completed successfully, expired, or was canceled.
Review operational timelines and execution details for troubleshooting or auditing.
Understand how traffic-shift actions have impacted the cluster over time.

The History view is useful for monitoring operational health, validating resiliency testing, and maintaining visibility into Day-2 zonal shift activities.

Start (Manual Zonal Shift)

Used to initiate a manual zonal shift.

Steps

Select the Availability Zone and set expiration duration.
Provide an optional comment and click Start to start the shift.

Behavior

Traffic moves away from the selected AZ.
Only one active shift per resource.
Shift automatically expires after the configured duration.

Auto Config

Used to configure automatic zonal autoshift behavior.

Enable Autoshift

Enables automatic traffic shifting.
Starts AWS-managed practice runs used to validate resiliency.

Configure Alarms

Alarms are used to control and evaluate Zonal Autoshift practice runs.

Outcome Alarms (Mandatory)

Outcome alarms define whether a practice run is successful or failed.

Evaluated during the practice run.
If an alarm enters the ALARM state, the practice run is marked as failed.
Used to validate workload stability when traffic is shifted away from an Availability Zone.

Blocking Alarms (Optional)

Blocking alarms prevent practice runs from starting when the environment is already unhealthy.

Checked before a practice run starts.
If active, the practice run does not run.

This helps avoid additional impact during ongoing issues.

Configure Run Windows

Run windows control when automatic Zonal Autoshift practice runs are allowed to execute.

Allowed windows define the time periods when practice runs can occur.
Blocked windows prevent practice runs from running during specific time ranges.

Run windows help schedule practice runs during appropriate periods and avoid execution during business-critical or peak usage hours.

Practice Run Behavior
When autoshift is enabled:

Practice runs occur approximately once per week.

Each run lasts about 30 minutes.

Traffic shifts away from a single Availability Zone.

Helps validate application resilience without requiring production incidents.

APIs

The following APIs allow you to programmatically manage ARC Zonal Shift for EKS clusters. Replace <project-id> with your project ID and <cluster-id> with your cluster ID in all requests.

Enable/Disable ARC Zonal Shift

curl --location 'https://console.rafay.dev/edge/v1/projects/<project-id>/edges/<cluster-id>/zonalshift/enable' \
--header 'Content-Type: text/plain' \
--header 'X-RAFAY-API-KEYID: <your-api-key>' \
--data '{
    "enabled": true
}'

Enable/Disable Auto Zonal Shift Config

curl --location 'https://console.rafay.dev/edge/v1/projects/<project-id>/edges/<cluster-id>/zonalshift/auto' \
--header 'Content-Type: text/plain' \
--header 'X-RAFAY-API-KEYID: <your-api-key>' \
--data '{
  "enabled": true,
  "allowedWindows": ["Sat:02:00-Sat:06:00"],
  "blockedWindows": ["Fri:02:00-Fri:06:00"],
  "blockingAlarms": ["arn:aws:cloudwatch:<region>:<account-id>:alarm:<alarm-name>"],
  "outcomeAlarms": ["arn:aws:cloudwatch:<region>:<account-id>:alarm:<alarm-name>"]
}'

Notes:

enabled and outcomeAlarms are mandatory fields.

Only one of allowedWindows or blockedWindows is allowed per request.

Start Zonal Shift

curl --location 'https://console.rafay.dev/edge/v1/projects/<project-id>/edges/<cluster-id>/zonalshift/start' \
--header 'Content-Type: text/plain' \
--header 'X-RAFAY-API-KEYID: <your-api-key>' \
--data '{
  "awayFrom": "<availability-zone>",
  "comment": "<optional-comment>",
  "expiresIn": "30m"
}'

Cancel Zonal Shift

curl --location 'https://console.rafay.dev/edge/v1/projects/<project-id>/edges/<cluster-id>/zonalshift/cancel' \
--header 'Content-Type: text/plain' \
--header 'X-RAFAY-API-KEYID: <your-api-key>' \
--data '{
    "zonalShiftId": "<zonal-shift-id>"
}'

List Zonal Shifts

curl --location 'https://console.rafay.dev/edge/v1/projects/<project-id>/edges/<cluster-id>/zonalshift/list' \
--header 'Content-Type: text/plain' \
--header 'X-RAFAY-API-KEYID: <your-api-key>'