# Config Examples

## Configuration Examples for EKS System Template

This document provides configuration snippets and usage patterns for customizing your Amazon EKS clusters using the **EKS System Template** in Rafay.

### Control Plane Subnet IDs

To specify the subnets for your EKS control plane, use the `Control Plane Subnet IDs` input parameter in your system template:

```
["subnet-070c1e8xxxxx", "subnet-02c612dd9yyyyy", "subnet-0b3bea6zzzzz", "subnet-0700b95vvvvvvv"]
```

### Managed Node Groups Configuration

To create managed node groups with custom configurations — such as instance types, AMI settings, subnet placement, and bootstrap arguments — use the `EKS Managed Node Group` input variable in your system template. This allows you to define multiple managed node groups with different configurations.

#### Example 1: Basic Managed Node Group Configuration

```
{
  "managed-ng-1": {
    "ami_type": "AL2_x86_64",
    "instance_types": ["t3.medium"],
    "cluster_version": "1.31",
    "min_size": 1,
    "max_size": 5,
    "desired_size": 2
  },
  "managed-ng-cami": {
    "ami_type": "AL2_x86_64",
    "instance_types": ["t3.medium"],
    "min_size": 1,
    "max_size": 5,
    "desired_size": 2,
    "ami_id": "ami-00ae057af58f63a80",
    "enable_bootstrap_user_data": true,
    "bootstrap_extra_args": "--container-runtime containerd --kubelet-extra-args '--max-pods=46' '--max-open-files=1000000' '--container-log-max-files=10'",
    "subnet_ids": [
      "subnet-0b3bea60007a1e322",
      "subnet-0700b95caf66b04da"
    ]
  }
}
```

#### Example 2: Managed Node Group with Custom Encrypted EBS Volumes

```
{
  "mg-node-group-01": {
    "ami_type": "AL2_x86_64",
    "instance_types": ["t3.medium"],
    "subnet_ids": ["subnet-09f11fbddxxxx", "subnet-0dc7b0289yyyyy", "subnet-01f66d22bzzz"],
    "min_size": 0,
    "max_size": 2,
    "desired_size": 2,
    "block_device_mappings": [{
      "device_name": "/dev/xvda",
      "ebs": {
        "encrypted": true,
        "kms_key_id": "arn:aws:kms:us-west-2:xxxxx:key/xxxx-xxxxx",
        "volume_size": 100,
        "volume_type": "gp2"
      }
    }]
  }
}
```

### Cluster Additional Security Group IDs

To attach additional security groups to the **EKS control plane**, use the `Cluster Additional Security Group IDs` field in your system template:

```
["sg-07f01xxxxxxxxx"]
```

### Node IAM Role ARN

To assign a custom **IAM role** to the EKS worker nodes (managed node groups), use the `Node IAM Role ARN` field. This is useful when you want nodes to assume a specific IAM role with fine-grained permissions:

```
"arn:aws:iam::xxxxxxxx:role/lan-AmazonEKSNodeRole"
```

### Cluster Addons Configuration

The `Cluster Addons` field allows you to configure and manage **EKS-managed add-ons** such as **CoreDNS**, **kube-proxy**, **VPC CNI**, and **CSI drivers** directly within the EKS System Template. You can also attach **IAM roles to service accounts** (using **Pod Identity Associations**) and provide **custom configurations or versions** for specific add-ons.

#### Example: Cluster Addons Configuration

```
{
  "coredns": {
    "name": "coredns",
    "pod_identity_association": []
  },
  "kube-proxy": {
    "name": "kube-proxy",
    "pod_identity_association": []
  },
  "eks-pod-identity-agent": {
    "name": "eks-pod-identity-agent",
    "pod_identity_association": []
  },
  "aws-ebs-csi-driver": {
    "name": "aws-ebs-csi-driver",
    "addon_version": "v1.38.1-eksbuild.2",
    "configuration_values": <<JSON
    {
      "defaultStorageClass": {
        "enabled": true
      }
    }
    JSON
  },
  "vpc-cni": {
    "name": "vpc-cni",
    "pod_identity_association": [
      {
        "role_arn": "arn:aws:iam::xxxxxxx:role/kalyan-vpc-cni-pia-role",
        "service_account": "aws-node"
      }
    ]
  }
}
```

### Node Security Group Additional Rules

Use the `Node Security Group Additional Rules` field to define **custom ingress or egress rules** for the node security group. This is useful when you need to open specific traffic flows between pods, the control plane, or other network components.

#### Example: Allow Pod-to-Control Plane Traffic

```
{
  "allow-traffic-from-pods-to-control-plane" = {
    description                   = "allow traffic from pods to control plane"
    protocol                      = "-1"
    from_port                     = 0
    to_port                       = 0
    type                          = "ingress"
    source_cluster_security_group = true
  }
}
```

**Key Fields:**

- **type**: Direction of the rule (ingress or egress)
- **protocol**: Protocol used (-1 means all protocols)
- **from\_port / to\_port**: Port range (0-0 for all ports)
- **source\_cluster\_security\_group**: true allows traffic from the cluster’s security group

### Pod Identity Association

To associate IAM policies with Kubernetes service accounts via Pod Identity Associations, use the following configuration format.

#### Example 1: Pod Identity Association for AWS Load Balancer Controller

```
{
  "aws-load-balancer-controller" = {
    additional_policy_arns = {
      "aws-load-balancer-controller-policy" = "arn:aws:iam::xxxxxx:policy/lb-aws"
    },
    associations = {
      aws-load-balancer-controller = {
        namespace       = "kube-system"
        service_account = "aws-load-balancer-controller"
      }
    }
  }
}
```

In the above example, the key `aws-load-balancer-controller-policy` refers to the `IAM Policy ARN`:

```
"arn:aws:iam::xxxxxx:policy/lb-aws"
```

This policy ensures that the AWS Load Balancer Controller has the necessary IAM permissions via pod identity.

#### Example 2: Pod Identity Association for Karpenter

```
{
  "karpenter-sa" = {
    additional_policy_arns = {
      "karpenter-policy" = "arn:aws:iam::xxxxxxx:policy/demo-karpenter-policy"
    },
    associations = {
      karpenter-sa = {
        namespace       = "karpenter"
        service_account = "karpenter"
      }
    }
  }
}
```

### Node Repair Support for EKS Managed Nodegroups

Enabling Node Repair for EKS managed nodegroups provides the following benefits:

- Automatically replaces or reboots unhealthy nodes to maintain cluster health
- Improves high availability and reduces manual intervention
- Ensures consistent performance in production environments

To enable AWS-native `Node Repair` for EKS managed nodegroups, add the following configuration in the node group spec:

```
"node_repair_config": {
  "enabled": true
}
```

**Example: Node Group Configuration**

```
"default-node-group": {
  "ami_type": "AL2_x86_64",
  "instance_types": ["t3.medium"],
  "min_size": 1,
  "max_size": 5,
  "desired_size": 2,
  "node_repair_config": {
    "enabled": true
  }
}
```

**Example: Cluster Addons Configuration**

```
"addons": {
  "coredns": {},
  "eks-pod-identity-agent": {},
  "kube-proxy": {},
  "vpc-cni": {},
  "eks-node-monitoring-agent": {
    "most_recent": true
  }
}
```