Learn KOP - EKS Add Node Group - Rafay Product Documentation

Part 3: Node Group

What Will You Do

In part 3, you will:

Add a node group to your EKS cluster increasing its capacity
Remove a node group from your EKS cluster

Watch a video of this exercise.

Assumptions

This part assumes that you have completed Part 1 of this series and have a successfully provisioned and healthy EKS cluster.

Add Node Group

In this step, we will scale the cluster by adding a new "spot instance" based node group to the cluster.

Select a method to provision and manage your EKS cluster from the tabs below.

Web Console | RCTL CLI | Terraform

In your project, navigate to Infrastructure -> Clusters
Click on the "cluster name" and then the "node groups" tab.
Click on "Add Node Group"
In the node group configuration, specify the following:
- Provide a name such as "spot-ng"
- Select "Use Spot Instances"
- Select "custom" for Instance type and enter "t3.large"
- Enter min nodes as 2 and max nodes as 4
- Enter "$0.03" for "Spot Max Price"
- Enter "50" for onDemandPercentageAboveBaseCapacity
Click on "Provision."

This will create the new node group and associated infrastructure and add it to your EKS cluster. The entire process can take a few minutes to complete.

Execute the following command to create the spot instance node group

./rctl apply -f eks-cluster-basics.yaml

Expected Output

Cluster: demo-eks-testing
{
    "taskset_id": "g29j3m0",
    "operations": [\
    {\
      "operation": "NodegroupCreation",\
      "resource_name": "spot-ng",\
      "status": "PROVISION_TASK_STATUS_PENDING"\
    }\
  ],
  "comments": "The status of the operations can be fetched using taskset_id",
  "status": "PROVISION_TASKSET_STATUS_PENDING"
}

Download the cluster specification for the existing cluster

Go to Infrastructure -> Clusters.
Click on the settings icon of the cluster and select "Download Cluster Config"
Update the downloaded specification file with the YAML for the additional node group.
Add the following node group configuration code to the previously applied cluster specification file

nodeGroups:
- name: spot-ng-1
  minSize: 2
  maxSize: 4
  instancesDistribution:
    maxPrice: 0.03
    instanceTypes: ["t3.large"]
    onDemandBaseCapacity: 0
    onDemandPercentageAboveBaseCapacity: 50
    spotInstancePools: 2

The fully updated cluster specification file including the newly added spot instance node group code will look like this:

kind: Cluster
metadata:
  name: demo-eks-testing
  project: defaultproject
spec:
  type: eks
  cloudprovider: demo
  blueprint: default
---
apiVersion: rafay.io/v1alpha5
kind: ClusterConfig
metadata:
  name: demo-eks-testing
  region: us-west-2
  version: "1.26"

managedNodeGroups:
  - name: ng-1
    instanceType: t3.large
    desiredCapacity: 2

nodeGroups:
- name: spot-ngs
  minSize: 2
  maxSize: 4
  instancesDistribution:
    maxPrice: 0.03
    instanceTypes: ["t3.large"]
    onDemandBaseCapacity: 0
    onDemandPercentageAboveBaseCapacity: 50
    spotInstancePools: 2

Execute the following command to create the spot instance node group

./rctl apply -f eks-cluster-basics.yaml

Expected Output

To add a node group:

Edit the terraform.tfvars file. The file location is /terraform/terraform.tfvars.
Copy and paste the following after pool1, then save the file.

  "pool2" = {
    ng_name         = "pool2"
    location        = "<CLUSTER_LOCATION/REGION>"
    node_count      = 1
    node_max_count  = 3
    node_min_count  = 1
    k8s_version     = "<K8S_VERSION>"
    instance_type   = "t3.xlarge"
  }

Open the terminal or command line.
Navigate to the terraform folder.
Run terraform apply. Enter yes when prompted.

It can take 20 minutes to add the node group to the cluster. Check the console for the node group status.

Verify Node Group

Once the new, spot based node group has been successfully added to the EKS cluster, you should be able to view the "new node group's details"

Users can also use the Zero Trust Kubectl shell to view details about the new node by using the following command.

kubectl get no

As you can see from this example, Kubectl is reporting "two new nodes" with a relatively recent "age". These are our Spot Instance based nodes from the node group.

Remove Node Group

In this step, we will remove the spot instance based node group we added recently. The process to remove a node group is very similar to the process of adding a node group.

Note: You can either use the web console OR use a declarative cluster specification with the RCTL CLI to manage your EKS cluster's lifecycle.

Web Console | RCTL CLI | Terraform

In your project, navigate to Infrastructure -> Clusters
Click on the "cluster name" and then the "node groups" tab.
Click on the gear icon next to the spot node group
Click on "Delete Node Group" and confirm it

This will "drain" the nodes in this node group and remove them from the EKS cluster. The entire process can take 5-10 minutes to complete.

Execute the following command to remove the spot instance node group

./rctl apply -f eks-cluster-basics.yaml

Download the cluster specification for the existing cluster

Go to Infrastructure -> Clusters.
Click on the settings icon of the cluster and select "Download Cluster Config"
Update the downloaded specification file with the YAML for the additional node group.
Remove the YAML for the spot node group

The fully updated cluster specification file including the newly added spot instance node group code will look like this:

managedNodeGroups:

name: ng-1 instanceType: t3.large desiredCapacity: 2


- Execute the following command to remove the spot instance node group

./rctl apply -f eks-cluster-basics.yaml


To remove a node group:

- Edit the `terraform.tfvars` file. The file location is **/terraform/terraform.tfvars**.
- Delete the `pool2` configuration, then save the file.
- Open the terminal or command line.
- Navigate to the `terraform` folder.
- Run `terraform apply`. Enter `yes` when prompted.

It can take 20 minutes to remove the node group to the cluster. Check the console to confirm the node group has been removed.

---

## Verify Deletion

Once the new, spot based node group has been successfully deleted from the EKS cluster, you should be able to confirm this on the web console.

---

## Recap

Congratulations!

You have successfully added a "spot instance" based node group to your EKS cluster to take advantage of discounted compute resources. As a final step, you also successfully removed the "spot" based node group from your EKS cluster.

---