Kubernetes Node Pool Autoscaling Should Be Enabled

More Info:

Ensure that node pool autoscaling is enabled

Risk Level

Medium

Address

Operational Excellence, Performance Efficiency, Reliability, Security

Compliance Standards

• Cloudanix Best Practice

Triage and Remediation

Remediation

Remediation

Using Console

To remediate the Kubernetes Node Pool Autoscaling misconfiguration for GCP using GCP console, follow these steps:

1. Open the Google Cloud Console and navigate to the Kubernetes Engine page.
2. Select the cluster that you want to remediate.
3. Click on the "Node pools" tab.
4. Select the node pool that you want to enable autoscaling for.
5. Click on the "Edit" button at the top of the page.
6. Scroll down to the "Autoscaling" section and click on the toggle button to enable autoscaling.
7. Set the minimum and maximum number of nodes that you want to use for the node pool.
8. Click on the "Save" button to apply the changes.

After following these steps, the Kubernetes Node Pool Autoscaling will be enabled for the selected node pool in GCP.

Using CLI

To remediate the misconfiguration "Kubernetes Node Pool Autoscaling Should Be Enabled" for GCP, you can follow these step-by-step instructions using GCP CLI:

1. Open the Cloud Shell in your GCP console.

2. Run the following command to get a list of all the node pools in your GKE cluster:

gcloud container node-pools list --cluster [CLUSTER_NAME] --zone [ZONE]

Replace [CLUSTER_NAME] and [ZONE] with your GKE cluster name and zone.

3. Choose the node pool that you want to enable autoscaling for.

4. Run the following command to enable autoscaling for the chosen node pool:

gcloud container node-pools update [NODE_POOL_NAME] --cluster [CLUSTER_NAME] --zone [ZONE] --autoscaling-enabled

Replace [NODE_POOL_NAME], [CLUSTER_NAME], and [ZONE] with the appropriate values.

5. Set the minimum and maximum number of nodes for the autoscaler using the following command:

gcloud container node-pools update [NODE_POOL_NAME] --cluster [CLUSTER_NAME] --zone [ZONE] --autoscaling-min-nodes=[MIN_NODES] --autoscaling-max-nodes=[MAX_NODES]

Replace [MIN_NODES] and [MAX_NODES] with the desired minimum and maximum number of nodes.

6. Verify that autoscaling is enabled for the node pool by running the following command:

gcloud container node-pools describe [NODE_POOL_NAME] --cluster [CLUSTER_NAME] --zone [ZONE] | grep autoscaling

This command should return "autoscaling: enabled".

With these steps, you have successfully remediated the misconfiguration "Kubernetes Node Pool Autoscaling Should Be Enabled" for GCP using GCP CLI.

Using Python

To remediate the misconfiguration "Kubernetes Node Pool Autoscaling Should Be Enabled" for GCP using Python, you can use the following steps:

1. Import the necessary libraries:

from google.cloud import container_v1
from google.oauth2 import service_account

2. Set up the credentials:

credentials = service_account.Credentials.from_service_account_file('path/to/credentials.json')

3. Define the project ID and cluster ID:

project_id = 'your-project-id'
zone = 'us-central1-a'
cluster_id = 'your-cluster-id'

4. Get the current node pool configuration:

client = container_v1.ClusterManagerClient(credentials=credentials)
cluster = client.get_cluster(project_id, zone, cluster_id)

node_pool = cluster.node_pools[0]

5. Check if autoscaling is enabled:

if not node_pool.autoscaling:
    node_pool.autoscaling = container_v1.types.NodePoolAutoscaling(enabled=True, min_node_count=1, max_node_count=10)
    update_request = container_v1.types.UpdateClusterRequest(project_id=project_id, zone=zone, cluster_id=cluster_id, update=cluster)
    operation = client.update_cluster(update_request)
    operation.result()

6. Update the node pool configuration:

update_request = container_v1.types.UpdateClusterRequest(project_id=project_id, zone=zone, cluster_id=cluster_id, update=cluster)
operation = client.update_cluster(update_request)
operation.result()

This will enable autoscaling for the node pool in your GCP Kubernetes cluster.

Using Terraform

resource "google_container_cluster" "PRIMARY_CLUSTER" {
  name     = "PRIMARY_CLUSTER_NAME"        # replace with your cluster name
  location = "GKE_LOCATION"                # e.g. "us-central1" (regional) or "us-central1-a" (zonal)
  project  = "GCP_PROJECT_ID"              # replace with your project ID

  # If you are currently using the default node pool, the common pattern
  # is to remove it and manage node pools explicitly.
  remove_default_node_pool = true
  initial_node_count       = 1

  # Cluster autoscaler must be enabled at the cluster level
  cluster_autoscaling {
    enabled = true

    auto_provisioning_defaults {
      service_account = "GKE_NODE_SERVICE_ACCOUNT_EMAIL" # optional; replace or remove as appropriate
    }

    resource_limits {
      resource_type = "cpu"
      minimum       = CLUSTER_MIN_VCPUS   # replace with an integer, e.g. 1
      maximum       = CLUSTER_MAX_VCPUS   # replace with an integer, e.g. 16
    }

    resource_limits {
      resource_type = "memory"
      minimum       = CLUSTER_MIN_GB_MEM  # replace with an integer, e.g. 4
      maximum       = CLUSTER_MAX_GB_MEM  # replace with an integer, e.g. 64
    }
  }

  autoscaling {
    profile = "OPTIMIZE_UTILIZATION"      # or "BALANCED", as required
  }

  networking_mode = "VPC_NATIVE"          # example; keep your existing setting
  network         = "VPC_NETWORK_NAME"    # replace with your VPC
  subnetwork      = "VPC_SUBNETWORK_NAME" # replace with your subnetwork
}

resource "google_container_node_pool" "PRIMARY_AUTOSCALED_POOL" {
  name     = "PRIMARY_AUTOSCALED_POOL_NAME" # replace with your node pool name
  project  = google_container_cluster.PRIMARY_CLUSTER.project
  location = google_container_cluster.PRIMARY_CLUSTER.location
  cluster  = google_container_cluster.PRIMARY_CLUSTER.name

  node_config {
    machine_type = "e2-standard-4"         # replace with your machine type
    oauth_scopes = [
      "https://www.googleapis.com/auth/cloud-platform",
    ]
    service_account = "GKE_NODE_SERVICE_ACCOUNT_EMAIL" # replace as appropriate
    labels = {
      "cluster" = google_container_cluster.PRIMARY_CLUSTER.name
    }
  }

  # This block enables node pool autoscaling – this is what the finding is about.
  autoscaling {
    min_node_count = NODE_POOL_MIN_NODES   # replace with integer threshold, e.g. 1
    max_node_count = NODE_POOL_MAX_NODES   # replace with integer threshold, e.g. 5
  }

  management {
    auto_upgrade = true
    auto_repair  = true
  }
}

Notes:

• If you change remove_default_node_pool from false to true on an existing google_container_cluster, Terraform will need to replace the cluster, which is an outage-causing operation. Plan this change carefully.
• Adjust CLUSTER_MIN_VCPUS, CLUSTER_MAX_VCPUS, CLUSTER_MIN_GB_MEM, CLUSTER_MAX_GB_MEM, NODE_POOL_MIN_NODES, and NODE_POOL_MAX_NODES to match any required thresholds for your environment.

Verification:

• terraform plan should show an in-place update to google_container_node_pool.PRIMARY_AUTOSCALED_POOL adding the autoscaling block (and enabling cluster_autoscaling on google_container_cluster.PRIMARY_CLUSTER if it was previously disabled), with no unexpected resource replacements other than any intentional change to remove_default_node_pool.

Additional Reading:

• https://cloud.google.com/kubernetes-engine/docs/concepts/cluster-autoscaler