Skip to main content

Node Pools Should Be Regional For High Availability

More Info:​

GKE node pools should be regional (multiple zones) for maximum nodes availability during zonal outages

Risk Level​

High

Address​

Operational Excellence, Performance Efficiency, Reliability, Security

Compliance Standards​

  • HIPAA
  • HITRUST CSF
  • NIST
  • NIST CSF
  • PCI
  • SOC2

Triage and Remediation​

Remediation​

Using Console

To remediate the misconfiguration "Node Pools Should Be Regional For High Availability" in GCP using GCP console, follow the below steps:

  1. Login to the GCP console.

  2. Navigate to the Kubernetes Engine section.

  3. Select the cluster that has the misconfigured node pool.

  4. Click on the "Nodes" tab on the left-hand side.

  5. Select the node pool that needs to be remediated.

  6. Click the "Edit" button at the top of the page.

  7. Under the "Location" section, select "Regional" from the drop-down menu.

  8. Choose the region where you want to create the node pool.

  9. Click "Save" to apply the changes.

  10. Wait for the node pool to be created in the selected region.

  11. Once the node pool is created, verify that the nodes are running and healthy.

By following these steps, you will have successfully remediated the misconfiguration "Node Pools Should Be Regional For High Availability" in GCP using GCP console.

Using CLI

To remediate the misconfiguration "Node Pools Should Be Regional For High Availability" for GCP using GCP CLI, follow the below steps:

  1. First, check the current status of your node pool using the following command:
gcloud container node-pools describe [NODE_POOL_NAME] --cluster=[CLUSTER_NAME] --zone=[ZONE_NAME]

Replace [NODE_POOL_NAME], [CLUSTER_NAME] and [ZONE_NAME] with your own values.

  1. If the node pool is not regional, create a new regional node pool using the following command:
gcloud container node-pools create [NEW_NODE_POOL_NAME] --cluster=[CLUSTER_NAME] --num-nodes=[NUMBER_OF_NODES] --region=[REGION_NAME]

Replace [NEW_NODE_POOL_NAME], [CLUSTER_NAME], [NUMBER_OF_NODES] and [REGION_NAME] with your own values.

  1. Verify that the new node pool has been created successfully using the following command:
gcloud container node-pools list --cluster=[CLUSTER_NAME] --region=[REGION_NAME]

Replace [CLUSTER_NAME] and [REGION_NAME] with your own values.

  1. Once the new node pool is created and verified, you can delete the old node pool using the following command:
gcloud container node-pools delete [OLD_NODE_POOL_NAME] --cluster=[CLUSTER_NAME] --zone=[ZONE_NAME]

Replace [OLD_NODE_POOL_NAME], [CLUSTER_NAME] and [ZONE_NAME] with your own values.

  1. Verify that the old node pool has been deleted successfully using the following command:
gcloud container node-pools list --cluster=[CLUSTER_NAME] --zone=[ZONE_NAME]

Replace [CLUSTER_NAME] and [ZONE_NAME] with your own values.

By following these steps, you can remediate the misconfiguration "Node Pools Should Be Regional For High Availability" for GCP using GCP CLI.

Using Python

To remediate the misconfiguration "Node Pools Should Be Regional For High Availability" for GCP using Python, you can follow the below steps:

Step 1: Import the required libraries and authenticate to GCP using service account credentials.

from google.oauth2 import service_account
from google.cloud import container_v1

credentials = service_account.Credentials.from_service_account_file(
    'path/to/service_account.json')
client = container_v1.ClusterManagerClient(credentials=credentials)

Step 2: Get the list of node pools in the cluster that are not regional.

project_id = 'your-project-id'
zone = 'your-zone'
cluster_id = 'your-cluster-id'

cluster = client.get_cluster(project_id, zone, cluster_id)
for node_pool in cluster.node_pools:
    if node_pool.locations[0] != zone:
        print(f"Node pool {node_pool.name} is not regional.")

Step 3: Delete the non-regional node pools and create new regional node pools.

# Delete non-regional node pools
for node_pool in cluster.node_pools:
    if node_pool.locations[0] != zone:
        client.delete_node_pool(project_id, zone, cluster_id, node_pool.name)

# Create new regional node pools
node_pool = {
    'name': 'your-node-pool-name',
    'config': {
        'machine_type': 'your-machine-type',
        'disk_size_gb': 100,
    },
    'initial_node_count': 1,
    'locations': ['your-zone'],
}
response = client.create_node_pool(project_id, zone, cluster_id, node_pool)

Note: Replace 'your-project-id', 'your-zone', 'your-cluster-id', 'your-node-pool-name', 'your-machine-type' and '100' with your actual values.

These steps will remediate the misconfiguration "Node Pools Should Be Regional For High Availability" for GCP using Python.

Using Terraform
resource "google_container_cluster" "REGIONAL_GKE_CLUSTER" {
  name     = "REGIONAL_CLUSTER_NAME"   # replace with your cluster name
  project  = "GCP_PROJECT_ID"          # replace with your project ID
  location = "us-central1"             # replace with your desired REGION (not a zone)

  # Ensure node pools are spread across multiple zones in this region
  node_locations = [
    "us-central1-a",
    "us-central1-b",
    "us-central1-c",
  ]                                     # replace with 2–3 zones in the same region

  # Either default node pool disabled and separate node_pools, or use initial_node_count
  remove_default_node_pool = true
  initial_node_count       = 1

  # Example managed node pool spanning the same zones
  node_pool {
    name = "PRIMARY_NODE_POOL_NAME"    # replace with your node pool name

    node_config {
      machine_type = "e2-medium"
      oauth_scopes = [
        "https://www.googleapis.com/auth/cloud-platform",
      ]
    }

    # This node pool will inherit the cluster's multi-zonal topology
  }
}

Changing a zonal google_container_cluster to regional (by switching location from a zone like us-central1-a to a region like us-central1 and adding node_locations) forces full cluster replacement, which is disruptive and will recreate all node pools and workloads; plan and migrate accordingly.

To verify, terraform plan should show the existing google_container_cluster being destroyed and a new one created with location set to a region and node_locations listing multiple zones in that region, and any node pools attached to this regional cluster.