Automatic Node Repair Should Be Enabled

More Info:

Ensures all Kubernetes cluster nodes have automatic repair enabled. When automatic repair on nodes is enabled, the Kubernetes engine performs health checks on all nodes, automatically repairing nodes that fail health checks. This ensures that the Kubernetes environment stays optimal.

Risk Level

Low

Address

Reliability, Security

Compliance Standards

• NIST CSF

Triage and Remediation

Remediation

Remediation

Using Console

To remediate the misconfiguration "Automatic Node Repair Should Be Enabled" for GCP, you can follow these steps using the GCP Console:

1. Open the GCP Console and go to the "Kubernetes Engine" section.
2. Select the cluster that you want to remediate.
3. Click on the "Edit" button at the top of the page.
4. Scroll down to the "Node Pools" section and click on the node pool that you want to remediate.
5. Scroll down to the "Node auto-repair" option and toggle it on.
6. Click on the "Save" button at the bottom of the page to apply the changes.

Once you have completed these steps, automatic node repair should be enabled for the selected node pool in your GCP Kubernetes cluster. This will ensure that any nodes that fail or become unresponsive are automatically repaired or replaced, helping to maintain the availability and reliability of your cluster.

Using CLI

To remediate the misconfiguration "Automatic Node Repair Should Be Enabled" in GCP using GCP CLI, follow the steps below:

1. Open the Cloud Shell in your GCP console.

2. Run the following command to enable automatic node repair for all node pools in a specific cluster:

gcloud container clusters update [CLUSTER_NAME] --enable-autorepair

Replace [CLUSTER_NAME] with the name of the cluster where you want to enable automatic node repair.

3. If you want to enable automatic node repair for a specific node pool in a cluster, run the following command:

gcloud container node-pools update [NODE_POOL_NAME] --cluster [CLUSTER_NAME] --enable-autorepair

Replace [NODE_POOL_NAME] with the name of the node pool where you want to enable automatic node repair, and [CLUSTER_NAME] with the name of the cluster where the node pool is located.

4. Verify that automatic node repair is enabled by running the following command:

gcloud container node-pools describe [NODE_POOL_NAME] --cluster [CLUSTER_NAME] | grep autorepair

This command will show you the status of automatic node repair for the specified node pool.

By following these steps, you can remediate the misconfiguration "Automatic Node Repair Should Be Enabled" in GCP using GCP CLI.

Using Python

To remediate the misconfiguration of "Automatic Node Repair Should Be Enabled" for GCP using Python, you can follow the below steps:

1. Import the necessary modules:

from googleapiclient import discovery
from oauth2client.client import GoogleCredentials

2. Set the project ID and zone where the misconfiguration exists:

project_id = 'YOUR_PROJECT_ID'
zone = 'YOUR_ZONE'

3. Create a client object to authenticate with the GCP API:

credentials = GoogleCredentials.get_application_default()
compute = discovery.build('compute', 'v1', credentials=credentials)

4. Get the instance group manager resource:

instance_group_manager = compute.instanceGroupManagers().get(project=project_id, zone=zone, instanceGroupManager='YOUR_INSTANCE_GROUP_MANAGER_NAME').execute()

5. Check if the "autoHealingPolicies" field exists in the instance group manager resource:

if 'autoHealingPolicies' not in instance_group_manager:
    instance_group_manager['autoHealingPolicies'] = []

6. Create an auto-healing policy dictionary object:

auto_healing_policy = {
    'healthCheck': 'YOUR_HEALTH_CHECK_URL',
    'initialDelaySec': 'YOUR_INITIAL_DELAY_IN_SECONDS',
    'maxUnavailable': 'YOUR_MAX_UNAVAILABLE_COUNT'
}

7. Append the auto-healing policy to the instance group manager's "autoHealingPolicies" field:

instance_group_manager['autoHealingPolicies'].append(auto_healing_policy)

8. Update the instance group manager resource with the new auto-healing policy:

compute.instanceGroupManagers().update(project=project_id, zone=zone, instanceGroupManager='YOUR_INSTANCE_GROUP_MANAGER_NAME', body=instance_group_manager).execute()

By following these steps, you can remediate the misconfiguration of "Automatic Node Repair Should Be Enabled" for GCP using Python.

Using Terraform

resource "google_container_cluster" "PRIMARY_CLUSTER" {
  name     = "PRIMARY_CLUSTER_NAME"   # replace with your cluster name
  location = "CLUSTER_LOCATION"       # e.g. "us-central1", "us-central1-a"
  project  = "GCP_PROJECT_ID"         # replace with your GCP project ID

  # If you still use the default node pool, enable auto-repair on it:
  node_config {
    machine_type = "e2-medium"

    # other node_config settings
  }

  management {
    auto_repair = true
    # auto_upgrade can be set independently, e.g.:
    # auto_upgrade = true
  }

  # other cluster settings
}

# For separately-managed node pools attached to the cluster:
resource "google_container_node_pool" "PRIMARY_NODE_POOL" {
  name       = "PRIMARY_NODE_POOL_NAME"  # replace with your node pool name
  cluster    = google_container_cluster.PRIMARY_CLUSTER.name
  location   = google_container_cluster.PRIMARY_CLUSTER.location
  project    = google_container_cluster.PRIMARY_CLUSTER.project
  node_count = 3

  node_config {
    machine_type = "e2-medium"

    # other node_config settings
  }

  management {
    auto_repair = true
    # auto_upgrade can be set independently, e.g.:
    # auto_upgrade = true
  }

  # other node pool settings
}

This change updates in place and should not force replacement of the cluster or node pools.

Verification: terraform plan should show management.auto_repair changing from false (or null) to true on each google_container_cluster/google_container_node_pool that previously lacked automatic repair.

Additional Reading:

• https://cloud.google.com/kubernetes-engine/docs/how-to/node-auto-repair