Skip to main content

Worker Pool Autoscaling Should Be Enabled

More Info:​

Ensure worker pool autoscaling is enabled

Risk Level​

Medium

Address​

Operational Excellence, Performance Efficiency, Reliability, Security

Compliance Standards​

  • Cloudanix Best Practice

Triage and Remediation​

Remediation​

Using Console

To remediate the misconfiguration "Worker Pool Autoscaling Should Be Enabled" in GCP using GCP console, follow these steps:

  1. Open the GCP console and select the appropriate project.

  2. Navigate to the Kubernetes Engine by selecting Kubernetes Engine from the left-hand side menu.

  3. Select the cluster that needs to be remediated.

  4. Click on the Edit button at the top of the page.

  5. Scroll down to the Node Pools section and click on the node pool that needs remediation.

  6. In the node pool settings, scroll down to the Autoscaling section.

  7. Toggle the button next to "Enable autoscaling" to the "On" position.

  8. Set the minimum and maximum number of nodes. It is recommended to set the minimum number of nodes to 1 and the maximum number of nodes based on the workload requirements.

  9. Click on the Save button to apply the changes.

With these steps, the "Worker Pool Autoscaling Should Be Enabled" misconfiguration in GCP using GCP console has been remediated.

Using CLI

To remediate the misconfiguration "Worker Pool Autoscaling Should Be Enabled" for GCP using GCP CLI, you can follow the below steps:

  1. Open the Cloud Shell in GCP Console.
  2. Run the command gcloud container clusters list to list all the clusters available in your project.
  3. Find the name of the cluster for which you want to enable the worker pool autoscaling.
  4. Run the command gcloud container clusters update CLUSTER_NAME --enable-autoscaling --min-nodes=1 --max-nodes=10 --num-nodes=3 to enable the worker pool autoscaling for the cluster.
    • Replace CLUSTER_NAME with the name of your cluster.
    • --enable-autoscaling enables the autoscaling feature.
    • --min-nodes=1 sets the minimum number of nodes to 1.
    • --max-nodes=10 sets the maximum number of nodes to 10.
    • --num-nodes=3 sets the initial number of nodes to 3.
  5. Verify the autoscaling feature is enabled by running the command gcloud container clusters describe CLUSTER_NAME.
  6. Check the output of the above command and ensure that the autoscaling field is set to enabled.

These steps will enable the worker pool autoscaling for the specified cluster in GCP, and remediate the misconfiguration "Worker Pool Autoscaling Should Be Enabled".

Using Python

To remediate the misconfiguration "Worker Pool Autoscaling Should Be Enabled" for GCP using Python, you can follow the below steps:

  1. First, you need to create an instance group for your worker nodes. You can use the following code to create an instance group:
from googleapiclient import discovery
from oauth2client.client import GoogleCredentials

credentials = GoogleCredentials.get_application_default()
compute = discovery.build('compute', 'v1', credentials=credentials)

project = 'your-project-id'
zone = 'us-central1-a'
name = 'instance-group-name'
size = 3
image_response = compute.images().getFromFamily(
    project='ubuntu-os-cloud', family='ubuntu-1804-lts').execute()
source_disk_image = image_response['selfLink']
machine_type = "n1-standard-1"
config = {
    'name': name,
    'instanceTemplate': f"projects/{project}/global/instanceTemplates/instance-template-name",
    'targetSize': size,
    'autoHealingPolicies': [
        {
            "healthCheck": "projects/{project}/global/healthChecks/health-check-name",
            "initialDelaySec": 300,
            "autoHealingPolicyMode": "RECREATE_INSTANCE"
        }
    ]
}
request = compute.instanceGroupManagers().insert(
    project=project,
    zone=zone,
    body=config)
response = request.execute()
  1. Next, you need to enable autoscaling for your instance group. You can use the following code to enable autoscaling:
autoscaler_config = {
    'name': 'autoscaler-name',
    'target': f"projects/{project}/zones/{zone}/instanceGroupManagers/{name}",
    'autoscalingPolicy': {
        'minNumReplicas': 1,
        'maxNumReplicas': 10,
        'coolDownPeriodSec': 60,
        'cpuUtilization': {
            'utilizationTarget': 0.6,
        }
    }
}
autoscaler_request = compute.autoscalers().insert(
    project=project,
    zone=zone,
    body=autoscaler_config)
autoscaler_response = autoscaler_request.execute()
  1. Finally, you need to verify that autoscaling is enabled for your instance group. You can use the following code to check the status of your autoscaler:
autoscaler_name = 'autoscaler-name'
autoscaler_request = compute.autoscalers().get(
    project=project,
    zone=zone,
    autoscaler=autoscaler_name)
autoscaler_response = autoscaler_request.execute()
print(autoscaler_response)

This should remediate the misconfiguration "Worker Pool Autoscaling Should Be Enabled" for GCP using Python.

Using Terraform
resource "google_dataflow_job" "DATAFLOW_JOB_NAME" {
  name              = "DATAFLOW_JOB_NAME"          # replace with your job name
  template_gcs_path = "gs://PATH/TO/TEMPLATE"      # replace with your template path
  temp_gcs_location = "gs://PATH/TO/TEMP/DIR"      # replace with your temp location
  region            = "DATAFLOW_REGION"            # replace with your region, e.g., us-central1

  # Initial worker count (can be 1 or higher, per your needs)
  num_workers = INITIAL_WORKER_COUNT               # e.g., 1

  # Enable worker pool autoscaling with a maximum number of workers (the threshold)
  autoscaling_algorithm = "AUTOSCALING_ALGORITHM_BASIC"
  max_workers           = MAX_WORKER_COUNT         # e.g., 10

  # Add any parameters your template requires
  parameters = {
    PARAM_KEY = "PARAM_VALUE"
  }

  on_delete = "cancel"
}
  • Replace DATAFLOW_JOB_NAME, PATH/TO/TEMPLATE, PATH/TO/TEMP/DIR, DATAFLOW_REGION, INITIAL_WORKER_COUNT, MAX_WORKER_COUNT, PARAM_KEY, and PARAM_VALUE with your actual values (set MAX_WORKER_COUNT to the autoscaling threshold you require).
  • Modifying autoscaling_algorithm, num_workers, or max_workers usually forces replacement of the Dataflow job; expect the existing job to be cancelled and a new one created on terraform apply.

For verification, terraform plan should show changes (or creation) where autoscaling_algorithm is set to "AUTOSCALING_ALGORITHM_BASIC" and max_workers = MAX_WORKER_COUNT on the google_dataflow_job.DATAFLOW_JOB_NAME resource.