K8s Stackdriver Logging Monitoring Enabled Remediation

Triage and Remediation

Remediation

Remediation

Using Console

To remediate the misconfiguration "Ensure Stackdriver Kubernetes Logging And Monitoring Is Enabled" for GCP using GCP console, follow the below steps:

Login to the GCP console with valid credentials.
Navigate to the Google Kubernetes Engine (GKE) cluster for which you want to enable Stackdriver Kubernetes Logging and Monitoring.
Click on the "Edit" button at the top of the page.
Scroll down to the "Stackdriver" section and click on it.
Under the "Stackdriver Logging" section, toggle the switch to "Enabled".
Under the "Stackdriver Monitoring" section, toggle the switch to "Enabled".
Click on the "Save" button at the bottom of the page.

Once the above steps are completed, Stackdriver Kubernetes Logging and Monitoring will be enabled for the GKE cluster. You can then view logs and metrics for your Kubernetes cluster in the Stackdriver Logging and Monitoring UI.

Using CLI

To remediate the misconfiguration of Stackdriver Kubernetes Logging and Monitoring not being enabled on GCP using GCP CLI, follow the below steps:

Install and configure the Google Cloud SDK (CLI) on your local machine.
Open the terminal or command prompt and authenticate to your GCP account using the following command:

gcloud auth login

Set the default project for the CLI to the project where the Kubernetes cluster is deployed using the following command:

gcloud config set project [PROJECT_ID]

Enable the Stackdriver Kubernetes Logging and Monitoring API for the project using the following command:

gcloud services enable container.googleapis.com stackdriver.googleapis.com

Verify that the Stackdriver Kubernetes Logging and Monitoring API is enabled by running the following command:

gcloud services list --enabled

If the API is enabled, you should see the following services listed:

NAME                                TITLE
container.googleapis.com            Kubernetes Engine API
logging.googleapis.com              Stackdriver Logging API
monitoring.googleapis.com           Stackdriver Monitoring API
stackdriver.googleapis.com          Stackdriver API

Once the API is enabled, you can configure the Kubernetes cluster to send logs and metrics to Stackdriver by following the official GCP documentation:

https://cloud.google.com/kubernetes-engine/docs/how-to/logging

https://cloud.google.com/monitoring/kubernetes-engine/

Using Python

To remediate the misconfiguration "Ensure Stackdriver Kubernetes Logging And Monitoring Is Enabled" for GCP using Python, you can follow these steps:

Install the google-cloud-monitoring and google-cloud-logging Python libraries using pip.

pip install google-cloud-monitoring google-cloud-logging

Create a service account with the necessary permissions to enable Stackdriver Kubernetes logging and monitoring.
Set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path of the service account key file.

export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service_account_key.json

Use the google-cloud-monitoring and google-cloud-logging libraries to enable Stackdriver Kubernetes logging and monitoring.

from google.cloud.monitoring_v3 import MetricServiceClient
from google.cloud.logging_v2 import LoggingServiceV2Client
from google.cloud.logging_v2.types import LogMetric

# Enable Kubernetes API metrics
metric_client = MetricServiceClient()
project_name = f"projects/{project_id}"
metric_descriptor = metric_client.get_metric_descriptor(
    name=f"{project_name}/metricDescriptors/kubernetes.io/container/cpu/usage_time"
)
metric_descriptor.enabled = True
metric_client.update_metric_descriptor(metric_descriptor)

# Create a Stackdriver Logging metric for Kubernetes logs
logging_client = LoggingServiceV2Client()
parent = logging_client.project_path(project_id)
log_metric = LogMetric(
    name="kubernetes_log",
    description="Kubernetes logs",
    filter='resource.type="k8s_container"',
    metric_descriptor_name=f"{project_name}/metricDescriptors/logging.googleapis.com/container/log_bytes",
    value_extractor="SUM(jsonPayload.message_size)",
)
logging_client.create_log_metric(parent=parent, metric=log_metric)

Note: Replace project_id with your GCP project ID.

Verify that Stackdriver Kubernetes logging and monitoring is enabled by checking the Stackdriver Metrics Explorer and Logs Viewer in the GCP Console.

That's it! You have now remediated the misconfiguration "Ensure Stackdriver Kubernetes Logging And Monitoring Is Enabled" for GCP using Python.

Using Terraform

resource "google_container_cluster" "GKE_CLUSTER" {
  name     = "GKE_CLUSTER_NAME"          # replace with your cluster name
  location = "GKE_CLUSTER_LOCATION"      # replace with your region or zone

  # ... other required cluster arguments (network, initial_node_count, etc.)

  # Ensure logs are sent to Cloud Logging (Stackdriver)
  logging_config {
    # Enable at least system and workload logs; add more components if required
    enable_components = [
      "SYSTEM_COMPONENTS",
      "WORKLOADS",
      # "APISERVER",
      # "SCHEDULER",
      # "CONTROLLER_MANAGER",
    ]
  }

  # Ensure metrics are sent to Cloud Monitoring (Stackdriver)
  monitoring_config {
    enable_components = [
      "SYSTEM_COMPONENTS",
      "WORKLOADS",
      # "APISERVER",
      # "SCHEDULER",
      # "CONTROLLER_MANAGER",
      # "STORAGE",
      # "HPA",
    ]

    # Optional but recommended if you use managed Prometheus
    # managed_prometheus {
    #   enabled = true
    # }
  }
}

Substitute:

GKE_CLUSTER_NAME with your cluster’s name.
GKE_CLUSTER_LOCATION with your cluster’s region or zone (for example, us-central1 or us-central1-a).

This change updates the existing cluster in place and does not normally force replacement; terraform plan should show an in‑place update to the google_container_cluster.GKE_CLUSTER resource with changes only to logging_config and monitoring_config.

Triage and Remediation​

Remediation​

Triage and Remediation

Remediation