​
Event Information

​
Meaning

  • The “Contact K8S API Server From Container” event in a Kubernetes cluster indicates that a container within the cluster attempted to communicate with the Kubernetes API server.
  • This event could be triggered by a legitimate action, such as a pod querying the API server for information or performing an operation like scaling or updating resources.
  • However, it could also indicate a potential security concern if the container is attempting unauthorized access or performing malicious activities against the API server.
To investigate further, you can:
  • Use the kubectl get pods command to list all the pods running in the cluster and check if any suspicious or unauthorized pods are present.
  • Review the logs of the container that triggered the event using kubectl logs <pod-name> -c <container-name> to identify any abnormal behavior or unauthorized API requests.
  • Monitor network traffic using tools like kubectl port-forward or kubectl proxy to capture and analyze the network communication between the container and the API server.

​
Remediation

  1. Create a Python script that uses the Kubernetes Python client library to interact with the Kubernetes API server.
  2. Use the script to create a Kubernetes ServiceAccount with limited permissions that can be used by the containers running in the cluster.
  3. Modify the deployment manifest of the container that triggered the event to use the newly created ServiceAccount.
Here’s an example of how the remediation script could look like: 
from kubernetes import client, config

# Load the Kubernetes configuration
config.load_kube_config()

# Create a Kubernetes API client
api_client = client.ApiClient()

# Create a ServiceAccount
service_account = client.V1ServiceAccount(
    metadata=client.V1ObjectMeta(name="restricted-service-account")
)

# Create the ServiceAccount in the cluster
api_instance = client.CoreV1Api(api_client)
api_instance.create_namespaced_service_account(
    namespace="your-namespace",
    body=service_account
)

# Modify the deployment manifest to use the newly created ServiceAccount
deployment_manifest = {
    "apiVersion": "apps/v1",
    "kind": "Deployment",
    "metadata": {
        "name": "your-deployment",
        "namespace": "your-namespace"
    },
    "spec": {
        "template": {
            "spec": {
                "serviceAccountName": "restricted-service-account",
                # Add other container specifications here
            }
        }
    }
}

# Update the deployment in the cluster
api_instance = client.AppsV1Api(api_client)
api_instance.patch_namespaced_deployment(
    name="your-deployment",
    namespace="your-namespace",
    body=deployment_manifest
)
Make sure to replace “your-namespace” with the actual namespace where the deployment is running, and “your-deployment” with the name of the deployment that triggered the event.