> ## Documentation Index
> Fetch the complete documentation index at: https://cloudanix.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Dataflow Job CMK Keys Should Be Set

### More Info:

Ensure Dataflow jobs are encrypted with customer managed kyes

### Risk Level

Low

### Address

Reliability, Security

### Compliance Standards

SOC2, GDPR, ISO27001, HIPAA, HITRUST, NISTCSF, PCIDSS

### Triage and Remediation

<Tabs>
  <Tab title="Remediation">
    ### Remediation

    <AccordionGroup>
      <Accordion title="Using Console" defaultOpen="true">
        To remediate the "Dataflow Job CMK Keys Should Be Set" misconfiguration in GCP using the GCP console, please follow these steps:

        1. Open the Google Cloud Console and go to the Dataflow page.
        2. Select the Dataflow job that is affected by the misconfiguration.
        3. Click on the "Edit" button to edit the job configuration.
        4. Scroll down to the "Security" section of the configuration page.
        5. Under the "Encryption" section, select "Customer-managed key" from the "Key source" dropdown menu.
        6. Choose the appropriate Cloud KMS key that you want to use to encrypt your data.
        7. Click on the "Save" button to save the changes.

        After following these steps, your Dataflow job will be configured to use customer-managed keys for encryption.

        #
      </Accordion>

      <Accordion title="Using CLI">
        To remediate the "Dataflow Job CMK Keys Should Be Set" misconfiguration in GCP using GCP CLI, you can follow the below steps:

        1. Open the Cloud Shell in your GCP Console.

        2. Run the following command to set the Cloud Key Management Service (KMS) key for Dataflow:

           ```
           gcloud dataflow jobs update <JOB_ID> --update-kms-key=<KMS_KEY>
           ```

           Replace `<JOB_ID>` with the ID of the Dataflow job and `<KMS_KEY>` with the ID or fully qualified name of the Cloud KMS key to be used for encrypting the Dataflow job's temporary files.

        3. Verify that the KMS key has been set by running the following command:

           ```
           gcloud dataflow jobs describe <JOB_ID> | grep kmsKeyName
           ```

           This command should return the name of the KMS key that was set in step 2.

        4. Repeat steps 2 and 3 for all Dataflow jobs that require a KMS key to be set.

        By following these steps, you can remediate the "Dataflow Job CMK Keys Should Be Set" misconfiguration in GCP using GCP CLI.
      </Accordion>

      <Accordion title="Using Python">
        To remediate the misconfiguration "Dataflow Job CMK Keys Should Be Set" for GCP using python, you can follow the below steps:

        1. First, you need to create a Cloud KMS key ring and key in the same region as your Dataflow job. You can use the following code to create a key ring and key:

        ```python theme={null}
        from google.cloud import kms_v1

        # Replace <project-id> with your GCP project ID
        # Replace <key-ring-name> with the name of the key ring you want to create
        # Replace <key-name> with the name of the key you want to create

        project_id = "<project-id>"
        key_ring_name = "<key-ring-name>"
        key_name = "<key-name>"

        # Create the Cloud KMS client
        client = kms_v1.KeyManagementServiceClient()

        # Create the key ring
        parent = f"projects/{project_id}/locations/{location}"
        key_ring = client.create_key_ring(request={"parent": parent, "key_ring_id": key_ring_name})

        # Create the key
        purpose = kms_v1.CryptoKey.CryptoKeyPurpose.ENCRYPT_DECRYPT
        crypto_key = {"purpose": purpose}
        response = client.create_crypto_key(request={"parent": key_ring.name, "crypto_key_id": key_name, "crypto_key": crypto_key})
        ```

        2. Once you have created the key ring and key, you can update your Dataflow job to use the key. You can use the following code to update your job:

        ```python theme={null}
        from googleapiclient.discovery import build
        from oauth2client.client import GoogleCredentials

        # Replace <project-id> with your GCP project ID
        # Replace <job-id> with the ID of the Dataflow job you want to update
        # Replace <location> with the region where your job is running
        # Replace <key-ring-name> with the name of the key ring you created
        # Replace <key-name> with the name of the key you created

        project_id = "<project-id>"
        job_id = "<job-id>"
        location = "<location>"
        key_ring_name = "<key-ring-name>"
        key_name = "<key-name>"

        # Authenticate with GCP
        credentials = GoogleCredentials.get_application_default()
        service = build('dataflow', 'v1b3', credentials=credentials)

        # Get the current job configuration
        job = service.projects().locations().jobs().get(projectId=project_id, location=location, jobId=job_id).execute()

        # Update the job configuration to use the Cloud KMS key
        job['environment']['userAgent']['additionalUserAgent'] = 'dataflow-kms-sample'
        job['environment']['workerPools'][0]['workerHarnessContainerImage'] = 'gcr.io/dataflow-kms-cloud/dataflow-kms-sample:latest'
        job['environment']['workerPools'][0]['environment']['KMS_KEY_NAME'] = f"projects/{project_id}/locations/{location}/keyRings/{key_ring_name}/cryptoKeys/{key_name}"

        # Update the job
        request = service.projects().locations().jobs().update(projectId=project_id, location=location, jobId=job_id, body=job)
        response = request.execute()
        ```

        3. Finally, you can verify that the job is using the Cloud KMS key by checking the logs. You should see a message similar to the following:

        ```
        INFO:root:Using Cloud KMS key projects/<project-id>/locations/<location>/keyRings/<key-ring-name>/cryptoKeys/<key-name> to encrypt data
        ```

        By following these steps, you should be able to remediate the misconfiguration "Dataflow Job CMK Keys Should Be Set" for GCP using python.
      </Accordion>
    </AccordionGroup>
  </Tab>
</Tabs>