> ## Documentation Index > Fetch the complete documentation index at: https://cloudanix.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # EMR In-Transit and At-Rest Encryption ### More Info: Ensure that your AWS Elastic MapReduce (EMR) clusters are encrypted in order to meet security and compliance requirements. Data encryption helps prevent unauthorized users from reading sensitive data available on your EMR clusters and their associated data storage systems. This includes data saved to persistent media, known as data at-rest, and data that can be intercepted as it travels through the network, known as data in-transit. ### Risk Level High ### Address Security, Cost Optimization ### Compliance Standards HIPAA,GDPR,NIST,AWSWAF ### Remediation How to enable in transit and at rest encryption for EMR. #### Using AWS Console 1. Log in to the AWS Management Console using your AWS account credentials. 2. Navigate to the Amazon EMR service by selecting "EMR" from the services menu. (In the Cloudanix Console, navigate to "Misconfig" page and look for Affected Assets for "EMR In-Transit and At-Rest Encryption" Policy.) 3. Click on the "Create cluster" button to create a new EMR cluster or select an existing cluster to modify its settings. 4. In the "Create Cluster" or "Cluster List" page, scroll down to the "Security and Encryption" section. 5. Under the "Encryption" tab, you will find options for both in-transit and at-rest encryption. 6. To enable in-transit encryption, select the "Enable" checkbox next to "Encryption in transit". 7. Choose the desired encryption option for in-transit encryption. You can select "SSL" for encryption using SSL/TLS or "Custom" to provide your own custom encryption settings. 8. To enable at-rest encryption, select the "Enable" checkbox next to "Encryption at rest". 9. Choose the desired encryption option for at-rest encryption. You can select "Default AWS Key Management Service (KMS) key" or "Custom AWS KMS key" for encryption using AWS KMS. If you choose the "Custom AWS KMS key" option, provide the ARN (Amazon Resource Name) of the KMS key you want to use. 10. Review the other configuration settings for your EMR cluster, such as instance types, storage, and software configuration. 11. Click on the "Create cluster" or "Modify cluster" button to start the cluster creation or modification process. 12. AWS will provision the EMR cluster with the specified encryption settings, enabling both in-transit and at-rest encryption. ### Additional Reading * \[[https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-data-encryption-options.html](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-data-encryption-options.html)] ### Triage and Remediation ### Remediation Below are step‑by‑step console instructions to enable **at‑rest** and **in‑transit** encryption for **Amazon Redshift**. *** ## 1. At‑Rest Encryption (Redshift) ### A. Check whether your existing cluster is encrypted 1. Sign in to the **AWS Management Console**. 2. Go to **Amazon Redshift** console. 3. In the left pane, choose **Clusters**. 4. Click your **cluster identifier**. 5. On the **Configuration** tab, look for: * **Encryption**: Enabled / Disabled * **KMS key**: Which KMS key is used, if enabled. If **Encryption = Enabled**, at‑rest encryption is already configured. > Note: Redshift cluster encryption **cannot be turned on or off for an existing cluster**. You must create a new encrypted cluster and move data. *** ### B. Create or choose a KMS Customer Managed Key (optional but recommended) If you don’t want to use the default AWS managed key: 1. Open the **AWS KMS** console. 2. Select **Customer managed keys** → **Create key**. 3. Choose **Symmetric** → **Next**. 4. Give it an alias (e.g., `alias/redshift-encryption-key`). 5. Configure **key administrators** and **key usage permissions** (grant Redshift account roles or IAM roles that will manage the cluster). 6. Finish key creation. *** ### C. Create a new encrypted Redshift cluster 1. In the **Redshift console**, go to **Clusters** → **Create cluster**. 2. Under **Cluster configuration**, set: * **Cluster identifier**, **Node type**, number of nodes as needed. 3. Under **Database configurations**, set admin username/password. 4. Under **Security and encryption** (or similar section): * Turn **Encryption**: **On**. * **KMS key**: Select either: * **Default AWS Key for Redshift**, or * The customer managed key you created (e.g., `alias/redshift-encryption-key`). 5. Configure networking/security groups as needed. 6. Choose **Create cluster**. This new cluster will now have **encryption at rest enabled**. *** ### D. Move data from unencrypted to encrypted cluster 1. Option 1 – **Snapshot and restore** (if original is encrypted with a different key or you’re cloning an already‑encrypted one): * For an unencrypted cluster you can’t create an encrypted snapshot directly; instead: * Create a new **encrypted cluster** (done above), * Use **UNLOAD** / **COPY** commands to migrate data. 2. Option 2 – **UNLOAD / COPY**: * From the source (unencrypted) cluster: * Use `UNLOAD` to export data to **S3** (optionally encrypt S3 objects with SSE‑S3 or SSE‑KMS). * From the target (encrypted) cluster: * Use `COPY` to load data from S3 into the new cluster. 3. After validation, point applications to the new encrypted cluster and decommission the old one. *** ## 2. In‑Transit Encryption (SSL/TLS) for Redshift Redshift supports SSL/TLS for connections. You must: * Ensure the cluster has **SSL enabled** (it is by default), * Optionally enforce SSL in a parameter group, * Update clients to use SSL. *** ### A. Verify or configure SSL requirement via parameter group 1. Go to the **Redshift** console → **Parameter groups**. 2. Find the parameter group associated with your cluster: * In **Clusters** → select cluster → **Properties** → note the **Parameter group**. 3. If using the **default** parameter group: * You cannot modify it. Create a **new parameter group**: 1. **Parameter groups** → **Create parameter group**. 2. Choose: * **Parameter group family** matching your engine version. * **Group name** and **Description**. 3. Click **Create**. 4. Select your (new) parameter group → **Edit parameters**. 5. Find parameter: `require_ssl`. * Set **Value** to `true`. 6. Save changes. *** ### B. Attach the parameter group to the cluster 1. In the **Redshift console**, go to **Clusters**. 2. Select your cluster → **Actions** → **Modify cluster**. 3. Under **Database configurations** (or **Cluster properties**): * Set **Parameter group** to the new parameter group where `require_ssl = true`. 4. Save/Apply changes and **reboot** the cluster if prompted: * Cluster → **Actions** → **Reboot** (if not done automatically). Once applied, the cluster will **reject non‑SSL connections**. *** ### C. Ensure clients connect using SSL For each application / client: 1. Get cluster endpoint: * Redshift console → **Clusters** → select cluster → **General information** → **Endpoint**. 2. Use **SSL-enabled connection strings**. Examples: * **psql**: ```bash theme={null} psql "host= port=5439 dbname= user= sslmode=require" ``` * **JDBC URL**: ```text theme={null} jdbc:redshift://:5439/?ssl=true&sslfactory=com.amazon.redshift.ssl.NonValidatingFactory ``` * **ODBC**: * In DSN configuration, enable **SSL** or **Require SSL**. 3. If you need certificate validation, download Redshift’s public certificate from AWS docs and configure client trust stores accordingly (optional but recommended). *** ## 3. Validate the Remediation * **At rest**: * Redshift console → **Clusters** → select cluster → **Configuration**: * Confirm **Encryption: Enabled** and correct **KMS key**. * **In transit**: * Confirm applications can only connect when SSL is enabled. * Try a test connection without SSL and verify it fails. This completes in‑transit and at‑rest encryption remediation for Amazon Redshift via the AWS console. For Amazon Redshift, “in‑transit” = SSL, and “at‑rest” = KMS encryption on the cluster.\ At‑rest encryption cannot be turned on for an existing unencrypted cluster; you must migrate to a new encrypted cluster. Below are step‑by‑step AWS CLI instructions. *** ## 1. At‑Rest Encryption (Redshift) ### 1.1. Create (or identify) a KMS key ```bash theme={null} aws kms create-key --description "Redshift encryption key" ``` Output will include `"KeyId": "arn:aws:kms:region:account-id:key/xxxxxxxx"` – save this ARN. Optionally add an alias: ```bash theme={null} aws kms create-alias \ --alias-name alias/redshift-kms \ --target-key-id ``` You can then use `alias/redshift-kms` instead of the full ARN. *** ### 1.2. Snapshot your existing unencrypted cluster Replace `my-redshift-cluster` and `my-redshift-snapshot`: ```bash theme={null} aws redshift create-cluster-snapshot \ --cluster-identifier my-redshift-cluster \ --snapshot-identifier my-redshift-snapshot ``` Wait until the snapshot is `available`: ```bash theme={null} aws redshift describe-cluster-snapshots \ --snapshot-identifier my-redshift-snapshot \ --query "Snapshots[0].Status" ``` *** ### 1.3. Restore a new encrypted cluster from the snapshot ```bash theme={null} aws redshift restore-from-cluster-snapshot \ --cluster-identifier my-redshift-cluster-encrypted \ --snapshot-identifier my-redshift-snapshot \ --encrypted \ --kms-key-id alias/redshift-kms ``` You can also specify node type, security groups, etc., if you want to override snapshot defaults (optional): ```bash theme={null} aws redshift restore-from-cluster-snapshot \ --cluster-identifier my-redshift-cluster-encrypted \ --snapshot-identifier my-redshift-snapshot \ --encrypted \ --kms-key-id alias/redshift-kms \ --node-type ra3.xlplus \ --cluster-subnet-group-name my-subnet-group \ --vpc-security-group-ids sg-xxxxxxxx \ --publicly-accessible false ``` Wait until the new cluster is `available`: ```bash theme={null} aws redshift describe-clusters \ --cluster-identifier my-redshift-cluster-encrypted \ --query "Clusters[0].ClusterStatus" ``` *** ### 1.4. Cut over and clean up 1. Update applications to point to the new cluster’s endpoint (from `describe-clusters`). 2. When fully migrated, delete the old unencrypted cluster: ```bash theme={null} aws redshift delete-cluster \ --cluster-identifier my-redshift-cluster \ --skip-final-cluster-snapshot ``` (Or omit `--skip-final-cluster-snapshot` and specify `--final-cluster-snapshot-identifier` if you want a final backup.) *** ## 2. In‑Transit Encryption (SSL) for Redshift To enforce SSL, configure a parameter group and apply it to the cluster. ### 2.1. Create a parameter group (if you don’t already have a custom one) ```bash theme={null} aws redshift create-cluster-parameter-group \ --parameter-group-name my-redshift-ssl-pg \ --parameter-group-family redshift-1.0 \ --description "Require SSL for Redshift" ``` (Use `redshift-1.0` or the family appropriate to your engine version.) *** ### 2.2. Set `require_ssl` to `true` ```bash theme={null} aws redshift modify-cluster-parameter-group \ --parameter-group-name my-redshift-ssl-pg \ --parameters "ParameterName=require_ssl,ParameterValue=true,Description=RequireSSL,ApplyType=static" ``` You can verify: ```bash theme={null} aws redshift describe-cluster-parameters \ --parameter-group-name my-redshift-ssl-pg \ --query "Parameters[?ParameterName=='require_ssl']" ``` *** ### 2.3. Attach the parameter group to your (encrypted) cluster ```bash theme={null} aws redshift modify-cluster \ --cluster-identifier my-redshift-cluster-encrypted \ --cluster-parameter-group-name my-redshift-ssl-pg ``` You must reboot for static parameters to take effect: ```bash theme={null} aws redshift reboot-cluster \ --cluster-identifier my-redshift-cluster-encrypted ``` *** ### 2.4. Ensure clients actually use SSL Redshift will now require SSL, but your clients must be configured correctly: * JDBC: `ssl=true` or `sslmode=require` in the connection string. * ODBC: enable SSL / require SSL in DSN settings. * psql: `sslmode=require` (or `verify-ca`/`verify-full` with appropriate CA). No extra CLI steps are needed here; this is client configuration. *** ## 3. Quick Verification **At‑rest encryption:** ```bash theme={null} aws redshift describe-clusters \ --cluster-identifier my-redshift-cluster-encrypted \ --query "Clusters[0].[Encrypted,KmsKeyId]" ``` Should return `true` and the KMS key. **In‑transit (SSL) enforcement:** ```bash theme={null} aws redshift describe-clusters \ --cluster-identifier my-redshift-cluster-encrypted \ --query "Clusters[0].ClusterParameterGroups" ``` Make sure `my-redshift-ssl-pg` is attached and `ParameterApplyStatus` is `in-sync`. Then test connections; non‑SSL connections should fail. If you share your current cluster identifier and whether it’s already encrypted, I can tailor exact CLI commands for your environment. For Amazon Redshift, “in-transit” = SSL/TLS for client connections; “at-rest” = KMS-encrypted data on disk (cluster + snapshots).\ Below is how to remediate both using Python (boto3). *** ## Prereqs ```bash theme={null} pip install boto3 ``` ```python theme={null} import boto3 redshift = boto3.client("redshift", region_name="us-east-1") # adjust region ``` Replace placeholders like ``, `` as needed. *** ## 1. At-Rest Encryption ### A. New cluster (easiest) You **cannot enable encryption on an existing unencrypted cluster**; you must create a new encrypted one and migrate. 1. Create an encrypted snapshot of the old cluster (unencrypted snapshot → restore as encrypted). ```python theme={null} # 1. Take snapshot of existing (unencrypted) cluster snapshot_id = "old-cluster-snapshot" redshift.create_cluster_snapshot( SnapshotIdentifier=snapshot_id, ClusterIdentifier="" ) ``` 2. Wait for snapshot to be available (poll or use waiter): ```python theme={null} waiter = redshift.get_waiter('snapshot_available') waiter.wait(SnapshotIdentifier=snapshot_id) ``` 3. Restore snapshot to a **new encrypted cluster**: ```python theme={null} redshift.restore_from_cluster_snapshot( ClusterIdentifier="", SnapshotIdentifier=snapshot_id, Encrypted=True, KmsKeyId="" # or omit to use AWS managed key ) ``` 4. Wait for new cluster to be available: ```python theme={null} waiter = redshift.get_waiter('cluster_available') waiter.wait(ClusterIdentifier="") ``` 5. Update clients / apps / connection strings to point to the new cluster’s endpoint, test, then delete the old cluster: ```python theme={null} redshift.delete_cluster( ClusterIdentifier="", SkipFinalClusterSnapshot=True # or create final snapshot if needed ) ``` Now your data at rest (cluster & snapshots) is encrypted. *** ### B. Ensuring snapshots are encrypted For an **already encrypted** cluster, snapshots are encrypted automatically with the same KMS key.\ To ensure cross-region snapshot copy is encrypted: ```python theme={null} redshift.enable_snapshot_copy( ClusterIdentifier="", DestinationRegion="us-west-2", RetentionPeriod=7, SnapshotCopyGrantName="", KmsKeyId="" ) ``` *** ## 2. In-Transit Encryption (SSL/TLS) You enable and enforce SSL via a parameter group (`require_ssl`). ### A. Create / update a parameter group with `require_ssl=1` 1. Create a new parameter group (if you don’t want to modify an existing one): ```python theme={null} param_group_name = "redshift-ssl-required" redshift.create_cluster_parameter_group( ParameterGroupName=param_group_name, ParameterGroupFamily="redshift-1.0", # check your engine family Description="Require SSL for all connections" ) ``` 2. Set `require_ssl = true` (1): ```python theme={null} redshift.modify_cluster_parameter_group( ParameterGroupName=param_group_name, Parameters=[ { "ParameterName": "require_ssl", "ParameterValue": "true", # or "1" "Description": "Force SSL for all DB connections", "Source": "user", "DataType": "boolean", "AllowedValues": "true,false", "IsModifiable": True, "ApplyType": "static" } ] ) ``` > `require_ssl` is a static parameter → cluster must be rebooted after the parameter group is associated. ### B. Attach the parameter group to your cluster ```python theme={null} redshift.modify_cluster( ClusterIdentifier="", ClusterParameterGroupName=param_group_name ) ``` Then reboot (or wait for Redshift to apply & manually reboot): ```python theme={null} redshift.reboot_cluster(ClusterIdentifier="") ``` *** ## 3. Enforce SSL From Clients After `require_ssl` is on, non-SSL connections will fail.\ Make sure your Python clients use SSL, for example with `psycopg2`: ```python theme={null} import psycopg2 conn = psycopg2.connect( dbname="", user="", password="", host="", port=5439, sslmode="require" # or "verify-full" with CA bundle ) ``` *** ## Summary 1. **At rest** (Redshift): * Existing unencrypted: snapshot → restore encrypted cluster (with `Encrypted=True`, `KmsKeyId=...`) → switch traffic → delete old cluster. * New clusters: always create with `Encrypted=True`. 2. **In transit**: * Create/modify parameter group: set `require_ssl=true`. * Attach parameter group, reboot cluster. * Ensure all clients connect with `sslmode=require` (or stronger). ```hcl theme={null} # At-rest encryption for an existing Redshift cluster # NOTE: Changing `encrypted` from false→true forces replacement of the cluster # (full recreate, data loss unless you restore from snapshot). resource "aws_redshift_cluster" "this" { cluster_identifier = "REDSHIFT_CLUSTER_IDENTIFIER" # e.g., "analytics-prod" node_type = "dc2.large" master_username = "MASTER_USERNAME" master_password = "MASTER_PASSWORD" # Enable encryption at rest encrypted = true # Optionally use a customer-managed KMS key (recommended) # Omit this line to use the default Redshift KMS key kms_key_id = "KMS_KEY_ARN_OR_ID" # Attach the parameter group that enforces SSL (see below) cluster_parameter_group_name = aws_redshift_parameter_group.ssl_enforced.name # ...other required arguments (subnets, security groups, etc) ... } # In-transit encryption: enforce SSL for all connections resource "aws_redshift_parameter_group" "ssl_enforced" { name = "REDSHIFT_SSL_REQUIRED_PARAMETER_GROUP_NAME" family = "redshift-1.0" description = "Require SSL for all client connections" parameter { name = "require_ssl" value = "true" } } ``` To verify, `terraform plan` should show `encrypted = true`, the desired `kms_key_id` on `aws_redshift_cluster.this`, and `parameter.require_ssl` changing/created as `value = "true"` on `aws_redshift_parameter_group.ssl_enforced`, with a note that the cluster will be replaced if it was previously unencrypted.