CVE-2023-6019

What it actually does

CVE-2023-6019 is an unauthenticated remote command injection vulnerability in the Ray dashboard. The vulnerable dashboard endpoint accepts a cpu_profile URL parameter and passes attacker-controlled input into operating-system command execution context without sufficient sanitization. Because the dashboard is commonly reachable over HTTP and the flaw requires no authentication, network access to the Ray dashboard can allow arbitrary shell command execution with the privileges of the Ray dashboard process. Successful exploitation can result in full compromise of the Ray head node or dashboard host, including data theft, modification of workloads, installation of malware, credential access, and disruption of Ray clusters.

Why it matters to the business

An exposed vulnerable Ray dashboard can let an external attacker take control of systems running distributed Python, AI, machine-learning, or data-processing workloads. This can lead to theft of models, datasets, credentials, and application secrets; tampering with compute jobs; cryptomining or malware deployment; service outages; and lateral movement into connected infrastructure. The risk is high because the CVSS score is 9.8, exploitation is weaponized, the EPSS score is high at 0.7463, and the issue has been public for a long time, increasing the likelihood that internet-exposed instances have already been scanned or attacked.

How an attack unfolds

An attacker first identifies an exposed Ray dashboard service, commonly by scanning for Ray dashboard HTTP ports or Ray-specific web responses. If the instance is running a vulnerable Ray version before 2.8.1 and the dashboard endpoint is reachable, the attacker sends a crafted HTTP request to the cpu_profile functionality with shell metacharacters or command substitutions embedded in the cpu_profile URL parameter. The vulnerable code incorporates that parameter into an operating-system command, causing the host to execute attacker-supplied commands without requiring login. Initial commands are often used to confirm execution, such as whoami, id, uname, curl, or outbound DNS/HTTP callbacks. After confirming code execution, the attacker may download payloads, add persistence, enumerate environment variables and cloud credentials, access Ray jobs and cluster resources, pivot to worker nodes, steal data, or deploy cryptominers, backdoors, or ransomware tooling.

How to fix it

Upgrade Ray to version 2.8.1 or later. Ray maintainers fixed the vulnerable command-injection behavior in 2.8.1+, so all deployments running earlier affected versions should be treated as exposed if the dashboard is reachable. If immediate upgrade is not possible, restrict Ray dashboard access to trusted administrative networks only, place it behind strong authentication and authorization, block public internet access, and use host/network firewalls or security groups to limit inbound access to the Ray dashboard port. Do not rely on the dashboard being obscure or unauthenticated endpoints being unused. Rotate credentials, API keys, tokens, and cloud credentials that may have been accessible to the Ray process if exploitation is suspected. Rebuild compromised nodes from trusted images rather than only removing observed payloads.

How to confirm the fix worked

Confirm the installed Ray version is 2.8.1 or later on all head nodes, dashboard hosts, worker images, containers, notebooks, and automation templates. Validate that no public or broadly reachable network path exists to the Ray dashboard unless it is intentionally protected by compensating controls. Use package inspection commands such as `python -c "import ray; print(ray.__version__)"`, `pip show ray`, container image package inventories, and software composition analysis to identify vulnerable Ray installations. Confirm firewall, load balancer, reverse proxy, Kubernetes ingress, and cloud security group rules do not expose the dashboard unnecessarily. After patching, restart Ray services and verify that crafted requests containing shell metacharacters in the cpu_profile parameter no longer result in command execution. Review access logs and system artifacts for historical exploitation before declaring the issue remediated.

Typical attack path

1. Scan the internet or internal network for exposed Ray dashboard services.
2. Fingerprint the service and infer whether it is a vulnerable Ray version prior to 2.8.1.
3. Send a crafted HTTP request to the Ray dashboard cpu_profile endpoint with command-injection syntax in the cpu_profile URL parameter.
4. Execute a low-noise validation command such as id, whoami, uname, curl, wget, nslookup, or a DNS callback.
5. Use the command-execution primitive to download and run a second-stage payload or interactive shell.
6. Enumerate the host, Ray cluster, environment variables, service credentials, cloud metadata, mounted volumes, and running workloads.
7. Move laterally to Ray worker nodes or adjacent infrastructure using discovered credentials or cluster trust relationships.
8. Exfiltrate sensitive data, tamper with workloads, deploy cryptomining malware, establish persistence, or disrupt Ray services.

How to detect it

To determine whether exploitation already occurred, review Ray dashboard HTTP access logs, reverse proxy logs, load balancer logs, WAF logs, and packet captures for requests to cpu_profile-related endpoints containing shell metacharacters, URL-encoded command separators, or suspicious command strings. Search endpoint telemetry for Ray or Python processes spawning shells or system utilities such as sh, bash, curl, wget, python, perl, nc, ncat, socat, chmod, crontab, nohup, or crypto-mining binaries. Review system logs, auditd, Sysmon-for-Linux, EDR process trees, container runtime logs, Kubernetes audit logs, and cloud workload logs for unexpected commands executed by the Ray dashboard process user. Check for outbound HTTP, HTTPS, DNS, or TCP callbacks shortly after suspicious dashboard requests, especially to rare domains or IPs. Examine filesystem locations commonly used for dropped payloads and persistence, including /tmp, /var/tmp, /dev/shm, user home directories, cron directories, systemd unit paths, shell profile files, SSH authorized_keys, and container writable layers. Investigate Ray job history, runtime environment changes, worker node activity, and cluster logs for unexpected jobs or commands. If exploitation is suspected, preserve logs and disk artifacts before rebuilding nodes, and rotate secrets that may have been accessible to the Ray process.

• cpu_profile=
• %3B
• %7C
• %26
• %60
• %24%28
• ;id
• ;whoami
• ;uname
• ;curl
• ;wget
• ;bash
• ;sh
• |id
• |whoami
• |uname
• |curl
• |wget
• `id`
• `whoami`
• $(id)
• $(whoami)
• /bin/sh
• /bin/bash
• /usr/bin/curl
• /usr/bin/wget
• /tmp/
• /var/tmp/
• /dev/shm/
• chmod +x
• nohup
• nc -e
• ncat -e
• bash -i
• /etc/cron.d/
• /var/spool/cron/
• /etc/systemd/system/
• authorized_keys
• ray dashboard
• raylet
• dashboard.py

What your security team should do

• Immediately inventory Ray deployments and prioritize any Ray dashboard exposed to the internet or untrusted networks.
• Upgrade all Ray installations to 2.8.1 or later and rebuild affected container images, VM images, and cluster templates.
• Restrict dashboard access to trusted administrative networks and require authentication through a secure reverse proxy or private access mechanism where possible.
• Hunt for prior exploitation using dashboard access logs, reverse proxy logs, endpoint telemetry, process execution logs, shell history, cloud audit logs, and DNS/HTTP egress logs.
• Look for suspicious child processes spawned by Ray dashboard or Python processes, especially shells, download utilities, cryptominers, reverse shells, and credential-discovery commands.
• Rotate secrets available to Ray workloads, including environment variables, cloud instance credentials, service account tokens, package repository credentials, and data platform credentials if compromise cannot be ruled out.
• Reimage or redeploy nodes where command execution, unknown payloads, or persistence mechanisms are found.
• Add continuous exposure monitoring for Ray dashboard services and alert on unauthenticated access to sensitive dashboard endpoints.
• Create SIEM detections for requests containing `cpu_profile` combined with command-injection metacharacters, shell commands, or outbound callback utilities.
• Monitor outbound connections from Ray nodes to unusual IPs, newly registered domains, paste sites, file-sharing services, mining pools, and command-and-control infrastructure.