Earlier this month, Profero published a write-up about their findings on Grok Bot, a personal AI assistant which provides an agent with its own computer in the cloud (with shell access, browser, relevant logins, etc.). Profero found that one command from an agent results in account takeover. Specifically:
- One command from the agent results in root access to the VM Grok uses as its sandbox environment. From there, they were able to steal credentials stored on the VM, execute commands within the sandbox environment from anywhere on the internet, and access files, connected apps, and browser sessions.
- All agents within a Grok account share the same credentials, user, and file system, so compromising one agent compromises all agents.
Profero reported their findings on Aug. 24 and claims to have still not heard back at the time of their write-up (75+ days later).
Importantly, the malicious command can be inserted via prompt injection - sneaking a command/instruction to the AI agent within seemingly innocuous content (Profero used a webpage, but it could also be in a cloned repo, installed package, etc.). No login required!
Reproducing Profero’s attack chain on Runlayer
Given the severity of these findings, we wanted to see how Runlayer Runtime Security might be able to detect and prevent such an attack. Spoiler alert: Runlayer detected the prompt injection attempt as well as the hijacking of the agent.
Prompt injection detected
Three separate scanners independently flagged the hidden instruction planted in the web page as prompt injection.
Behavioral hijack detected
Even when the agent provides seemingly innocuous descriptions of what it’s doing, Runlayer was able to detect when the agent was no longer following the user’s instructions/goals and instead following the prompt injection. When the agent viewed the same page without following the prompt injection, Runlayer did not flag any suspicious activity.
Runlayer also detected the malicious command the agent generated to exfiltrate credentials/secrets before it was executed.
No false positives
Throughout the entire process, Runlayer did not flag any of the agent’s seemingly innocuous actions (docker commands, package installations, API calls, etc.) that were similar to commands/actions performed in Profero’s attack chain.
Lastly, we verified that the sandbox escape technique detailed by Profero is not possible on the agent sandbox/runtime that powers Runlayer Agents.
Note that we cannot secure Grok’s (or any other agent product’s) sandbox, and once an attacker has root access to Grok’s VM, there’s nothing we (or anyone else!) can do about it. This is why detection & prevention of the initial prompt injection attack is so important!
Lessons learned (aka how to keep your AI agents safe)
Profero’s attack chain was possible because Grok’s architecture assumes that commands generated by its AI agents can be trusted, which is a dangerous assumption considering that agents frequently perform actions/instructions specified by external content (websites, documents, other tools’ responses, skills, etc.). Here are a couple of ways Runlayer can help keep your agents (and your data) safe:
1. Inspect all inputs to your AI agents at runtime for prompt injections, malicious instructions, etc. Runlayer’s proprietary models & scanners detect prompt injections, data exfiltration, agent manipulation, intent drift, and more during agent execution, regardless of model provider, client, agent harness, etc. used. Start in alert-only mode to understand what’s normal for your use case/environment and turn on block mode to secure your agents.
2. Use unique identities/logins/permissions for each agent. As mentioned by Profero, this limits the blast radius of a compromised agent to that specific account instead of taking down everything accessible via SSO.
Profero’s blog post on this attack chain explicitly recommends using unique logins/credentials for agents:
Give the agent its own identity. Dedicated accounts, narrow connectors, nothing it does not need. A compromised agent should cost you one account, not your single sign-on.
At Runlayer, we create a unique identity for each agent and allow admins to specify if the agent should perform actions using its own login/credentials or on behalf of a user. Admins can also apply policies to restrict tool/connector usage by agents.
3. Log everything (outside of the agent’s execution environment). Profero recommends log & activity monitoring outside of the agent’s VM/sandbox since, as they discovered, there were no logs of their attack once inside Grok’s sandbox environment & anyone with root access to a VM has access to logs generated/stored on that VM.
Runlayer logs everything (outside of the agent’s execution environment) including traces of each prompt, thought/reasoning step, tool call, & response made by an agent. These logs can be sent to your SIEM or accessed via API for easy threat detection & incident response/investigation (especially helpful when an agent claims everything is ok).
Don’t leave AI agent security up to chance; make the safe path the golden path.
These best practices aren’t specific to Grok. AI agents will always interact with & take actions specified by untrusted content, and enterprises need AI agent security solutions that are effective across all agents used by employees. That’s exactly what Runlayer’s agent security, AI governance & management features were designed for.
Runlayer’s AI control plane for agents unifies AI enablement, management & security so that all employees have a golden path to delegate meaningful work to AI agents securely. See how enterprises like AngelList, Gusto, Lemonade, and more use Runlayer to confidently scale AI → book a demo.
