diff --git a/docs/research/hn-agent-safety-discourse-july-2026.md b/docs/research/hn-agent-safety-discourse-july-2026.md index 8c7d193..ff6fd5b 100644 --- a/docs/research/hn-agent-safety-discourse-july-2026.md +++ b/docs/research/hn-agent-safety-discourse-july-2026.md @@ -20,10 +20,15 @@ README files) rather than exploiting the isolation boundary directly. bot-bottle's architecture holds up well against the direct-escape class (Firecracker/Apple Container default backends, credentials never in the -agent's env, harness entirely on the host). It is less strong against -the trusted-channel injection class, where the only runtime defense is -the inbound DLP scanner, which is explicitly described as naive. That -gap is acknowledged but not yet closed. +agent's env, harness entirely on the host). The remaining gap is prompt +injection — attacker-controlled data interpreted as model instructions. +Egress controls and prompt injection defenses are orthogonal: egress +limits what the agent can *send out*; injection is about what it is +*told to do*. The two don't substitute for each other. Inside a tightly- +egressed sandbox a successful injection can't exfiltrate to unknown +hosts, but it can still corrupt the work product, push malicious commits +past a secret scanner, or use allowlisted channels for exfiltration. +Those residual risks are addressed below. ## The sandboxing boom sets the stage @@ -40,11 +45,13 @@ this and were shipping anyway. ## The June–July attack cascade -Six distinct attack classes broke in quick succession. Together they -form the argument that the community's framing was wrong: the threat -model for agents isn't just "code that escapes its container" — it's -also "code that doesn't need to escape because it arrived via a trusted -channel." +Six attack patterns broke in quick succession. Together they form the +argument that the community's framing was wrong: the threat model for +agents isn't just "code that escapes its container" — it's also prompt +injection, where attacker-controlled data is interpreted as model +instructions regardless of whether any isolation boundary was crossed. +Sections 2–4 below are all the same attack class; the "trusted channel" +label describes the delivery vector, not a different threat. ### 1. Sandbox escape CVEs (DuneSlide, CVE-2026-39861) @@ -66,7 +73,7 @@ All patched, but the pattern holds: any application-level sandbox that takes attacker-influenced values as path parameters is reachable from a prompt injection. -### 2. Agentjacking via trusted external data +### 2. Prompt injection via MCP data (Agentjacking) Tenet's "Agentjacking" technique planted a fake bug report in Sentry's MCP output. When an agent queries Sentry to fix open issues, the @@ -74,7 +81,9 @@ malicious event is rendered as structured content visually indistinguishable from a real Sentry event, and the agent executes the embedded instructions with the developer's full privileges. Hit rate across Claude Code and Cursor: **85%**. The route is entirely through a -legitimately-authorized MCP channel — no sandbox boundary is crossed. +legitimately-authorized MCP channel — no isolation boundary is crossed; +the injection arrives inbound through a channel the sandbox explicitly +trusts. The Cloud Security Alliance's summary: treat observability, bug-report, and integration data as **untrusted agent input**, not neutral @@ -89,12 +98,13 @@ Claude Sonnet 4.6 and GPT-5.5 said no. A payload written for Sonnet 4.6 transferred unchanged to Sonnet 5, Opus 4.8, and GPT-5.5. The attack surface is every repo an agent is asked to work in. -### 4. MCP tool description poisoning +### 4. Prompt injection via MCP tool descriptions Microsoft research (June 30) showed that attacker-controlled MCP tool -description fields can silently redirect agent behavior. The exfiltration -instruction is embedded in metadata the model reads during tool -selection, before any sandbox enforcement or egress check runs. +description fields can silently redirect agent behavior. The injection +is embedded in metadata the model reads during tool selection — before +any sandbox enforcement or egress check runs, and entirely on the +inbound path that egress controls cannot touch. ### 5. MCP STDIO command injection (10 CVEs) @@ -215,43 +225,71 @@ A compromised third-party proxy is not in the architecture. ### Where it is weaker -**Model-layer prompt injection** +**Prompt injection** + +Egress controls and prompt injection defenses are orthogonal. Egress +limits what the agent can *send out* (outbound leg); prompt injection +is about what attacker-controlled data *tells the agent to do* (inbound +leg). The two don't substitute for each other and must be treated +separately. -README injection, MCP description poisoning, and Agentjacking all -succeed by convincing the model, not by crossing a transport boundary. The inbound DLP scanner (`inbound_detectors: [naive_injection_detection]`) -scans MCP and HTTP responses for injection patterns, but it is described -as naive — pattern-matching is not a reliable defense against a -sufficiently crafted payload. There is no semantic / intent-level gate -between what the model decides and what the agent executes. +is the only runtime defense against injection arriving through allowlisted +channels — Sentry MCP responses, MCP tool descriptions, README content. +It is explicitly pattern-matching and will not catch a sufficiently +crafted payload. There is no semantic / intent-level gate between what +the model decides and what the agent executes. -**Trusted-channel data injection from allowlisted hosts** +**Blast radius within the permitted scope** -If Sentry, GitHub, or another allowlisted service returns a -malicious payload, the egress allowlist provides no protection (the host -is trusted) and the inbound DLP scanner is the only remaining runtime -defense. This is the gap the Agentjacking research targets. The -architecture doesn't have a mechanism for the operator to say "trust HTTP -responses from this host for data, but treat any instruction-shaped -content as untrusted." +Inside a tightly-egressed sandbox a successful injection can't +exfiltrate to unknown hosts, but it still has real options: -**Content-visible exfiltration through allowlisted channels** +- *Work product corruption.* The agent can modify, delete, or backdoor + files in the working directory. This is within its permitted scope; + egress controls have nothing to say about it. -An agent with legitimate access to a GitHub remote can open a PR to -an attacker-controlled fork if that remote is in the egress allowlist. -The outbound DLP scanner catches tokens and secrets, but structured -business data — source code, private issue content — is not in its -threat model. +- *Malicious commits past the git-gate.* The git-gate scans outbound + refs for secrets (gitleaks), not for semantic code intent. A prompt- + injected agent can commit subtly malicious code — logic bombs, + backdoored auth paths, code that exfiltrates data through the + application's own HTTP clients at runtime — that looks clean to a + secret scanner. + +- *Exfiltration through allowlisted channels.* If an attacker knows or + can predict what hosts are in the egress allowlist, those channels are + available for exfiltration. A GitHub remote being allowlisted means + "push to an attacker-controlled fork" is viable. A logging endpoint + being allowlisted means structured data can leave through it. The + outbound DLP scanner catches credential tokens and known secrets but + not arbitrary business data. + +- *Dependency installation within the sandbox.* An agent that runs + `npm install` or `pip install` on attacker-specified packages executes + code inside the sandbox with the same capabilities the agent has: + filesystem access, tool calls, calls to allowlisted hosts. Supply chain + injection via package names is in the same injection family, triggered + by the same prompt-injection path. ### What would close the remaining gaps -The "harness outside the sandbox" thread's hypervisor-layer idea maps -onto a capability the architecture could grow: a semantic policy layer -in the egress addon that classifies request intent (file write vs. data -exfiltration vs. external communication) rather than just scanning for -token patterns. The per-route `dlp.outbound_on_match: supervise` -mechanism is already the right shape for human-in-the-loop review; what -it lacks is awareness of context beyond the outbound request itself. +The blast-radius risks above point at two distinct mitigations that +don't yet exist in bot-bottle: + +- *Outbound intent classification.* The egress addon today scans + outbound request content for token patterns. What it lacks is + awareness of context — it can't distinguish "agent is pushing a + legitimate commit" from "agent was injected and is pushing a backdoor." + The `supervise` policy is already the right shape for human-in-the-loop + review on sensitive outbound actions; extending it with context from + the agent's recent tool calls (what files were touched, what was the + triggering task) would narrow the gap. + +- *Semantic code review on git push.* gitleaks is the wrong tool for + catching injected logic. A review step on outbound commits — even a + simple diff summary surfaced in `cli.py supervise` before the push is + forwarded — would close the malicious-commit path without requiring + the agent to be fully trusted. ## Sources