docs(research): clarify prompt injection framing and blast-radius risks
Collapse "trusted-channel data injection" into prompt injection throughout — the trusted channel is a delivery vector, not a distinct attack class. Add explicit inbound/outbound orthogonality framing. Replace the two redundant "weaker" bullets with a single prompt injection section and a new blast-radius breakdown covering work product corruption, malicious commits past gitleaks, exfiltration through allowlisted channels, and dependency-install injection.
This commit is contained in:
@@ -20,10 +20,15 @@ README files) rather than exploiting the isolation boundary directly.
|
||||
|
||||
bot-bottle's architecture holds up well against the direct-escape class
|
||||
(Firecracker/Apple Container default backends, credentials never in the
|
||||
agent's env, harness entirely on the host). It is less strong against
|
||||
the trusted-channel injection class, where the only runtime defense is
|
||||
the inbound DLP scanner, which is explicitly described as naive. That
|
||||
gap is acknowledged but not yet closed.
|
||||
agent's env, harness entirely on the host). The remaining gap is prompt
|
||||
injection — attacker-controlled data interpreted as model instructions.
|
||||
Egress controls and prompt injection defenses are orthogonal: egress
|
||||
limits what the agent can *send out*; injection is about what it is
|
||||
*told to do*. The two don't substitute for each other. Inside a tightly-
|
||||
egressed sandbox a successful injection can't exfiltrate to unknown
|
||||
hosts, but it can still corrupt the work product, push malicious commits
|
||||
past a secret scanner, or use allowlisted channels for exfiltration.
|
||||
Those residual risks are addressed below.
|
||||
|
||||
## The sandboxing boom sets the stage
|
||||
|
||||
@@ -40,11 +45,13 @@ this and were shipping anyway.
|
||||
|
||||
## The June–July attack cascade
|
||||
|
||||
Six distinct attack classes broke in quick succession. Together they
|
||||
form the argument that the community's framing was wrong: the threat
|
||||
model for agents isn't just "code that escapes its container" — it's
|
||||
also "code that doesn't need to escape because it arrived via a trusted
|
||||
channel."
|
||||
Six attack patterns broke in quick succession. Together they form the
|
||||
argument that the community's framing was wrong: the threat model for
|
||||
agents isn't just "code that escapes its container" — it's also prompt
|
||||
injection, where attacker-controlled data is interpreted as model
|
||||
instructions regardless of whether any isolation boundary was crossed.
|
||||
Sections 2–4 below are all the same attack class; the "trusted channel"
|
||||
label describes the delivery vector, not a different threat.
|
||||
|
||||
### 1. Sandbox escape CVEs (DuneSlide, CVE-2026-39861)
|
||||
|
||||
@@ -66,7 +73,7 @@ All patched, but the pattern holds: any application-level sandbox that
|
||||
takes attacker-influenced values as path parameters is reachable from a
|
||||
prompt injection.
|
||||
|
||||
### 2. Agentjacking via trusted external data
|
||||
### 2. Prompt injection via MCP data (Agentjacking)
|
||||
|
||||
Tenet's "Agentjacking" technique planted a fake bug report in Sentry's
|
||||
MCP output. When an agent queries Sentry to fix open issues, the
|
||||
@@ -74,7 +81,9 @@ malicious event is rendered as structured content visually
|
||||
indistinguishable from a real Sentry event, and the agent executes the
|
||||
embedded instructions with the developer's full privileges. Hit rate
|
||||
across Claude Code and Cursor: **85%**. The route is entirely through a
|
||||
legitimately-authorized MCP channel — no sandbox boundary is crossed.
|
||||
legitimately-authorized MCP channel — no isolation boundary is crossed;
|
||||
the injection arrives inbound through a channel the sandbox explicitly
|
||||
trusts.
|
||||
|
||||
The Cloud Security Alliance's summary: treat observability, bug-report,
|
||||
and integration data as **untrusted agent input**, not neutral
|
||||
@@ -89,12 +98,13 @@ Claude Sonnet 4.6 and GPT-5.5 said no. A payload written for Sonnet
|
||||
4.6 transferred unchanged to Sonnet 5, Opus 4.8, and GPT-5.5. The
|
||||
attack surface is every repo an agent is asked to work in.
|
||||
|
||||
### 4. MCP tool description poisoning
|
||||
### 4. Prompt injection via MCP tool descriptions
|
||||
|
||||
Microsoft research (June 30) showed that attacker-controlled MCP tool
|
||||
description fields can silently redirect agent behavior. The exfiltration
|
||||
instruction is embedded in metadata the model reads during tool
|
||||
selection, before any sandbox enforcement or egress check runs.
|
||||
description fields can silently redirect agent behavior. The injection
|
||||
is embedded in metadata the model reads during tool selection — before
|
||||
any sandbox enforcement or egress check runs, and entirely on the
|
||||
inbound path that egress controls cannot touch.
|
||||
|
||||
### 5. MCP STDIO command injection (10 CVEs)
|
||||
|
||||
@@ -215,43 +225,71 @@ A compromised third-party proxy is not in the architecture.
|
||||
|
||||
### Where it is weaker
|
||||
|
||||
**Model-layer prompt injection**
|
||||
**Prompt injection**
|
||||
|
||||
Egress controls and prompt injection defenses are orthogonal. Egress
|
||||
limits what the agent can *send out* (outbound leg); prompt injection
|
||||
is about what attacker-controlled data *tells the agent to do* (inbound
|
||||
leg). The two don't substitute for each other and must be treated
|
||||
separately.
|
||||
|
||||
README injection, MCP description poisoning, and Agentjacking all
|
||||
succeed by convincing the model, not by crossing a transport boundary.
|
||||
The inbound DLP scanner (`inbound_detectors: [naive_injection_detection]`)
|
||||
scans MCP and HTTP responses for injection patterns, but it is described
|
||||
as naive — pattern-matching is not a reliable defense against a
|
||||
sufficiently crafted payload. There is no semantic / intent-level gate
|
||||
between what the model decides and what the agent executes.
|
||||
is the only runtime defense against injection arriving through allowlisted
|
||||
channels — Sentry MCP responses, MCP tool descriptions, README content.
|
||||
It is explicitly pattern-matching and will not catch a sufficiently
|
||||
crafted payload. There is no semantic / intent-level gate between what
|
||||
the model decides and what the agent executes.
|
||||
|
||||
**Trusted-channel data injection from allowlisted hosts**
|
||||
**Blast radius within the permitted scope**
|
||||
|
||||
If Sentry, GitHub, or another allowlisted service returns a
|
||||
malicious payload, the egress allowlist provides no protection (the host
|
||||
is trusted) and the inbound DLP scanner is the only remaining runtime
|
||||
defense. This is the gap the Agentjacking research targets. The
|
||||
architecture doesn't have a mechanism for the operator to say "trust HTTP
|
||||
responses from this host for data, but treat any instruction-shaped
|
||||
content as untrusted."
|
||||
Inside a tightly-egressed sandbox a successful injection can't
|
||||
exfiltrate to unknown hosts, but it still has real options:
|
||||
|
||||
**Content-visible exfiltration through allowlisted channels**
|
||||
- *Work product corruption.* The agent can modify, delete, or backdoor
|
||||
files in the working directory. This is within its permitted scope;
|
||||
egress controls have nothing to say about it.
|
||||
|
||||
An agent with legitimate access to a GitHub remote can open a PR to
|
||||
an attacker-controlled fork if that remote is in the egress allowlist.
|
||||
The outbound DLP scanner catches tokens and secrets, but structured
|
||||
business data — source code, private issue content — is not in its
|
||||
threat model.
|
||||
- *Malicious commits past the git-gate.* The git-gate scans outbound
|
||||
refs for secrets (gitleaks), not for semantic code intent. A prompt-
|
||||
injected agent can commit subtly malicious code — logic bombs,
|
||||
backdoored auth paths, code that exfiltrates data through the
|
||||
application's own HTTP clients at runtime — that looks clean to a
|
||||
secret scanner.
|
||||
|
||||
- *Exfiltration through allowlisted channels.* If an attacker knows or
|
||||
can predict what hosts are in the egress allowlist, those channels are
|
||||
available for exfiltration. A GitHub remote being allowlisted means
|
||||
"push to an attacker-controlled fork" is viable. A logging endpoint
|
||||
being allowlisted means structured data can leave through it. The
|
||||
outbound DLP scanner catches credential tokens and known secrets but
|
||||
not arbitrary business data.
|
||||
|
||||
- *Dependency installation within the sandbox.* An agent that runs
|
||||
`npm install` or `pip install` on attacker-specified packages executes
|
||||
code inside the sandbox with the same capabilities the agent has:
|
||||
filesystem access, tool calls, calls to allowlisted hosts. Supply chain
|
||||
injection via package names is in the same injection family, triggered
|
||||
by the same prompt-injection path.
|
||||
|
||||
### What would close the remaining gaps
|
||||
|
||||
The "harness outside the sandbox" thread's hypervisor-layer idea maps
|
||||
onto a capability the architecture could grow: a semantic policy layer
|
||||
in the egress addon that classifies request intent (file write vs. data
|
||||
exfiltration vs. external communication) rather than just scanning for
|
||||
token patterns. The per-route `dlp.outbound_on_match: supervise`
|
||||
mechanism is already the right shape for human-in-the-loop review; what
|
||||
it lacks is awareness of context beyond the outbound request itself.
|
||||
The blast-radius risks above point at two distinct mitigations that
|
||||
don't yet exist in bot-bottle:
|
||||
|
||||
- *Outbound intent classification.* The egress addon today scans
|
||||
outbound request content for token patterns. What it lacks is
|
||||
awareness of context — it can't distinguish "agent is pushing a
|
||||
legitimate commit" from "agent was injected and is pushing a backdoor."
|
||||
The `supervise` policy is already the right shape for human-in-the-loop
|
||||
review on sensitive outbound actions; extending it with context from
|
||||
the agent's recent tool calls (what files were touched, what was the
|
||||
triggering task) would narrow the gap.
|
||||
|
||||
- *Semantic code review on git push.* gitleaks is the wrong tool for
|
||||
catching injected logic. A review step on outbound commits — even a
|
||||
simple diff summary surfaced in `cli.py supervise` before the push is
|
||||
forwarded — would close the malicious-commit path without requiring
|
||||
the agent to be fully trusted.
|
||||
|
||||
## Sources
|
||||
|
||||
|
||||
Reference in New Issue
Block a user