github-actions[bot]
171 lines
6.3 KiB
Markdown
171 lines
6.3 KiB
Markdown
# PRD 0056: Extended outbound DLP scan surfaces
|
|
|
|
- **Status:** Active
|
|
- **Author:** claude
|
|
- **Created:** 2026-06-06
|
|
- **Issue:** #204
|
|
|
|
## Summary
|
|
|
|
The outbound DLP scan introduced in PRD 0052 covers only the request body and
|
|
the `Authorization` header. An agent can exfiltrate a provisioned secret or
|
|
credential token via several other request surfaces that are currently
|
|
unscanned: arbitrary HTTP headers, URL query parameters, URL path segments,
|
|
and DNS-level hostnames (DNS tunnelling). This PRD closes those gaps by
|
|
extending the scan corpus to cover all four surfaces.
|
|
|
|
## Problem
|
|
|
|
After PRD 0052, `egress_addon.py` builds the DLP scan text as:
|
|
|
|
```python
|
|
auth_header = flow.request.headers.get("authorization", "")
|
|
scan_text = body
|
|
if auth_header:
|
|
scan_text = auth_header + "\n" + body
|
|
```
|
|
|
|
This leaves the following attack surfaces unscanned:
|
|
|
|
| Surface | Example exfiltration vector |
|
|
|---|---|
|
|
| Other request headers | `X-Api-Key: <secret>`, `Cookie: token=<secret>` |
|
|
| Query parameters | `GET /search?api_key=<secret>` |
|
|
| URL path segments | `GET /proxy/<base64-secret>/endpoint` |
|
|
| DNS hostname | `<base64-secret>.attacker.com` (DNS tunnelling) |
|
|
|
|
Each surface allows a determined agent to bypass the current scan and leak
|
|
a provisioned `EGRESS_TOKEN_*` value or a known token pattern to an
|
|
upstream attacker.
|
|
|
|
## Goals / Success Criteria
|
|
|
|
1. All four surfaces (headers, query params, path, hostname) are included in
|
|
the outbound DLP scan text for every route that has outbound scanning
|
|
enabled.
|
|
2. A pure helper `build_outbound_scan_text(host, path, query, headers, body)`
|
|
in `egress_addon_core.py` assembles the scan corpus so the logic is fully
|
|
unit-testable without a mitmproxy dependency.
|
|
3. Unit tests demonstrate that `scan_outbound` blocks a request when a known
|
|
token pattern or provisioned secret appears in each surface independently.
|
|
4. No manifest schema changes — the `dlp` block's `outbound_detectors`
|
|
field continues to control which detectors run; all surfaces are scanned
|
|
by whichever detectors are active.
|
|
5. The auth-strip ordering invariant from PRD 0052 is preserved: the
|
|
outbound scan sees the original `Authorization` header before the addon
|
|
strips it.
|
|
|
|
## Non-goals
|
|
|
|
- Raw UDP/DNS queries — these bypass the HTTP proxy entirely and require a
|
|
network-level DNS sinkhole (tracked separately in issue #205).
|
|
- Structured query-param parsing — scanning the raw query string is
|
|
sufficient.
|
|
- Changes to the `dlp` block schema or detector names.
|
|
- Scanning outbound request bodies for prompt injection (inbound only,
|
|
per PRD 0052 design).
|
|
- LLM-based semantic detection or entropy-based secret scanning (deferred,
|
|
per PRD 0052 non-goals).
|
|
|
|
## Design
|
|
|
|
### `build_outbound_scan_text` in `egress_addon_core.py`
|
|
|
|
A new pure function assembles all request surfaces into a single newline-
|
|
delimited string suitable for passing to `scan_outbound`:
|
|
|
|
```python
|
|
def build_outbound_scan_text(
|
|
host: str,
|
|
path: str,
|
|
query: str,
|
|
headers: typing.Mapping[str, str],
|
|
body: str,
|
|
) -> str:
|
|
parts: list[str] = [host, path]
|
|
if query:
|
|
parts.append(query)
|
|
for name, value in headers.items():
|
|
parts.append(f"{name}: {value}")
|
|
if body:
|
|
parts.append(body)
|
|
return "\n".join(parts)
|
|
```
|
|
|
|
**Why hostname in the scan corpus?**
|
|
DNS tunnelling encodes data into subdomain labels
|
|
(`<base64-secret>.attacker.com`). The mitmproxy `request` hook sees the
|
|
`pretty_host` field before the TCP connection is fully established, so
|
|
scanning it catches this vector. Both the `token_patterns` and
|
|
`known_secrets` detectors handle encoded variants (raw, base64, URL-encoded,
|
|
hex), so the existing encoding-variant logic in `_encoded_variants` already
|
|
covers common DNS-tunnelling encodings.
|
|
|
|
### `egress_addon.py` update
|
|
|
|
The narrow scan-text construction is replaced with a call to
|
|
`build_outbound_scan_text`, which the addon has already split `path` and
|
|
`query` from `flow.request.path` at the top of `request()`:
|
|
|
|
```python
|
|
# Build full scan corpus: hostname + path + query + all headers + body
|
|
body = flow.request.get_text(strict=False) or ""
|
|
scan_text = build_outbound_scan_text(
|
|
flow.request.pretty_host,
|
|
request_path,
|
|
query,
|
|
dict(flow.request.headers),
|
|
body,
|
|
)
|
|
dlp_result = scan_outbound(route, scan_text, os.environ)
|
|
```
|
|
|
|
The `Authorization` header is present in `flow.request.headers` at this
|
|
point (the strip happens below on line 115), so the auth-strip ordering
|
|
invariant is automatically preserved.
|
|
|
|
### `build_inbound_scan_text` in `egress_addon_core.py`
|
|
|
|
An analogous helper assembles the inbound response corpus (all response
|
|
headers + body) for `scan_inbound`. The `response()` hook now passes this
|
|
combined text instead of the body alone, closing the response-header
|
|
injection vector.
|
|
|
|
### WebSocket frame scanning
|
|
|
|
A new `websocket_message` hook in `EgressAddon` scans every frame after the
|
|
HTTP 101 upgrade. Outbound frames (`from_client=True`) are scanned for
|
|
credential patterns and known secrets; inbound frames are scanned for prompt
|
|
injection. On a block the entire WebSocket connection is killed via
|
|
`flow.kill()` (there is no HTTP response surface to write to after upgrade).
|
|
|
|
### Extended encoding variants in `_encoded_variants`
|
|
|
|
`_encoded_variants` is extended from 4 to 9 encoding forms:
|
|
|
|
| Added encoding | Rationale |
|
|
|---|---|
|
|
| Standard base64 without padding | Common in log lines where `=` is stripped |
|
|
| URL-safe base64 with padding | JWT / OAuth standard alphabet |
|
|
| URL-safe base64 without padding | Same, padding stripped |
|
|
| Hex uppercase | Complements existing hex-lowercase variant |
|
|
| Base32 | TOTP seeds; some DNS-exfil channels use base32 subdomains |
|
|
| gzip + base64 | Recognisable by `H4sI` prefix; naive compression before encode |
|
|
|
|
### OpenAI project key pattern
|
|
|
|
`TOKEN_PATTERNS` gains `sk-proj-[A-Za-z0-9_\-]{48,}` covering OpenAI's
|
|
newer project-scoped API key format.
|
|
|
|
## Implementation
|
|
|
|
Delivered across three commits on the same branch:
|
|
|
|
1. **Outbound scan surfaces** — `build_outbound_scan_text`, `egress_addon.py`
|
|
`request()` rewrite, `TestBuildOutboundScanText`, `TestScanOutbound`.
|
|
2. **Remaining gaps** — extended `_encoded_variants`, `sk-proj-` pattern,
|
|
`build_inbound_scan_text`, response-header scanning, `websocket_message`
|
|
hook, and matching unit tests.
|
|
3. **PRD flip** — `Status: Draft → Active` (committed with the first
|
|
implementation commit; updated here to reflect final scope).
|