bot-bottle/docs/prds/0062-egress-supervisor-token-override.md

# PRD 0062: Supervisor override for egress token blocks

- **Status:** Active
- **Author:** claude
- **Created:** 2026-06-24
- **Issue:** #261

## Summary

Give each egress route a policy for what happens when an outbound DLP detector
matches a token, via `dlp.outbound_on_match: block | redact | supervise`
(default `supervise`):

- **`supervise`** (default) — route the block through the existing supervisor
  approval queue instead of returning `403` immediately. The proxy holds the
  request open until the operator approves or rejects it. On approval the
  matched token is added to an in-memory "safe tokens" set so the request — and
  any later request carrying the same token — flows through without
  re-prompting.
- **`redact`** — scrub the matched value(s) from the request and forward it,
  no operator in the loop. For routes where a token-shaped value is noise the
  upstream doesn't need (telemetry/log sinks). Fails closed if a match lands on
  a surface redaction can't rewrite (the hostname).
- **`block`** — the original hard `403`; never overridable. For routes where a
  detected token must always stop.

The motivating goal is reducing friction from false positives without weakening
the default-deny posture: supervise keeps a human in the loop, redact is an
explicit per-route opt-in, and block stays available for sensitive routes.

## Problem

The outbound DLP detectors (`token_patterns`, `known_secrets`) are
deliberately aggressive: any string that looks like a credential is blocked
before it leaves the bottle. That is the right default, but it produces false
positives — a token-shaped value that is not actually a secret, or a credential
the agent legitimately needs to send to a declared host. Today the only
recovery is for the operator to notice the `egress DLP` 403 in the logs and
hand-edit the route's `dlp.outbound_detectors`, which disables the detector for
the whole route rather than allowing the one value.

The operator has no in-the-loop signal that a token block happened and no
fine-grained way to say "this specific value is fine."

## Goals / Success Criteria

1. An outbound DLP **token** block (a `ScanResult` carrying a matched secret
   value) creates a supervisor proposal instead of an immediate `403`.
2. The egress proxy holds the blocked request open, polling for the operator's
   response up to a bounded timeout.
3. The proposal shows the operator the host, method, path, the detector reason,
   and a **redacted** context snippet — never the raw token value.
4. On `approved`/`modified`, the matched token value is added to an in-memory
   safe-tokens set and the request proceeds normally; later requests carrying
   the same value skip the block.
5. On `rejected`, timeout, malformed response, or missing supervisor wiring,
   the request fails closed with the same `403` as today.
6. Structural blocks that carry no token value (CRLF injection) and the
   route-not-allowlisted / git blocks are unchanged — they stay hard `403`s and
   keep their existing agent-driven `allow` / `egress-block` MCP path.
7. The proxy event loop is not stalled while waiting: the wait is asynchronous,
   so other flows keep being served.

## Non-goals

- Persisting the safe-tokens set across egress restarts. It lives in process
  memory only; a restart re-prompts. (The issue explicitly defers persistence.)
- Supervising inbound (prompt-injection) blocks or WebSocket frame blocks.
  WebSocket frames still honour the safe-tokens set for already-approved values
  but cannot wait for approval (there is no response surface after upgrade).
- Generalising an approved secret across encodings. The safe-tokens set matches
  the exact value the detector found.
- Replacing the per-route `dlp.outbound_detectors` override. That remains the
  way to turn a detector off wholesale.
- Making `redact` the default. Silent redaction of a true false positive
  corrupts legitimate data, so it is opt-in per route; `supervise` (human in
  the loop) stays the default.

## Scope

### In scope

The minimum cut that ships, in build order:

1. **Core** — `ScanResult.matched`; thread `safe_tokens` through
   `scan_outbound` / the token detectors; `build_token_allow_payload`.
2. **Supervise + TUI** — `TOOL_EGRESS_TOKEN_ALLOW`; TUI suffix, modify guard,
   required approval reason.
3. **Addon glue** — async `request`, safe-tokens set, proposal write + async
   poll, allow/block decision; pass `safe_tokens` into the WebSocket path.
4. **On-match policy** — `dlp.outbound_on_match` through manifest → render →
   addon; `redact` surface scrub with fail-closed re-scan; policy dispatch in
   the addon's outbound handler.
5. **Tests + docs** — core/supervise/TUI/manifest/render unit tests; README
   egress + supervisor notes.

### Out of scope

The deferrals enumerated under **Non-goals** — restart persistence, inbound /
WebSocket-frame supervision, cross-encoding generalisation, replacing
`dlp.outbound_detectors`, and making `redact` the default.

## Proposed Design

### New services / components

A new proposal tool constant `egress-token-allow` (`TOOL_EGRESS_TOKEN_ALLOW`)
is added to `supervise.TOOLS`, and the egress addon gains an in-memory
safe-tokens set plus the policy-dispatch path that drives it.

On an outbound block the addon dispatches on the resolved policy:

- **Structural blocks always 403.** A `ScanResult` with no `matched` value
  (CRLF injection) is a hard `403` regardless of policy — there is nothing to
  redact or safelist.
- **`redact`** runs `redact_tokens` over the body, non-`host` header values,
  and path/query, then re-scans. If the re-scan is clean the (rewritten)
  request is forwarded; if a block-severity match remains (e.g. in the
  hostname, or a unicode-evasion token redaction can't reach) it fails closed
  with a `403`.
- **`block`** writes the `403` immediately.
- **`supervise`** runs the queue-and-wait loop, falling back to `block` when
  supervise isn't wired for the bottle.

For `supervise`, the addon writes the proposal directly to
`SUPERVISE_QUEUE_DIR` (the queue is bind-mounted into the sidecar bundle and
shared by every daemon, exactly as git-gate's `gitleaks-allow` proposal in PRD
0061 does). The proposal's `proposed_file` is a human-readable text payload
built by `build_token_allow_payload`:

```
egress blocked an outbound request carrying a detected token
host: api.example.com
method: POST
path: /v1/ingest
detector: OpenAI API key found in body
context: ...before ******** after...
```

The justification tells the operator to approve only if the value is a false
positive or a credential the request legitimately needs. The addon then polls
`<proposal-id>.response.json` for `EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS` (default
300). `approved`/`modified` allow the request and add the value to the
safe-tokens set; `rejected`, malformed responses, and timeout fail the request
closed. The proposal + response are archived to `processed/` after a decision.
Because the wait happens inside mitmproxy's asyncio loop, the addon's `request`
hook is async and polls with `asyncio.sleep`, so concurrent flows are
unaffected.

### Existing code touched

- **Policy threading.** `dlp.outbound_on_match` is a per-route enum threaded
  from the bottle manifest (`manifest_egress`) through the resolved route
  (`egress.EgressRoute`), the rendered `routes.yaml` (`egress_render_routes`),
  and the addon's `Route` (`egress_addon_core`). Unset renders nothing and
  resolves to `supervise` at request time. The `list-egress-routes`
  introspection endpoint round-trips it so the agent's proposals preserve it.
- **Provider-route default.** Agent-provider routes (the agent talking to its
  own LLM API — `api.anthropic.com`, the Codex backend, etc.) are the worst
  source of token-shaped false positives because the whole conversation payload
  flows through them. `egress_routes_for_bottle` fills `outbound_on_match=redact`
  on any provider route that doesn't set it explicitly; a provider that sets the
  policy keeps its choice, and manifest routes are unaffected (they default to
  `supervise`).
- **Scanners.** `scan_outbound` (and the token detectors `scan_token_patterns`
  / `scan_known_secrets` it calls) accept a `safe_tokens` set. A match whose
  value is in `safe_tokens` is skipped, so an approved token no longer blocks;
  the scanners keep searching past a safelisted match so a second, un-approved
  secret in the same request is still caught. The WebSocket path is passed the
  same `safe_tokens` set.
- **Supervisor UI.** `cli/supervise.py` renders `egress-token-allow` like
  `gitleaks-allow`: the text payload is shown, modify is unavailable (there is
  no file patch to edit), and approval prompts for a non-empty reason recorded
  in the response notes. There is no on-disk config diff, so — like
  `gitleaks-allow` and `capability-block` — it writes no egress audit-log entry.
- **Failure handling.** If `SUPERVISE_QUEUE_DIR` / `SUPERVISE_BOTTLE_SLUG` are
  unset (supervise disabled for the bottle), the addon skips the queue and
  returns the existing `403`. Any error writing the proposal or reading the
  response also fails closed.

### Data model changes

- New per-route manifest field `dlp.outbound_on_match: block | redact |
  supervise`, rendered into `routes.yaml` (omitted when unset).
- `ScanResult` gains a `matched: str = ""` field carrying the raw substring the
  detector matched. The token detectors populate it; the structural CRLF
  detector leaves it empty. The value stays inside the egress sidecar process —
  never written to a log line (logs use the redacted `context`) nor to the
  proposal file.
- Proposal text payload (above) plus `<proposal-id>.response.json` in
  `SUPERVISE_QUEUE_DIR`, archived to `processed/` after a decision.
- New env var `EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS` (default 300).

### External dependencies

None. Reuses the existing supervisor queue (`SUPERVISE_QUEUE_DIR`) and the
mitmproxy addon framework already in the egress sidecar.

## Open questions

- Should `known_secrets` (provisioned `EGRESS_TOKEN_*` exfiltration) be
  override-able at all, or only `token_patterns`? This PRD allows both —
  approval is an explicit operator decision and the safe-tokens set matches the
  exact found value — but a future revision could restrict `known_secrets` to
  reject-only.

## References

- Issue #261
- PRD 0061 — `gitleaks-allow` supervisor proposal pattern this reuses.