From d2d50be65acfb5faa6fa80b630dfea6a39e2c66b Mon Sep 17 00:00:00 2001 From: didericis Date: Wed, 24 Jun 2026 21:10:31 -0400 Subject: [PATCH] Restructure PRD 0062 to the init-prd template Conform the PRD to the standard PRD-new skeleton: add a Scope section (In scope / Out of scope), rename Design -> Proposed Design and split its prose into New services / Existing code touched / Data model changes / External dependencies, fold the old Implementation chunks into In scope, and add a References section. No change in substance. Co-Authored-By: Claude Opus 4.8 --- .../0062-egress-supervisor-token-override.md | 160 ++++++++++-------- 1 file changed, 89 insertions(+), 71 deletions(-) diff --git a/docs/prds/0062-egress-supervisor-token-override.md b/docs/prds/0062-egress-supervisor-token-override.md index 6844d47..741b127 100644 --- a/docs/prds/0062-egress-supervisor-token-override.md +++ b/docs/prds/0062-egress-supervisor-token-override.md @@ -76,25 +76,37 @@ fine-grained way to say "this specific value is fine." corrupts legitimate data, so it is opt-in per route; `supervise` (human in the loop) stays the default. -## Design +## Scope -### On-match policy +### In scope -`dlp.outbound_on_match` is a per-route enum threaded from the bottle manifest -(`manifest_egress`) through the resolved route (`egress.EgressRoute`), the -rendered `routes.yaml` (`egress_render_routes`), and the addon's `Route` -(`egress_addon_core`). Unset renders nothing and resolves to `supervise` at -request time. The `list-egress-routes` introspection endpoint round-trips it so -the agent's proposals preserve it. +The minimum cut that ships, in build order: -**Provider routes default to `redact`.** Agent-provider routes (the agent -talking to its own LLM API — `api.anthropic.com`, the Codex backend, etc.) are -the worst source of token-shaped false positives because the whole -conversation payload flows through them. `egress_routes_for_bottle` fills -`outbound_on_match=redact` on any provider route that doesn't set it -explicitly, so a match there is scrubbed and forwarded rather than blocked or -queued. A provider that sets the policy keeps its choice; manifest routes are -unaffected (they default to `supervise`). +1. **Core** — `ScanResult.matched`; thread `safe_tokens` through + `scan_outbound` / the token detectors; `build_token_allow_payload`. +2. **Supervise + TUI** — `TOOL_EGRESS_TOKEN_ALLOW`; TUI suffix, modify guard, + required approval reason. +3. **Addon glue** — async `request`, safe-tokens set, proposal write + async + poll, allow/block decision; pass `safe_tokens` into the WebSocket path. +4. **On-match policy** — `dlp.outbound_on_match` through manifest → render → + addon; `redact` surface scrub with fail-closed re-scan; policy dispatch in + the addon's outbound handler. +5. **Tests + docs** — core/supervise/TUI/manifest/render unit tests; README + egress + supervisor notes. + +### Out of scope + +The deferrals enumerated under **Non-goals** — restart persistence, inbound / +WebSocket-frame supervision, cross-encoding generalisation, replacing +`dlp.outbound_detectors`, and making `redact` the default. + +## Proposed Design + +### New services / components + +A new proposal tool constant `egress-token-allow` (`TOOL_EGRESS_TOKEN_ALLOW`) +is added to `supervise.TOOLS`, and the egress addon gains an in-memory +safe-tokens set plus the policy-dispatch path that drives it. On an outbound block the addon dispatches on the resolved policy: @@ -107,30 +119,14 @@ On an outbound block the addon dispatches on the resolved policy: hostname, or a unicode-evasion token redaction can't reach) it fails closed with a `403`. - **`block`** writes the `403` immediately. -- **`supervise`** runs the queue-and-wait loop below, falling back to `block` - when supervise isn't wired for the bottle. +- **`supervise`** runs the queue-and-wait loop, falling back to `block` when + supervise isn't wired for the bottle. -### Detected-value plumbing - -`ScanResult` gains a `matched: str = ""` field carrying the raw substring the -detector matched. The token detectors (`scan_token_patterns`, -`scan_known_secrets`) populate it; the structural CRLF detector leaves it -empty. The value stays inside the egress sidecar process — it is never written -to a log line (logs already use the redacted `context`) nor to the proposal -file. - -`scan_outbound` (and the token detectors it calls) accept a `safe_tokens` -set. A match whose value is in `safe_tokens` is skipped, so an approved token -no longer blocks. The scanners keep searching past a safelisted match so a -second, un-approved secret in the same request is still caught. - -### Supervisor proposal - -A new proposal tool constant `egress-token-allow` is added to -`supervise.TOOLS`. The egress addon writes the proposal directly to +For `supervise`, the addon writes the proposal directly to `SUPERVISE_QUEUE_DIR` (the queue is bind-mounted into the sidecar bundle and shared by every daemon, exactly as git-gate's `gitleaks-allow` proposal in PRD -0061 does). The proposal's `proposed_file` is a human-readable text payload: +0061 does). The proposal's `proposed_file` is a human-readable text payload +built by `build_token_allow_payload`: ``` egress blocked an outbound request carrying a detected token @@ -142,46 +138,63 @@ context: ...before ******** after... ``` The justification tells the operator to approve only if the value is a false -positive or a credential the request legitimately needs. +positive or a credential the request legitimately needs. The addon then polls +`.response.json` for `EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS` (default +300). `approved`/`modified` allow the request and add the value to the +safe-tokens set; `rejected`, malformed responses, and timeout fail the request +closed. The proposal + response are archived to `processed/` after a decision. +Because the wait happens inside mitmproxy's asyncio loop, the addon's `request` +hook is async and polls with `asyncio.sleep`, so concurrent flows are +unaffected. -The addon then polls `.response.json` for -`EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS` (default 300). `approved`/`modified` -allow the request and add the value to the safe-tokens set; `rejected`, -malformed responses, and timeout fail the request closed. The proposal + -response are archived to `processed/` after a decision. +### Existing code touched -Because the wait happens inside mitmproxy's asyncio loop, the addon's -`request` hook is async and polls with `asyncio.sleep`, so concurrent flows -are unaffected. +- **Policy threading.** `dlp.outbound_on_match` is a per-route enum threaded + from the bottle manifest (`manifest_egress`) through the resolved route + (`egress.EgressRoute`), the rendered `routes.yaml` (`egress_render_routes`), + and the addon's `Route` (`egress_addon_core`). Unset renders nothing and + resolves to `supervise` at request time. The `list-egress-routes` + introspection endpoint round-trips it so the agent's proposals preserve it. +- **Provider-route default.** Agent-provider routes (the agent talking to its + own LLM API — `api.anthropic.com`, the Codex backend, etc.) are the worst + source of token-shaped false positives because the whole conversation payload + flows through them. `egress_routes_for_bottle` fills `outbound_on_match=redact` + on any provider route that doesn't set it explicitly; a provider that sets the + policy keeps its choice, and manifest routes are unaffected (they default to + `supervise`). +- **Scanners.** `scan_outbound` (and the token detectors `scan_token_patterns` + / `scan_known_secrets` it calls) accept a `safe_tokens` set. A match whose + value is in `safe_tokens` is skipped, so an approved token no longer blocks; + the scanners keep searching past a safelisted match so a second, un-approved + secret in the same request is still caught. The WebSocket path is passed the + same `safe_tokens` set. +- **Supervisor UI.** `cli/supervise.py` renders `egress-token-allow` like + `gitleaks-allow`: the text payload is shown, modify is unavailable (there is + no file patch to edit), and approval prompts for a non-empty reason recorded + in the response notes. There is no on-disk config diff, so — like + `gitleaks-allow` and `capability-block` — it writes no egress audit-log entry. +- **Failure handling.** If `SUPERVISE_QUEUE_DIR` / `SUPERVISE_BOTTLE_SLUG` are + unset (supervise disabled for the bottle), the addon skips the queue and + returns the existing `403`. Any error writing the proposal or reading the + response also fails closed. -### Supervisor UI +### Data model changes -`cli/supervise.py` renders `egress-token-allow` like `gitleaks-allow`: the -text payload is shown, modify is unavailable (there is no file patch to edit), -and approval prompts for a non-empty reason that is recorded in the response -notes. There is no on-disk config diff, so — like `gitleaks-allow` and -`capability-block` — it writes no egress audit-log entry. +- New per-route manifest field `dlp.outbound_on_match: block | redact | + supervise`, rendered into `routes.yaml` (omitted when unset). +- `ScanResult` gains a `matched: str = ""` field carrying the raw substring the + detector matched. The token detectors populate it; the structural CRLF + detector leaves it empty. The value stays inside the egress sidecar process — + never written to a log line (logs use the redacted `context`) nor to the + proposal file. +- Proposal text payload (above) plus `.response.json` in + `SUPERVISE_QUEUE_DIR`, archived to `processed/` after a decision. +- New env var `EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS` (default 300). -### Failure handling +### External dependencies -If `SUPERVISE_QUEUE_DIR` / `SUPERVISE_BOTTLE_SLUG` are unset (supervise -disabled for the bottle), the addon skips the queue and returns the existing -`403`. Any error writing the proposal or reading the response also fails -closed. - -## Implementation chunks - -1. **Core** — `ScanResult.matched`; thread `safe_tokens` through - `scan_outbound` / token detectors; `build_token_allow_payload`. -2. **Supervise + TUI** — `TOOL_EGRESS_TOKEN_ALLOW`; TUI suffix, modify guard, - required approval reason. -3. **Addon glue** — async `request`, safe-tokens set, proposal write + async - poll, allow/block decision; pass `safe_tokens` into the WebSocket path. -4. **On-match policy** — `dlp.outbound_on_match` through manifest → render → - addon; `redact` surface scrub with fail-closed re-scan; policy dispatch in - the addon's outbound handler. -5. **Tests + docs** — core/supervise/TUI/manifest/render unit tests; README - egress + supervisor notes. +None. Reuses the existing supervisor queue (`SUPERVISE_QUEUE_DIR`) and the +mitmproxy addon framework already in the egress sidecar. ## Open questions @@ -190,3 +203,8 @@ closed. approval is an explicit operator decision and the safe-tokens set matches the exact found value — but a future revision could restrict `known_secrets` to reject-only. + +## References + +- Issue #261 +- PRD 0061 — `gitleaks-allow` supervisor proposal pattern this reuses.