didericis/bot-bottle
1
0
Fork 0
Code Issues 7 Pull Requests 15 Actions Packages Projects Releases Wiki Activity

PRD 0053: Egress DLP addon #196

Merged
didericis merged 8 commits from prd-0053-egress-dlp-addon into main 2026-06-06 00:09:21 -04:00
Conversation 11 Commits 8 Files Changed 19
+2645 -651
didericis-claude commented 2026-06-04 20:35:30 -04:00
Collaborator
Copy Link
Copy Source

Closes #195.

PRD 0053

Summary

With pipelock removed (PR #193), the egress proxy has no DLP scanning. This PRD defines the replacement: a per-route DLP addon built directly into the mitmproxy egress sidecar, following the architecture recommended in the research doc (PR #192).

Three implementation chunks after the manifest schema:

  1. Manifest dlp block — new dlp: key on egress.routes entries controls which detectors run outbound/inbound per route; omitting dlp keeps all detectors on (default-on, backward-compatible).
  2. Phase 1a — Token pattern detection — token_patterns detector (regex for AWS/GitHub/Anthropic/etc. keys) blocks outbound requests containing known credential formats.
  3. Phase 1b — Known secret detection — known_secrets detector checks provisioned EGRESS_TOKEN_* env vars and encoded variants in outbound request bodies.
  4. Phase 2 — Naive prompt injection — naive_injection_detection detector blocks/warns on inbound responses containing credential+disclosure combos or multiple jailbreak phrases.

All detector logic is pure Python in egress_addon_core.py / a new dlp_detectors.py, unit-testable on the host without a mitmproxy dependency.

Also replaces path_allowlist with Gateway API HTTPRoute match vocabulary (matches block with paths/methods/headers and AND/OR semantics).

Merge rule(s)

All chunks implemented in a single commit. PRD status flipped Draft → Active.

Closes #195. [PRD 0053](https://gitea.dideric.is/didericis/bot-bottle/src/commit/726713d0810bee66634b50b531280842a6598a36/docs/prds/0053-egress-dlp-addon.md) ## Summary With pipelock removed (PR #193), the egress proxy has no DLP scanning. This PRD defines the replacement: a per-route DLP addon built directly into the mitmproxy egress sidecar, following the architecture recommended in the [research doc (PR #192)](https://gitea.dideric.is/didericis/bot-bottle/pulls/192). Three implementation chunks after the manifest schema: 1. **Manifest `dlp` block** — new `dlp:` key on `egress.routes` entries controls which detectors run outbound/inbound per route; omitting `dlp` keeps all detectors on (default-on, backward-compatible). 2. **Phase 1a — Token pattern detection** — `token_patterns` detector (regex for AWS/GitHub/Anthropic/etc. keys) blocks outbound requests containing known credential formats. 3. **Phase 1b — Known secret detection** — `known_secrets` detector checks provisioned `EGRESS_TOKEN_*` env vars and encoded variants in outbound request bodies. 4. **Phase 2 — Naive prompt injection** — `naive_injection_detection` detector blocks/warns on inbound responses containing credential+disclosure combos or multiple jailbreak phrases. All detector logic is pure Python in `egress_addon_core.py` / a new `dlp_detectors.py`, unit-testable on the host without a mitmproxy dependency. Also replaces `path_allowlist` with Gateway API HTTPRoute match vocabulary (`matches` block with paths/methods/headers and AND/OR semantics). ## Merge rule(s) All chunks implemented in a single commit. PRD status flipped Draft → Active.
didericis-claude added 1 commit 2026-06-04 20:35:31 -04:00
docs: PRD 0053 — egress DLP addon (token, secret, injection detection)
test / unit (pull_request) Successful in 30s
Details
test / integration (pull_request) Successful in 46s
Details
f145203eee 
Adds the product requirements document for replacing pipelock's DLP
capability with a per-route mitmproxy addon. Covers three implementation
chunks: token-pattern detection, known-secret detection, and naive prompt
injection scanning. References the research in PR #192 and issue #195.
didericis added 1 commit 2026-06-04 20:41:28 -04:00
docs: research on YAML route matching formats (paths, headers, methods)
test / unit (pull_request) Successful in 28s
Details
test / integration (pull_request) Successful in 46s
Details
035ed430ba
didericis reviewed 2026-06-04 20:45:50 -04:00
docs/research/yaml-route-matching-formats.md Outdated
@@ -0,0 +473,4 @@
1. **Backward compatibility:** `path_allowlist` is the current field. If
adopting a `match`/`matches` structure, keep `path_allowlist` as a
deprecated alias? Or treat this as a breaking manifest version bump?
didericis commented 2026-06-04 20:45:50 -04:00
Owner
Copy Link
Copy Source

Treat it as a breaking version bump/do not preserve anything about previous behavior or manifests. Also do not bother creating loud exceptions or looking for the old format.

Treat it as a breaking version bump/do not preserve anything about previous behavior or manifests. Also do not bother creating loud exceptions or looking for the old format.
didericis marked this conversation as resolved
didericis reviewed 2026-06-04 20:47:40 -04:00
docs/research/yaml-route-matching-formats.md Outdated
@@ -0,0 +476,4 @@
deprecated alias? Or treat this as a breaking manifest version bump?
2. **Glob segment semantics:** adopt shell convention (`*` = intra-segment,
`**` = cross-segment) or ALB convention (`*` = anything including `/`)?
The shell convention is safer; ALB's is simpler.
didericis commented 2026-06-04 20:47:40 -04:00
Owner
Copy Link
Copy Source

Drop the glob actually, not strictly necessary/can just use regex

Drop the glob actually, not strictly necessary/can just use regex
didericis marked this conversation as resolved
didericis reviewed 2026-06-04 20:48:10 -04:00
docs/research/yaml-route-matching-formats.md Outdated
@@ -0,0 +481,4 @@
header values. ALB allows multiple values in one condition. Which is
less surprising for bot-bottle operators? The ALB approach is more
concise for the common case (e.g., `Content-Type: [application/json,
application/x-www-form-urlencoded]`).
didericis commented 2026-06-04 20:48:10 -04:00
Owner
Copy Link
Copy Source

Stick with Gateway API

Stick with Gateway API
didericis marked this conversation as resolved
didericis reviewed 2026-06-04 20:48:22 -04:00
docs/research/yaml-route-matching-formats.md Outdated
@@ -0,0 +483,4 @@
concise for the common case (e.g., `Content-Type: [application/json,
application/x-www-form-urlencoded]`).
4. **Case sensitivity on method names:** normalize to uppercase at parse
time (fail on unrecognised values) or case-insensitively?
didericis commented 2026-06-04 20:48:22 -04:00
Owner
Copy Link
Copy Source

Case insensitivity

Case insensitivity
didericis marked this conversation as resolved
didericis-claude reviewed 2026-06-04 20:50:06 -04:00
docs/research/yaml-route-matching-formats.md
didericis-claude commented 2026-06-04 20:50:06 -04:00
Author
Collaborator
Copy Link
Copy Source

Got it — hard cutover, no compatibility shim, no fallback parsing for the old path_allowlist shape.

Got it — hard cutover, no compatibility shim, no fallback parsing for the old `path_allowlist` shape.
didericis-claude commented 2026-06-04 20:50:06 -04:00
Author
Collaborator
Copy Link
Copy Source

Dropping glob. Will update the research doc and PRD to remove it — regex type covers every case glob would handle.

Dropping glob. Will update the research doc and PRD to remove it — `regex` type covers every case glob would handle.
didericis-claude commented 2026-06-04 20:50:06 -04:00
Author
Collaborator
Copy Link
Copy Source

Sticking with Gateway API — OR across header values means a separate match entry, not multiple values in one header block.

Sticking with Gateway API — OR across header values means a separate match entry, not multiple values in one header block.
didericis-claude commented 2026-06-04 20:50:06 -04:00
Author
Collaborator
Copy Link
Copy Source

Case-insensitive at parse time — get, GET, Get all accepted and normalised to uppercase internally.

Case-insensitive at parse time — `get`, `GET`, `Get` all accepted and normalised to uppercase internally.
didericis marked this conversation as resolved
didericis added 1 commit 2026-06-04 20:53:04 -04:00
docs: address PR #196 review; update research decisions and PRD
test / unit (pull_request) Successful in 30s
Details
test / integration (pull_request) Successful in 41s
Details
5265e25f9b 
Research doc: close open questions with decisions from review — hard
cutover on path_allowlist, drop glob (regex sufficient), stick with
Gateway API OR semantics for headers, case-insensitive method names.

PRD 0053: adopt Gateway API HTTPRoute match vocabulary (paths, methods,
headers) as the route schema replacement for path_allowlist. Add
MatchEntry / PathMatch / HeaderMatch types to EgressRoute design; cite
the route matching research doc; fold match restructure into chunk 1
alongside the dlp block.
didericis added 1 commit 2026-06-05 22:37:02 -04:00
feat(egress): implement PRD 0053 — DLP addon with Gateway API matches
lint / lint (push) Failing after 1m43s
Details
test / unit (pull_request) Successful in 40s
Details
test / integration (pull_request) Successful in 50s
Details
726713d081 
Replace path_allowlist with Gateway API HTTPRoute match vocabulary
(paths, methods, headers with AND/OR semantics) and add DLP scanning
to the egress proxy:

- Token pattern detection (AWS, GitHub, Anthropic, OpenAI, Stripe, JWT)
- Known secret detection (EGRESS_TOKEN_* with base64/URL/hex variants)
- Naive prompt injection detection (disclosure + credential, jailbreak)
- Per-route DLP configuration via manifest dlp block
- Inbound response scanning with block/warn severity

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
didericis added 1 commit 2026-06-05 22:44:57 -04:00
fix: remove unused ScanResult import in test_egress_addon_core
lint / lint (push) Failing after 1m45s
Details
test / unit (pull_request) Successful in 42s
Details
test / integration (pull_request) Successful in 53s
Details
4c60779fac 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
didericis reviewed 2026-06-05 22:59:28 -04:00
bot_bottle/backend/docker/egress_apply.py
@@ -11,3 +5,1 @@
Raises EgressApplyError on any failure — the dashboard
surfaces the message and keeps the proposal pending so the
operator can retry.
egress-block proposal. Fetches current routes.yaml, validates,
didericis commented 2026-06-05 22:59:28 -04:00
Owner
Copy Link
Copy Source

Reading through all of egress_apply.py makes me think we should just nix this... if a user needs to change the routes for a bottle (or anything else for the bottle), I think they should just change the bottle config. Couple reasons for that:

  1. Ephemeral changes to a running bottle are less visible/you have to look at the logs or a running bottle config to see what it can access
  2. It adds a bunch of surface area we need to maintain (like this)
  3. The features risks the user clicking y to everything the bottle asks for/friction isn't exactly bad

I think a better solution for adding something to a bottle during a workflow is probably more like:

  1. Create a new bottle with the changes you want
  2. Apply them to the sidecar for a running bottle

This means there will need to be a strict separation between what in the manifest results in a sidecar change vs what in the manifest results in an agent image change. But I think even agent changes could probably just be applied to the running bottle? Maybe? Will need to think more about that.

Reading through all of `egress_apply.py` makes me think we should just nix this... if a user needs to change the routes for a bottle (or anything else for the bottle), I think they should just change the bottle config. Couple reasons for that: 1. Ephemeral changes to a running bottle are less visible/you have to look at the logs or a running bottle config to see what it can access 2. It adds a bunch of surface area we need to maintain (like this) 3. The features risks the user clicking `y` to everything the bottle asks for/friction isn't exactly bad I think a better solution for adding something to a bottle during a workflow is probably more like: 1. Create a new bottle with the changes you want 2. Apply them to the sidecar for a running bottle This means there will need to be a strict separation between what in the manifest results in a sidecar change vs what in the manifest results in an agent image change. But I think even agent changes could probably just be applied to the running bottle? Maybe? Will need to think more about that.
didericis commented 2026-06-05 23:06:24 -04:00
Owner
Copy Link
Copy Source

@didericis-claude yes, open a new issue.

@didericis-claude yes, open a new issue.
didericis-claude commented 2026-06-05 23:07:03 -04:00
Author
Collaborator
Copy Link
Copy Source

Opened #198.

Opened #198.
didericis added 1 commit 2026-06-05 23:08:30 -04:00
fix: remove unused _yaml_scalar and redundant isinstance guard
lint / lint (push) Failing after 1m32s
Details
test / unit (pull_request) Successful in 34s
Details
test / integration (pull_request) Successful in 42s
Details
1c7812fa9f 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
didericis reviewed 2026-06-05 23:16:03 -04:00
bot_bottle/dlp_detectors.py
@@ -0,0 +111,4 @@
def scan_naive_injection(text: str) -> ScanResult | None:
disclosure = any(p.search(text) for p in DISCLOSURE_PHRASES)
token = scan_token_patterns(text) is not None
didericis commented 2026-06-05 23:16:03 -04:00
Owner
Copy Link
Copy Source

Anything with a token will already be caught by the token detector. Calling it again in here is silly. Let's do the following instead:

  1. Warn on any disclosure phrases
  2. Warn on any jailbreak phrases
  3. Block when disclosure and jailbreak phrases are within a certain distance
Anything with a token will already be caught by the token detector. Calling it again in here is silly. Let's do the following instead: 1. Warn on any disclosure phrases 2. Warn on any jailbreak phrases 3. Block when disclosure and jailbreak phrases are within a certain distance
didericis marked this conversation as resolved
didericis added 1 commit 2026-06-05 23:17:57 -04:00
fix(dlp): rework naive injection to proximity-based disclosure+jailbreak
lint / lint (push) Failing after 1m24s
Details
test / unit (pull_request) Successful in 30s
Details
test / integration (pull_request) Successful in 44s
Details
abcb336e7c 
Token detection is already handled by the token_patterns detector
running separately — calling it again from scan_naive_injection was
redundant. New logic:

- Warn on any disclosure phrase
- Warn on any jailbreak phrase
- Block when both appear within 500 chars of each other

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
didericis reviewed 2026-06-05 23:18:30 -04:00
bot_bottle/egress.py
@@ -123,1 +65,3 @@
routes are merged."""
from .egress_addon_core import MatchEntry as CoreMatchEntry
from .egress_addon_core import PathMatch as CorePathMatch
from .egress_addon_core import HeaderMatch as CoreHeaderMatch
didericis commented 2026-06-05 23:18:30 -04:00
Owner
Copy Link
Copy Source

Why not import these at the top of the module?

Why not import these at the top of the module?
didericis marked this conversation as resolved
didericis added 1 commit 2026-06-05 23:19:46 -04:00
refactor(egress): move core type imports to module level
test / unit (pull_request) Successful in 37s
Details
test / integration (pull_request) Successful in 55s
Details
lint / lint (push) Failing after 1m38s
Details
test / unit (push) Failing after 37s
Details
test / integration (push) Successful in 50s
Details
52820278fd 
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
didericis reviewed 2026-06-05 23:44:03 -04:00
didericis-claude closed this pull request 2026-06-05 23:53:04 -04:00
didericis-claude deleted branch prd-0053-egress-dlp-addon 2026-06-05 23:53:04 -04:00
didericis-claude reopened this pull request 2026-06-05 23:53:24 -04:00
didericis force-pushed prd-0053-egress-dlp-addon from 0c285b1712 to 52820278fd 2026-06-06 00:08:39 -04:00 Compare
didericis merged commit 52820278fd into main 2026-06-06 00:09:21 -04:00
didericis deleted branch prd-0053-egress-dlp-addon 2026-06-06 00:09:22 -04:00
Sign in to join this conversation.
Reviewers
No Reviewers
didericis-claude
didericis
Labels
Clear labels
Compat/Breaking
Breaking change that won't be backward compatible
 Kind/Bug
Something is not working
 Kind/Documentation
Documentation changes
 Kind/Enhancement
Improve existing functionality
 Kind/Feature
New functionality
 Kind/Security
This is security issue
 Kind/Testing
Issue or pull request related to testing
Priority
Critical
1
The priority is critical
Priority
High
2
The priority is high
Priority
Low
4
The priority is low
Priority
Medium
3
The priority is medium
Reviewed
Confirmed
1
Issue has been confirmed
Reviewed
Duplicate
2
This issue or pull request already exists
Reviewed
Invalid
3
Invalid issue
Reviewed
Won't Fix
3
This issue won't be fixed
Status
Abandoned
3
Somebody has started to work on this but abandoned work
Status
Blocked
1
Something is blocking this issue or pull request
Status
Need More Info
2
Feedback is required to reproduce issue or to continue work
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
didericis-claude (didericis (claude)) didericis-codex (didericis (codex)) didericis-gemma (gemma) didericis
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: didericis/bot-bottle#196
Delete Branch "prd-0053-egress-dlp-addon"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?