Add dlp.outbound_on_match policy (block | redact | supervise)
lint / lint (push) Successful in 1m41s
test / unit (pull_request) Successful in 30s
test / integration (pull_request) Successful in 18s

Give each egress route a policy for what the proxy does when an outbound
DLP detector matches a token, defaulting to the supervise flow added in
the previous commit. The goal is cutting false-positive friction without
weakening default-deny.

- redact: scrub the matched value(s) from the body, non-host headers, and
  path/query via redact_tokens, then re-scan. Forward if clean; fail
  closed with a 403 if a match remains on a surface redaction can't
  rewrite (the hostname, or a unicode-evasion token). For routes where a
  token-shaped value is noise the upstream doesn't need.
- block: the original hard 403, never overridable.
- supervise (default, unset): hold the request for operator approval.

Structural blocks (CRLF, no safelist-able value) stay hard 403s under
every policy.

Threads outbound_on_match from the bottle manifest (manifest_egress)
through the resolved EgressRoute and rendered routes.yaml (egress.py) to
the addon's Route (egress_addon_core), and round-trips it via the
list-egress-routes introspection endpoint. The allow/egress-block tool
descriptions document the new key.

Tests: manifest parse/validation, core parse/validation, full
manifest->render->addon round-trip for redact. README + PRD 0062 updated.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HnvBjPZC5V7qeQpFbQdDmS
This commit is contained in:
2026-06-24 16:50:13 -04:00
parent 7f2352287e
commit cdfaaa3de8
10 changed files with 291 additions and 53 deletions
+3 -2
View File
@@ -15,7 +15,7 @@
## Features ## Features
- **Per-bottle egress allowlist** — TLS-bumped HTTP/HTTPS chokepoint with a per-manifest host allowlist; per-route path/method/header `matches` filtering; outbound DLP scanning for known tokens and secrets, inbound DLP scanning for prompt-injection attempts; DoH and arbitrary hosts blocked by default. - **Per-bottle egress allowlist** — TLS-bumped HTTP/HTTPS chokepoint with a per-manifest host allowlist; per-route path/method/header `matches` filtering; outbound DLP scanning for known tokens and secrets, inbound DLP scanning for prompt-injection attempts; DoH and arbitrary hosts blocked by default.
- **Supervisor override for token blocks** — when the outbound DLP catches a token, the request is held and surfaced in `./cli.py supervise` instead of failing outright. The operator approves or rejects; an approved value is remembered for the life of the egress proxy so the request — and later ones carrying it — flow through. Fails closed on rejection or timeout. - **Per-route token-match policy** — each egress route picks what happens when the outbound DLP catches a token via `dlp.outbound_on_match`: `supervise` (default) holds the request and surfaces it in `./cli.py supervise` for approval (an approved value is remembered for the life of the proxy); `redact` scrubs the value and forwards; `block` is a hard `403`. Cuts false-positive friction without weakening default-deny.
- **Tokens the agent never sees** — host secrets live in a sidecar; the agent dials `http://sidecar:9099/<path>` and the proxy strips inbound `Authorization` and injects the real token before forwarding. `printenv` in the agent shows proxy URLs only. - **Tokens the agent never sees** — host secrets live in a sidecar; the agent dials `http://sidecar:9099/<path>` and the proxy strips inbound `Authorization` and injects the real token before forwarding. `printenv` in the agent shows proxy URLs only.
- **Gitleaks-scanned push (git-gate)** — `bottle.git` remotes route through a per-bottle `git daemon` that gitleaks-scans incoming refs pre-receive and forwards clean refs upstream over SSH. The agent never holds the upstream credential. - **Gitleaks-scanned push (git-gate)** — `bottle.git` remotes route through a per-bottle `git daemon` that gitleaks-scans incoming refs pre-receive and forwards clean refs upstream over SSH. The agent never holds the upstream credential.
- **Manifest-scoped skills + secrets** — each bottle declares its skills, env, git identity, remotes, and egress routes; unknown keys die at load. - **Manifest-scoped skills + secrets** — each bottle declares its skills, env, git identity, remotes, and egress routes; unknown keys die at load.
@@ -149,9 +149,10 @@ You help maintain Gitea-hosted projects.
| `dlp` | no | Per-route DLP overrides. Omit to use defaults (all detectors on). | | `dlp` | no | Per-route DLP overrides. Omit to use defaults (all detectors on). |
| `dlp.outbound_detectors` | no | `false` disables outbound scanning; list restricts to named detectors (`token_patterns`, `known_secrets`). | | `dlp.outbound_detectors` | no | `false` disables outbound scanning; list restricts to named detectors (`token_patterns`, `known_secrets`). |
| `dlp.inbound_detectors` | no | `false` disables inbound scanning; list restricts to named detectors (`naive_injection_detection`). | | `dlp.inbound_detectors` | no | `false` disables inbound scanning; list restricts to named detectors (`naive_injection_detection`). |
| `dlp.outbound_on_match` | no | What to do when an outbound token is detected: `supervise` (default — hold for operator approval), `redact` (scrub the value and forward), or `block` (hard 403). |
| `git.fetch` | no | `true` permits smart HTTP clone/fetch (`git-upload-pack`) for this host. Push (`git-receive-pack`) remains blocked. | | `git.fetch` | no | `true` permits smart HTTP clone/fetch (`git-upload-pack`) for this host. Push (`git-receive-pack`) remains blocked. |
When an outbound request is blocked because a DLP detector matched a token, the proxy queues an `egress-token-allow` proposal for the operator's `./cli.py supervise` TUI and holds the request open until it is answered (or `EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS`, default 300s, elapses — after which it fails closed). The operator never sees the raw token, only the host, method, path, and a redacted snippet. Approving adds the value to an in-memory safelist for the life of the egress proxy. Structural blocks (CRLF injection) and not-in-allowlist host blocks stay hard `403`s. When an outbound DLP detector matches a token, the route's `dlp.outbound_on_match` policy decides what happens. Under the default `supervise`, the proxy queues an `egress-token-allow` proposal for the operator's `./cli.py supervise` TUI and holds the request open until it is answered (or `EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS`, default 300s, elapses — after which it fails closed). The operator never sees the raw token, only the host, method, path, and a redacted snippet; approving adds the value to an in-memory safelist for the life of the egress proxy. Under `redact`, the matched value is scrubbed from the body, headers, and path and the request is forwarded (failing closed if a match lands somewhere unredactable, like the hostname). Under `block` it stays a hard `403`. Structural blocks (CRLF injection) and not-in-allowlist host blocks are always hard `403`s regardless of policy.
More examples in `examples/`. Full design lives under `docs/prds/`; the trust-boundary rationale is in `docs/prds/0011-per-file-md-manifest.md`. More examples in `examples/`. Full design lives under `docs/prds/`; the trust-boundary rationale is in `docs/prds/0011-per-file-md-manifest.md`.
+10 -1
View File
@@ -95,6 +95,7 @@ def egress_manifest_routes(
git_fetch=r.GitFetch, git_fetch=r.GitFetch,
outbound_detectors=r.OutboundDetectors, outbound_detectors=r.OutboundDetectors,
inbound_detectors=r.InboundDetectors, inbound_detectors=r.InboundDetectors,
outbound_on_match=r.OutboundOnMatch,
)) ))
return tuple(out) return tuple(out)
@@ -177,7 +178,11 @@ def _route_to_yaml_fields(r: Route) -> dict[str, object]:
fields["matches"] = matches_data fields["matches"] = matches_data
if r.git_fetch: if r.git_fetch:
fields["git"] = {"fetch": True} fields["git"] = {"fetch": True}
if r.outbound_detectors is not None or r.inbound_detectors is not None: if (
r.outbound_detectors is not None
or r.inbound_detectors is not None
or r.outbound_on_match
):
dlp: dict[str, object] = {} dlp: dict[str, object] = {}
if r.outbound_detectors is not None: if r.outbound_detectors is not None:
dlp["outbound_detectors"] = ( dlp["outbound_detectors"] = (
@@ -189,6 +194,8 @@ def _route_to_yaml_fields(r: Route) -> dict[str, object]:
False if not r.inbound_detectors False if not r.inbound_detectors
else list(r.inbound_detectors) else list(r.inbound_detectors)
) )
if r.outbound_on_match:
dlp["outbound_on_match"] = r.outbound_on_match
fields["dlp"] = dlp fields["dlp"] = dlp
return fields return fields
@@ -260,6 +267,8 @@ def egress_render_routes(
elif isinstance(dv, list): elif isinstance(dv, list):
items_str = ", ".join(f'"{x}"' for x in dv) items_str = ", ".join(f'"{x}"' for x in dv)
lines.append(f" {dk}: [{items_str}]") lines.append(f" {dk}: [{items_str}]")
elif isinstance(dv, str):
lines.append(f' {dk}: "{dv}"')
return "\n".join(lines) + "\n" return "\n".join(lines) + "\n"
+112 -30
View File
@@ -17,7 +17,11 @@ from mitmproxy import http # type: ignore[import-not-found] # pylint: disable=
from egress_addon_core import ( # type: ignore[import-not-found] # pylint: disable=import-error from egress_addon_core import ( # type: ignore[import-not-found] # pylint: disable=import-error
LOG_BLOCKS, LOG_BLOCKS,
LOG_FULL, LOG_FULL,
DEFAULT_OUTBOUND_ON_MATCH,
ON_MATCH_BLOCK,
ON_MATCH_REDACT,
Config, Config,
Route,
ScanResult, ScanResult,
build_inbound_scan_text, build_inbound_scan_text,
build_outbound_scan_text, build_outbound_scan_text,
@@ -189,37 +193,11 @@ class EgressAddon:
# Hostname is included to catch DNS-tunnelling exfiltration attempts. # Hostname is included to catch DNS-tunnelling exfiltration attempts.
route = match_route(self.config.routes, flow.request.pretty_host) route = match_route(self.config.routes, flow.request.pretty_host)
if route is not None: if route is not None:
body = flow.request.get_text(strict=False) or "" if not await self._handle_outbound_dlp(flow, route):
# Re-scan after each operator approval so a second, un-approved
# token in the same request is still caught (PRD 0062).
while True:
scan_text = build_outbound_scan_text(
flow.request.pretty_host,
request_path,
query,
outbound_scan_headers(route, dict(flow.request.headers)),
body,
)
dlp_result = scan_outbound(
route, scan_text, os.environ, safe_tokens=self.safe_tokens,
)
if dlp_result is None or dlp_result.severity != "block":
break
# Token blocks (a match with a safelist-able value) can be
# routed to the operator; structural blocks (CRLF, matched="")
# and any block when supervise is disabled stay hard 403s.
if dlp_result.matched and self._supervise_available():
approved = await self._supervise_token_block(
flow, request_path, dlp_result,
)
if approved:
continue # re-scan; matched value now in safe_tokens
return # _supervise_token_block wrote the 403 response
ctx = self._req_ctx(flow)
if dlp_result.context:
ctx = {**ctx, "context": dlp_result.context}
self._block(flow, f"egress DLP: {dlp_result.reason}", ctx=ctx)
return return
# The redact policy may have rewritten the request line; recompute
# the path/query the git checks below rely on.
request_path, _, query = flow.request.path.partition("?")
if is_git_push_request(request_path, query): if is_git_push_request(request_path, query):
self._block( self._block(
@@ -269,6 +247,110 @@ class EgressAddon:
if self.config.log >= LOG_FULL: if self.config.log >= LOG_FULL:
self._log_request(flow) self._log_request(flow)
def _block_dlp(self, flow: http.HTTPFlow, result: ScanResult) -> None:
ctx = self._req_ctx(flow)
if result.context:
ctx = {**ctx, "context": result.context}
self._block(flow, f"egress DLP: {result.reason}", ctx=ctx)
async def _handle_outbound_dlp(
self,
flow: http.HTTPFlow,
route: Route,
) -> bool:
"""Scan the outbound request and apply the route's on-match policy
(PRD 0062). Returns True if the request may be forwarded, False if a
403 response has been written to `flow`.
Loops so the supervise policy can re-scan after each approval a
second, un-approved token in the same request is still caught."""
while True:
request_path, _, query = flow.request.path.partition("?")
body = flow.request.get_text(strict=False) or ""
scan_text = build_outbound_scan_text(
flow.request.pretty_host,
request_path,
query,
outbound_scan_headers(route, dict(flow.request.headers)),
body,
)
result = scan_outbound(
route, scan_text, os.environ, safe_tokens=self.safe_tokens,
)
if result is None or result.severity != "block":
return True
# Structural blocks (CRLF, no safelist-able value) are always a
# hard 403, regardless of the route's on-match policy.
if not result.matched:
self._block_dlp(flow, result)
return False
policy = route.outbound_on_match or DEFAULT_OUTBOUND_ON_MATCH
if policy == ON_MATCH_REDACT:
if self._redact_outbound(flow, route):
if self.config.log >= LOG_BLOCKS:
sys.stderr.write(json.dumps({
"event": "egress_redacted",
"reason": f"egress DLP: {result.reason}",
**self._req_ctx(flow),
}) + "\n")
return True
self._block(
flow,
f"egress DLP: {result.reason}; redaction could not remove "
"all matches (e.g. a match in the hostname)",
ctx=self._req_ctx(flow),
)
return False
if policy == ON_MATCH_BLOCK:
self._block_dlp(flow, result)
return False
# supervise (default): hold the request for operator approval.
# Fall back to a hard 403 when supervise isn't wired for the bottle.
if not self._supervise_available():
self._block_dlp(flow, result)
return False
approved = await self._supervise_token_block(flow, request_path, result)
if not approved:
return False # _supervise_token_block wrote the 403 response
# loop: the approved value is now in safe_tokens; re-scan.
def _redact_outbound(self, flow: http.HTTPFlow, route: Route) -> bool:
"""Scrub detected tokens from the mutable request surfaces (body,
headers, path/query) and re-scan. Returns True if the request is now
clean; False if a block-severity match remains on a surface redaction
cannot rewrite (the hostname) so the caller fails closed."""
body = flow.request.get_text(strict=False)
if body:
redacted_body = redact_tokens(body, env=os.environ)
if redacted_body != body:
flow.request.text = redacted_body
for name, value in list(flow.request.headers.items()):
if name.lower() == "host":
continue # routing-critical; never a legitimate token
redacted = redact_tokens(value, env=os.environ)
if redacted != value:
flow.request.headers[name] = redacted
redacted_path = redact_tokens(flow.request.path, env=os.environ)
if redacted_path != flow.request.path:
flow.request.path = redacted_path
request_path, _, query = flow.request.path.partition("?")
new_body = flow.request.get_text(strict=False) or ""
scan_text = build_outbound_scan_text(
flow.request.pretty_host,
request_path,
query,
outbound_scan_headers(route, dict(flow.request.headers)),
new_body,
)
result = scan_outbound(route, scan_text, os.environ)
return result is None or result.severity != "block"
async def _supervise_token_block( async def _supervise_token_block(
self, self,
flow: http.HTTPFlow, flow: http.HTTPFlow,
+37 -7
View File
@@ -37,6 +37,15 @@ VALID_METHODS = frozenset({
OUTBOUND_DETECTOR_NAMES = frozenset({"token_patterns", "known_secrets"}) OUTBOUND_DETECTOR_NAMES = frozenset({"token_patterns", "known_secrets"})
INBOUND_DETECTOR_NAMES = frozenset({"naive_injection_detection"}) INBOUND_DETECTOR_NAMES = frozenset({"naive_injection_detection"})
# Per-route policy for what the proxy does when an outbound DLP detector
# matches a token (PRD 0062).
ON_MATCH_BLOCK = "block" # hard 403, never overridable
ON_MATCH_REDACT = "redact" # scrub the matched value, forward the request
ON_MATCH_SUPERVISE = "supervise" # queue for operator approval, hold the request
OUTBOUND_ON_MATCH_VALUES = (ON_MATCH_BLOCK, ON_MATCH_REDACT, ON_MATCH_SUPERVISE)
# Unset resolves to supervise (fall back to block when supervise is not wired).
DEFAULT_OUTBOUND_ON_MATCH = ON_MATCH_SUPERVISE
@dataclass(frozen=True) @dataclass(frozen=True)
class PathMatch: class PathMatch:
@@ -69,6 +78,8 @@ class Route:
git_fetch: bool = False git_fetch: bool = False
outbound_detectors: tuple[str, ...] | None = None outbound_detectors: tuple[str, ...] | None = None
inbound_detectors: tuple[str, ...] | None = None inbound_detectors: tuple[str, ...] | None = None
# "" means unset → DEFAULT_OUTBOUND_ON_MATCH. See OUTBOUND_ON_MATCH_VALUES.
outbound_on_match: str = ""
LOG_OFF = 0 # no logging LOG_OFF = 0 # no logging
@@ -223,12 +234,12 @@ def _parse_detectors(
idx: int, idx: int,
host: str, host: str,
raw_dict: dict[str, object], raw_dict: dict[str, object],
) -> tuple[tuple[str, ...] | None, tuple[str, ...] | None]: ) -> tuple[tuple[str, ...] | None, tuple[str, ...] | None, str]:
"""Parse the optional `dlp` block on a route, returning """Parse the optional `dlp` block on a route, returning
(outbound_detectors, inbound_detectors).""" (outbound_detectors, inbound_detectors, outbound_on_match)."""
dlp_raw = raw_dict.get("dlp") dlp_raw = raw_dict.get("dlp")
if dlp_raw is None: if dlp_raw is None:
return None, None return None, None, ""
label = f"route[{idx}] ({host})" label = f"route[{idx}] ({host})"
if not isinstance(dlp_raw, dict): if not isinstance(dlp_raw, dict):
raise ValueError(f"{label}: 'dlp' must be an object") raise ValueError(f"{label}: 'dlp' must be an object")
@@ -265,13 +276,24 @@ def _parse_detectors(
outbound = _parse_detector_field("outbound_detectors", OUTBOUND_DETECTOR_NAMES) outbound = _parse_detector_field("outbound_detectors", OUTBOUND_DETECTOR_NAMES)
inbound = _parse_detector_field("inbound_detectors", INBOUND_DETECTOR_NAMES) inbound = _parse_detector_field("inbound_detectors", INBOUND_DETECTOR_NAMES)
on_match = ""
on_match_raw = dlp.get("outbound_on_match")
if on_match_raw is not None:
if not isinstance(on_match_raw, str) or on_match_raw not in OUTBOUND_ON_MATCH_VALUES:
raise ValueError(
f"{label}: dlp.outbound_on_match must be one of "
f"{', '.join(OUTBOUND_ON_MATCH_VALUES)} (got {on_match_raw!r})"
)
on_match = on_match_raw
for k in dlp: for k in dlp:
if k not in ("outbound_detectors", "inbound_detectors"): if k not in ("outbound_detectors", "inbound_detectors", "outbound_on_match"):
raise ValueError( raise ValueError(
f"{label}: dlp has unknown key {k!r}; accepted keys " f"{label}: dlp has unknown key {k!r}; accepted keys "
f"are 'outbound_detectors', 'inbound_detectors'" f"are 'outbound_detectors', 'inbound_detectors', "
f"'outbound_on_match'"
) )
return outbound, inbound return outbound, inbound, on_match
def parse_routes(payload: object) -> tuple[Route, ...]: def parse_routes(payload: object) -> tuple[Route, ...]:
@@ -342,7 +364,7 @@ def _parse_one(idx: int, raw: object) -> Route:
) )
# dlp detectors # dlp detectors
outbound_detectors, inbound_detectors = _parse_detectors( outbound_detectors, inbound_detectors, outbound_on_match = _parse_detectors(
idx, host, raw_dict, idx, host, raw_dict,
) )
@@ -361,6 +383,7 @@ def _parse_one(idx: int, raw: object) -> Route:
git_fetch=git_fetch, git_fetch=git_fetch,
outbound_detectors=outbound_detectors, outbound_detectors=outbound_detectors,
inbound_detectors=inbound_detectors, inbound_detectors=inbound_detectors,
outbound_on_match=outbound_on_match,
) )
@@ -409,6 +432,8 @@ def route_to_yaml_dict(r: Route) -> dict[str, object]:
dlp["outbound_detectors"] = list(r.outbound_detectors) dlp["outbound_detectors"] = list(r.outbound_detectors)
if r.inbound_detectors is not None: if r.inbound_detectors is not None:
dlp["inbound_detectors"] = list(r.inbound_detectors) dlp["inbound_detectors"] = list(r.inbound_detectors)
if r.outbound_on_match:
dlp["outbound_on_match"] = r.outbound_on_match
if dlp: if dlp:
d["dlp"] = dlp d["dlp"] = dlp
return d return d
@@ -781,6 +806,11 @@ __all__ = [
"route_to_yaml_dict", "route_to_yaml_dict",
"LOG_FULL", "LOG_FULL",
"LOG_OFF", "LOG_OFF",
"ON_MATCH_BLOCK",
"ON_MATCH_REDACT",
"ON_MATCH_SUPERVISE",
"OUTBOUND_ON_MATCH_VALUES",
"DEFAULT_OUTBOUND_ON_MATCH",
"Config", "Config",
"Decision", "Decision",
"HeaderMatch", "HeaderMatch",
+22 -5
View File
@@ -21,6 +21,9 @@ VALID_METHODS = frozenset({
OUTBOUND_DETECTOR_NAMES = frozenset({"token_patterns", "known_secrets"}) OUTBOUND_DETECTOR_NAMES = frozenset({"token_patterns", "known_secrets"})
INBOUND_DETECTOR_NAMES = frozenset({"naive_injection_detection"}) INBOUND_DETECTOR_NAMES = frozenset({"naive_injection_detection"})
# What the proxy does on an outbound token match (PRD 0062).
OUTBOUND_ON_MATCH_VALUES = ("block", "redact", "supervise")
def validate_egress_routes( def validate_egress_routes(
bottle_name: str, bottle_name: str,
@@ -67,6 +70,7 @@ class ManifestEgressRoute:
GitFetch: bool = False GitFetch: bool = False
OutboundDetectors: tuple[str, ...] | None = None OutboundDetectors: tuple[str, ...] | None = None
InboundDetectors: tuple[str, ...] | None = None InboundDetectors: tuple[str, ...] | None = None
OutboundOnMatch: str = ""
@classmethod @classmethod
def from_dict(cls, bottle_name: str, idx: int, raw: object) -> "ManifestEgressRoute": def from_dict(cls, bottle_name: str, idx: int, raw: object) -> "ManifestEgressRoute":
@@ -161,8 +165,9 @@ class ManifestEgressRoute:
# --- dlp --- # --- dlp ---
outbound_detectors: tuple[str, ...] | None = None outbound_detectors: tuple[str, ...] | None = None
inbound_detectors: tuple[str, ...] | None = None inbound_detectors: tuple[str, ...] | None = None
outbound_on_match = ""
if "dlp" in d: if "dlp" in d:
outbound_detectors, inbound_detectors = _parse_dlp_block( outbound_detectors, inbound_detectors, outbound_on_match = _parse_dlp_block(
label, d.get("dlp"), label, d.get("dlp"),
) )
@@ -201,6 +206,7 @@ class ManifestEgressRoute:
GitFetch=git_fetch, GitFetch=git_fetch,
OutboundDetectors=outbound_detectors, OutboundDetectors=outbound_detectors,
InboundDetectors=inbound_detectors, InboundDetectors=inbound_detectors,
OutboundOnMatch=outbound_on_match,
) )
@@ -323,7 +329,7 @@ def _parse_header_match(
def _parse_dlp_block( def _parse_dlp_block(
route_label: str, route_label: str,
raw: object, raw: object,
) -> tuple[tuple[str, ...] | None, tuple[str, ...] | None]: ) -> tuple[tuple[str, ...] | None, tuple[str, ...] | None, str]:
label = f"{route_label} dlp" label = f"{route_label} dlp"
d = as_json_object(raw, label) d = as_json_object(raw, label)
@@ -358,13 +364,24 @@ def _parse_dlp_block(
outbound = _parse_field("outbound_detectors", OUTBOUND_DETECTOR_NAMES) outbound = _parse_field("outbound_detectors", OUTBOUND_DETECTOR_NAMES)
inbound = _parse_field("inbound_detectors", INBOUND_DETECTOR_NAMES) inbound = _parse_field("inbound_detectors", INBOUND_DETECTOR_NAMES)
on_match = ""
on_match_raw = d.get("outbound_on_match")
if on_match_raw is not None:
if not isinstance(on_match_raw, str) or on_match_raw not in OUTBOUND_ON_MATCH_VALUES:
raise ManifestError(
f"{label} outbound_on_match must be one of "
f"{', '.join(OUTBOUND_ON_MATCH_VALUES)} (got {on_match_raw!r})"
)
on_match = on_match_raw
for k in d: for k in d:
if k not in ("outbound_detectors", "inbound_detectors"): if k not in ("outbound_detectors", "inbound_detectors", "outbound_on_match"):
raise ManifestError( raise ManifestError(
f"{label} has unknown key {k!r}; accepted keys are " f"{label} has unknown key {k!r}; accepted keys are "
f"'outbound_detectors', 'inbound_detectors'" f"'outbound_detectors', 'inbound_detectors', "
f"'outbound_on_match'"
) )
return outbound, inbound return outbound, inbound, on_match
LOG_LEVELS = frozenset({0, 1, 2}) LOG_LEVELS = frozenset({0, 1, 2})
+2
View File
@@ -187,6 +187,7 @@ TOOL_DEFINITIONS: list[dict[str, object]] = [
" dlp: (optional DLP scanner overrides)\n" " dlp: (optional DLP scanner overrides)\n"
" outbound_detectors: [token_patterns, known_secrets]\n" " outbound_detectors: [token_patterns, known_secrets]\n"
" inbound_detectors: [naive_injection_detection]\n" " inbound_detectors: [naive_injection_detection]\n"
" outbound_on_match: block|redact|supervise (default supervise)\n"
"Omit any key that should use its default. " "Omit any key that should use its default. "
"`list-egress-routes` returns routes in this same format." "`list-egress-routes` returns routes in this same format."
), ),
@@ -228,6 +229,7 @@ TOOL_DEFINITIONS: list[dict[str, object]] = [
" dlp: (optional DLP scanner overrides)\n" " dlp: (optional DLP scanner overrides)\n"
" outbound_detectors: [token_patterns, known_secrets]\n" " outbound_detectors: [token_patterns, known_secrets]\n"
" inbound_detectors: [naive_injection_detection]\n" " inbound_detectors: [naive_injection_detection]\n"
" outbound_on_match: block|redact|supervise (default supervise)\n"
"Omit any key that should use its default. " "Omit any key that should use its default. "
"`list-egress-routes` returns routes in this same format." "`list-egress-routes` returns routes in this same format."
), ),
@@ -7,12 +7,26 @@
## Summary ## Summary
When the egress proxy blocks an outbound request because a DLP detector Give each egress route a policy for what happens when an outbound DLP detector
matched a token/secret, route that block through the existing supervisor matches a token, via `dlp.outbound_on_match: block | redact | supervise`
(default `supervise`):
- **`supervise`** (default) — route the block through the existing supervisor
approval queue instead of returning `403` immediately. The proxy holds the approval queue instead of returning `403` immediately. The proxy holds the
request open until the operator approves or rejects it. On approval, the request open until the operator approves or rejects it. On approval the
matched token is added to an in-memory "safe tokens" set so the request — and matched token is added to an in-memory "safe tokens" set so the request — and
any later request carrying the same token — flows through without re-prompting. any later request carrying the same token — flows through without
re-prompting.
- **`redact`** — scrub the matched value(s) from the request and forward it,
no operator in the loop. For routes where a token-shaped value is noise the
upstream doesn't need (telemetry/log sinks). Fails closed if a match lands on
a surface redaction can't rewrite (the hostname).
- **`block`** — the original hard `403`; never overridable. For routes where a
detected token must always stop.
The motivating goal is reducing friction from false positives without weakening
the default-deny posture: supervise keeps a human in the loop, redact is an
explicit per-route opt-in, and block stays available for sensitive routes.
## Problem ## Problem
@@ -58,9 +72,35 @@ fine-grained way to say "this specific value is fine."
the exact value the detector found. the exact value the detector found.
- Replacing the per-route `dlp.outbound_detectors` override. That remains the - Replacing the per-route `dlp.outbound_detectors` override. That remains the
way to turn a detector off wholesale. way to turn a detector off wholesale.
- Making `redact` the default. Silent redaction of a true false positive
corrupts legitimate data, so it is opt-in per route; `supervise` (human in
the loop) stays the default.
## Design ## Design
### On-match policy
`dlp.outbound_on_match` is a per-route enum threaded from the bottle manifest
(`manifest_egress`) through the resolved route (`egress.EgressRoute`), the
rendered `routes.yaml` (`egress_render_routes`), and the addon's `Route`
(`egress_addon_core`). Unset renders nothing and resolves to `supervise` at
request time. The `list-egress-routes` introspection endpoint round-trips it so
the agent's proposals preserve it.
On an outbound block the addon dispatches on the resolved policy:
- **Structural blocks always 403.** A `ScanResult` with no `matched` value
(CRLF injection) is a hard `403` regardless of policy — there is nothing to
redact or safelist.
- **`redact`** runs `redact_tokens` over the body, non-`host` header values,
and path/query, then re-scans. If the re-scan is clean the (rewritten)
request is forwarded; if a block-severity match remains (e.g. in the
hostname, or a unicode-evasion token redaction can't reach) it fails closed
with a `403`.
- **`block`** writes the `403` immediately.
- **`supervise`** runs the queue-and-wait loop below, falling back to `block`
when supervise isn't wired for the bottle.
### Detected-value plumbing ### Detected-value plumbing
`ScanResult` gains a `matched: str = ""` field carrying the raw substring the `ScanResult` gains a `matched: str = ""` field carrying the raw substring the
@@ -128,8 +168,11 @@ closed.
required approval reason. required approval reason.
3. **Addon glue** — async `request`, safe-tokens set, proposal write + async 3. **Addon glue** — async `request`, safe-tokens set, proposal write + async
poll, allow/block decision; pass `safe_tokens` into the WebSocket path. poll, allow/block decision; pass `safe_tokens` into the WebSocket path.
4. **Tests + docs** — core/supervise/TUI unit tests; README egress + supervisor 4. **On-match policy**`dlp.outbound_on_match` through manifest → render →
notes. addon; `redact` surface scrub with fail-closed re-scan; policy dispatch in
the addon's outbound handler.
5. **Tests + docs** — core/supervise/TUI/manifest/render unit tests; README
egress + supervisor notes.
## Open questions ## Open questions
+17
View File
@@ -329,6 +329,23 @@ class TestRenderRoutes(unittest.TestCase):
self.assertEqual(("token_patterns",), addon_routes[0].outbound_detectors) self.assertEqual(("token_patterns",), addon_routes[0].outbound_detectors)
self.assertEqual((), addon_routes[0].inbound_detectors) self.assertEqual((), addon_routes[0].inbound_detectors)
def test_outbound_on_match_round_trips(self):
from bot_bottle.egress_addon_core import load_routes
b = _bottle([{"host": "logs.example", "dlp": {
"outbound_on_match": "redact",
}}])
routes = egress_routes_for_bottle(b)
rendered = egress_render_routes(routes)
self.assertIn('outbound_on_match: "redact"', rendered)
addon_routes = load_routes(rendered)
self.assertEqual("redact", addon_routes[0].outbound_on_match)
def test_outbound_on_match_default_omitted_from_render(self):
b = _bottle([{"host": "x.example"}])
routes = egress_routes_for_bottle(b)
rendered = egress_render_routes(routes)
self.assertNotIn("outbound_on_match", rendered)
def test_git_fetch_policy_round_trips(self): def test_git_fetch_policy_round_trips(self):
from bot_bottle.egress_addon_core import load_routes from bot_bottle.egress_addon_core import load_routes
b = _bottle([{"host": "github.com", "git": {"fetch": True}}]) b = _bottle([{"host": "github.com", "git": {"fetch": True}}])
+19
View File
@@ -269,6 +269,25 @@ class TestParseDlp(unittest.TestCase):
"dlp": {"wat": True}, "dlp": {"wat": True},
}]}) }]})
def test_outbound_on_match_default_empty(self):
routes = parse_routes({"routes": [{"host": "x.example"}]})
self.assertEqual("", routes[0].outbound_on_match)
def test_outbound_on_match_parsed(self):
for policy in ("block", "redact", "supervise"):
routes = parse_routes({"routes": [{
"host": "x.example",
"dlp": {"outbound_on_match": policy},
}]})
self.assertEqual(policy, routes[0].outbound_on_match)
def test_outbound_on_match_invalid_rejected(self):
with self.assertRaises(ValueError):
parse_routes({"routes": [{
"host": "x.example",
"dlp": {"outbound_on_match": "nope"},
}]})
# --- load_routes --------------------------------------------------------- # --- load_routes ---------------------------------------------------------
+18
View File
@@ -302,6 +302,24 @@ class TestDlp(unittest.TestCase):
"bogus": True, "bogus": True,
}}]) }}])
def test_outbound_on_match_omitted_is_empty(self):
b = _bottle([{"host": "x.example"}])
self.assertEqual("", b.egress.routes[0].OutboundOnMatch)
def test_outbound_on_match_accepts_policies(self):
for policy in ("block", "redact", "supervise"):
with self.subTest(policy=policy):
b = _bottle([{"host": "x.example", "dlp": {
"outbound_on_match": policy,
}}])
self.assertEqual(policy, b.egress.routes[0].OutboundOnMatch)
def test_outbound_on_match_rejects_unknown_value(self):
with self.assertRaises(ManifestError):
_bottle([{"host": "x.example", "dlp": {
"outbound_on_match": "allow",
}}])
class TestGitPolicy(unittest.TestCase): class TestGitPolicy(unittest.TestCase):
def test_omitted_means_https_git_fetch_disabled(self): def test_omitted_means_https_git_fetch_disabled(self):