Consolidated egress ran every bottle through one process but keyed the
supervise proposal queue off a single SUPERVISE_BOTTLE_SLUG env and kept
one *global* DLP safelist — so in the shared gateway an operator's token
approval for bottle A would (a) be attributed to the wrong bottle and
(b) leak into bottle B's DLP scan (A's approved secret passes B's egress).
This slice keys both per bottle, resolved by source IP.
- policy_resolver: add `resolve_policy_and_bottle_id` — policy + bottle id
in one `/resolve`, so egress keys routing *and* the supervise
queue/safelist from a single round-trip. Fail-closed (403 -> (None,None)).
- egress_addon_core: add `resolve_client_context` (+ `ContextResolverLike`)
returning `(Config, bottle_id)`, sharing the fail-closed parse with
`resolve_client_config` via `_config_from_policy`.
- egress_addon: `_active_config` -> `_resolve_flow` returns `(Config, slug)`;
`safe_tokens` set -> per-bottle `_safe_tokens_for(slug)`; the token-allow
write/await/archive + the approved-token add all use the resolved slug.
Single-tenant (no resolver) unchanged — slug = the env SUPERVISE_BOTTLE_SLUG.
New tests cover the resolver, the fail-closed context matrix, and the
cross-tenant isolation (an approval lands only in the calling bottle's
safelist; the proposal is keyed by the source-IP-attributed bottle;
unattributed IPs can't supervise).
Out of scope (noted): the git-gate gitleaks-allow hook + supervise_server
agent-proposal paths, and websocket DLP (still self.config-only, inert in
consolidated mode) — follow-up slices.
pyright 0 errors; pylint 9.83/10; unit suite green (1700 tests; the 13
test_sidecar_init /bin/sleep errors are pre-existing NixOS-local noise).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
Retire "sidecar" for the consolidated per-host path (PRD 0070 naming
decision): the orchestrator is the umbrella/control plane, and the
egress/git/supervise data-plane unit it runs is the "gateway".
- git mv sidecar.py -> gateway.py and the two integration + one unit test
files; DockerSidecar->DockerGateway, Sidecar->Gateway,
SidecarError->GatewayError, SIDECAR_*->GATEWAY_*, ensure_sidecar->
ensure_gateway, sidecar_status->gateway_status, container name
bot-bottle-orch-sidecar->bot-bottle-orch-gateway.
- Prose rename across broker/registry/egress/policy_resolver + PRD 0070.
- Preserved: the image name bot-bottle-sidecars, the
BOT_BOTTLE_SIDECAR_IMAGE env var, Dockerfile.sidecars, and PRD 0069's
own stage-name cross-references (that doc still uses "sidecar").
No behavior change. Full unit suite green (1679 tests; the 13
test_sidecar_init /bin/sleep errors are pre-existing NixOS-local noise).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
The egress addon now selects each request's Config by the calling bottle's
source IP, so one shared sidecar serves every bottle. Opt-in and fail-closed;
single-tenant behaviour is unchanged.
Orchestrator side (source-IP-primary attribution, per the PRD invariant):
* registry: `by_source_ip` (the single active bottle at a source IP —
network-layer attribution); `attribute` now composes it + the token.
* service: `resolve(source_ip, token="")` — with a token, strict
attribution; without, source IP alone.
* control_plane: `POST /resolve`'s identity_token is now OPTIONAL (absent
→ source-IP-only); split cleanly from the token-required `/attribute`.
* policy_resolver: `resolve` token now optional.
Egress side:
* egress_addon_core: `resolve_client_config(resolver, client_ip, token)` —
fetches + parses the client's Config, **fail-closed**: unattributed, a
resolver error, or an unparseable policy all yield deny-all (no routes).
Host-testable; `PolicyResolverLike` Protocol keeps it import-free.
* egress_addon: consolidated mode when `BOT_BOTTLE_ORCHESTRATOR_URL` is
set → `_active_config(flow)` resolves per client IP (reads + strips the
`x-bot-bottle-identity` header); `request()` uses it. Unset → the static
routes file, exactly as before. `PolicyResolver` added to the bundle.
Security note: source-IP-only resolution is safe where the IP is unspoofable
(Firecracker /31 + nft) AND the control plane is reachable only by the
trusted sidecar; the identity token, when the agent injects it, strengthens
it on weaker backends.
Scope note: the egress data plane is now multi-tenant. Remaining to be fully
live: the network topology routing every bottle's proxy to the one shared
sidecar, git-gate multitenancy, and agent-side identity-token injection.
Tests: registry by_source_ip; orchestrator resolve (with/without token);
control-plane /resolve token-optional; resolver token-optional;
resolve_client_config fail-closed matrix. All 182 egress tests still pass
(single-tenant unchanged). Full suite green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
_log_request and _log_response wrote headers and bodies to stderr verbatim.
_log_request also included the sidecar-injected upstream Authorization value,
exposing live bearer tokens on every allowed request under LOG_FULL.
Apply redact_tokens to all header values and bodies in both log functions;
exclude the authorization header from _log_request entirely since its value
is always a live sidecar-injected credential by the time _log_request runs.
Closes#257
A 403 "egress DLP: URL-encoded CRLF (%0d%0a)" was firing on legitimate
requests (e.g. the Claude Code login flow) and bypassing the on-match
policy entirely, because CRLF blocks carry no matched value and were
routed straight to a hard 403.
Root cause: CRLF injection is only an attack in the request line and
headers. An HTTP body is delimited by Content-Length, so CRLF bytes in
the body cannot split the request — but the scan flattened the body into
the same blob it checked, so form-encoded / multi-line body content
(which legitimately contains %0d%0a) tripped it.
Fix:
- scan_outbound takes a crlf_text param; the addon scans CRLF only over
the body-excluded request line + headers. crlf_text=None keeps the
old full-blob behavior for host-side callers/tests; the websocket path
passes "" since a data frame is not a request line.
- The redact policy now also scrubs CRLF (new strip_crlf helper) from the
path and headers, so redact is a complete escape hatch and structural
CRLF in the URL/headers can be forwarded when a route opts into it.
Tests: strip_crlf unit tests; scan_outbound crlf_text body-exclusion and
backward-compat tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HnvBjPZC5V7qeQpFbQdDmS
Give each egress route a policy for what the proxy does when an outbound
DLP detector matches a token, defaulting to the supervise flow added in
the previous commit. The goal is cutting false-positive friction without
weakening default-deny.
- redact: scrub the matched value(s) from the body, non-host headers, and
path/query via redact_tokens, then re-scan. Forward if clean; fail
closed with a 403 if a match remains on a surface redaction can't
rewrite (the hostname, or a unicode-evasion token). For routes where a
token-shaped value is noise the upstream doesn't need.
- block: the original hard 403, never overridable.
- supervise (default, unset): hold the request for operator approval.
Structural blocks (CRLF, no safelist-able value) stay hard 403s under
every policy.
Threads outbound_on_match from the bottle manifest (manifest_egress)
through the resolved EgressRoute and rendered routes.yaml (egress.py) to
the addon's Route (egress_addon_core), and round-trips it via the
list-egress-routes introspection endpoint. The allow/egress-block tool
descriptions document the new key.
Tests: manifest parse/validation, core parse/validation, full
manifest->render->addon round-trip for redact. README + PRD 0062 updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HnvBjPZC5V7qeQpFbQdDmS
When the outbound DLP catches a token, route the block through the
existing supervisor approval queue instead of returning 403 outright.
The egress proxy holds the request open until the operator answers, then
remembers an approved value for the life of the proxy so the request --
and later ones carrying it -- flow through. Fails closed on rejection,
timeout, malformed response, or when supervise is disabled.
- ScanResult.matched carries the raw matched substring (sidecar-only;
never logged or written to the proposal). scan_outbound and the token
detectors take a safe_tokens set and skip approved values, continuing
past a safelisted match so a second secret in the same request is
still caught.
- New egress-token-allow proposal tool, written directly to the queue by
the addon (the gitleaks-allow pattern from PRD 0061). build_token_allow
_payload renders host/method/path/detector reason + redacted context.
- Async request hook polls the queue without stalling the proxy event
loop; EGRESS_TOKEN_ALLOW_TIMEOUT_SECONDS (default 300) bounds the wait.
- Supervisor TUI renders egress-token-allow like gitleaks-allow: report
only, modify unavailable, approval requires a recorded reason.
- Unit tests for the matched/safe-tokens plumbing, payload builder, tool
constant round-trip, and TUI paths; README + PRD 0062.
Closes#261.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HnvBjPZC5V7qeQpFbQdDmS
- Set http_proxy/https_proxy (lowercase) alongside uppercase variants in smolmachines guest env for tools that only check lowercase
- Replace dataclasses.asdict with route_to_yaml_dict in /allowlist introspection so returned routes use YAML-schema-compatible keys
- Expand routes_yaml tool description in supervise_server to document all accepted route keys, making the round-trip from list-egress-routes to propose/apply explicit
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each DLP block/warn now reports where the match was found (body,
authorization header, response body) and includes a context snippet:
SNIPPET_CONTEXT chars before and after the match, with the matched
value replaced by REDACT ("********").
scan_token_patterns/scan_known_secrets/scan_naive_injection all gain
`location` and `context` fields on their ScanResult returns. The
outbound scanner takes `auth_header` as a separate kwarg so the two
locations are scanned and reported independently.
redact_tokens() is added to dlp_detectors and used in egress_addon.py
to scrub token patterns and provisioned secrets from host/path fields
before they appear in any log output (level 1 and 2).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Level 0 (off, default): no stderr output beyond boot line.
Level 1 (blocks): each block/warn emitted as JSON with reason and
request context (host, method, path, response_status for inbound).
Level 2 (full): level-1 events + egress_request and egress_response
JSON lines for every forwarded connection.
Block logging at level 1+ replaces the previous plain-text stderr write.
DLP warn logging is also gated on level 1+. All block call sites now pass
_req_ctx(flow) so the blocked request is visible in the log entry.
Boot message shows log level label (off/blocks/full).
Adds PRD 0053 documenting wire format, manifest format, and all log event
shapes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds a top-level `log: true` option to the egress config that logs the
full request (method, path, headers, body) and response (status, headers,
body) for every forwarded connection as JSON lines on stderr.
Wire format: `log: true` at the root of routes.yaml, parsed into the new
`Config` dataclass alongside `routes`. The sidecar addon switches from
`self.routes` to `self.config` and writes `_log_request` / `_log_response`
JSON lines when `self.config.log` is set.
Manifest: `egress.log: true` in bottle YAML flows through `EgressConfig.Log`
→ `Egress.prepare()` → `egress_render_routes(..., log=)` → routes.yaml.
`EgressPlan` also carries the flag for introspection.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove 35+ unused imports across 20+ files (W0611). Wrap 19 lines
to fit under 100 character limit (C0301). Add type casts and
annotations in egress_addon_core.py to resolve pyright errors
caused by JSON parsing of untyped objects.
Key changes:
- Remove unused imports (abstractmethod, mock utilities, etc)
- Split long lines at logical breaks (method calls, error messages)
- Add typing.cast() for proper type inference in JSON parsing
- Explicit type annotations for dict/list accesses
Results:
- Pylint rating: 8.73/10
- egress_addon_core.py: 0 pyright errors (was 15)
- All W0611 and C0301 issues fixed
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
didericis
didericis-codex
didericis-claude