Group the gateway's data-plane modules into three service sub-packages
mirroring the host-side trio (bot_bottle.egress / .supervisor / .git_gate):
gateway/egress/ addon_core, addon, dlp_config, dlp_detectors
gateway/supervisor/ server (was supervise_server)
gateway/git_gate/ render, http_backend
Prefix-stripped filenames now that the package namespaces them; each
sub-package has a thin docstring __init__ (no eager imports, cheap leaf
loads). The two cross-cutting files stay at the gateway root:
policy_resolver (shared per-client lookup) and gateway_init, renamed to
bootstrap now that gateway/ already namespaces it.
Updated all importers (bot_bottle + tests), the in-VM/container `-m`
launch strings, the Dockerfile.gateway addon shim + ENTRYPOINT, and the
five gateway entries in scripts/critical-modules.txt. Full unit suite
green (2243).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Separate the gateway (data plane) from the orchestrator (control plane) at the
module level. The gateway runtime files move out of the package root — and the
backend-neutral Gateway lifecycle ABC + GATEWAY_* constants move out of
orchestrator/ — into a new bot_bottle/gateway/ package:
gateway/__init__.py (was orchestrator/gateway.py: Gateway ABC + consts
+ rotate_gateway_ca)
gateway/gateway_init.py (the PID-1 daemon supervisor)
gateway/egress_addon.py, egress_addon_core.py, egress_dlp_config.py,
dlp_detectors.py (the egress mitmproxy daemon)
gateway/git_http_backend.py (the git-http daemon)
gateway/git_gate_render.py (the git-gate pre-receive rendering)
gateway/supervise_server.py (the supervise MCP daemon)
gateway/policy_resolver.py (the data-plane control-plane RPC client)
orchestrator/ now holds only control-plane files. The shared plan/types/auth
layer (egress.py=EgressPlan, git_gate.py=GitGatePlan, supervise.py,
supervise_types.py, control_auth.py) and the launch-time git-gate provisioning
helpers stay at root, so orchestrator/ and backend/ still own them.
Because these daemons are invoked as `python3 -m bot_bottle.<name>`, loaded flat
by mitmproxy, and referenced in Dockerfile.gateway, the move updates more than
Python imports: the `-m` invocations (firecracker/macOS infra scripts), the
Dockerfile.gateway addon shim + ENTRYPOINT, gateway_init's _DAEMONS module
paths, and the git-gate CGI heredocs all now point at bot_bottle.gateway.*.
No behavior change; full unit suite green (2251).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`/resolve` fail-closes on a missing/ambiguous registry row *and* on a
request whose identity token doesn't match. The message named only the
first, so a bottle that was registered correctly but sent no token read as
"not registered" and sent the reader looking for a deregistered bottle.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An unattributed bottle, an unreachable orchestrator, and an unparseable
policy all become a deny-all Config, and a deny-all is indistinguishable
from "policy loaded, host not allowed" at the decision point — both are just
"no matching route". So every one of them reported `host X is not in the
bottle's egress.routes allowlist`, which reads as a config problem and sends
the operator hunting for a route that was never missing. Diagnosing a
bricked registration cost hours for exactly this reason.
Carry the structural reason on Config and prefer it in decide(). A genuine
allowlist miss — a policy that loaded and simply lacks the host — keeps the
original wording, so the message now tells the two cases apart.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The cut-over dropped the per-bottle token flow, so an authed egress route on
the shared gateway failed with 'env var EGRESS_TOKEN_0 is unset' — the gateway
reads the token from its env, but a shared gateway has no per-bottle env.
Now the bottle's egress auth tokens travel to the gateway over /resolve and
the addon injects from them, mirroring what the per-bottle sidecar's env did:
- launch resolves the token values from the host env and hands them to the
orchestrator, which holds them IN MEMORY (keyed by bottle_id, never written
to the registry DB) and serves them on /resolve;
- PolicyResolver.resolve_policy_and_bottle_id + resolve_client_context now
return the token map alongside policy + bottle_id (one round-trip);
- the egress addon overlays the process env with the bottle's tokens per
request and uses that env for auth injection AND DLP — the agent never sees
the credential.
Secrets stay off disk (validated: /resolve returns the token, the registry DB
does not contain it). SecretProvider (#355) is the future hardening.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
Consolidated egress ran every bottle through one process but keyed the
supervise proposal queue off a single SUPERVISE_BOTTLE_SLUG env and kept
one *global* DLP safelist — so in the shared gateway an operator's token
approval for bottle A would (a) be attributed to the wrong bottle and
(b) leak into bottle B's DLP scan (A's approved secret passes B's egress).
This slice keys both per bottle, resolved by source IP.
- policy_resolver: add `resolve_policy_and_bottle_id` — policy + bottle id
in one `/resolve`, so egress keys routing *and* the supervise
queue/safelist from a single round-trip. Fail-closed (403 -> (None,None)).
- egress_addon_core: add `resolve_client_context` (+ `ContextResolverLike`)
returning `(Config, bottle_id)`, sharing the fail-closed parse with
`resolve_client_config` via `_config_from_policy`.
- egress_addon: `_active_config` -> `_resolve_flow` returns `(Config, slug)`;
`safe_tokens` set -> per-bottle `_safe_tokens_for(slug)`; the token-allow
write/await/archive + the approved-token add all use the resolved slug.
Single-tenant (no resolver) unchanged — slug = the env SUPERVISE_BOTTLE_SLUG.
New tests cover the resolver, the fail-closed context matrix, and the
cross-tenant isolation (an approval lands only in the calling bottle's
safelist; the proposal is keyed by the source-IP-attributed bottle;
unattributed IPs can't supervise).
Out of scope (noted): the git-gate gitleaks-allow hook + supervise_server
agent-proposal paths, and websocket DLP (still self.config-only, inert in
consolidated mode) — follow-up slices.
pyright 0 errors; pylint 9.83/10; unit suite green (1700 tests; the 13
test_sidecar_init /bin/sleep errors are pre-existing NixOS-local noise).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
The egress addon now selects each request's Config by the calling bottle's
source IP, so one shared sidecar serves every bottle. Opt-in and fail-closed;
single-tenant behaviour is unchanged.
Orchestrator side (source-IP-primary attribution, per the PRD invariant):
* registry: `by_source_ip` (the single active bottle at a source IP —
network-layer attribution); `attribute` now composes it + the token.
* service: `resolve(source_ip, token="")` — with a token, strict
attribution; without, source IP alone.
* control_plane: `POST /resolve`'s identity_token is now OPTIONAL (absent
→ source-IP-only); split cleanly from the token-required `/attribute`.
* policy_resolver: `resolve` token now optional.
Egress side:
* egress_addon_core: `resolve_client_config(resolver, client_ip, token)` —
fetches + parses the client's Config, **fail-closed**: unattributed, a
resolver error, or an unparseable policy all yield deny-all (no routes).
Host-testable; `PolicyResolverLike` Protocol keeps it import-free.
* egress_addon: consolidated mode when `BOT_BOTTLE_ORCHESTRATOR_URL` is
set → `_active_config(flow)` resolves per client IP (reads + strips the
`x-bot-bottle-identity` header); `request()` uses it. Unset → the static
routes file, exactly as before. `PolicyResolver` added to the bundle.
Security note: source-IP-only resolution is safe where the IP is unspoofable
(Firecracker /31 + nft) AND the control plane is reachable only by the
trusted sidecar; the identity token, when the agent injects it, strengthens
it on weaker backends.
Scope note: the egress data plane is now multi-tenant. Remaining to be fully
live: the network topology routing every bottle's proxy to the one shared
sidecar, git-gate multitenancy, and agent-side identity-token injection.
Tests: registry by_source_ip; orchestrator resolve (with/without token);
control-plane /resolve token-optional; resolver token-optional;
resolve_client_config fail-closed matrix. All 182 egress tests still pass
(single-tenant unchanged). Full suite green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
didericis