Compare commits

..

20 Commits

Author SHA1 Message Date
didericis d3e08cf039 feat(orchestrator+egress): slice 8 — multi-tenant egress via the resolver (#352)
lint / lint (push) Successful in 2m8s
test / unit (pull_request) Successful in 1m1s
test / integration (pull_request) Successful in 25s
test / coverage (pull_request) Successful in 1m25s
The egress addon now selects each request's Config by the calling bottle's
source IP, so one shared sidecar serves every bottle. Opt-in and fail-closed;
single-tenant behaviour is unchanged.

Orchestrator side (source-IP-primary attribution, per the PRD invariant):
  * registry: `by_source_ip` (the single active bottle at a source IP —
    network-layer attribution); `attribute` now composes it + the token.
  * service: `resolve(source_ip, token="")` — with a token, strict
    attribution; without, source IP alone.
  * control_plane: `POST /resolve`'s identity_token is now OPTIONAL (absent
    → source-IP-only); split cleanly from the token-required `/attribute`.
  * policy_resolver: `resolve` token now optional.

Egress side:
  * egress_addon_core: `resolve_client_config(resolver, client_ip, token)` —
    fetches + parses the client's Config, **fail-closed**: unattributed, a
    resolver error, or an unparseable policy all yield deny-all (no routes).
    Host-testable; `PolicyResolverLike` Protocol keeps it import-free.
  * egress_addon: consolidated mode when `BOT_BOTTLE_ORCHESTRATOR_URL` is
    set → `_active_config(flow)` resolves per client IP (reads + strips the
    `x-bot-bottle-identity` header); `request()` uses it. Unset → the static
    routes file, exactly as before. `PolicyResolver` added to the bundle.

Security note: source-IP-only resolution is safe where the IP is unspoofable
(Firecracker /31 + nft) AND the control plane is reachable only by the
trusted sidecar; the identity token, when the agent injects it, strengthens
it on weaker backends.

Scope note: the egress data plane is now multi-tenant. Remaining to be fully
live: the network topology routing every bottle's proxy to the one shared
sidecar, git-gate multitenancy, and agent-side identity-token injection.

Tests: registry by_source_ip; orchestrator resolve (with/without token);
control-plane /resolve token-optional; resolver token-optional;
resolve_client_config fail-closed matrix. All 182 egress tests still pass
(single-tenant unchanged). Full suite green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 17:48:46 -04:00
didericis de9da29027 refactor(orchestrator): drop the PolicyResolver cache (#352)
lint / lint (push) Successful in 2m0s
test / unit (pull_request) Successful in 1m1s
test / integration (pull_request) Successful in 20s
test / coverage (pull_request) Successful in 1m12s
Review: the resolver is called rarely enough that a round-trip doesn't
matter, and correctness beats speed — always fetching means a revocation,
policy change, or teardown the orchestrator knows about is honored
immediately instead of lingering for a cache TTL. Remove the TTL cache
(and `invalidate`); `resolve` now hits the orchestrator every call. Noted
in the docstring that any future caching should use orchestrator-driven
invalidation, not a blind TTL.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 17:32:05 -04:00
didericis 72ea500342 feat(orchestrator): slice 7 — sidecar-side PolicyResolver (#352)
lint / lint (push) Successful in 2m2s
test / unit (pull_request) Successful in 1m3s
test / integration (pull_request) Successful in 22s
test / coverage (pull_request) Successful in 1m9s
The data-plane bridge that lets the consolidated sidecar apply each
bottle's policy per request. `PolicyResolver` resolves a client's policy
from the orchestrator's `POST /resolve` keyed on (source_ip, identity
token) and caches it briefly (short TTL) so it isn't a round-trip per
request; `invalidate()` drops an entry on teardown / live reload.

Fail-closed: an unattributed client (orchestrator answers 403) resolves to
None so the caller denies; unreachable / unexpected status raises so the
caller can fail closed too rather than serve stale/empty policy. Stdlib
only and free of bot-bottle imports, so it can be COPYed flat into the
sidecar bundle.

Scope note: this is the sidecar-side *client*. Wiring it into the live
egress mitmproxy addon (select `Config` per client IP in the request path)
and git-gate, plus routing all bottles' egress to the one shared sidecar,
are the remaining data-plane pieces — a heavier change to the sidecar
bundle's adversarial-input code, taken next.

Tests: resolve returns/caches/expires/invalidates; 403 -> None (fail
closed); other HTTP status + unreachable raise; missing policy -> empty;
posts source_ip + identity_token. Full suite green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 17:21:27 -04:00
didericis f754c575d7 feat(orchestrator): slice 6 — source-IP-keyed multi-tenant policy (#352)
lint / lint (push) Successful in 2m1s
test / unit (pull_request) Successful in 1m5s
test / integration (pull_request) Successful in 21s
test / coverage (pull_request) Successful in 1m16s
The orchestrator side of the multi-tenant consolidated sidecar: hold each
bottle's sidecar policy and serve it by verified source IP, with live
reload. One shared sidecar can now get per-bottle config keyed on who's
calling.

  * registry.py — a `policy` column (migration v3, opaque JSON the sidecar
    interprets) on BottleRecord; `register(..., policy=)` stores it,
    `set_policy(bottle_id, policy)` updates it live, and `attribute` returns
    it (the source-IP-keyed resolution).
  * service.py — `launch_bottle(..., policy=)` and `set_policy`.
  * control_plane.py — `POST /bottles` accepts `policy`; `PUT
    /bottles/<id>/policy` live-reloads it; `POST /resolve` returns
    {bottle_id, policy} for a verified (source_ip, token) — the per-request
    call the multi-tenant sidecar makes; `/attribute` stays identity-only.

Scope note: this is the control-plane / state half. The data-plane half —
the egress mitmproxy addon (and git-gate) selecting allowlist / DLP /
token-injection per client IP by calling `/resolve` — is the next slice
(route agent bottles through the shared sidecar). The orchestrator stays
policy-agnostic: it stores and serves the blob verbatim.

Tests: registry policy store/update/persist; Orchestrator launch-with-policy
+ live set_policy; control-plane resolve returns policy (403 on bad token),
PUT policy updates / 404 / 400. Verified live over HTTP (launch -> resolve
-> PUT reload -> resolve reflects). Full suite green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 17:09:32 -04:00
didericis a092d00312 feat(orchestrator): slice 5 — build the consolidated sidecar bundle image (#352)
lint / lint (push) Successful in 2m7s
test / unit (pull_request) Successful in 1m5s
test / integration (pull_request) Successful in 20s
test / coverage (pull_request) Successful in 1m12s
Answers "where do we build the consolidated sidecar": nowhere, until now.

  * sidecar.py — `Sidecar.ensure_built()` (default no-op) + `DockerSidecar`
    now defaults its image to the real bundle (`bot-bottle-sidecars`) and
    `ensure_built()` builds it from `Dockerfile.sidecars` when
    `docker image inspect` shows it's missing (no-op when present or when no
    dockerfile is configured, e.g. a pre-pulled image). `image_exists()`
    added.
  * service.py — `ensure_sidecar()` now builds then runs.
  * __main__.py — `--sidecar` runs the consolidated bundle (build-if-missing).

Scope note: this builds + launches the bundle *container*; making the
running instance functional across bottles needs the per-bottle,
source-IP-keyed multi-tenant config + registration/reload, and routing
agent bottles to it — the next slices (added to PRD 0070's roadmap).

Tests: unit (docker mocked) — image_exists, ensure_built builds when
missing / no-op when present / no-op without a dockerfile / raises on build
failure; ensure_sidecar builds-then-runs; integration (gated, no heavy
build) — image_exists reflects real docker state. Full suite green (only
pre-existing /bin/sleep errors).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 17:00:18 -04:00
didericis 67652899eb refactor(orchestrator): move run_docker to a lean top-level docker_cmd (#352)
lint / lint (push) Successful in 2m2s
test / unit (pull_request) Successful in 1m0s
test / integration (pull_request) Successful in 18s
test / coverage (pull_request) Successful in 1m7s
Review feedback: don't bury a parallel docker util in the orchestrator.
But reusing backend.docker.util (docker_mod) isn't right either — importing
it runs backend/__init__.py, which eagerly loads all three backends
(docker + firecracker + macos) plus the manifest/egress/git-gate/supervise
framework (~76 modules), so every orchestrator import would drag the whole
backend layer in.

Compromise: promote the helper to a top-level, framework-free
bot_bottle/docker_cmd.py (single stdlib import), a proper shared home the
orchestrator's docker components use now and backend.docker.util can adopt
later. Verified `import bot_bottle.orchestrator` stays lean (12 modules, no
firecracker/macos backends).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 16:24:37 -04:00
didericis f85cbdeebf feat(orchestrator): slice 4 — consolidated per-host sidecar (#352)
lint / lint (push) Successful in 1m58s
test / unit (pull_request) Successful in 58s
test / integration (pull_request) Successful in 18s
test / coverage (pull_request) Successful in 1m7s
The core consolidation win: one persistent sidecar per host, shared by
every bottle, instead of a sidecar bundle per bottle. Safe to share
because the attribution invariant (source IP + identity token) lets the
sidecar map each request to the right bottle.

  * orchestrator/sidecar.py — a backend-neutral `Sidecar` lifecycle
    contract (mirrors LaunchBroker) + a `DockerSidecar` impl. The defining
    behaviour is idempotent singleton: `ensure_running` starts the instance
    if absent and is a no-op if it's already up, so N launches never spawn
    N sidecars; `stop` is idempotent.
  * orchestrator/dockerutil.py — a shared `run_docker` helper; DockerBroker
    now uses it too (DRY with slice 3).
  * service.py — the Orchestrator holds an optional `Sidecar`, exposes
    `ensure_sidecar()` + `sidecar_status()`.
  * control_plane.py — `GET /sidecar` reports it; __main__ gains
    `--sidecar-image` and ensures the single sidecar on startup.

Tests: unit (docker mocked) — is_running, ensure idempotent (no-op when up,
starts when absent), failure raises, stop idempotent; Orchestrator sidecar
wiring/status; control-plane /sidecar; integration (gated) — ensure is a
real idempotent singleton (one container after two ensures), stop removes.
Full suite green (only pre-existing /bin/sleep errors); integration
verified locally against real docker.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 15:28:29 -04:00
didericis 44e611d14e fix(orchestrator): pyright — pass explicit LaunchRequest in the docker integration test (#352)
lint / lint (push) Successful in 1m58s
test / unit (pull_request) Successful in 59s
test / integration (pull_request) Successful in 20s
test / coverage (pull_request) Successful in 1m6s
The **kw unpacking put str into slot: int|None. Pass explicit
LaunchRequests instead.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 15:05:55 -04:00
didericis 4fb0f64249 feat(orchestrator): slice 3 — real Docker launch broker (#352)
lint / lint (push) Failing after 1m59s
test / unit (pull_request) Successful in 58s
test / integration (pull_request) Successful in 22s
test / coverage (pull_request) Successful in 1m6s
The first concrete LaunchBroker, proving the orchestrator -> backend seam
on the cheapest backend (the sidecar bundle is already containers):

  * orchestrator/docker_broker.py — DockerBroker runs a container on a
    verified launch (`docker run --detach --name <bottle> --label ...
    <image_ref>`) and removes it on teardown (`docker rm --force`,
    idempotent on an already-absent container). The argv is built only from
    the request's static ids/flags, so nothing free-form reaches docker;
    provenance/schema verification is inherited from LaunchBroker.submit.
  * __main__.py gains `--broker {stub,docker}` so the harness can drive real
    containers.

Slice 3 launches a single container from image_ref (the seam); the full
agent + sidecar bundle is a later slice.

Tests: unit (docker mocked) — argv from static fields, launch/teardown call
the right commands, missing-image and docker-failure raise, teardown
idempotent on missing, forged token never touches docker; integration
(gated on a reachable daemon) — launch creates a real container, teardown
removes it. Full suite green (only pre-existing /bin/sleep errors);
integration verified locally against real docker.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 15:02:11 -04:00
didericis 9226d45041 feat(orchestrator): slice 2 — launch lifecycle + signed launch-broker (#352)
lint / lint (push) Successful in 2m2s
test / unit (pull_request) Successful in 57s
test / integration (pull_request) Successful in 18s
test / coverage (pull_request) Successful in 1m6s
Second slice of PRD 0070, still a backend-neutral dev-harness:

  * orchestrator/broker.py — the launch-broker contract. A LaunchRequest is
    structured (static ids/flags only — bottle id, pool slot, a
    content-addressed image_ref; never a path/argv) and signed as a compact
    HS256 JWT so the broker verifies PROVENANCE before acting: a compromised
    co-located component can't forge a launch without the shared secret.
    verify_request is fail-closed (bad sig / malformed / off-schema -> raise).
    Stdlib only (no runtime deps). Ships a StubBroker that records verified
    requests for the harness/tests.
  * orchestrator/service.py — the Orchestrator: owns the registry and brokers
    the lifecycle. launch_bottle mints the bottle + sends a signed launch,
    rolling the registry entry back if the launch fails (no orphans);
    teardown_bottle brokers teardown then deregisters; attribute delegates.
  * control_plane.py — POST /bottles now launches, DELETE tears down (both go
    through the Orchestrator + broker). dispatch/server take an Orchestrator.
  * __main__.py wires an ephemeral secret + StubBroker for the harness.

Tests: broker sign/verify round-trip, tamper/wrong-secret/malformed/off-schema
rejection, StubBroker fail-closed; Orchestrator launch->registry->attribute,
teardown, rollback-on-broker-failure; control-plane updated for launch/teardown.
Full suite green (only the pre-existing /bin/sleep errors); harness does
launch -> attribute -> teardown over HTTP.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 14:42:35 -04:00
didericis 6847fdf0ab refactor: drop the vestigial bot_bottle_root/host_db_path re-exports (#352)
lint / lint (push) Successful in 1m58s
test / unit (pull_request) Successful in 1m0s
test / integration (pull_request) Successful in 19s
test / coverage (pull_request) Successful in 1m6s
paths is the single home now, so stop re-exporting the path helpers from
supervise: remove host_db_path from supervise's imports + __all__ (it was
re-export-only) and drop bot_bottle_root from __all__ (kept as an import,
still used by audit_dir). supervise_types was already clean. Repoint the
last readers (test_supervise imports host_db_path from paths;
test_supervise_edge calls paths.bot_bottle_root) and refresh the doc
mentions. No supervise.bot_bottle_root / supervise.host_db_path references
remain.

Behavior-preserving: full unit suite unchanged (only the pre-existing
/bin/sleep sidecar-init errors).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 14:30:13 -04:00
didericis 421d31c32f refactor: move bot_bottle_root/host_db_path to paths; kill the monkeypatch (#352)
lint / lint (push) Successful in 1m57s
test / unit (pull_request) Successful in 55s
test / integration (pull_request) Successful in 15s
test / coverage (pull_request) Successful in 1m1s
Create bot_bottle/paths.py as the canonical home for the app-root path
helpers (bot_bottle_root, host_db_path, HOST_DB_FILENAME) — foundational,
not supervise- or db-specific. `bot_bottle_root()` now honours a
BOT_BOTTLE_ROOT env override.

Repoint every consumer (supervise, supervise_types, db_store, queue_store,
audit_store, store_manager, bottle_state, cli/supervise, docker/cleanup,
orchestrator/registry) at paths; remove the definitions (and supervise's
duplicate host_db_path) and the now-dead `import sys`. Add paths.py to the
sidecar bundle (Dockerfile.sidecars) for the flat-import copies.

Tests: replace ~12 files' monkeypatching of supervise.bot_bottle_root (and
the flat/pkg/supervise_types triple-patch dance) with a single
`use_bottle_root()` helper that sets BOT_BOTTLE_ROOT — every module and
flat/package copy reads the same env var, so one override covers them all.
Net -97 lines. Behaviour-preserving: full unit suite unchanged (only the
pre-existing /bin/sleep sidecar-init errors remain).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 14:04:23 -04:00
didericis b4b4e08f62 refactor: move host_db_path/HOST_DB_FILENAME to db_store (#352)
lint / lint (push) Successful in 1m58s
test / unit (pull_request) Successful in 57s
test / integration (pull_request) Successful in 16s
test / coverage (pull_request) Successful in 1m0s
host_db_path is shared DB infrastructure, not supervise-specific, so its
canonical home is db_store (alongside DbStore). It resolves bot_bottle_root
from supervise_types lazily inside the function — no load-time cycle, and a
monkey-patch of supervise_types.bot_bottle_root still propagates.
supervise_types re-exports both names for the historical import path
(queue_store/audit_store unchanged); the orchestrator registry now imports
from db_store. Drops the now-unused `import sys` from supervise_types.

Behavior-preserving: full unit suite unchanged (only the pre-existing
/bin/sleep sidecar-init errors remain); monkeypatch propagation verified.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 13:36:19 -04:00
didericis b174442c60 feat(orchestrator): registry co-tenants the shared bot-bottle.db (#352)
lint / lint (push) Successful in 1m57s
test / unit (pull_request) Successful in 59s
test / integration (pull_request) Successful in 18s
test / coverage (pull_request) Successful in 1m5s
Per review: use one shared bot-bottle.db for all runtime state including
the registry (DbStore namespaces by schema_key), so it's one queryable
file for backup/console. default_db_path() -> host_db_path(). Drop the
unilateral WAL flip — WAL on the shared DB affects supervise/audit and is
finicky over guest shares, so it's a deliberate future change; keep a
busy_timeout for lock contention.

PRD State section updated: integrity now by SOLE ownership (only the
orchestrator opens bot-bottle.db; data plane + console reach state via the
control-plane RPC, never a file handle) rather than ro/rw mount-splitting,
which one shared file can't do. Notes the transitional caveat that the
supervise sidecar currently rw-mounts bot-bottle.db.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 13:26:03 -04:00
didericis ed7307e0e3 docs(prd): 0070 registry DB is host-resident with rw/ro access split (#352)
test / unit (pull_request) Successful in 55s
test / integration (pull_request) Successful in 16s
test / coverage (pull_request) Successful in 1m1s
Per review: the SQLite runtime-state DB lives on the host, not owned
inside the orchestrator unit. Durability (re-adoption must survive an
orchestrator restart) + integrity via access-scoping (control-plane rw,
data-plane ro) rather than location. Note the WAL-over-guest-share wrinkle
for the VM slices (may want a host-side DB owner reached over the RPC).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 13:17:05 -04:00
didericis 1702664b81 feat(orchestrator): slice 1 — registry + attribution + HTTP control plane (#352)
lint / lint (push) Successful in 1m58s
test / unit (pull_request) Successful in 56s
test / integration (pull_request) Successful in 17s
test / coverage (pull_request) Successful in 1m2s
First implementation slice of PRD 0070, the backend-neutral consolidation
core as a plain-process dev-harness (no VM packaging yet):

  * orchestrator/registry.py — SQLite (WAL) runtime-state store on the
    existing DbStore/TableMigrations base. Live bottle registry keyed by
    source IP + per-bottle identity token, with fail-closed attribution:
    a request resolves to a bottle only when its source IP AND identity
    token both match exactly one active record (unknown/ambiguous IP,
    empty token, or token mismatch all deny). Tokens are 256-bit urandom.
  * orchestrator/control_plane.py — the HTTP control plane (the universal
    transport chosen in 0070): register / deregister / list / attribute /
    health. Routing is a pure dispatch() so it is socket-free testable;
    Handler/ControlPlaneServer/make_server are a thin stdlib adapter.
    register/deregister are the live-reload path; listing redacts tokens.
  * orchestrator/__main__.py — `python -m bot_bottle.orchestrator` harness.

Launch/teardown, the launch broker, and the egress/git/supervise data
plane come in later slices. 24 unit tests (attribution matrix, persistence,
dispatch, one real-socket round-trip).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 13:00:04 -04:00
didericis 8d54fc38ea docs(prd): 0070 VM-to-VM routing is not a blocker (#352)
Per review: resolvable at implementation time (host tweak acceptable, as
the pool setup already needs on NixOS); the earlier sidecars-in-VMs spike
showed feasibility. No need to hunt the spike now.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 12:46:08 -04:00
didericis 73a7582abe docs(prd): 0070 fold in review decisions (#352)
Resolve open questions from review: consolidate egress (hardened minimal
surface + identity token + vault mitigations); HTTP control-plane
transport; broker schema = signed-JWT JSON of static flags+ids (provenance
+ un-coercible); state re-adoption procedure (singleton orchestrator →
wait-healthy → adopt via SQLite + process inspection before serving, with
write-ahead intent to close the in-flight-launch race). Add a per-bottle
identity-token defense-in-depth layer on the attribution invariant.
Remaining open: VM-to-VM routing (per-backend wire(), pending spike link),
live-reload protocol, identity-token delivery.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 12:39:06 -04:00
didericis 095896817c docs(prd): 0070 secret-handling as a future pattern (SecretProvider, #355)
lint / lint (push) Successful in 2m3s
Add a "Secret handling — FUTURE pattern (not v1)" subsection: vault as a
separate trust domain holding long-lived roots, deriving short-lived
scoped creds where the upstream allows (with the honest limit that a
compromised proxy can still abuse currently-authorized access). The
mechanism is a generic, user-extensible SecretProvider that generalizes
PRD 0048's DeployKeyProvisioner and drops into the manifest wherever a raw
token is accepted — discovered from ~/.bot-bottle/contrib like user
AgentProviders. Marked explicitly as not required for the initial
orchestrator; tracked as #355.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 12:14:25 -04:00
didericis a30dd49967 docs(prd): 0070 per-host orchestrator service
Fold the per-bottle sidecar bundle into a single persistent per-host
orchestrator: runs the sidecar functions (egress/git-gate/supervise),
coordinates with the console, and brokers agent launches. Virtualized
from the start with backend-native isolation (fc VM / apple ctr / docker
ctr), fronted by a single backend-agnostic contract; per-backend
variation lives on BottleBackend, not a parallel Orchestrator hierarchy.

Leads with the security review (secret concentration, shared fate, the
launch-broker-as-new-privileged-core, and the source-IP attribution
invariant each backend must enforce). Proposes one SQLite DB owned by the
orchestrator for runtime state (slot leases, approvals, registry) —
distinct from build-time constants (flat .env) and user config
(declarative ~/.bot-bottle). Sequences docker -> firecracker -> macos,
developing the service as a plain-process dev-harness before the VM.

Supersedes 0069's Stage-1/4 sidecar framing; depends on 0069's nix-built
fixed images. Tracking issue #351.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 12:14:25 -04:00
119 changed files with 3474 additions and 3950 deletions
+3 -3
View File
@@ -8,10 +8,10 @@ broad permissions inside a sandbox, so a misbehaving agent cannot reach the
host. A Python CLI (entry point `cli.py`, package `bot_bottle/`) orchestrates host. A Python CLI (entry point `cli.py`, package `bot_bottle/`) orchestrates
the runtime lifecycle and the copying of skills and env vars into it. the runtime lifecycle and the copying of skills and env vars into it.
The default backend on compatible macOS hosts is macos-container: The default backend on compatible macOS hosts is macos-container:
agents and gateways run through Apple's `container` CLI without agents and sidecar bundles run through Apple's `container` CLI without
requiring Docker. On KVM-capable Linux hosts the default is firecracker: requiring Docker. On KVM-capable Linux hosts the default is firecracker:
agents run in a Firecracker microVM reached over SSH on a point-to-point agents run in a Firecracker microVM reached over SSH on a point-to-point
TAP, while the gateway still uses Docker. The legacy Docker TAP, while the sidecar bundle still uses Docker. The legacy Docker
backend remains available with `BOT_BOTTLE_BACKEND=docker` or backend remains available with `BOT_BOTTLE_BACKEND=docker` or
`--backend=docker`. `--backend=docker`.
@@ -59,7 +59,7 @@ backend remains available with `BOT_BOTTLE_BACKEND=docker` or
in a PRD, research note, or decision record. in a PRD, research note, or decision record.
- Low dependencies by default. The project is Python, stdlib-first (no - Low dependencies by default. The project is Python, stdlib-first (no
runtime pip dependencies in the package itself; the only language runtime pip dependencies in the package itself; the only language
runtime is the Python 3.13 used by the CLI + companion containers). Ask before runtime is the Python 3.13 used by the CLI + sidecars). Ask before
adding new tools, runtimes, or package managers. adding new tools, runtimes, or package managers.
- Commit messages follow [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/): - Commit messages follow [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/):
`<type>[(scope)][!]: <description>`, where `<type>` is one of `feat`, `fix`, `<type>[(scope)][!]: <description>`, where `<type>` is one of `feat`, `fix`,
-27
View File
@@ -1,27 +0,0 @@
# Orchestrator control-plane image (PRD 0070, #384).
#
# The per-host orchestrator runs `python3 -m bot_bottle.orchestrator`.
# The `bot_bottle` package is **stdlib-only** by design, so the control
# plane needs nothing but a Python runtime — none of the gateway's
# mitmproxy / git / gitleaks payload (that is the separate
# `bot-bottle-gateway` image, Dockerfile.gateway). Splitting them keeps
# the secret-dense control plane (it concentrates every bottle's egress
# tokens — see PRD 0070's "secret concentration") on a minimal
# dependency surface.
#
# The repo is bind-mounted read-only into the container at run time (see
# `orchestrator/lifecycle.py`), so the source is NOT copied in here: the
# image is just the runtime. `ensure_running` recreates the container
# only when the bind-mounted source hash changes (#381), which is why
# the code stays a mount rather than a baked layer.
FROM python:3.12-slim
# No third-party deps to install — stdlib only. Kept as an explicit,
# self-documenting stage so a future confinement step (baking the
# package, dropping the bind mount) has an obvious home.
WORKDIR /app
# Documentation only; lifecycle.py overrides the entrypoint to
# `python3 -m bot_bottle.orchestrator` with the runtime flags.
ENTRYPOINT ["python3", "-m", "bot_bottle.orchestrator"]
+6 -13
View File
@@ -1,15 +1,8 @@
# Gateway data-plane image (PRD 0024 bundle shape; PRD 0070 gateway). # Per-bottle sidecar bundle image (PRD 0024).
# #
# The egress / git-gate / supervise *data plane* — one image, run by # Collapses the prior per-sidecar images (egress, git-gate,
# the consolidated per-host gateway (PRD 0070). It is NOT the
# orchestrator control plane: that is the separate, lean
# `bot-bottle-orchestrator` image (Dockerfile.orchestrator, #384), which
# ships only python + the stdlib-only `bot_bottle` package and none of
# this image's mitmproxy / git / gitleaks payload.
#
# Collapses the prior per-daemon images (egress, git-gate,
# supervise) into one. A small stdlib-Python init supervisor at # supervise) into one. A small stdlib-Python init supervisor at
# /app/gateway_init.py spawns all daemons, forwards SIGTERM, and # /app/sidecar_init.py spawns all daemons, forwards SIGTERM, and
# propagates per-daemon stdout/stderr to the container log with a # propagates per-daemon stdout/stderr to the container log with a
# `[name]` prefix. See PRD 0024 for the rationale. # `[name]` prefix. See PRD 0024 for the rationale.
# #
@@ -19,7 +12,7 @@
# /app/egress_addon.py + siblings mitmproxy addon (egress) # /app/egress_addon.py + siblings mitmproxy addon (egress)
# /app/egress-entrypoint.sh mitmdump launcher # /app/egress-entrypoint.sh mitmdump launcher
# /app/supervise_server.py + .py supervise MCP server # /app/supervise_server.py + .py supervise MCP server
# /app/gateway_init.py PID 1 supervisor # /app/sidecar_init.py PID 1 supervisor
# /etc/egress/routes.yaml bind-mounted at run time # /etc/egress/routes.yaml bind-mounted at run time
# /etc/git-gate/pre-receive docker-cp'd at start time # /etc/git-gate/pre-receive docker-cp'd at start time
# /git-gate-entrypoint.sh docker-cp'd at start time # /git-gate-entrypoint.sh docker-cp'd at start time
@@ -83,7 +76,7 @@ COPY bot_bottle/audit_store.py /app/audit_store.py
COPY bot_bottle/store_manager.py /app/store_manager.py COPY bot_bottle/store_manager.py /app/store_manager.py
COPY bot_bottle/supervise.py /app/supervise.py COPY bot_bottle/supervise.py /app/supervise.py
COPY bot_bottle/supervise_server.py /app/supervise_server.py COPY bot_bottle/supervise_server.py /app/supervise_server.py
COPY bot_bottle/gateway_init.py /app/gateway_init.py COPY bot_bottle/sidecar_init.py /app/sidecar_init.py
COPY bot_bottle/git_http_backend.py /app/git_http_backend.py COPY bot_bottle/git_http_backend.py /app/git_http_backend.py
COPY bot_bottle/egress_entrypoint.sh /app/egress-entrypoint.sh COPY bot_bottle/egress_entrypoint.sh /app/egress-entrypoint.sh
RUN chmod +x /app/egress-entrypoint.sh RUN chmod +x /app/egress-entrypoint.sh
@@ -109,4 +102,4 @@ WORKDIR /app
# PID 1 is the supervisor. It owns signal handling and exit-code # PID 1 is the supervisor. It owns signal handling and exit-code
# propagation; no `exec` chain in the entrypoint itself. # propagation; no `exec` chain in the entrypoint itself.
ENTRYPOINT ["python3", "/app/gateway_init.py"] ENTRYPOINT ["python3", "/app/sidecar_init.py"]
+10 -10
View File
@@ -5,7 +5,7 @@
# bot-bottle # bot-bottle
[![test](https://gitea.dideric.is/didericis/bot-bottle/actions/workflows/test.yml/badge.svg?branch=main)](https://gitea.dideric.is/didericis/bot-bottle/actions?workflow=test.yml) [![test](https://gitea.dideric.is/didericis/bot-bottle/actions/workflows/test.yml/badge.svg?branch=main)](https://gitea.dideric.is/didericis/bot-bottle/actions?workflow=test.yml)
[![coverage](https://img.shields.io/badge/coverage-84%25-brightgreen)](https://coverage.readthedocs.io/) [![coverage](https://img.shields.io/badge/coverage-82%25-brightgreen)](https://coverage.readthedocs.io/)
[![core coverage](https://img.shields.io/badge/core%20coverage-95%25-brightgreen)](https://gitea.dideric.is/didericis/bot-bottle/src/branch/main/docs/decisions/0004-coverage-policy.md) [![core coverage](https://img.shields.io/badge/core%20coverage-95%25-brightgreen)](https://gitea.dideric.is/didericis/bot-bottle/src/branch/main/docs/decisions/0004-coverage-policy.md)
**Problem:** Developer wants to run a coding agent without supervision, but they don't want a prompt injected or misbehaving agent wrecking their environment or exfiltrating sensitive data. **Problem:** Developer wants to run a coding agent without supervision, but they don't want a prompt injected or misbehaving agent wrecking their environment or exfiltrating sensitive data.
@@ -16,7 +16,7 @@
- **Per-bottle egress allowlist** — TLS-bumped HTTP/HTTPS chokepoint with a per-manifest host allowlist; per-route path/method/header `matches` filtering; outbound DLP scanning for known tokens and secrets, inbound DLP scanning for prompt-injection attempts; DoH and arbitrary hosts blocked by default. - **Per-bottle egress allowlist** — TLS-bumped HTTP/HTTPS chokepoint with a per-manifest host allowlist; per-route path/method/header `matches` filtering; outbound DLP scanning for known tokens and secrets, inbound DLP scanning for prompt-injection attempts; DoH and arbitrary hosts blocked by default.
- **Per-route token-match policy** — each egress route picks what happens when the outbound DLP catches a token via `dlp.outbound_on_match`: `supervise` (default) holds the request and surfaces it in `./cli.py supervise` for approval (an approved value is remembered for the life of the proxy); `redact` scrubs the value and forwards; `block` is a hard `403`. Cuts false-positive friction without weakening default-deny. - **Per-route token-match policy** — each egress route picks what happens when the outbound DLP catches a token via `dlp.outbound_on_match`: `supervise` (default) holds the request and surfaces it in `./cli.py supervise` for approval (an approved value is remembered for the life of the proxy); `redact` scrubs the value and forwards; `block` is a hard `403`. Cuts false-positive friction without weakening default-deny.
- **Tokens the agent never sees** — host secrets live in a gateway; the agent dials `http://gateway:9099/<path>` and the proxy strips inbound `Authorization` and injects the real token before forwarding. `printenv` in the agent shows proxy URLs only. - **Tokens the agent never sees** — host secrets live in a sidecar; the agent dials `http://sidecar:9099/<path>` and the proxy strips inbound `Authorization` and injects the real token before forwarding. `printenv` in the agent shows proxy URLs only.
- **Gitleaks-scanned push (git-gate)** — `bottle.git` remotes route through a per-bottle `git daemon` that gitleaks-scans incoming refs pre-receive and forwards clean refs upstream over SSH. The agent never holds the upstream credential. - **Gitleaks-scanned push (git-gate)** — `bottle.git` remotes route through a per-bottle `git daemon` that gitleaks-scans incoming refs pre-receive and forwards clean refs upstream over SSH. The agent never holds the upstream credential.
- **Manifest-scoped skills + secrets** — each bottle declares its skills, env, git identity, remotes, and egress routes; unknown keys die at load. - **Manifest-scoped skills + secrets** — each bottle declares its skills, env, git identity, remotes, and egress routes; unknown keys die at load.
- **Trust boundary at `$HOME`** — bottles (credentials, egress, remotes) live only under `~/.bot-bottle/bottles/`. Repos may ship agents but not bottles, so a cloned repo can't redirect an env var to an attacker host. - **Trust boundary at `$HOME`** — bottles (credentials, egress, remotes) live only under `~/.bot-bottle/bottles/`. Repos may ship agents but not bottles, so a cloned repo can't redirect an env var to an attacker host.
@@ -24,17 +24,17 @@
- **Parallel, isolated bottles** — each bottle runs in its own backend-owned isolation boundary; bottles don't share state or talk to each other. - **Parallel, isolated bottles** — each bottle runs in its own backend-owned isolation boundary; bottles don't share state or talk to each other.
- **Provider templates (Claude, Codex)** — `Dockerfile.claude` / `Dockerfile.codex`, or a bottle-supplied Dockerfile. Claude auth via long-lived OAuth token; Codex via opt-in host device-auth forwarding. - **Provider templates (Claude, Codex)** — `Dockerfile.claude` / `Dockerfile.codex`, or a bottle-supplied Dockerfile. Claude auth via long-lived OAuth token; Codex via opt-in host device-auth forwarding.
- **gVisor auto-detect** — on Linux hosts where `runsc` is registered with Docker, every bottle launches under it for a userspace syscall barrier; no manifest config required. - **gVisor auto-detect** — on Linux hosts where `runsc` is registered with Docker, every bottle launches under it for a userspace syscall barrier; no manifest config required.
- **Apple Container backend (macOS default when available)** — runs the agent and gateway with Apple's `container` CLI, using a host-only agent network plus a separate gateway egress network. - **Apple Container backend (macOS default when available)** — runs the agent and sidecar bundle with Apple's `container` CLI, using a host-only agent network plus a separate sidecar egress network.
- **Firecracker backend (Linux default when available)** — runs the agent in a KVM Firecracker microVM reached over SSH on a point-to-point TAP, with the gateway in Docker. A dedicated, fail-closed `nftables` table isolates the guest, closing the raw DNS/IP exfiltration gap that exists in the legacy Docker backend. Requires KVM (`/dev/kvm`) and a one-time privileged network-pool setup. - **Firecracker backend (Linux default when available)** — runs the agent in a KVM Firecracker microVM reached over SSH on a point-to-point TAP, with the sidecar bundle in Docker. A dedicated, fail-closed `nftables` table isolates the guest, closing the raw DNS/IP exfiltration gap that exists in the legacy Docker backend. Requires KVM (`/dev/kvm`) and a one-time privileged network-pool setup.
- **Legacy Docker backend** — still available for examples, CI, and hosts without Apple Container or KVM via `BOT_BOTTLE_BACKEND=docker` or `--backend=docker`. - **Legacy Docker backend** — still available for examples, CI, and hosts without Apple Container or KVM via `BOT_BOTTLE_BACKEND=docker` or `--backend=docker`.
## Architecture ## Architecture
On the default macOS Apple Container backend, a bottle is an agent container on a host-only internal network plus a gateway attached to both that internal network and a NAT egress network. The agent gets HTTP(S)_PROXY and CA bundle env vars pointing at the gateway's internal-network IP, so HTTP/HTTPS traffic flows through the gateway instead of direct egress. `bottle.git` / git-gate is intentionally deferred on this backend until a safe Apple Container key-delivery path exists. On the default macOS Apple Container backend, a bottle is an agent container on a host-only internal network plus a sidecar bundle attached to both that internal network and a NAT egress network. The agent gets HTTP(S)_PROXY and CA bundle env vars pointing at the sidecar's internal-network IP, so HTTP/HTTPS traffic flows through the sidecar instead of direct egress. `bottle.git` / git-gate is intentionally deferred on this backend until a safe Apple Container key-delivery path exists.
On the Firecracker backend, a bottle is an agent microVM plus a Docker gateway for egress, git-gate, and supervise. The VM reaches the gateway over a per-bottle point-to-point TAP link; a dedicated fail-closed `nftables` table (`inet bot_bottle_fc`) confines the guest to that link, so nothing leaves the box except through the gateway. The TAP pool and nft table are provisioned once (root); per-launch needs no privilege. On the Firecracker backend, a bottle is an agent microVM plus a Docker sidecar bundle for egress, git-gate, and supervise. The VM reaches the sidecars over a per-bottle point-to-point TAP link; a dedicated fail-closed `nftables` table (`inet bot_bottle_fc`) confines the guest to that link, so nothing leaves the box except through the sidecars. The TAP pool and nft table are provisioned once (root); per-launch needs no privilege.
On the legacy Docker backend, the same logical bottle is two containers per agent: an `agent` container and a `companion containers` container. They share a per-agent Docker `--internal` network; the agent has no default route off-box. On the legacy Docker backend, the same logical bottle is two containers per agent: an `agent` container and a `sidecars` container. They share a per-agent Docker `--internal` network; the agent has no default route off-box.
The Docker topology looks like this: The Docker topology looks like this:
@@ -67,11 +67,11 @@ The Docker topology looks like this:
└─────────────────────────────────────────────────────────────────────┘ └─────────────────────────────────────────────────────────────────────┘
``` ```
When the agent exits, `cli.py` tears down every gateway and both networks; nothing about a bottle persists between runs. When the agent exits, `cli.py` tears down every sidecar and both networks; nothing about a bottle persists between runs.
## Quickstart ## Quickstart
On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The Firecracker backend (Linux) requires Docker on the host for the gateway plus the `firecracker` binary and KVM. The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`. On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The Firecracker backend (Linux) requires Docker on the host for the sidecar bundle plus the `firecracker` binary and KVM. The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
Use `BOT_BOTTLE_BACKEND=docker ./cli.py start <agent>` on hosts where neither Apple Container nor KVM is available and Docker is the desired backend. Use `BOT_BOTTLE_BACKEND=docker ./cli.py start <agent>` on hosts where neither Apple Container nor KVM is available and Docker is the desired backend.
@@ -81,7 +81,7 @@ On Linux, a KVM-capable host defaults to the Firecracker backend. It needs:
- **`/dev/kvm`** present and accessible. Load `kvm-intel` or `kvm-amd` (and enable virtualization in BIOS/firmware). The invoking user must be in the `kvm` group: `sudo usermod -aG kvm "$USER"` then re-login. bot-bottle preflights this and reports exactly what's missing. - **`/dev/kvm`** present and accessible. Load `kvm-intel` or `kvm-amd` (and enable virtualization in BIOS/firmware). The invoking user must be in the `kvm` group: `sudo usermod -aG kvm "$USER"` then re-login. bot-bottle preflights this and reports exactly what's missing.
- **`firecracker`** on `PATH`: grab a release from <https://github.com/firecracker-microvm/firecracker/releases>. Start flows print this pointer when the binary is missing. - **`firecracker`** on `PATH`: grab a release from <https://github.com/firecracker-microvm/firecracker/releases>. Start flows print this pointer when the binary is missing.
- **Docker** for the gateway and image build. - **Docker** for the sidecar bundle and image build.
- **A one-time privileged network setup** — the per-bottle TAP pool plus the fail-closed `nftables` isolation table. Run `./cli.py backend setup --backend=firecracker` for the host-appropriate config (a NixOS module, a `sudo` script elsewhere); `./cli.py backend status --backend=firecracker` reports what's present, including whether the pool range collides with an existing route. The pool defaults to `10.243.0.0/16` (an obscure RFC-1918 block that dodges docker/libvirt/LAN and, deliberately, Tailscale's `100.64.0.0/10` CGNAT range); override with `BOT_BOTTLE_FC_IP_BASE` if it clashes on your host. - **A one-time privileged network setup** — the per-bottle TAP pool plus the fail-closed `nftables` isolation table. Run `./cli.py backend setup --backend=firecracker` for the host-appropriate config (a NixOS module, a `sudo` script elsewhere); `./cli.py backend status --backend=firecracker` reports what's present, including whether the pool range collides with an existing route. The pool defaults to `10.243.0.0/16` (an obscure RFC-1918 block that dodges docker/libvirt/LAN and, deliberately, Tailscale's `100.64.0.0/10` CGNAT range); override with `BOT_BOTTLE_FC_IP_BASE` if it clashes on your host.
```sh ```sh
+2 -2
View File
@@ -204,9 +204,9 @@ class AgentProvider(ABC):
bottle: "Bottle", bottle: "Bottle",
supervise_url: str, supervise_url: str,
) -> None: ) -> None:
"""Register the per-bottle supervise daemon as an MCP server """Register the per-bottle supervise sidecar as an MCP server
in the provider's in-guest config. Called by the backend after in the provider's in-guest config. Called by the backend after
the supervise daemon is reachable. No-op when the supervise sidecar is reachable. No-op when
`plan.supervise_plan is None`.""" `plan.supervise_plan is None`."""
@abstractmethod @abstractmethod
+5 -5
View File
@@ -199,7 +199,7 @@ class ActiveAgent:
bottle is the container, the agent is what runs in it.) bottle is the container, the agent is what runs in it.)
Fields are deliberately backend-neutral. `services` is the set Fields are deliberately backend-neutral. `services` is the set
of gateway daemons currently up for this bottle (`egress`, of sidecar daemons currently up for this bottle (`egress`,
`git-gate`, `supervise`); the dashboard uses it to `git-gate`, `supervise`); the dashboard uses it to
gate edit verbs. `backend_name` is the matching key in gate edit verbs. `backend_name` is the matching key in
`_BACKENDS` (`docker` / `firecracker` / `macos-container`) — used by the active- `_BACKENDS` (`docker` / `firecracker` / `macos-container`) — used by the active-
@@ -251,7 +251,7 @@ class Bottle(ABC):
`user` (default `node`, matching the agent image's USER `user` (default `node`, matching the agent image's USER
directive) and return the captured stdout/stderr/returncode. directive) and return the captured stdout/stderr/returncode.
The bottle's environment (including HTTPS_PROXY pointing at The bottle's environment (including HTTPS_PROXY pointing at
the egress daemon) is inherited by the child. Non-zero the egress sidecar) is inherited by the child. Non-zero
exit does not raise — callers inspect `returncode` exit does not raise — callers inspect `returncode`
themselves. themselves.
@@ -454,7 +454,7 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
declarative provision-plan apply, supervise MCP registration) declarative provision-plan apply, supervise MCP registration)
live on the `AgentProvider` plugin. The backend only owns the live on the `AgentProvider` plugin. The backend only owns the
steps that are about backend infrastructure (CA, workspace, steps that are about backend infrastructure (CA, workspace,
git) and surfaces the supervise daemon URL its launch step git) and surfaces the supervise sidecar URL its launch step
knows about via `supervise_mcp_url`. knows about via `supervise_mcp_url`.
PRD 0017: cred-proxy's agent-side dotfile rewrites (~/.npmrc, PRD 0017: cred-proxy's agent-side dotfile rewrites (~/.npmrc,
@@ -502,7 +502,7 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
def supervise_mcp_url(self, plan: PlanT) -> str: def supervise_mcp_url(self, plan: PlanT) -> str:
"""Return the agent-side URL of the per-bottle supervise """Return the agent-side URL of the per-bottle supervise
gateway, or "" when this bottle has no gateway. The provider sidecar, or "" when this bottle has no sidecar. The provider
plugin's `provision_supervise_mcp` uses it to register the plugin's `provision_supervise_mcp` uses it to register the
MCP entry inside the guest. MCP entry inside the guest.
@@ -524,7 +524,7 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
def enumerate_active(self) -> Sequence[ActiveAgent]: def enumerate_active(self) -> Sequence[ActiveAgent]:
"""Return every currently-running agent on this backend. """Return every currently-running agent on this backend.
Empty when none. Backend-specific: docker queries `docker Empty when none. Backend-specific: docker queries `docker
compose ls`; firecracker cross-references its running gateway compose ls`; firecracker cross-references its running sidecar
containers against per-bottle metadata.""" containers against per-bottle metadata."""
@classmethod @classmethod
+1 -6
View File
@@ -106,16 +106,11 @@ class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanup
yield bottle yield bottle
def supervise_mcp_url(self, plan: DockerBottlePlan) -> str: def supervise_mcp_url(self, plan: DockerBottlePlan) -> str:
"""Docker bottles reach the supervise daemon via the """Docker bottles reach the supervise sidecar via the
compose-network alias `supervise:9100`. No per-bottle URL compose-network alias `supervise:9100`. No per-bottle URL
plumbing needed; the alias resolves inside the bridge.""" plumbing needed; the alias resolves inside the bridge."""
if plan.supervise_plan is None: if plan.supervise_plan is None:
return "" return ""
# Consolidated: the supervise daemon lives on the shared gateway, so
# the agent registers the gateway address (NO_PROXY bypasses egress),
# not the per-bottle `supervise` alias.
if plan.agent_supervise_url:
return plan.agent_supervise_url
return f"http://{SUPERVISE_HOSTNAME}:{SUPERVISE_PORT}/" return f"http://{SUPERVISE_HOSTNAME}:{SUPERVISE_PORT}/"
def prepare_cleanup(self) -> DockerBottleCleanupPlan: def prepare_cleanup(self) -> DockerBottleCleanupPlan:
-19
View File
@@ -28,30 +28,11 @@ class DockerBottlePlan(BottlePlan):
# accidental log of the plan dataclass. # accidental log of the plan dataclass.
forwarded_env: dict[str, str] = field(repr=False) forwarded_env: dict[str, str] = field(repr=False)
use_runsc: bool use_runsc: bool
# Consolidated mode (PRD 0070): the agent reaches git-gate over HTTP at the
# shared gateway's address (`http://<gateway_ip>:9420`), set at launch.
# Empty → single-tenant defaults (the `git-gate` alias over git://).
agent_git_gate_url: str = ""
# Likewise the supervise MCP endpoint at the gateway (`http://<gw>:9100/`);
# empty → the single-tenant `supervise` alias.
agent_supervise_url: str = ""
@property @property
def container_name(self) -> str: def container_name(self) -> str:
return self.agent_provision.instance_name return self.agent_provision.instance_name
@property
def git_gate_insteadof_host(self) -> str:
if self.agent_git_gate_url.startswith("http://"):
return self.agent_git_gate_url.removeprefix("http://").rstrip("/")
return super().git_gate_insteadof_host
@property
def git_gate_insteadof_scheme(self) -> str:
if self.agent_git_gate_url.startswith("http://"):
return "http"
return super().git_gate_insteadof_scheme
@property @property
def image(self) -> str: def image(self) -> str:
return self.agent_provision.image return self.agent_provision.image
+243 -6
View File
@@ -1,10 +1,20 @@
"""Docker compose lifecycle helpers (PRD 0018). """Compose-spec rendering for a Docker bottle (PRD 0018, chunk 1).
Serialize a compose spec to disk, drive `docker compose up/down`, `bottle_plan_to_compose(plan)` returns a Compose v2 spec dict
dump the merged log on teardown, and enumerate `bot-bottle-*` describing the per-bottle container topology — one project per
projects. The spec itself is built by `consolidated_compose.py` bottle instance, services for the agent + every applicable sidecar,
(the consolidated per-host gateway topology); this module owns the two networks, no named volumes.
I/O side that persists and runs it.
Pure function. No I/O, no subprocess. Expects every launch-time
field (network names, CA host paths, etc.) on the plan's inner
plans to be populated; chunks 2+3 own that ordering.
Conditional services follow the plan content:
- agent + sidecars bundle: always.
- git-gate: iff plan.git_gate_plan.upstreams.
- egress: iff plan.egress_plan.routes.
- supervise: iff plan.supervise_plan is not None.
""" """
from __future__ import annotations from __future__ import annotations
@@ -15,7 +25,233 @@ import sys
from pathlib import Path from pathlib import Path
from typing import Any from typing import Any
from ...egress import (
EGRESS_HOSTNAME,
EGRESS_ROUTES_IN_CONTAINER,
egress_agent_env_entries,
egress_sidecar_env_entries,
)
from ...git_gate import GIT_GATE_HOSTNAME
from ...log import die, warn from ...log import die, warn
from ...supervise import (
DB_PATH_IN_CONTAINER,
SUPERVISE_HOSTNAME,
SUPERVISE_PORT,
)
from ...util import expand_tilde
from ..util import AGENT_CA_BUNDLE, AGENT_CA_PATH
from .bottle_plan import DockerBottlePlan
from .egress import (
EGRESS_CA_IN_CONTAINER,
EGRESS_PORT,
)
from .git_gate import (
GIT_GATE_ACCESS_HOOK_IN_CONTAINER,
GIT_GATE_CREDS_DIR_IN_CONTAINER,
GIT_GATE_ENTRYPOINT_IN_CONTAINER,
GIT_GATE_HOOK_IN_CONTAINER,
)
from . import network as network_mod
from .sidecar_bundle import (
SIDECAR_BUNDLE_DOCKERFILE,
SIDECAR_BUNDLE_IMAGE,
sidecar_bundle_container_name,
)
# Repo root, used as the build context for the bundle Dockerfile.
_REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent)
def bottle_plan_to_compose(plan: DockerBottlePlan) -> dict[str, Any]:
"""Render a Compose v2 spec dict from a fully-resolved
DockerBottlePlan.
The plan must have its inner plans (`git_gate_plan`,
`egress_plan`, `supervise_plan`) populated with launch-time
fields — network names, CA host paths. The renderer doesn't
validate; callers feed it a fully-resolved plan or get an
incomplete compose spec back.
"""
project = f"bot-bottle-{plan.slug}"
services: dict[str, Any] = {
"sidecars": _sidecar_bundle_service(plan),
"agent": _agent_service(plan),
}
return {
"name": project,
"services": services,
"networks": _networks(plan),
}
def _networks(plan: DockerBottlePlan) -> dict[str, Any]:
"""Compose-managed networks with explicit `name:` matching the
existing slug-suffixed convention. Compose creates them on `up`
and destroys them on `down`. The internal one is `--internal`
(no default gateway); the egress one is a normal user-defined
bridge."""
return {
"internal": {
"name": network_mod.network_name_for_slug(plan.slug),
"internal": True,
},
"egress": {
"name": network_mod.network_egress_name_for_slug(plan.slug),
},
}
def _bind(host: str | Path, target: str, *, read_only: bool = True) -> dict[str, Any]:
"""One bind-mount entry in the long-form `volumes:` shape.
Long form is preferred over `host:target:ro` strings because
it's easier to inspect in tests and survives whitespace in
host paths."""
return {
"type": "bind",
"source": str(host),
"target": target,
"read_only": read_only,
}
def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
"""The `sidecars` service: one container per bottle, bundle
image, all daemons under a Python init supervisor.
Daemon subset narrows via `BOT_BOTTLE_SIDECAR_DAEMONS` env.
egress is always present; git-gate / supervise are conditional.
"""
daemons: list[str] = ["egress"]
if plan.git_gate_plan.upstreams:
daemons.append("git-gate")
if plan.supervise_plan is not None:
daemons.append("supervise")
env: list[str] = [f"BOT_BOTTLE_SIDECAR_DAEMONS={','.join(daemons)}"]
volumes: list[dict[str, Any]] = []
# --- egress -------------------------------------------------------
ep = plan.egress_plan
volumes.append(_bind(ep.mitmproxy_ca_host_path, EGRESS_CA_IN_CONTAINER))
if ep.routes:
volumes.append(_bind(ep.routes_path.parent, str(Path(EGRESS_ROUTES_IN_CONTAINER).parent)))
env.extend(egress_sidecar_env_entries(ep))
# --- git-gate -----------------------------------------------------
gp = plan.git_gate_plan
if gp.upstreams:
volumes += [
_bind(gp.entrypoint_script, GIT_GATE_ENTRYPOINT_IN_CONTAINER),
_bind(gp.hook_script, GIT_GATE_HOOK_IN_CONTAINER),
_bind(gp.access_hook_script, GIT_GATE_ACCESS_HOOK_IN_CONTAINER),
]
for u in gp.upstreams:
keypath = expand_tilde(u.identity_file)
volumes.append(_bind(
keypath,
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{u.name}-key",
))
if u.known_hosts_file:
volumes.append(_bind(
u.known_hosts_file,
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{u.name}-known_hosts",
))
# --- supervise ----------------------------------------------------
sp = plan.supervise_plan
if sp is not None:
env += [
f"SUPERVISE_BOTTLE_SLUG={plan.slug}",
f"SUPERVISE_DB_PATH={DB_PATH_IN_CONTAINER}",
f"SUPERVISE_PORT={SUPERVISE_PORT}",
]
volumes.append({
"type": "bind",
"source": str(sp.db_path),
"target": DB_PATH_IN_CONTAINER,
"read_only": False,
})
internal_aliases = [EGRESS_HOSTNAME]
if gp.upstreams:
internal_aliases.append(GIT_GATE_HOSTNAME)
if sp is not None:
internal_aliases.append(SUPERVISE_HOSTNAME)
service: dict[str, Any] = {
"image": SIDECAR_BUNDLE_IMAGE,
"build": {
"context": _REPO_DIR,
"dockerfile": SIDECAR_BUNDLE_DOCKERFILE,
},
"container_name": sidecar_bundle_container_name(plan.slug),
"networks": {
"internal": {"aliases": internal_aliases},
"egress": None,
},
"environment": env,
"volumes": volumes,
}
return service
def _agent_service(plan: DockerBottlePlan) -> dict[str, Any]:
"""Agent container. Runs `sleep infinity`; claude is `docker
exec -it`'d into it later. HTTP_PROXY/HTTPS_PROXY point at the
egress sidecar."""
proxy_url = _agent_proxy_url(plan)
no_proxy = _agent_no_proxy(plan)
env: list[str] = [
f"HTTPS_PROXY={proxy_url}",
f"HTTP_PROXY={proxy_url}",
f"https_proxy={proxy_url}",
f"http_proxy={proxy_url}",
f"NO_PROXY={no_proxy}",
f"no_proxy={no_proxy}",
f"NODE_EXTRA_CA_CERTS={AGENT_CA_PATH}",
f"SSL_CERT_FILE={AGENT_CA_BUNDLE}",
f"REQUESTS_CA_BUNDLE={AGENT_CA_BUNDLE}",
]
for name, value in sorted(plan.agent_provision.guest_env.items()):
env.append(f"{name}={value}")
# Forwarded vars (OAuth token, manifest host-interpolations):
# bare name → inherits from compose-up process env, value
# never lands on argv or in the compose file.
for name in sorted(plan.forwarded_env.keys()):
env.append(name)
env.extend(egress_agent_env_entries(plan.egress_plan))
service: dict[str, Any] = {
"image": plan.image,
"container_name": plan.container_name,
"command": ["sleep", "infinity"],
"networks": {"internal": None},
"environment": env,
}
if plan.use_runsc:
service["runtime"] = "runsc"
# The init supervisor inside the bundle owns intra-bundle
# daemon ordering, so the agent only waits for the bundle
# container itself.
service["depends_on"] = ["sidecars"]
return service
def _agent_proxy_url(plan: DockerBottlePlan) -> str:
"""Agent's HTTP_PROXY — always points at egress."""
return f"http://{EGRESS_HOSTNAME}:{EGRESS_PORT}"
def _agent_no_proxy(plan: DockerBottlePlan) -> str:
"""NO_PROXY for the agent: loopback always; supervise hostname
when the supervise sidecar is up (MCP long-poll must bypass
the egress proxy)."""
hosts = ["localhost", "127.0.0.1"]
if plan.supervise_plan is not None:
hosts.append(SUPERVISE_HOSTNAME)
return ",".join(hosts)
# --- Lifecycle helpers (PRD 0018 chunk 3) ---------------------------------- # --- Lifecycle helpers (PRD 0018 chunk 3) ----------------------------------
@@ -206,6 +442,7 @@ __all__ = [
"COMPOSE_FILE_NAME", "COMPOSE_FILE_NAME",
"COMPOSE_LOG_NAME", "COMPOSE_LOG_NAME",
"COMPOSE_PROJECT_PREFIX", "COMPOSE_PROJECT_PREFIX",
"bottle_plan_to_compose",
"compose_down", "compose_down",
"compose_dump_logs", "compose_dump_logs",
"compose_file_path", "compose_file_path",
@@ -1,78 +0,0 @@
"""Agent-only compose for the consolidated docker backend (PRD 0070).
The per-bottle model rendered a compose project with the agent *and* a
gateway on two per-bottle networks. In the consolidated model the
per-bottle companion containers are gone — one shared gateway serves every bottle — so this renders
just the agent, attached to the **external shared gateway network** with the
pinned source IP the orchestrator allocated, and pointed at the gateway's
address for egress (and, around the proxy, for git-http / supervise).
Pure: it takes the launch-time `LaunchContext` values (gateway address,
source IP, network) and the prepared plan, and returns a compose dict — no
docker, so it's testable in isolation.
"""
from __future__ import annotations
from typing import Any
from ...egress import egress_agent_env_entries
from ..util import AGENT_CA_BUNDLE, AGENT_CA_PATH
from .bottle_plan import DockerBottlePlan
from .egress import EGRESS_PORT
def consolidated_agent_compose(
plan: DockerBottlePlan,
*,
gateway_ip: str,
source_ip: str,
network: str,
) -> dict[str, Any]:
"""A compose spec with only the agent service, on the external gateway
network at `source_ip`, proxying egress through `gateway_ip`."""
proxy_url = f"http://{gateway_ip}:{EGRESS_PORT}"
# git-http + supervise live on the gateway too and must NOT go through the
# egress proxy — the agent reaches them directly by the gateway address.
no_proxy = f"localhost,127.0.0.1,{gateway_ip}"
env: list[str] = [
f"HTTPS_PROXY={proxy_url}",
f"HTTP_PROXY={proxy_url}",
f"https_proxy={proxy_url}",
f"http_proxy={proxy_url}",
f"NO_PROXY={no_proxy}",
f"no_proxy={no_proxy}",
f"NODE_EXTRA_CA_CERTS={AGENT_CA_PATH}",
f"SSL_CERT_FILE={AGENT_CA_BUNDLE}",
f"REQUESTS_CA_BUNDLE={AGENT_CA_BUNDLE}",
]
for name, value in sorted(plan.agent_provision.guest_env.items()):
env.append(f"{name}={value}")
# Forwarded vars: bare name → inherits from the compose-up process env so
# the secret value never lands on argv or in the compose file.
for name in sorted(plan.forwarded_env.keys()):
env.append(name)
env.extend(egress_agent_env_entries(plan.egress_plan))
service: dict[str, Any] = {
"image": plan.image,
"container_name": plan.container_name,
"command": ["sleep", "infinity"],
# Pinned address on the shared gateway network — the orchestrator
# registered this IP, and the gateway attributes the bottle by it.
"networks": {network: {"ipv4_address": source_ip}},
"environment": env,
}
if plan.use_runsc:
service["runtime"] = "runsc"
return {
"name": f"bot-bottle-{plan.slug}",
"services": {"agent": service},
# The gateway network is created + owned by the orchestrator; compose
# attaches to it (external) and must not create or destroy it.
"networks": {network: {"external": True}},
}
__all__ = ["consolidated_agent_compose"]
@@ -1,153 +0,0 @@
"""Consolidated bottle launch sequence for the docker backend (PRD 0070).
Composes the orchestrator primitives into the register/teardown sequence that
replaces the per-bottle gateway:
1. ensure the orchestrator control plane + shared gateway are up;
2. allocate the bottle a pinned source IP on the gateway network (the
attribution key), skipping the gateway's own address + live bottles;
3. register it (egress policy blob + slug metadata) → bottle id + identity
token;
4. provision its git-gate repos/creds into the running gateway.
It returns a `LaunchContext` with everything the agent container needs to
attach — network, pinned IP, the gateway's address (its proxy target), the
orchestrator URL, and the identity token. The agent `docker run` itself is
the backend's job (it owns provider provisioning); this owns the
orchestrator-facing wiring so that sequence stays testable in isolation.
"""
from __future__ import annotations
from dataclasses import dataclass
from ...docker_cmd import run_docker
from ...egress import EgressPlan
from ...git_gate import GitGatePlan
from ...orchestrator.client import OrchestratorClient
from ...orchestrator.gateway import GATEWAY_NAME, GATEWAY_NETWORK
from ...orchestrator.lifecycle import OrchestratorService
from ...orchestrator.registration import registration_inputs
from .gateway_net import next_free_ip
from .gateway_provision import deprovision_git_gate, provision_git_gate
class ConsolidatedLaunchError(RuntimeError):
"""The consolidated register/provision sequence could not complete."""
@dataclass(frozen=True)
class LaunchContext:
"""What the agent container needs to join the shared gateway."""
bottle_id: str
identity_token: str
source_ip: str # the agent's pinned address (attribution key)
network: str # the shared gateway network to attach to
gateway_ip: str # the gateway's address — the agent's proxy target
orchestrator_url: str
def _network_cidr(network: str) -> str:
"""The gateway network's IPv4 subnet, or raise."""
proc = run_docker([
"docker", "network", "inspect",
"--format", "{{range .IPAM.Config}}{{.Subnet}}{{end}}", network,
])
cidr = proc.stdout.strip()
if proc.returncode != 0 or not cidr:
raise ConsolidatedLaunchError(
f"gateway network {network} has no subnet: {proc.stderr.strip()}"
)
return cidr
def _container_ip(name: str, network: str) -> str:
"""A container's IPv4 address on `network`, or raise."""
proc = run_docker([
"docker", "inspect", "--format",
f'{{{{(index .NetworkSettings.Networks "{network}").IPAddress}}}}', name,
])
ip = proc.stdout.strip()
if proc.returncode != 0 or not ip:
raise ConsolidatedLaunchError(
f"gateway {name} has no address on {network}: {proc.stderr.strip()}"
)
return ip
def _network_container_ips(network: str) -> list[str]:
"""Every address currently assigned on the gateway network — the ground
truth for "in use": the gateway + orchestrator infrastructure containers
and every live agent. Read from the network so a new bottle can't collide
with anything actually attached (a registry-only view would miss the
orchestrator/gateway containers)."""
proc = run_docker([
"docker", "network", "inspect", "--format",
"{{range .Containers}}{{.IPv4Address}} {{end}}", network,
])
ips: list[str] = []
for entry in proc.stdout.split():
# entries look like "172.20.0.2/16" — keep the address.
ips.append(entry.split("/", 1)[0])
return ips
def launch_consolidated(
egress_plan: EgressPlan,
git_gate_plan: GitGatePlan,
*,
image_ref: str = "",
tokens: dict[str, str] | None = None,
service: OrchestratorService | None = None,
gateway_name: str = GATEWAY_NAME,
network: str = GATEWAY_NETWORK,
) -> LaunchContext:
"""Ensure the orchestrator + gateway are up, allocate + register the
bottle, and provision its git-gate state. Returns the agent's attach
context. Raises `ConsolidatedLaunchError` (or the primitives' own errors)
if any step fails — the caller tears down on failure."""
service = service or OrchestratorService()
url = service.ensure_running()
client = OrchestratorClient(url)
cidr = _network_cidr(network)
gateway_ip = _container_ip(gateway_name, network)
source_ip = next_free_ip(cidr, _network_container_ips(network))
inputs = registration_inputs(egress_plan)
reg = client.register_bottle(
source_ip, image_ref=image_ref, policy=inputs.policy,
metadata=inputs.metadata, tokens=tokens,
)
try:
provision_git_gate(gateway_name, reg.bottle_id, git_gate_plan)
except Exception:
# Roll the registration back so a provisioning failure leaves no orphan.
client.teardown_bottle(reg.bottle_id)
raise
return LaunchContext(
bottle_id=reg.bottle_id,
identity_token=reg.identity_token,
source_ip=source_ip,
network=network,
gateway_ip=gateway_ip,
orchestrator_url=url,
)
def teardown_consolidated(
bottle_id: str, *, orchestrator_url: str, gateway_name: str = GATEWAY_NAME,
) -> None:
"""Deregister the bottle and remove its git-gate state from the gateway.
Both steps are idempotent so this is safe from a cleanup trap."""
OrchestratorClient(orchestrator_url).teardown_bottle(bottle_id)
deprovision_git_gate(gateway_name, bottle_id)
__all__ = [
"LaunchContext",
"launch_consolidated",
"teardown_consolidated",
"ConsolidatedLaunchError",
]
+1 -1
View File
@@ -4,7 +4,7 @@ prepare-time routes-yaml rendering itself lives on the
platform-neutral `Egress` ABC — backends instantiate it directly. platform-neutral `Egress` ABC — backends instantiate it directly.
The per-container `.start()` / `.stop()` lifecycle was removed in The per-container `.start()` / `.stop()` lifecycle was removed in
PRD 0024 chunk 3; the gateway (PRD 0024) runs egress PRD 0024 chunk 3; the sidecar bundle (PRD 0024) runs egress
under its python init supervisor.""" under its python init supervisor."""
from __future__ import annotations from __future__ import annotations
+44 -13
View File
@@ -1,29 +1,60 @@
"""Host-side egress route-apply for the docker backend. """Host-side helper for egress sidecar inspection and live updates.
The per-bottle companion container this used to signal (`docker kill The approve path uses this module to validate a proposed routes file,
--signal HUP <container>`) was removed in the companion-container removal (#385). write it to the bottle's live egress state dir, and signal the sidecar
In the consolidated model the shared gateway resolves egress policy bundle so the mitmproxy addon reloads it.
per-request against the orchestrator rather than reloading a per-bottle
routes file, so the live per-bottle reload is not supported here and
fails closed until the gateway-side apply lands.
""" """
from __future__ import annotations from __future__ import annotations
import os
import subprocess
from ...egress import EGRESS_ROUTES_IN_CONTAINER
from ...log import warn
from ..egress_apply import EgressApplicator, EgressApplyError from ..egress_apply import EgressApplicator, EgressApplyError
from .sidecar_bundle import sidecar_bundle_container_name
def fetch_current_routes(slug: str) -> str:
container = sidecar_bundle_container_name(slug)
r = subprocess.run(
["docker", "exec", container, "cat", EGRESS_ROUTES_IN_CONTAINER],
capture_output=True, text=True, check=False,
)
if r.returncode != 0:
raise EgressApplyError(
f"could not read routes.yaml from {container}: "
f"{(r.stderr or '').strip() or 'container not running?'}"
)
return r.stdout
class DockerEgressApplicator(EgressApplicator): class DockerEgressApplicator(EgressApplicator):
def _signal_bundle_reload(self, slug: str) -> None: def _signal_bundle_reload(self, slug: str) -> None:
del slug container = sidecar_bundle_container_name(slug)
raise EgressApplyError( result = subprocess.run(
"live egress route-apply was removed with the per-bottle " ["docker", "kill", "--signal", "HUP", container],
"companion container (#385); route changes will flow through " capture_output=True, text=True, check=False, env=os.environ,
"the consolidated gateway in a follow-up."
) )
if result.returncode != 0:
last_error = (result.stderr or "").strip() or (result.stdout or "").strip()
warn(
f"egress: routes updated on disk for {slug}, but bundle reload failed: "
f"{last_error or 'docker kill failed'}"
)
raise EgressApplyError(
f"could not reload egress bundle {container}: "
f"{last_error or 'docker kill failed'}"
)
applicator = DockerEgressApplicator() applicator = DockerEgressApplicator()
__all__ = ["DockerEgressApplicator", "EgressApplyError", "applicator"] __all__ = [
"DockerEgressApplicator",
"EgressApplyError",
"applicator",
"fetch_current_routes",
]
-47
View File
@@ -1,47 +0,0 @@
"""Shared-gateway source-IP allocation for the consolidated docker backend
(PRD 0070).
In the consolidated model one gateway container serves every bottle over a
single shared docker network, and each agent bottle attaches with a pinned,
deterministic address that the gateway uses as its **attribution key**. This
allocates those addresses from the network's subnet, skipping the reserved
ones — the network address and broadcast (excluded by `hosts()`), docker's
router `.1`, and everything already in use (`taken`: the gateway container
plus every live bottle, which the caller reads from the registry).
Pure `ipaddress` logic — the docker-specific bits (the subnet CIDR, the
gateway container's own address) are gathered by the caller and passed in, so
this stays testable without docker.
"""
from __future__ import annotations
import ipaddress
from collections.abc import Iterable
class NoFreeAddressError(RuntimeError):
"""The shared gateway network's subnet is exhausted — every host address
is reserved or already assigned to a bottle."""
def next_free_ip(cidr: str, taken: Iterable[str]) -> str:
"""The lowest host address in `cidr` not in `taken` and not docker's
router (`.1`). `taken` must include the gateway container's own address
and every live bottle's. Raises `NoFreeAddressError` if the subnet is
full."""
net = ipaddress.ip_network(cidr, strict=False)
reserved = {str(a) for a in taken}
# Docker assigns the network's first host (.1) to the bridge router; a
# bottle must never be handed that address.
reserved.add(str(net.network_address + 1))
for host in net.hosts(): # hosts() already excludes network + broadcast
candidate = str(host)
if candidate not in reserved:
return candidate
raise NoFreeAddressError(
f"no free address in {cidr} ({len(reserved)} reserved/assigned)"
)
__all__ = ["next_free_ip", "NoFreeAddressError"]
@@ -1,100 +0,0 @@
"""Provision one bottle's git-gate state into the running shared gateway
(PRD 0070, docker slice).
The consolidated gateway serves every bottle's repos under `/git/<bottle_id>/`
with per-repo credentials under `/git-gate/creds/<bottle_id>/`. When a bottle
is registered the launcher must place *its* deploy keys + known_hosts into
that per-bottle creds dir and init its bare repos there — so this copies the
credential files into the live gateway container and runs the (namespaced,
init-only) provisioning script produced by `git_gate_render_provision`.
Isolating each bottle's creds dir + repo root by id is what keeps one
bottle's push credentials out of another's repos on the shared gateway.
"""
from __future__ import annotations
import re
from ...docker_cmd import run_docker
from ...git_gate import GitGatePlan, git_gate_render_provision
# bottle ids index the gateway's per-bottle repo + creds dirs; they land in
# `docker cp`/`rm` path arguments, so validate before any path is built (a
# traversal id like "../etc" must never reach the container). Registry ids are
# token_hex — this is defense in depth at the docker boundary.
_SAFE_BOTTLE_ID = re.compile(r"[A-Za-z0-9_-]+")
class GatewayProvisionError(RuntimeError):
"""A git-gate provisioning step against the running gateway failed."""
def _require_safe(bottle_id: str) -> None:
if not _SAFE_BOTTLE_ID.fullmatch(bottle_id):
raise GatewayProvisionError(f"unsafe bottle id {bottle_id!r}")
def _creds_dir(bottle_id: str) -> str:
return f"/git-gate/creds/{bottle_id}"
def _exec(gateway: str, argv: list[str]) -> None:
"""`docker exec` a command in the gateway, raising on non-zero exit."""
proc = run_docker(["docker", "exec", gateway, *argv])
if proc.returncode != 0:
raise GatewayProvisionError(
f"gateway exec {argv!r} failed: {proc.stderr.strip()}"
)
def _cp_into(gateway: str, src: str, dest: str) -> None:
"""`docker cp` a host file into the gateway, raising on non-zero exit."""
proc = run_docker(["docker", "cp", src, f"{gateway}:{dest}"])
if proc.returncode != 0:
raise GatewayProvisionError(
f"gateway cp {src} -> {dest} failed: {proc.stderr.strip()}"
)
def provision_git_gate(gateway: str, bottle_id: str, plan: GitGatePlan) -> None:
"""Place `bottle_id`'s git-gate credentials into the running `gateway`
container and init its bare repos under `/git/<bottle_id>/`.
Copies each upstream's identity key (and known_hosts, when present) into
`/git-gate/creds/<bottle_id>/`, then runs the namespaced provisioning
script. No-op for a bottle with no git upstreams."""
_require_safe(bottle_id)
if not plan.upstreams:
return
# The pre-receive + access hooks are bottle-agnostic and shared by every
# bottle's repos; install them into the gateway (idempotent — same content
# each time). The per-bottle model cp'd these into each bundle at start.
_exec(gateway, ["mkdir", "-p", "/etc/git-gate"])
_cp_into(gateway, str(plan.hook_script), "/etc/git-gate/pre-receive")
_cp_into(gateway, str(plan.access_hook_script), "/etc/git-gate/access-hook")
creds = _creds_dir(bottle_id)
_exec(gateway, ["mkdir", "-p", creds])
for u in plan.upstreams:
if u.identity_file:
_cp_into(gateway, u.identity_file, f"{creds}/{u.name}-key")
known_hosts = str(u.known_hosts_file)
if known_hosts and known_hosts != ".":
_cp_into(gateway, known_hosts, f"{creds}/{u.name}-known_hosts")
# Init the bare repos + per-repo credential config for this namespace.
script = git_gate_render_provision(bottle_id, plan.upstreams)
_exec(gateway, ["sh", "-c", script])
def deprovision_git_gate(gateway: str, bottle_id: str) -> None:
"""Remove a bottle's repos + creds from the gateway on teardown. Idempotent
— an already-absent namespace is a clean no-op (best effort; a stray dir
can't leak, since attribution is by source IP and the bottle is gone)."""
_require_safe(bottle_id)
run_docker([
"docker", "exec", gateway, "rm", "-rf",
f"/git/{bottle_id}", _creds_dir(bottle_id),
])
__all__ = ["provision_git_gate", "deprovision_git_gate", "GatewayProvisionError"]
+1 -1
View File
@@ -2,7 +2,7 @@
bind-mounts target + the listening port. The prepare-time entrypoint bind-mounts target + the listening port. The prepare-time entrypoint
/ hook render lives on the platform-neutral `GitGate` ABC — backends / hook render lives on the platform-neutral `GitGate` ABC — backends
instantiate it directly. The git-gate daemon's container lifecycle instantiate it directly. The git-gate daemon's container lifecycle
is owned by the gateway (PRD 0024).""" is owned by the sidecar bundle (PRD 0024)."""
from __future__ import annotations from __future__ import annotations
+62 -57
View File
@@ -5,7 +5,7 @@ PRD 0018 chunk 3: each instance is one `docker compose` project.
The flow is: The flow is:
1. Build the agent image from the provider Dockerfile (compose 1. Build the agent image from the provider Dockerfile (compose
builds the gateway image on first up). builds the sidecar images via the `build:` directive on first up).
2. Mint the per-bottle egress CA (chunk 2 writes it under 2. Mint the per-bottle egress CA (chunk 2 writes it under
state/<slug>/egress/). state/<slug>/egress/).
3. Populate the inner plans with launch-time fields so the 3. Populate the inner plans with launch-time fields so the
@@ -42,6 +42,7 @@ from ...git_gate import (
revoke_git_gate_provisioned_keys, revoke_git_gate_provisioned_keys,
) )
from ...log import info, warn from ...log import info, warn
from . import network as network_mod
from . import util as docker_mod from . import util as docker_mod
from .bottle import DockerBottle from .bottle import DockerBottle
from .bottle_plan import DockerBottlePlan from .bottle_plan import DockerBottlePlan
@@ -52,6 +53,7 @@ from ...bottle_state import (
read_committed_image, read_committed_image,
) )
from .compose import ( from .compose import (
bottle_plan_to_compose,
compose_down, compose_down,
compose_dump_logs, compose_dump_logs,
compose_file_path, compose_file_path,
@@ -60,9 +62,7 @@ from .compose import (
compose_up, compose_up,
write_compose_file, write_compose_file,
) )
from .consolidated_compose import consolidated_agent_compose from .egress import egress_tls_init
from .consolidated_launch import launch_consolidated, teardown_consolidated
from ...orchestrator.gateway import DockerGateway
# Where the repo root lives, for `docker build` context. Computed once. # Where the repo root lives, for `docker build` context. Computed once.
@@ -97,7 +97,8 @@ def launch(
try: try:
# Step 1: agent image. Use a committed snapshot when one exists # Step 1: agent image. Use a committed snapshot when one exists
# and is present in the local daemon; otherwise build from the # and is present in the local daemon; otherwise build from the
# Dockerfile. (The gateway image is built by the orchestrator.) # Dockerfile. Sidecar images get built lazily by `docker compose
# up` via the renderer's `build:` directives.
committed = read_committed_image(plan.slug) committed = read_committed_image(plan.slug)
if committed and docker_mod.image_exists(committed): if committed and docker_mod.image_exists(committed):
info(f"using committed image {committed!r}") info(f"using committed image {committed!r}")
@@ -111,82 +112,82 @@ def launch(
dockerfile=plan.dockerfile_path, dockerfile=plan.dockerfile_path,
) )
# Step 2: mint the git-gate dynamic (gitea) deploy keys, if any, before internal_network = network_mod.network_name_for_slug(plan.slug)
# provisioning the bottle's repos into the shared gateway. egress_network = network_mod.network_egress_name_for_slug(plan.slug)
egress_ca_host, egress_ca_cert_only = egress_tls_init(
egress_state_dir(plan.slug),
)
git_gate_plan = plan.git_gate_plan git_gate_plan = plan.git_gate_plan
if git_gate_plan.upstreams: if git_gate_plan.upstreams:
git_gate_plan = provision_git_gate_dynamic_keys( git_gate_plan = provision_git_gate_dynamic_keys(
plan.manifest.bottle, git_gate_plan, git_gate_state_dir(plan.slug), plan.manifest.bottle,
git_gate_plan,
git_gate_state_dir(plan.slug),
)
git_gate_plan = dataclasses.replace(
git_gate_plan,
internal_network=internal_network,
egress_network=egress_network,
) )
# Step 3: register on the orchestrator + provision this bottle's
# git-gate state into the shared gateway; get the agent's attach
# context (pinned source IP, gateway address, shared network). The
# per-bottle egress auth tokens are resolved from the host env now and
# handed to the orchestrator (in memory) for the gateway to inject —
# the agent never sees them.
effective_env = {**os.environ, **plan.agent_provision.provisioned_env}
token_values = egress_resolve_token_values(
plan.egress_plan.token_env_map, effective_env,
)
ctx = launch_consolidated(
plan.egress_plan, git_gate_plan, image_ref=plan.image, tokens=token_values,
)
stack.callback(
teardown_consolidated, ctx.bottle_id, orchestrator_url=ctx.orchestrator_url,
)
# Step 4: install the SHARED gateway CA into the agent (replaces the
# per-bottle CA) — read it out of the running gateway.
ca_dir = egress_state_dir(plan.slug) / "gateway-ca"
ca_dir.mkdir(parents=True, exist_ok=True)
ca_file = ca_dir / "gateway-ca.pem"
ca_file.write_text(DockerGateway(network=ctx.network).ca_cert_pem())
egress_plan = dataclasses.replace( egress_plan = dataclasses.replace(
plan.egress_plan, plan.egress_plan,
mitmproxy_ca_host_path=ca_file, internal_network=internal_network,
mitmproxy_ca_cert_only_host_path=ca_file, egress_network=egress_network,
) mitmproxy_ca_host_path=egress_ca_host,
# Point the agent's git-gate insteadOf rewrites at the shared gateway's mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
# HTTP git endpoint (9420) instead of the dead per-bottle `git-gate`
# alias. git-http + supervise on the gateway bypass the egress proxy
# (NO_PROXY includes the gateway address).
git_gate_url = (
f"http://{ctx.gateway_ip}:9420" if git_gate_plan.upstreams else ""
)
supervise_url = (
f"http://{ctx.gateway_ip}:9100/" if plan.supervise_plan is not None else ""
) )
supervise_plan = plan.supervise_plan
if supervise_plan is not None:
supervise_plan = dataclasses.replace(
supervise_plan,
internal_network=internal_network,
)
plan = dataclasses.replace( plan = dataclasses.replace(
plan, plan,
git_gate_plan=git_gate_plan, git_gate_plan=git_gate_plan,
egress_plan=egress_plan, egress_plan=egress_plan,
agent_git_gate_url=git_gate_url, supervise_plan=supervise_plan,
agent_supervise_url=supervise_url,
) )
# Step 5: render + up the agent-only compose, pinned on the shared # Step 6: render + write the compose file. metadata.json
# gateway network and proxied through the gateway's address. # was written at prepare time and already carries
# compose_project; nothing to update here.
state_dir = bottle_state_dir(plan.slug) state_dir = bottle_state_dir(plan.slug)
spec = consolidated_agent_compose( spec = bottle_plan_to_compose(plan)
plan, gateway_ip=ctx.gateway_ip, source_ip=ctx.source_ip, network=ctx.network,
)
compose_file = write_compose_file(spec, compose_file_path(state_dir)) compose_file = write_compose_file(spec, compose_file_path(state_dir))
project = compose_project_name(plan.slug) project = compose_project_name(plan.slug)
# Forwarded vars (OAuth token, host interpolations) flow through the
# subprocess env as bare names so values never land in the file. # Step 7: compose up. Token values + the OAuth placeholder
compose_env: dict[str, str] = {**os.environ, **plan.forwarded_env} # flow through subprocess env; the compose file holds only
# bare names for the secret-carrying entries.
effective_env = {**dict(os.environ), **plan.agent_provision.provisioned_env}
token_values = egress_resolve_token_values(
plan.egress_plan.token_env_map, effective_env,
)
compose_env: dict[str, str] = {
**os.environ,
**plan.forwarded_env,
**token_values,
}
info( info(
f"docker compose up -d (project {project}, agent on shared " f"docker compose up -d (project {project}, "
f"gateway {ctx.gateway_ip}, ip {ctx.source_ip})" f"{len(spec['services'])} services)"
) )
compose_up(project, compose_file, env=compose_env) compose_up(project, compose_file, env=compose_env)
# Register teardown in reverse order: log dump first, then
# `compose down`. Networks come down last via callbacks
# registered in step 2.
stack.callback(compose_down, project, compose_file) stack.callback(compose_down, project, compose_file)
stack.callback( stack.callback(
compose_dump_logs, project, compose_file, compose_log_path(state_dir), compose_dump_logs, project, compose_file, compose_log_path(state_dir),
) )
# Step 6: provision (CA install now uses the gateway CA) + yield. # Step 8: provision. Create the bottle first so provisioners
# can use bottle.exec / bottle.cp_in; set the prompt path
# returned by provision_prompt after the fact.
bottle = DockerBottle( bottle = DockerBottle(
plan.container_name, plan.container_name,
teardown, teardown,
@@ -199,6 +200,10 @@ def launch(
agent_workdir=plan.workspace_plan.workdir, agent_workdir=plan.workspace_plan.workdir,
) )
bottle.prompt_path = provision(plan, bottle) bottle.prompt_path = provision(plan, bottle)
# Step 9: yield. exec_agent continues to use `docker exec -it`
# — the agent runs `sleep infinity` per the renderer's
# service spec.
yield bottle yield bottle
finally: finally:
teardown() teardown()
+1 -1
View File
@@ -76,7 +76,7 @@ def network_create_internal(slug: str) -> str:
def network_create_egress(slug: str) -> str: def network_create_egress(slug: str) -> str:
"""Create a per-agent user-defined bridge (NOT the legacy `bridge`) """Create a per-agent user-defined bridge (NOT the legacy `bridge`)
so the egress daemon has working DNS for upstream hostnames.""" so the egress sidecar has working DNS for upstream hostnames."""
return _network_create_with_prefix(network_egress_name_for_slug(slug), internal=False) return _network_create_with_prefix(network_egress_name_for_slug(slug), internal=False)
+3 -3
View File
@@ -1,7 +1,7 @@
"""Host setup + status for the Docker backend. """Host setup + status for the Docker backend.
Unlike Firecracker, the Docker backend needs no privileged one-time Unlike Firecracker, the Docker backend needs no privileged one-time
host provisioning (no TAP pool / nft table) — networks and the gateway host provisioning (no TAP pool / nft table) — networks and the sidecar
bundle are created per-launch. So `setup()` is mostly an install/daemon bundle are created per-launch. So `setup()` is mostly an install/daemon
pointer, and `status()` reports whether docker is usable. pointer, and `status()` reports whether docker is usable.
@@ -45,7 +45,7 @@ def setup() -> int:
return 1 return 1
sys.stderr.write( sys.stderr.write(
"Docker backend: no privileged host setup required — networks and " "Docker backend: no privileged host setup required — networks and "
"the gateway are created per-launch.\n" "the sidecar bundle are created per-launch.\n"
) )
if not _daemon_reachable(): if not _daemon_reachable():
sys.stderr.write( sys.stderr.write(
@@ -64,7 +64,7 @@ def setup() -> int:
def teardown() -> int: def teardown() -> int:
sys.stderr.write( sys.stderr.write(
"Docker backend: nothing to undo — it provisions no privileged host " "Docker backend: nothing to undo — it provisions no privileged host "
"state (networks and the gateway are per-launch and are " "state (networks and the sidecar bundle are per-launch and are "
"removed by `./cli.py cleanup`). Docker itself is left installed.\n" "removed by `./cli.py cleanup`). Docker itself is left installed.\n"
) )
return 0 return 0
@@ -0,0 +1,30 @@
"""Sidecar bundle constants + helpers for the Docker backend
(PRD 0024).
The bundle image (built by Dockerfile.sidecars, PRD 0024 chunk 1)
runs egress + git-gate + supervise as one container per bottle
under a small Python init supervisor. As of chunk 5 the bundle
is the only shape — the legacy four-sidecar topology and its
`BOT_BOTTLE_SIDECAR_BUNDLE` feature flag are gone."""
from __future__ import annotations
import os
# Bundle image. Defaults to a built-locally tag (built from the
# repo's Dockerfile.sidecars via compose `build:`). Operators
# pinning to a published digest can override via env.
SIDECAR_BUNDLE_IMAGE = os.environ.get(
"BOT_BOTTLE_SIDECAR_IMAGE",
"bot-bottle-sidecars:latest",
)
SIDECAR_BUNDLE_DOCKERFILE = "Dockerfile.sidecars"
def sidecar_bundle_container_name(slug: str) -> str:
"""`bot-bottle-sidecars-<slug>`. Same prefix scheme as the
per-sidecar containers it replaces, so the dashboard's
discovery-by-prefix logic keeps working."""
return f"bot-bottle-sidecars-{slug}"
+1 -1
View File
@@ -7,7 +7,7 @@ session. `ssh -t` forwards the host terminal's SIGWINCH to the remote
PTY natively, so no separate resize bridge is needed. PTY natively, so no separate resize bridge is needed.
Commands run as the image's `node` user via `runuser`, with HOME/USER/ Commands run as the image's `node` user via `runuser`, with HOME/USER/
PATH and the bottle env (HTTPS_PROXY at the gateway, CA paths, …) set PATH and the bottle env (HTTPS_PROXY at the sidecar, CA paths, …) set
per-invocation through `env` (the VM itself just runs the init; it has per-invocation through `env` (the VM itself just runs the init; it has
no baked-in process env like a `docker run` container would). no baked-in process env like a `docker run` container would).
""" """
@@ -10,9 +10,10 @@ from .. import BottleCleanupPlan
@dataclass(frozen=True) @dataclass(frozen=True)
class FirecrackerBottleCleanupPlan(BottleCleanupPlan): class FirecrackerBottleCleanupPlan(BottleCleanupPlan):
# PIDs of orphaned firecracker VMM processes and the per-bottle run # PIDs of orphaned firecracker VMM processes and the sidecar
# dirs left behind by previous bottles. # containers left behind by previous bottles.
vm_pids: tuple[int, ...] = () vm_pids: tuple[int, ...] = ()
containers: tuple[str, ...] = ()
run_dirs: tuple[str, ...] = () run_dirs: tuple[str, ...] = ()
def print(self) -> None: def print(self) -> None:
@@ -21,9 +22,11 @@ class FirecrackerBottleCleanupPlan(BottleCleanupPlan):
return return
for pid in self.vm_pids: for pid in self.vm_pids:
info(f"firecracker VM process: pid {pid}") info(f"firecracker VM process: pid {pid}")
for name in self.containers:
info(f"firecracker sidecar container: {name}")
for path in self.run_dirs: for path in self.run_dirs:
info(f"firecracker run dir: {path}") info(f"firecracker run dir: {path}")
@property @property
def empty(self) -> bool: def empty(self) -> bool:
return not (self.vm_pids or self.run_dirs) return not (self.vm_pids or self.containers or self.run_dirs)
@@ -13,7 +13,7 @@ from .. import BottlePlan
class FirecrackerBottlePlan(BottlePlan): class FirecrackerBottlePlan(BottlePlan):
slug: str slug: str
forwarded_env: dict[str, str] = field(repr=False) forwarded_env: dict[str, str] = field(repr=False)
# Stamped by launch once the gateway is up and its ports are # Stamped by launch once the sidecar is up and its ports are
# published on the host-side TAP IP (empty at prepare time). # published on the host-side TAP IP (empty at prepare time).
agent_proxy_url: str = "" agent_proxy_url: str = ""
agent_git_gate_url: str = "" agent_git_gate_url: str = ""
@@ -21,7 +21,7 @@ class FirecrackerBottlePlan(BottlePlan):
@property @property
def container_name(self) -> str: def container_name(self) -> str:
"""Instance name, reused for the gateway container + VM run dir. """Instance name, reused for the sidecar container + VM run dir.
Matches the `bot-bottle-<slug>` convention the other backends Matches the `bot-bottle-<slug>` convention the other backends
use so cleanup/enumerate discovery-by-prefix keeps working.""" use so cleanup/enumerate discovery-by-prefix keeps working."""
return self.agent_provision.instance_name return self.agent_provision.instance_name
+23 -2
View File
@@ -1,8 +1,9 @@
"""Cleanup for the Firecracker backend. """Cleanup for the Firecracker backend.
Orphans are: firecracker VMM processes whose config lives under our run Orphans are: firecracker VMM processes whose config lives under our run
dir, and the per-bottle run dirs. TAP slots free themselves (the flock dir, the `bot-bottle-sidecars-*` containers, and the per-bottle run
drops when the launcher exits), so there is nothing to reclaim there. dirs. TAP slots free themselves (the flock drops when the launcher
exits), so there is nothing to reclaim there.
""" """
from __future__ import annotations from __future__ import annotations
@@ -17,6 +18,8 @@ from ...log import info
from . import util from . import util
from .bottle_cleanup_plan import FirecrackerBottleCleanupPlan from .bottle_cleanup_plan import FirecrackerBottleCleanupPlan
_SIDECAR_PREFIX = "bot-bottle-sidecars-"
def _run_root() -> Path: def _run_root() -> Path:
return util.cache_dir() / "run" return util.cache_dir() / "run"
@@ -43,6 +46,17 @@ def _orphan_vm_pids() -> list[int]:
return pids return pids
def _sidecar_containers() -> list[str]:
result = subprocess.run(
["docker", "ps", "-a", "--format", "{{.Names}}",
"--filter", f"name={_SIDECAR_PREFIX}"],
capture_output=True, text=True, check=False,
)
if result.returncode != 0:
return []
return sorted(n.strip() for n in result.stdout.splitlines() if n.strip())
def _run_dirs() -> list[str]: def _run_dirs() -> list[str]:
run_root = _run_root() run_root = _run_root()
if not run_root.is_dir(): if not run_root.is_dir():
@@ -53,6 +67,7 @@ def _run_dirs() -> list[str]:
def prepare_cleanup() -> FirecrackerBottleCleanupPlan: def prepare_cleanup() -> FirecrackerBottleCleanupPlan:
return FirecrackerBottleCleanupPlan( return FirecrackerBottleCleanupPlan(
vm_pids=tuple(_orphan_vm_pids()), vm_pids=tuple(_orphan_vm_pids()),
containers=tuple(_sidecar_containers()),
run_dirs=tuple(_run_dirs()), run_dirs=tuple(_run_dirs()),
) )
@@ -64,6 +79,12 @@ def cleanup(plan: FirecrackerBottleCleanupPlan) -> None:
os.kill(pid, signal.SIGTERM) os.kill(pid, signal.SIGTERM)
except ProcessLookupError: except ProcessLookupError:
pass pass
for name in plan.containers:
info(f"docker rm -f {name}")
subprocess.run(
["docker", "rm", "-f", name],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
)
for path in plan.run_dirs: for path in plan.run_dirs:
info(f"rm -rf {path}") info(f"rm -rf {path}")
shutil.rmtree(path, ignore_errors=True) shutil.rmtree(path, ignore_errors=True)
+33 -4
View File
@@ -1,14 +1,43 @@
"""Active-agent enumeration for the Firecracker backend. """Active-agent enumeration for the Firecracker backend.
The backend is disabled during the companion-container removal (#385) — it can't The agent runs in a VM (no container to list), so a live bottle is
launch bottles, so there are none to enumerate. Real enumeration returns identified by its running sidecar container `bot-bottle-sidecars-<slug>`
with the backend's consolidated relaunch (#354). the same discovery-by-prefix the other backends use.
""" """
from __future__ import annotations from __future__ import annotations
import subprocess
from ...bottle_state import read_metadata
from .. import ActiveAgent from .. import ActiveAgent
_SIDECAR_PREFIX = "bot-bottle-sidecars-"
def enumerate_active() -> list[ActiveAgent]: def enumerate_active() -> list[ActiveAgent]:
return [] result = subprocess.run(
["docker", "ps", "--format", "{{.Names}}",
"--filter", f"name={_SIDECAR_PREFIX}"],
capture_output=True, text=True, check=False,
)
if result.returncode != 0:
return []
out: list[ActiveAgent] = []
for name in sorted(n.strip() for n in result.stdout.splitlines() if n.strip()):
slug = name[len(_SIDECAR_PREFIX):]
metadata = read_metadata(slug)
if metadata is None or metadata.backend != "firecracker":
# Skip sidecars owned by another backend (docker shares the
# container-name prefix).
continue
out.append(ActiveAgent(
backend_name="firecracker",
slug=slug,
agent_name=metadata.agent_name,
started_at=metadata.started_at,
services=(),
label=metadata.label,
color=metadata.color,
))
return out
+385 -20
View File
@@ -1,39 +1,404 @@
"""Launch flow for the Firecracker backend — temporarily disabled (#385). """Launch flow for the Firecracker backend.
The firecracker backend launched a per-bottle companion container (the Per bottle:
egress / git-gate / supervise data plane) alongside each microVM. That 1. mint the egress CA, build the agent image (docker), export it to a
per-bottle-companion architecture was removed in the companion-container removal; cached ext4 rootfs;
firecracker's replacement — the consolidated per-host gateway — lands in 2. claim a free TAP pool slot (rootless flock);
its own cutover (#354). 3. bring up the Docker sidecar bundle, publishing egress / git-gate /
supervise on the slot's host-side TAP IP at fixed ports;
4. boot the microVM on that TAP; wait for SSH;
5. provision (CA, prompt, skills, workspace, git, supervise) over SSH.
Until that lands, launching a firecracker bottle fails closed rather than Isolation is enforced by the operator-provisioned nft table (checked
silently running the removed path. `prepare` / `status` / cleanup still fail-closed in preflight): a VM reaches only its sidecar (DNAT'd from
work, so `backend status --backend=firecracker` and orphan cleanup are the host TAP IP) and nothing else. The agent's HTTPS_PROXY therefore
unaffected. points at `http://<host_tap_ip>:9099`, its only route to the world.
""" """
from __future__ import annotations from __future__ import annotations
from contextlib import contextmanager import dataclasses
import os
import subprocess
from contextlib import ExitStack, contextmanager
from pathlib import Path
from typing import Callable, Generator from typing import Callable, Generator
from ...log import die from ...bottle_state import (
egress_state_dir,
git_gate_state_dir,
read_committed_image,
)
from ...egress import (
EGRESS_ROUTES_IN_CONTAINER,
egress_agent_env_entries,
egress_resolve_token_values,
egress_sidecar_env_entries,
)
from ...git_gate import (
provision_git_gate_dynamic_keys,
revoke_git_gate_provisioned_keys,
)
from ...log import die, info, warn
from ...supervise import DB_PATH_IN_CONTAINER, SUPERVISE_PORT
from ...util import expand_tilde
from ..docker.egress import (
EGRESS_CA_IN_CONTAINER,
EGRESS_PORT,
egress_tls_init,
)
from ..docker.git_gate import (
GIT_GATE_ACCESS_HOOK_IN_CONTAINER,
GIT_GATE_CREDS_DIR_IN_CONTAINER,
GIT_GATE_ENTRYPOINT_IN_CONTAINER,
GIT_GATE_HOOK_IN_CONTAINER,
)
from ..docker.sidecar_bundle import (
SIDECAR_BUNDLE_DOCKERFILE,
SIDECAR_BUNDLE_IMAGE,
)
from ..util import AGENT_CA_BUNDLE, AGENT_CA_PATH
from . import firecracker_vm, isolation_probe, netpool, util
from .bottle import FirecrackerBottle from .bottle import FirecrackerBottle
from .bottle_plan import FirecrackerBottlePlan from .bottle_plan import FirecrackerBottlePlan
_REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent)
_GIT_HTTP_PORT = 9420
_GIT_GATE_READY_FILE = "/run/git-gate/ready"
def sidecar_container_name(slug: str) -> str:
return f"bot-bottle-sidecars-{slug}"
@contextmanager @contextmanager
def launch( def launch(
plan: FirecrackerBottlePlan, plan: FirecrackerBottlePlan,
*, *,
provision: Callable[[FirecrackerBottlePlan, "FirecrackerBottle"], str | None], provision: Callable[[FirecrackerBottlePlan, "FirecrackerBottle"], str | None],
) -> Generator[FirecrackerBottle, None, None]: ) -> Generator[FirecrackerBottle, None, None]:
"""Fail closed: the firecracker backend is disabled while its stack = ExitStack()
consolidated (gateway-backed) launch is built in #354.""" bottle_for_revoke = plan.manifest.bottle
del plan, provision git_gate_dir_for_revoke = git_gate_state_dir(plan.slug)
die(
"the firecracker backend is temporarily disabled during the " def teardown() -> None:
"companion-container removal (#385); its consolidated relaunch " teardown_exc: BaseException | None = None
"lands in #354. Use --backend=docker for now." try:
stack.close()
except BaseException as exc: # noqa: W0718 - teardown must continue
teardown_exc = exc
warn(f"firecracker teardown failed: {exc!r}")
revoke_git_gate_provisioned_keys(bottle_for_revoke, git_gate_dir_for_revoke)
if teardown_exc is not None:
raise teardown_exc
try:
plan = _mint_certs(plan)
plan = _build_agent_image(plan)
# Claim a TAP slot; the flock is held until teardown closes it.
slot, lock = netpool.allocate(plan.slug)
stack.callback(lock.close)
info(f"firecracker slot {slot.iface}: host={slot.host_ip} "
f"guest={slot.guest_ip}")
plan = _provision_git_gate_keys(plan)
sidecar_name = sidecar_container_name(plan.slug)
_force_remove_container(sidecar_name)
_start_sidecar_bundle(plan, sidecar_name, slot.host_ip)
stack.callback(_force_remove_container, sidecar_name)
_stage_git_gate(plan, sidecar_name)
plan = _stamp_agent_urls(plan, slot.host_ip)
# Build the per-bottle rootfs + SSH key, then boot.
base_dir = util.build_base_rootfs_dir(plan.image)
run_dir = util.cache_dir() / "run" / plan.slug
run_dir.mkdir(parents=True, exist_ok=True)
rootfs = run_dir / "rootfs.ext4"
util.build_rootfs_ext4(base_dir, rootfs)
private_key, pubkey = util.generate_keypair(run_dir)
vm = firecracker_vm.boot(
name=plan.container_name,
rootfs=rootfs,
tap=slot.iface,
guest_ip=slot.guest_ip,
host_ip=slot.host_ip,
pubkey=pubkey,
run_dir=run_dir,
)
stack.callback(vm.terminate)
firecracker_vm.wait_for_ssh(vm, private_key)
# Authoritative fail-closed egress-boundary check, before the
# agent runs: prove the VM cannot reach the host directly.
isolation_probe.verify_isolation(private_key, slot.guest_ip)
bottle = FirecrackerBottle(
plan.container_name,
private_key=private_key,
guest_ip=slot.guest_ip,
guest_env=_agent_guest_env(plan, slot.host_ip),
agent_command=plan.agent_command,
agent_prompt_mode=plan.agent_prompt_mode,
agent_provider_template=plan.agent_provider_template,
terminal_title=(
f"{plan.spec.label} ({plan.spec.agent_name})"
if plan.spec.label else plan.spec.agent_name
),
terminal_color=plan.spec.color,
agent_workdir=plan.workspace_plan.workdir,
)
bottle.prompt_path = provision(plan, bottle)
yield bottle
finally:
teardown()
def _mint_certs(plan: FirecrackerBottlePlan) -> FirecrackerBottlePlan:
egress_ca_host, egress_ca_cert_only = egress_tls_init(egress_state_dir(plan.slug))
egress_plan = dataclasses.replace(
plan.egress_plan,
mitmproxy_ca_host_path=egress_ca_host,
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
) )
yield # unreachable — `die` raises; keeps this a generator/contextmanager return dataclasses.replace(plan, egress_plan=egress_plan)
def _build_agent_image(plan: FirecrackerBottlePlan) -> FirecrackerBottlePlan:
_docker_build(SIDECAR_BUNDLE_IMAGE, _REPO_DIR, dockerfile=SIDECAR_BUNDLE_DOCKERFILE)
committed = read_committed_image(plan.slug)
if committed and _image_exists(committed):
info(f"using committed image {committed!r}")
return dataclasses.replace(
plan,
agent_provision=dataclasses.replace(plan.agent_provision, image=committed),
)
_docker_build(plan.image, _REPO_DIR, dockerfile=plan.dockerfile_path)
return plan
def _provision_git_gate_keys(plan: FirecrackerBottlePlan) -> FirecrackerBottlePlan:
if not plan.git_gate_plan.upstreams:
return plan
git_gate_plan = provision_git_gate_dynamic_keys(
plan.manifest.bottle, plan.git_gate_plan, git_gate_state_dir(plan.slug),
)
return dataclasses.replace(plan, git_gate_plan=git_gate_plan)
def _stamp_agent_urls(
plan: FirecrackerBottlePlan, host_ip: str,
) -> FirecrackerBottlePlan:
proxy_url = f"http://{host_ip}:{EGRESS_PORT}"
supervise_url = (
f"http://{host_ip}:{SUPERVISE_PORT}/" if plan.supervise_plan is not None else ""
)
git_gate_url = (
f"http://{host_ip}:{_GIT_HTTP_PORT}" if plan.git_gate_plan.upstreams else ""
)
return dataclasses.replace(
plan,
agent_proxy_url=proxy_url,
agent_git_gate_url=git_gate_url,
agent_supervise_url=supervise_url,
)
# --- sidecar bundle (Docker) ----------------------------------------
def _start_sidecar_bundle(
plan: FirecrackerBottlePlan, sidecar_name: str, host_ip: str,
) -> None:
argv = ["docker", "run", "--name", sidecar_name, "--detach", "--rm",
"-e", f"BOT_BOTTLE_SIDECAR_DAEMONS={','.join(_sidecar_daemons(plan))}"]
for entry in _sidecar_env_entries(plan):
argv += ["-e", entry]
for host_path, container_path, read_only in _sidecar_mounts(plan):
argv += ["-v", f"{host_path}:{container_path}{':ro' if read_only else ''}"]
# Publish on the slot's host TAP IP at fixed ports — each bottle
# has a distinct host_ip, so fixed ports never collide, and the VM
# reaches them at a stable, well-known address (its only route out).
for port in _sidecar_ports(plan):
argv += ["-p", f"{host_ip}:{port}:{port}"]
argv.append(SIDECAR_BUNDLE_IMAGE)
effective_env = {**dict(os.environ), **plan.agent_provision.provisioned_env}
token_values = egress_resolve_token_values(
plan.egress_plan.token_env_map, effective_env,
)
env = {**os.environ, **token_values}
info(f"docker run sidecar bundle {sidecar_name} (published on {host_ip})")
result = subprocess.run(argv, capture_output=True, text=True, env=env, check=False)
if result.returncode != 0:
die(f"docker run for sidecar bundle {sidecar_name} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}")
def _sidecar_daemons(plan: FirecrackerBottlePlan) -> tuple[str, ...]:
daemons = ["egress"]
if plan.git_gate_plan.upstreams:
daemons += ["git-gate", "git-http"]
if plan.supervise_plan is not None:
daemons.append("supervise")
return tuple(daemons)
def _sidecar_ports(plan: FirecrackerBottlePlan) -> tuple[int, ...]:
ports = [EGRESS_PORT]
if plan.git_gate_plan.upstreams:
ports.append(_GIT_HTTP_PORT)
if plan.supervise_plan is not None:
ports.append(SUPERVISE_PORT)
return tuple(ports)
def _sidecar_env_entries(plan: FirecrackerBottlePlan) -> tuple[str, ...]:
env: list[str] = list(egress_sidecar_env_entries(plan.egress_plan))
if plan.git_gate_plan.upstreams:
env.append(f"BOT_BOTTLE_GIT_GATE_READY_FILE={_GIT_GATE_READY_FILE}")
if plan.supervise_plan is not None:
env += [
f"SUPERVISE_BOTTLE_SLUG={plan.slug}",
f"SUPERVISE_DB_PATH={DB_PATH_IN_CONTAINER}",
f"SUPERVISE_PORT={SUPERVISE_PORT}",
]
return tuple(env)
def _sidecar_mounts(
plan: FirecrackerBottlePlan,
) -> tuple[tuple[str, str, bool], ...]:
mounts: list[tuple[str, str, bool]] = []
ep = plan.egress_plan
mounts.append((str(ep.mitmproxy_ca_host_path.parent),
str(Path(EGRESS_CA_IN_CONTAINER).parent), False))
if ep.routes:
mounts.append((str(ep.routes_path.parent),
str(Path(EGRESS_ROUTES_IN_CONTAINER).parent), True))
sp = plan.supervise_plan
if sp is not None:
mounts.append((str(sp.db_path.parent),
str(Path(DB_PATH_IN_CONTAINER).parent), False))
return tuple(mounts)
def _stage_git_gate(plan: FirecrackerBottlePlan, sidecar_name: str) -> None:
gp = plan.git_gate_plan
if not gp.upstreams:
return
_docker_exec(sidecar_name, [
"mkdir", "-p",
str(Path(GIT_GATE_HOOK_IN_CONTAINER).parent),
GIT_GATE_CREDS_DIR_IN_CONTAINER, "/git",
str(Path(_GIT_GATE_READY_FILE).parent),
])
for host_path, container_path in _git_gate_files(plan):
_docker_cp(host_path, f"{sidecar_name}:{container_path}")
_docker_exec(sidecar_name, [
"sh", "-c",
f"chmod 755 {GIT_GATE_ENTRYPOINT_IN_CONTAINER} "
f"{GIT_GATE_HOOK_IN_CONTAINER} {GIT_GATE_ACCESS_HOOK_IN_CONTAINER} && "
f"chmod 600 {GIT_GATE_CREDS_DIR_IN_CONTAINER}/* && "
f"touch {_GIT_GATE_READY_FILE}",
])
def _git_gate_files(plan: FirecrackerBottlePlan) -> tuple[tuple[str, str], ...]:
gp = plan.git_gate_plan
files: list[tuple[str, str]] = [
(str(gp.entrypoint_script), GIT_GATE_ENTRYPOINT_IN_CONTAINER),
(str(gp.hook_script), GIT_GATE_HOOK_IN_CONTAINER),
(str(gp.access_hook_script), GIT_GATE_ACCESS_HOOK_IN_CONTAINER),
]
for upstream in gp.upstreams:
files.append((expand_tilde(upstream.identity_file),
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{upstream.name}-key"))
if upstream.known_hosts_file:
files.append((str(upstream.known_hosts_file),
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{upstream.name}-known_hosts"))
return tuple(files)
# --- agent guest env -------------------------------------------------
def _agent_guest_env(plan: FirecrackerBottlePlan, host_ip: str) -> dict[str, str]:
"""Env injected into every agent/exec call over SSH. The VM has no
baked process env (it just runs init), so the proxy/CA/git/supervise
wiring is applied per-invocation."""
proxy_url = f"http://{host_ip}:{EGRESS_PORT}"
no_proxy = f"localhost,127.0.0.1,{host_ip}"
env: dict[str, str] = {
"HTTPS_PROXY": proxy_url, "HTTP_PROXY": proxy_url,
"https_proxy": proxy_url, "http_proxy": proxy_url,
"NO_PROXY": no_proxy, "no_proxy": no_proxy,
"NODE_EXTRA_CA_CERTS": AGENT_CA_PATH,
"SSL_CERT_FILE": AGENT_CA_BUNDLE,
"REQUESTS_CA_BUNDLE": AGENT_CA_BUNDLE,
}
if plan.agent_git_gate_url:
env["GIT_GATE_URL"] = plan.agent_git_gate_url
if plan.agent_supervise_url:
env["MCP_SUPERVISE_URL"] = plan.agent_supervise_url
for entry in egress_agent_env_entries(plan.egress_plan):
key, _, value = entry.partition("=")
env[key] = value
env.update(plan.agent_provision.guest_env)
# Forwarded (bare-name) env: resolve host values now, since the VM
# can't inherit them from a `docker run --env NAME`.
for name in plan.forwarded_env:
value = os.environ.get(name)
if value is not None:
env[name] = value
return env
# --- docker helpers --------------------------------------------------
def _docker_build(ref: str, context: str, *, dockerfile: str = "") -> None:
info(f"docker build {ref}")
args = ["docker", "build", "-t", ref]
if dockerfile:
if not os.path.isabs(dockerfile):
dockerfile = os.path.join(context, dockerfile)
args += ["-f", dockerfile]
args.append(context)
result = subprocess.run(args, check=False)
if result.returncode != 0:
die(f"docker build for {ref!r} failed")
def _image_exists(ref: str) -> bool:
return subprocess.run(
["docker", "image", "inspect", ref],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
).returncode == 0
def _force_remove_container(name: str) -> None:
subprocess.run(
["docker", "rm", "-f", name],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
)
def _docker_exec(name: str, argv: list[str]) -> None:
result = subprocess.run(
["docker", "exec", name, *argv], capture_output=True, text=True, check=False,
)
if result.returncode != 0:
die(f"docker exec in {name} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}")
def _docker_cp(host_path: str, dest: str) -> None:
result = subprocess.run(
["docker", "cp", host_path, dest], capture_output=True, text=True, check=False,
)
if result.returncode != 0:
die(f"docker cp {host_path} -> {dest} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}")
+3 -3
View File
@@ -18,7 +18,7 @@ Topology (per slot i):
* a /31 host<->guest link: host = base + 2i (the gateway the VM * a /31 host<->guest link: host = base + 2i (the gateway the VM
routes through), guest = base + 2i + 1 (the VM's address). routes through), guest = base + 2i + 1 (the VM's address).
* isolation via ``table inet bot_bottle_fc``: a VM reaches only its * isolation via ``table inet bot_bottle_fc``: a VM reaches only its
own gateway (DNAT'd from the host TAP IP) and nothing else. own sidecar (DNAT'd from the host TAP IP) and nothing else.
The default IP block is ``10.243.0.0/16`` — an intentionally obscure The default IP block is ``10.243.0.0/16`` — an intentionally obscure
corner of RFC-1918 private space. RFC-1918 is the range *designated* corner of RFC-1918 private space. RFC-1918 is the range *designated*
@@ -88,10 +88,10 @@ def ip_base() -> str:
return _cfg("BOT_BOTTLE_FC_IP_BASE") return _cfg("BOT_BOTTLE_FC_IP_BASE")
# Gateway ports the VM reaches at its host-side TAP IP. Kept in sync # Sidecar ports the VM reaches at its host-side TAP IP. Kept in sync
# with the backend constants (egress 9099, supervise 9100, git-http # with the backend constants (egress 9099, supervise 9100, git-http
# 9420); rendered into the setup output for operator visibility. # 9420); rendered into the setup output for operator visibility.
GATEWAY_PORTS = (9099, 9100, 9420) SIDECAR_PORTS = (9099, 9100, 9420)
@dataclass(frozen=True) @dataclass(frozen=True)
@@ -2,7 +2,7 @@
Selectable via `BOT_BOTTLE_BACKEND=macos-container`. This package owns Selectable via `BOT_BOTTLE_BACKEND=macos-container`. This package owns
the Apple `container` CLI integration; launch remains gated until the the Apple `container` CLI integration; launch remains gated until the
gateway network enforcement shape is implemented. sidecar network enforcement shape is implemented.
""" """
from .backend import MacosContainerBottleBackend from .backend import MacosContainerBottleBackend
@@ -9,6 +9,7 @@ from . import util as container_mod
from .bottle_cleanup_plan import MacosContainerBottleCleanupPlan from .bottle_cleanup_plan import MacosContainerBottleCleanupPlan
_PREFIX = "bot-bottle-" _PREFIX = "bot-bottle-"
_BUNDLE_PREFIX = "bot-bottle-sidecars-"
def _list_prefixed_containers() -> list[str]: def _list_prefixed_containers() -> list[str]:
@@ -23,7 +24,7 @@ def _list_prefixed_containers() -> list[str]:
return [] return []
return sorted( return sorted(
name for name in (line.strip() for line in result.stdout.splitlines()) name for name in (line.strip() for line in result.stdout.splitlines())
if name.startswith(_PREFIX) if name.startswith(_PREFIX) or name.startswith(_BUNDLE_PREFIX)
) )
@@ -1,24 +1,36 @@
"""Host-side egress route-apply for the macos-container backend. """Host-side egress apply for the macos-container backend.
The per-bottle companion container this used to signal (`container kill Uses `container kill --signal HUP` (Apple Container framework) instead
--signal HUP <container>`) was removed in the companion-container removal (#385), of `docker kill` to signal the sidecar bundle.
along with the disabled macOS launch path. Fails closed until the macOS
backend grows the consolidated gateway.
""" """
from __future__ import annotations from __future__ import annotations
import os
import subprocess
from ...log import warn
from ..egress_apply import EgressApplicator, EgressApplyError from ..egress_apply import EgressApplicator, EgressApplyError
from .launch import sidecar_container_name
class MacOSContainerEgressApplicator(EgressApplicator): class MacOSContainerEgressApplicator(EgressApplicator):
def _signal_bundle_reload(self, slug: str) -> None: def _signal_bundle_reload(self, slug: str) -> None:
del slug container = sidecar_container_name(slug)
raise EgressApplyError( result = subprocess.run(
"live egress route-apply was removed with the per-bottle " ["container", "kill", "--signal", "HUP", container],
"companion container (#385); the macos-container backend is " capture_output=True, text=True, check=False, env=os.environ,
"disabled until it uses the consolidated gateway."
) )
if result.returncode != 0:
last_error = (result.stderr or "").strip() or (result.stdout or "").strip()
warn(
f"egress: routes updated on disk for {slug}, but bundle reload failed: "
f"{last_error or 'container kill failed'}"
)
raise EgressApplyError(
f"could not reload egress bundle {container}: "
f"{last_error or 'container kill failed'}"
)
applicator = MacOSContainerEgressApplicator() applicator = MacOSContainerEgressApplicator()
@@ -1,14 +1,40 @@
"""Active-agent enumeration for the macOS Apple Container backend. """Active-agent enumeration for the macOS Apple Container backend."""
The backend is disabled during the companion-container removal (#385) — it can't
launch bottles, so there are none to enumerate. Enumeration returns when
the backend grows the consolidated gateway.
"""
from __future__ import annotations from __future__ import annotations
import subprocess
from ...bottle_state import read_metadata
from .. import ActiveAgent from .. import ActiveAgent
_PREFIX = "bot-bottle-"
_SIDECAR_PREFIX = "bot-bottle-sidecars-"
def enumerate_active() -> list[ActiveAgent]: def enumerate_active() -> list[ActiveAgent]:
return [] result = subprocess.run(
["container", "list", "--quiet"],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
return []
out: list[ActiveAgent] = []
for name in sorted(line.strip() for line in result.stdout.splitlines()):
if not name.startswith(_PREFIX):
continue
if name.startswith(_SIDECAR_PREFIX):
continue
slug = name[len(_PREFIX):]
metadata = read_metadata(slug)
out.append(ActiveAgent(
backend_name="macos-container",
slug=slug,
agent_name=metadata.agent_name if metadata else "?",
started_at=metadata.started_at if metadata else "",
services=(),
label=metadata.label if metadata else "",
color=metadata.color if metadata else "",
))
return out
+440 -21
View File
@@ -1,39 +1,458 @@
"""Launch flow for the macOS Apple Container backend — disabled (#385). """Launch flow for the macOS Apple Container backend.
This backend launched a per-bottle companion container (the egress / This backend keeps the explicit proxy-env enforcement model for v1:
git-gate / supervise data plane) alongside the agent container, with the the agent container is attached only to a host-only Apple Container
agent's proxy env pointed at the companion's host-only IP. That network, while the sidecar bundle is attached to a NAT network first
per-bottle-companion architecture was removed in the companion-container removal; and the host-only network second. The sidecar's host-only IP is
the macOS backend will be re-enabled once it grows the consolidated discovered from `container inspect` and stamped into the agent's
per-host gateway the docker backend already uses. HTTP_PROXY / HTTPS_PROXY env vars.
Until then, launching a macOS bottle fails closed. `prepare` / `status`
/ cleanup still work.
""" """
from __future__ import annotations from __future__ import annotations
from contextlib import contextmanager import dataclasses
import os
import subprocess
from contextlib import ExitStack, contextmanager
from pathlib import Path
from typing import Callable, Generator from typing import Callable, Generator
from ...log import die from ...bottle_state import (
egress_state_dir,
git_gate_state_dir,
read_committed_image,
)
from ...egress import (
EGRESS_ROUTES_IN_CONTAINER,
egress_agent_env_entries,
egress_resolve_token_values,
egress_sidecar_env_entries,
)
from ...git_gate import (
provision_git_gate_dynamic_keys,
revoke_git_gate_provisioned_keys,
)
from ...log import die, info, warn
from ...supervise import DB_PATH_IN_CONTAINER, SUPERVISE_PORT
from ...util import expand_tilde
from ..docker.egress import EGRESS_CA_IN_CONTAINER, EGRESS_PORT
from ..docker.git_gate import (
GIT_GATE_ACCESS_HOOK_IN_CONTAINER,
GIT_GATE_CREDS_DIR_IN_CONTAINER,
GIT_GATE_ENTRYPOINT_IN_CONTAINER,
GIT_GATE_HOOK_IN_CONTAINER,
)
from ..docker.sidecar_bundle import (
SIDECAR_BUNDLE_DOCKERFILE,
SIDECAR_BUNDLE_IMAGE,
)
from ..docker.egress import egress_tls_init
from ..util import AGENT_CA_BUNDLE, AGENT_CA_PATH
from . import util as container_mod
from .bottle import MacosContainerBottle from .bottle import MacosContainerBottle
from .bottle_plan import MacosContainerBottlePlan from .bottle_plan import MacosContainerBottlePlan
_REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent)
_AGENT_SLEEP_SECONDS = "2147483647"
_GIT_HTTP_PORT = 9420
_GIT_GATE_READY_FILE = "/run/git-gate/ready"
def internal_network_name(slug: str) -> str:
return f"bot-bottle-net-{slug}"
def egress_network_name(slug: str) -> str:
return f"bot-bottle-egress-{slug}"
def sidecar_container_name(slug: str) -> str:
return f"bot-bottle-sidecars-{slug}"
@contextmanager @contextmanager
def launch( def launch(
plan: MacosContainerBottlePlan, plan: MacosContainerBottlePlan,
*, *,
provision: Callable[[MacosContainerBottlePlan, "MacosContainerBottle"], str | None], provision: Callable[[MacosContainerBottlePlan, "MacosContainerBottle"], str | None],
) -> Generator[MacosContainerBottle, None, None]: ) -> Generator[MacosContainerBottle, None, None]:
"""Fail closed: the macOS backend is disabled until it grows the """Build, run, provision, and yield an Apple Container bottle."""
consolidated per-host gateway (the companion-container path it used stack = ExitStack()
was removed in #385).""" bottle_for_revoke = plan.manifest.bottle
del plan, provision git_gate_dir_for_revoke = git_gate_state_dir(plan.slug)
die(
"the macos-container backend is temporarily disabled during the " def teardown() -> None:
"companion-container removal (#385); it will return once it uses " teardown_exc: BaseException | None = None
"the consolidated gateway. Use --backend=docker for now." try:
stack.close()
except BaseException as exc: # noqa: W0718 - teardown must continue
teardown_exc = exc
warn(f"macos-container teardown failed: {exc!r}")
revoke_git_gate_provisioned_keys(bottle_for_revoke, git_gate_dir_for_revoke)
if teardown_exc is not None:
raise teardown_exc
try:
plan = _mint_certs(plan)
plan = _build_images(plan)
internal_network = internal_network_name(plan.slug)
egress_network = egress_network_name(plan.slug)
_create_networks(internal_network, egress_network, stack)
plan = _provision_git_gate_keys(plan)
sidecar_name = sidecar_container_name(plan.slug)
container_mod.force_remove_container(sidecar_name)
_start_sidecar_bundle(plan, sidecar_name, internal_network, egress_network)
stack.callback(container_mod.force_remove_container, sidecar_name)
_stage_git_gate(plan, sidecar_name)
sidecar_ip = container_mod.container_ipv4_on_network(
sidecar_name, internal_network,
)
plan = _stamp_agent_urls(plan, sidecar_ip)
container_mod.force_remove_container(plan.container_name)
_start_agent(plan, internal_network, sidecar_ip)
stack.callback(container_mod.force_remove_container, plan.container_name)
bottle = MacosContainerBottle(
plan.container_name,
teardown,
None,
agent_command=plan.agent_command,
agent_prompt_mode=plan.agent_prompt_mode,
agent_provider_template=plan.agent_provider_template,
terminal_title=f"{plan.spec.label} ({plan.spec.agent_name})" if plan.spec.label else plan.spec.agent_name,
terminal_color=plan.spec.color,
agent_workdir=plan.workspace_plan.workdir,
)
bottle.prompt_path = provision(plan, bottle)
yield bottle
finally:
teardown()
def _mint_certs(plan: MacosContainerBottlePlan) -> MacosContainerBottlePlan:
egress_ca_host, egress_ca_cert_only = egress_tls_init(
egress_state_dir(plan.slug),
) )
yield # unreachable — `die` raises; keeps this a generator/contextmanager egress_plan = dataclasses.replace(
plan.egress_plan,
mitmproxy_ca_host_path=egress_ca_host,
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
)
return dataclasses.replace(plan, egress_plan=egress_plan)
def _build_images(plan: MacosContainerBottlePlan) -> MacosContainerBottlePlan:
container_mod.build_image(
SIDECAR_BUNDLE_IMAGE,
_REPO_DIR,
dockerfile=SIDECAR_BUNDLE_DOCKERFILE,
)
committed = read_committed_image(plan.slug)
if committed and container_mod.image_exists(committed):
info(f"using committed image {committed!r}")
return dataclasses.replace(
plan,
agent_provision=dataclasses.replace(
plan.agent_provision,
image=committed,
),
)
container_mod.build_image(
plan.image,
_REPO_DIR,
dockerfile=plan.dockerfile_path,
)
return plan
def _create_networks(
internal_network: str,
egress_network: str,
stack: ExitStack,
) -> None:
container_mod.create_network(internal_network, internal=True)
stack.callback(container_mod.remove_network, internal_network)
container_mod.create_network(egress_network)
stack.callback(container_mod.remove_network, egress_network)
def _start_sidecar_bundle(
plan: MacosContainerBottlePlan,
sidecar_name: str,
internal_network: str,
egress_network: str,
) -> None:
argv = _sidecar_run_argv(plan, sidecar_name, internal_network, egress_network)
effective_env = {**dict(os.environ), **plan.agent_provision.provisioned_env}
token_values = egress_resolve_token_values(
plan.egress_plan.token_env_map, effective_env,
)
env = {**os.environ, **token_values}
info(f"container run sidecar bundle {sidecar_name}")
result = subprocess.run(
argv, capture_output=True, text=True, env=env, check=False,
)
if result.returncode != 0:
die(
f"container run for sidecar bundle {sidecar_name} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
def _start_agent(
plan: MacosContainerBottlePlan,
internal_network: str,
sidecar_ip: str,
) -> None:
argv = _agent_run_argv(plan, internal_network, sidecar_ip)
env = {
**os.environ,
**plan.forwarded_env,
}
info(f"container run agent {plan.container_name}")
result = subprocess.run(
argv, capture_output=True, text=True, env=env, check=False,
)
if result.returncode != 0:
die(
f"container run for agent {plan.container_name} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
def _stamp_agent_urls(
plan: MacosContainerBottlePlan,
sidecar_ip: str,
) -> MacosContainerBottlePlan:
proxy_url = f"http://{sidecar_ip}:{EGRESS_PORT}"
supervise_url = ""
if plan.supervise_plan is not None:
supervise_url = f"http://{sidecar_ip}:{SUPERVISE_PORT}/"
git_gate_url = ""
if plan.git_gate_plan.upstreams:
git_gate_url = f"http://{sidecar_ip}:{_GIT_HTTP_PORT}"
return dataclasses.replace(
plan,
agent_proxy_url=proxy_url,
agent_git_gate_url=git_gate_url,
agent_supervise_url=supervise_url,
)
def _provision_git_gate_keys(
plan: MacosContainerBottlePlan,
) -> MacosContainerBottlePlan:
if not plan.git_gate_plan.upstreams:
return plan
git_gate_plan = provision_git_gate_dynamic_keys(
plan.manifest.bottle,
plan.git_gate_plan,
git_gate_state_dir(plan.slug),
)
return dataclasses.replace(plan, git_gate_plan=git_gate_plan)
def _stage_git_gate(plan: MacosContainerBottlePlan, sidecar_name: str) -> None:
gp = plan.git_gate_plan
if not gp.upstreams:
return
container_mod.exec_container(
sidecar_name,
[
"mkdir",
"-p",
str(Path(GIT_GATE_HOOK_IN_CONTAINER).parent),
GIT_GATE_CREDS_DIR_IN_CONTAINER,
"/git",
str(Path(_GIT_GATE_READY_FILE).parent),
],
)
for host_path, container_path in _git_gate_files(plan):
container_mod.copy_into_container(
sidecar_name, host_path, container_path,
)
container_mod.exec_container(
sidecar_name,
[
"sh",
"-c",
"chmod 755 "
f"{GIT_GATE_ENTRYPOINT_IN_CONTAINER} "
f"{GIT_GATE_HOOK_IN_CONTAINER} "
f"{GIT_GATE_ACCESS_HOOK_IN_CONTAINER} && "
f"chmod 600 {GIT_GATE_CREDS_DIR_IN_CONTAINER}/* && "
f"touch {_GIT_GATE_READY_FILE}",
],
)
def _git_gate_files(
plan: MacosContainerBottlePlan,
) -> tuple[tuple[str, str], ...]:
gp = plan.git_gate_plan
files: list[tuple[str, str]] = [
(str(gp.entrypoint_script), GIT_GATE_ENTRYPOINT_IN_CONTAINER),
(str(gp.hook_script), GIT_GATE_HOOK_IN_CONTAINER),
(str(gp.access_hook_script), GIT_GATE_ACCESS_HOOK_IN_CONTAINER),
]
for upstream in gp.upstreams:
files.append((
expand_tilde(upstream.identity_file),
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{upstream.name}-key",
))
if upstream.known_hosts_file:
files.append((
str(upstream.known_hosts_file),
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{upstream.name}-known_hosts",
))
return tuple(files)
def _sidecar_run_argv(
plan: MacosContainerBottlePlan,
sidecar_name: str,
internal_network: str,
egress_network: str,
) -> list[str]:
argv = [
"container", "run",
"--name", sidecar_name,
"--detach",
"--rm",
"--network", egress_network,
"--network", internal_network,
"--dns", _sidecar_dns(),
"--env", f"BOT_BOTTLE_SIDECAR_DAEMONS={','.join(_sidecar_daemons(plan))}",
]
for entry in _sidecar_env_entries(plan):
argv += ["--env", entry]
for host_path, container_path, read_only in _sidecar_mounts(plan):
argv += ["--mount", _mount_spec(host_path, container_path, read_only)]
argv.append(SIDECAR_BUNDLE_IMAGE)
return argv
def _agent_run_argv(
plan: MacosContainerBottlePlan,
internal_network: str,
sidecar_ip: str,
) -> list[str]:
argv = [
"container", "run",
"--name", plan.container_name,
"--detach",
"--network", internal_network,
]
for entry in _agent_env_entries(plan, sidecar_ip):
argv += ["--env", entry]
argv += [plan.image, "sleep", _AGENT_SLEEP_SECONDS]
return argv
def _sidecar_dns() -> str:
return container_mod.dns_server()
def _sidecar_daemons(plan: MacosContainerBottlePlan) -> tuple[str, ...]:
daemons = ["egress"]
if plan.git_gate_plan.upstreams:
daemons += ["git-gate", "git-http"]
if plan.supervise_plan is not None:
daemons.append("supervise")
return tuple(daemons)
def _sidecar_env_entries(plan: MacosContainerBottlePlan) -> tuple[str, ...]:
env: list[str] = list(egress_sidecar_env_entries(plan.egress_plan))
if plan.git_gate_plan.upstreams:
env.append(f"BOT_BOTTLE_GIT_GATE_READY_FILE={_GIT_GATE_READY_FILE}")
if plan.supervise_plan is not None:
env += [
f"SUPERVISE_BOTTLE_SLUG={plan.slug}",
f"SUPERVISE_DB_PATH={DB_PATH_IN_CONTAINER}",
f"SUPERVISE_PORT={SUPERVISE_PORT}",
]
return tuple(env)
def _sidecar_mounts(
plan: MacosContainerBottlePlan,
) -> tuple[tuple[str, str, bool], ...]:
mounts: list[tuple[str, str, bool]] = []
ep = plan.egress_plan
mounts.append((
str(ep.mitmproxy_ca_host_path.parent),
str(Path(EGRESS_CA_IN_CONTAINER).parent),
False,
))
if ep.routes:
mounts.append((
str(ep.routes_path.parent),
str(Path(EGRESS_ROUTES_IN_CONTAINER).parent),
True,
))
sp = plan.supervise_plan
if sp is not None:
# `container run --mount type=bind` only accepts directory
# sources (a file source fails with "is not a directory") —
# mount db_path's dedicated parent dir instead of the file
# itself, same as the CA/routes mounts above.
mounts.append((
str(sp.db_path.parent),
str(Path(DB_PATH_IN_CONTAINER).parent),
False,
))
return tuple(mounts)
def _mount_spec(host_path: str, container_path: str, read_only: bool) -> str:
spec = f"type=bind,source={host_path},target={container_path}"
if read_only:
spec += ",readonly"
return spec
def _agent_env_entries(
plan: MacosContainerBottlePlan,
sidecar_ip: str,
) -> tuple[str, ...]:
proxy_url = f"http://{sidecar_ip}:{EGRESS_PORT}"
no_proxy = _agent_no_proxy(plan, sidecar_ip)
env = [
f"HTTPS_PROXY={proxy_url}",
f"HTTP_PROXY={proxy_url}",
f"https_proxy={proxy_url}",
f"http_proxy={proxy_url}",
f"NO_PROXY={no_proxy}",
f"no_proxy={no_proxy}",
f"NODE_EXTRA_CA_CERTS={AGENT_CA_PATH}",
f"SSL_CERT_FILE={AGENT_CA_BUNDLE}",
f"REQUESTS_CA_BUNDLE={AGENT_CA_BUNDLE}",
]
if plan.agent_git_gate_url:
env.append(f"GIT_GATE_URL={plan.agent_git_gate_url}")
if plan.agent_supervise_url:
env.append(f"MCP_SUPERVISE_URL={plan.agent_supervise_url}")
for name, value in sorted(plan.agent_provision.guest_env.items()):
env.append(f"{name}={value}")
for name in sorted(plan.forwarded_env.keys()):
env.append(name)
env.extend(egress_agent_env_entries(plan.egress_plan))
return tuple(env)
def _agent_no_proxy(plan: MacosContainerBottlePlan, sidecar_ip: str) -> str:
hosts = ["localhost", "127.0.0.1", sidecar_ip]
return ",".join(hosts)
+1 -1
View File
@@ -94,7 +94,7 @@ def prepare_egress(
def prepare_supervise(bottle: ManifestBottle, slug: str) -> SupervisePlan | None: def prepare_supervise(bottle: ManifestBottle, slug: str) -> SupervisePlan | None:
"""Prepare the supervise daemon state dir. Returns None when """Prepare the supervise sidecar state dir. Returns None when
bottle.supervise is falsy.""" bottle.supervise is falsy."""
if not bottle.supervise: if not bottle.supervise:
return None return None
+9 -9
View File
@@ -44,16 +44,16 @@ _STATE_SUBDIR = "state"
_PER_BOTTLE_DOCKERFILE_NAME = "Dockerfile" _PER_BOTTLE_DOCKERFILE_NAME = "Dockerfile"
_COMMITTED_IMAGE_NAME = "committed-image" _COMMITTED_IMAGE_NAME = "committed-image"
_TRANSCRIPT_SUBDIR = "transcript" _TRANSCRIPT_SUBDIR = "transcript"
# Per-daemon scratch subdirs. PRD 0018 chunk 2: bind-mount sources # Per-sidecar scratch subdirs. PRD 0018 chunk 2: bind-mount sources
# live here so chunk 3's `docker compose up` can find them at stable # live here so chunk 3's `docker compose up` can find them at stable
# paths. Each daemon's `prepare()` writes config + CAs into its own # paths. Each sidecar's `prepare()` writes config + CAs into its own
# subdir; the launch step is unchanged today (still `docker cp`). # subdir; the launch step is unchanged today (still `docker cp`).
_EGRESS_SUBDIR = "egress" _EGRESS_SUBDIR = "egress"
_GIT_GATE_SUBDIR = "git-gate" _GIT_GATE_SUBDIR = "git-gate"
_SUPERVISE_SUBDIR = "supervise" _SUPERVISE_SUBDIR = "supervise"
_AGENT_SUBDIR = "agent" _AGENT_SUBDIR = "agent"
_METADATA_NAME = "metadata.json" _METADATA_NAME = "metadata.json"
# Live-config dir bind-mounted into the supervise daemon (read-only). # Live-config dir bind-mounted into the supervise sidecar (read-only).
# Host's apply paths keep these files fresh so supervise's # Host's apply paths keep these files fresh so supervise's
# `list-egress-routes` MCP tool returns the current state — # `list-egress-routes` MCP tool returns the current state —
# not a snapshot from launch time. # not a snapshot from launch time.
@@ -222,7 +222,7 @@ def per_bottle_image_tag(identity: str) -> str:
def live_config_dir(identity: str) -> Path: def live_config_dir(identity: str) -> Path:
"""Per-bottle live-config dir. Bind-mounted read-only into the """Per-bottle live-config dir. Bind-mounted read-only into the
supervise daemon; the host's apply paths refresh the files on supervise sidecar; the host's apply paths refresh the files on
every operator approval so the agent's `list-*` MCP tools always every operator approval so the agent's `list-*` MCP tools always
return current state.""" return current state."""
return bottle_state_dir(identity) / _LIVE_CONFIG_SUBDIR return bottle_state_dir(identity) / _LIVE_CONFIG_SUBDIR
@@ -260,9 +260,9 @@ def transcript_snapshot_dir(identity: str) -> Path:
return bottle_state_dir(identity) / _TRANSCRIPT_SUBDIR return bottle_state_dir(identity) / _TRANSCRIPT_SUBDIR
# --- Per-daemon scratch subdirs (PRD 0018 chunk 2) ------------------------ # --- Per-sidecar scratch subdirs (PRD 0018 chunk 2) ------------------------
# #
# Each daemon gets its own subdir under the bottle's state dir for # Each sidecar gets its own subdir under the bottle's state dir for
# bind-mount sources (config, CAs, hooks, etc.). Prepare-time writes # bind-mount sources (config, CAs, hooks, etc.). Prepare-time writes
# land here; the state dir's normal cleanup (`cleanup_state`) reaps # land here; the state dir's normal cleanup (`cleanup_state`) reaps
# them along with everything else when the bottle session ends and # them along with everything else when the bottle session ends and
@@ -270,20 +270,20 @@ def transcript_snapshot_dir(identity: str) -> Path:
def egress_state_dir(identity: str) -> Path: def egress_state_dir(identity: str) -> Path:
"""State subdir for the egress daemon: routes.yaml + the """State subdir for the egress sidecar: routes.yaml + the
per-bottle mitmproxy CA. Bind-mount source from chunk 3 onward.""" per-bottle mitmproxy CA. Bind-mount source from chunk 3 onward."""
return bottle_state_dir(identity) / _EGRESS_SUBDIR return bottle_state_dir(identity) / _EGRESS_SUBDIR
def git_gate_state_dir(identity: str) -> Path: def git_gate_state_dir(identity: str) -> Path:
"""State subdir for the git-gate daemon: entrypoint + hooks + """State subdir for the git-gate sidecar: entrypoint + hooks +
per-upstream known_hosts. Bind-mount source from chunk 3 per-upstream known_hosts. Bind-mount source from chunk 3
onward.""" onward."""
return bottle_state_dir(identity) / _GIT_GATE_SUBDIR return bottle_state_dir(identity) / _GIT_GATE_SUBDIR
def supervise_state_dir(identity: str) -> Path: def supervise_state_dir(identity: str) -> Path:
"""State subdir reserved for supervise daemon bind-mount sources. """State subdir reserved for supervise sidecar bind-mount sources.
Runtime queue/audit rows live in the host-level bot-bottle SQLite Runtime queue/audit rows live in the host-level bot-bottle SQLite
database, so they survive state-dir cleanup.""" database, so they survive state-dir cleanup."""
return bottle_state_dir(identity) / _SUPERVISE_SUBDIR return bottle_state_dir(identity) / _SUPERVISE_SUBDIR
+2 -2
View File
@@ -2,8 +2,8 @@
Walks every registered backend (docker, firecracker, macos-container) Walks every registered backend (docker, firecracker, macos-container)
so a single `./cli.py cleanup` reaps every backend's leftovers — a so a single `./cli.py cleanup` reaps every backend's leftovers — a
firecracker bottle's VM processes and run dirs won't survive a firecracker bottle's sidecars won't survive a docker-only cleanup pass
docker-only cleanup pass (issue addressed alongside #77). (issue addressed alongside #77).
Each backend's `prepare_cleanup` enumerates its own resources; Each backend's `prepare_cleanup` enumerates its own resources;
docker's `_list_orphan_state_dirs` consults docker's `_list_orphan_state_dirs` consults
+2 -2
View File
@@ -4,7 +4,7 @@ The Claude-specific behavior previously inlined under
`agent_provider.agent_provision_plan` (claude.json trust marker, `agent_provider.agent_provision_plan` (claude.json trust marker,
api.anthropic.com egress route, OAuth-token placeholder), plus api.anthropic.com egress route, OAuth-token placeholder), plus
the `claude mcp add` invocation that registers the supervise the `claude mcp add` invocation that registers the supervise
gateway in claude-code's user config (PRD 0013).""" sidecar in claude-code's user config (PRD 0013)."""
from __future__ import annotations from __future__ import annotations
@@ -293,7 +293,7 @@ class ClaudeAgentProvider(AgentProvider):
supervise_url: str, supervise_url: str,
) -> None: ) -> None:
"""Run `claude mcp add` inside the agent guest to register the """Run `claude mcp add` inside the agent guest to register the
supervise daemon in claude-code's user config (~/.claude.json). supervise sidecar in claude-code's user config (~/.claude.json).
Failure is logged but not fatal the bottle still works without Failure is logged but not fatal the bottle still works without
the entry; the operator can register it manually.""" the entry; the operator can register it manually."""
+2 -2
View File
@@ -4,7 +4,7 @@ The Codex-specific behavior previously inlined under
`agent_provider.agent_provision_plan` (config.toml trust marker, `agent_provider.agent_provision_plan` (config.toml trust marker,
chatgpt.com / api.openai.com egress routes, optional host-credential chatgpt.com / api.openai.com egress routes, optional host-credential
forwarding with dummy-auth.json + verify), plus the `codex mcp add` forwarding with dummy-auth.json + verify), plus the `codex mcp add`
invocation that registers the supervise daemon in Codex's invocation that registers the supervise sidecar in Codex's
~/.codex/config.toml (PRD 0050).""" ~/.codex/config.toml (PRD 0050)."""
from __future__ import annotations from __future__ import annotations
@@ -266,7 +266,7 @@ class CodexAgentProvider(AgentProvider):
supervise_url: str, supervise_url: str,
) -> None: ) -> None:
"""Run `codex mcp add` inside the agent guest to register the """Run `codex mcp add` inside the agent guest to register the
supervise daemon in Codex's user config (~/.codex/config.toml). supervise sidecar in Codex's user config (~/.codex/config.toml).
Mirrors the Claude provider's `claude mcp add` flow — failure Mirrors the Claude provider's `claude mcp add` flow — failure
is logged but not fatal.""" is logged but not fatal."""
+1 -1
View File
@@ -3,7 +3,7 @@
Pure Python, no mitmproxy dependency. Each detector is a module-level Pure Python, no mitmproxy dependency. Each detector is a module-level
function returning `ScanResult | None`. function returning `ScanResult | None`.
Ships flat into the gateway image alongside Ships flat into the sidecar bundle image alongside
`egress_addon_core.py` both this file and the package source use `egress_addon_core.py` both this file and the package source use
the same try/except import shim pattern. the same try/except import shim pattern.
""" """
+6 -6
View File
@@ -2,7 +2,7 @@
This module defines the abstract proxy (`Egress`), its plan This module defines the abstract proxy (`Egress`), its plan
dataclass (`EgressPlan`), and the resolved per-route shape dataclass (`EgressPlan`), and the resolved per-route shape
(`EgressRoute`). The gateway's start/stop lifecycle is backend- (`EgressRoute`). The sidecar's start/stop lifecycle is backend-
specific and lives on concrete subclasses (see specific and lives on concrete subclasses (see
`bot_bottle/backend/docker/egress.py`). `bot_bottle/backend/docker/egress.py`).
""" """
@@ -63,8 +63,8 @@ def _random_canary_env() -> str:
return f"{first}_{second}_SECRET" return f"{first}_{second}_SECRET"
def egress_gateway_env_entries(plan: "EgressPlan") -> tuple[str, ...]: def egress_sidecar_env_entries(plan: "EgressPlan") -> tuple[str, ...]:
"""Return gateway env entries needed by egress across all backends.""" """Return sidecar env entries needed by egress across all backends."""
env: list[str] = [] env: list[str] = []
if plan.routes: if plan.routes:
env.extend(sorted(plan.token_env_map.keys())) env.extend(sorted(plan.token_env_map.keys()))
@@ -87,7 +87,7 @@ class EgressRoute(Route):
Inherits `host`, `matches`, `auth_scheme`, and `token_env` Inherits `host`, `matches`, `auth_scheme`, and `token_env`
from `egress_addon_core.Route` those are the fields that cross the from `egress_addon_core.Route` those are the fields that cross the
YAML wire into the gateway. The fields below are host-only and YAML wire into the sidecar. The fields below are host-only and
are never serialised to the addon. are never serialised to the addon.
`token_ref` is the host env var the CLI reads at launch and forwards `token_ref` is the host env var the CLI reads at launch and forwards
@@ -386,7 +386,7 @@ class Egress(ABC):
routes_path.write_text(egress_render_routes(routes, log=log)) routes_path.write_text(egress_render_routes(routes, log=log))
routes_path.chmod(0o600) routes_path.chmod(0o600)
# Generate a per-session fake secret under a plausible random env name. # Generate a per-session fake secret under a plausible random env name.
# The gateway marks that exact env name as sensitive for known-secret # The sidecar marks that exact env name as sensitive for known-secret
# scanning; the agent receives the same name/value as exfil bait. # scanning; the agent receives the same name/value as exfil bait.
canary = secrets.token_urlsafe(32) canary = secrets.token_urlsafe(32)
return EgressPlan( return EgressPlan(
@@ -412,6 +412,6 @@ __all__ = [
"egress_resolve_token_values", "egress_resolve_token_values",
"egress_routes_for_bottle", "egress_routes_for_bottle",
"egress_agent_env_entries", "egress_agent_env_entries",
"egress_gateway_env_entries", "egress_sidecar_env_entries",
"egress_token_env_map", "egress_token_env_map",
] ]
+47 -83
View File
@@ -1,4 +1,4 @@
"""mitmproxy addon entrypoint for the egress gateway (PRD 0017, PRD 0053). """mitmproxy addon entrypoint for the egress sidecar (PRD 0017, PRD 0053).
Loaded by `mitmdump -s /app/egress_addon.py` inside the Loaded by `mitmdump -s /app/egress_addon.py` inside the
egress container.""" egress container."""
@@ -10,7 +10,6 @@ import json
import os import os
import signal import signal
import sys import sys
import typing
from pathlib import Path from pathlib import Path
from mitmproxy import http # type: ignore[import-not-found] # pylint: disable=import-error from mitmproxy import http # type: ignore[import-not-found] # pylint: disable=import-error
@@ -33,7 +32,7 @@ from egress_addon_core import ( # type: ignore[import-not-found] # pylint: dis
is_git_push_request, is_git_push_request,
load_config, load_config,
match_route, match_route,
resolve_client_context, resolve_client_config,
outbound_scan_headers, outbound_scan_headers,
route_to_yaml_dict, route_to_yaml_dict,
scan_inbound, scan_inbound,
@@ -100,29 +99,17 @@ class EgressAddon:
# Absent → single-tenant (static routes file); behaviour unchanged. # Absent → single-tenant (static routes file); behaviour unchanged.
orch_url = os.environ.get(ORCHESTRATOR_URL_ENV, "").strip() orch_url = os.environ.get(ORCHESTRATOR_URL_ENV, "").strip()
self._resolver = PolicyResolver(orch_url) if orch_url else None self._resolver = PolicyResolver(orch_url) if orch_url else None
# Tokens the operator has approved this session (PRD 0062), keyed by # Tokens the operator has approved this session (PRD 0062). In-memory
# bottle so the shared gateway keeps each bottle's safelist separate — # only — a restart re-prompts. Mutated only from the asyncio loop that
# a global set would let bottle A's approved secret pass bottle B's DLP # runs the addon hooks, so no lock is needed.
# scan. In-memory only (a restart re-prompts); mutated only from the self.safe_tokens: set[str] = set()
# asyncio loop that runs the addon hooks, so no lock is needed.
self._safe_tokens: dict[str, set[str]] = {}
self._supervise_slug = os.environ.get("SUPERVISE_BOTTLE_SLUG", "").strip() self._supervise_slug = os.environ.get("SUPERVISE_BOTTLE_SLUG", "").strip()
self._token_allow_timeout = _token_allow_timeout_from_env(os.environ) self._token_allow_timeout = _token_allow_timeout_from_env(os.environ)
self._reload(initial=True) self._reload(initial=True)
self._install_sighup() self._install_sighup()
@staticmethod def _supervise_available(self) -> bool:
def _supervise_available(slug: str) -> bool: return bool(self._supervise_slug)
"""Supervise is reachable for this request iff we resolved a bottle to
attribute its proposals to (single-tenant env slug, or a source-IP
-attributed bottle id). Empty fail closed (no queue to write to)."""
return bool(slug)
def _safe_tokens_for(self, slug: str) -> set[str]:
"""This bottle's operator-approved DLP safelist (PRD 0062), created on
first use. Keyed by bottle so the shared gateway never leaks one
bottle's approved token into another's scan."""
return self._safe_tokens.setdefault(slug, set())
def _reload(self, *, initial: bool = False) -> None: def _reload(self, *, initial: bool = False) -> None:
try: try:
@@ -231,27 +218,19 @@ class EgressAddon:
+ "\n" + "\n"
) )
def _resolve_flow( def _active_config(self, flow: http.HTTPFlow) -> Config:
self, flow: http.HTTPFlow, """The Config to apply to this request. Single-tenant → the static
) -> "tuple[Config, str, typing.Mapping[str, str]]": `self.config`. Consolidated the calling bottle's Config, resolved
"""The `(Config, supervise slug, env)` to apply to this request. by source IP (fail-closed to deny-all if unattributed). The identity
Single-tenant the static `self.config`, the env slug, and the process token, if the agent injected one, is read then stripped so it never
env. Consolidated the calling bottle's Config + bottle id + auth leaks upstream."""
tokens, resolved by source IP in one round-trip (fail-closed to deny-all
+ empty slug if unattributed); `env` is the process env overlaid with
the bottle's tokens, so upstream-auth injection (and DLP) use *this*
bottle's credentials — exactly what the per-bottle gateway daemon's env did.
The identity token, if the agent injected one, is read then stripped so
it never leaks upstream."""
if self._resolver is None: if self._resolver is None:
return self.config, self._supervise_slug, os.environ return self.config
conn = flow.client_conn conn = flow.client_conn
client_ip = conn.peername[0] if conn and conn.peername else "" client_ip = conn.peername[0] if conn and conn.peername else ""
token = flow.request.headers.get(IDENTITY_HEADER, "") token = flow.request.headers.get(IDENTITY_HEADER, "")
flow.request.headers.pop(IDENTITY_HEADER, None) flow.request.headers.pop(IDENTITY_HEADER, None)
config, slug, tokens = resolve_client_context(self._resolver, client_ip, token) return resolve_client_config(self._resolver, client_ip, token)
env = {**os.environ, **tokens} if tokens else os.environ
return config, slug, env
async def request(self, flow: http.HTTPFlow) -> None: async def request(self, flow: http.HTTPFlow) -> None:
request_path, _, query = flow.request.path.partition("?") request_path, _, query = flow.request.path.partition("?")
@@ -260,14 +239,14 @@ class EgressAddon:
self._serve_introspection(flow, request_path) self._serve_introspection(flow, request_path)
return return
config, slug, env = self._resolve_flow(flow) config = self._active_config(flow)
# DLP outbound scan BEFORE stripping auth — catches tokens the # DLP outbound scan BEFORE stripping auth — catches tokens the
# agent tried to smuggle in any header, path, query param, or body. # agent tried to smuggle in any header, path, query param, or body.
# Hostname is included to catch DNS-tunnelling exfiltration attempts. # Hostname is included to catch DNS-tunnelling exfiltration attempts.
route = match_route(config.routes, flow.request.pretty_host) route = match_route(config.routes, flow.request.pretty_host)
if route is not None: if route is not None:
if not await self._handle_outbound_dlp(flow, route, slug, env): if not await self._handle_outbound_dlp(flow, route):
return return
# The redact policy may have rewritten the request line; recompute # The redact policy may have rewritten the request line; recompute
# the path/query the git checks below rely on. # the path/query the git checks below rely on.
@@ -296,7 +275,7 @@ class EgressAddon:
return return
# Strip agent-set Authorization after DLP scan so smuggled tokens # Strip agent-set Authorization after DLP scan so smuggled tokens
# are caught above; the route may inject gateway-owned auth below. # are caught above; the route may inject sidecar-owned auth below.
flow.request.headers.pop("authorization", None) flow.request.headers.pop("authorization", None)
# Build headers mapping for match evaluation # Build headers mapping for match evaluation
@@ -306,7 +285,7 @@ class EgressAddon:
config.routes, config.routes,
flow.request.pretty_host, flow.request.pretty_host,
request_path, request_path,
env, os.environ,
request_method=flow.request.method, request_method=flow.request.method,
request_headers=req_headers, request_headers=req_headers,
) )
@@ -331,13 +310,10 @@ class EgressAddon:
self, self,
flow: http.HTTPFlow, flow: http.HTTPFlow,
route: Route, route: Route,
slug: str,
env: "typing.Mapping[str, str]",
) -> bool: ) -> bool:
"""Scan the outbound request and apply the route's on-match policy """Scan the outbound request and apply the route's on-match policy
(PRD 0062). `env` is the per-bottle env overlay (process env + this (PRD 0062). Returns True if the request may be forwarded, False if a
bottle's tokens) used for DLP detection. Returns True if the request may 403 response has been written to `flow`.
be forwarded, False if a 403 response has been written to `flow`.
Loops so the supervise policy can re-scan after each approval a Loops so the supervise policy can re-scan after each approval a
second, un-approved token in the same request is still caught.""" second, un-approved token in the same request is still caught."""
@@ -354,8 +330,8 @@ class EgressAddon:
flow.request.pretty_host, request_path, query, headers, "", flow.request.pretty_host, request_path, query, headers, "",
) )
result = scan_outbound( result = scan_outbound(
route, scan_text, env, route, scan_text, os.environ,
safe_tokens=self._safe_tokens_for(slug), crlf_text=crlf_text, safe_tokens=self.safe_tokens, crlf_text=crlf_text,
) )
if result is None or result.severity != "block": if result is None or result.severity != "block":
return True return True
@@ -365,7 +341,7 @@ class EgressAddon:
# redact scrubs every detection (tokens and structural CRLF) and # redact scrubs every detection (tokens and structural CRLF) and
# forwards; it fails closed only if a match survives the scrub. # forwards; it fails closed only if a match survives the scrub.
if policy == ON_MATCH_REDACT: if policy == ON_MATCH_REDACT:
if self._redact_outbound(flow, route, env): if self._redact_outbound(flow, route):
if self.config.log >= LOG_BLOCKS: if self.config.log >= LOG_BLOCKS:
sys.stderr.write(json.dumps({ sys.stderr.write(json.dumps({
"event": "egress_redacted", "event": "egress_redacted",
@@ -390,34 +366,32 @@ class EgressAddon:
# supervise (default): hold the request for operator approval. # supervise (default): hold the request for operator approval.
# Fall back to a hard 403 when supervise isn't wired for the bottle. # Fall back to a hard 403 when supervise isn't wired for the bottle.
if not self._supervise_available(slug): if not self._supervise_available():
self._block_dlp(flow, result) self._block_dlp(flow, result)
return False return False
approved = await self._supervise_token_block(flow, request_path, result, slug, env) approved = await self._supervise_token_block(flow, request_path, result)
if not approved: if not approved:
return False # _supervise_token_block wrote the 403 response return False # _supervise_token_block wrote the 403 response
# loop: the approved value is now in safe_tokens; re-scan. # loop: the approved value is now in safe_tokens; re-scan.
def _redact_outbound( def _redact_outbound(self, flow: http.HTTPFlow, route: Route) -> bool:
self, flow: http.HTTPFlow, route: Route, env: "typing.Mapping[str, str]",
) -> bool:
"""Scrub detected tokens (and CRLF injection sequences) from the mutable """Scrub detected tokens (and CRLF injection sequences) from the mutable
request surfaces (body, headers, path/query) and re-scan. `env` is the request surfaces (body, headers, path/query) and re-scan. Returns True
per-bottle env overlay. Returns True if the request is now clean; False if the request is now clean; False if a block-severity match remains on
if a block-severity match remains on a surface redaction cannot rewrite a surface redaction cannot rewrite (the hostname) so the caller fails
(the hostname) so the caller fails closed.""" closed."""
body = flow.request.get_text(strict=False) body = flow.request.get_text(strict=False)
if body: if body:
redacted_body = redact_tokens(body, env=env) redacted_body = redact_tokens(body, env=os.environ)
if redacted_body != body: if redacted_body != body:
flow.request.text = redacted_body flow.request.text = redacted_body
for name, value in list(flow.request.headers.items()): for name, value in list(flow.request.headers.items()):
if name.lower() == "host": if name.lower() == "host":
continue # routing-critical; never a legitimate token continue # routing-critical; never a legitimate token
redacted = strip_crlf(redact_tokens(value, env=env)) redacted = strip_crlf(redact_tokens(value, env=os.environ))
if redacted != value: if redacted != value:
flow.request.headers[name] = redacted flow.request.headers[name] = redacted
redacted_path = strip_crlf(redact_tokens(flow.request.path, env=env)) redacted_path = strip_crlf(redact_tokens(flow.request.path, env=os.environ))
if redacted_path != flow.request.path: if redacted_path != flow.request.path:
flow.request.path = redacted_path flow.request.path = redacted_path
@@ -430,7 +404,7 @@ class EgressAddon:
crlf_text = build_outbound_scan_text( crlf_text = build_outbound_scan_text(
flow.request.pretty_host, request_path, query, headers, "", flow.request.pretty_host, request_path, query, headers, "",
) )
result = scan_outbound(route, scan_text, env, crlf_text=crlf_text) result = scan_outbound(route, scan_text, os.environ, crlf_text=crlf_text)
return result is None or result.severity != "block" return result is None or result.severity != "block"
async def _supervise_token_block( async def _supervise_token_block(
@@ -438,27 +412,21 @@ class EgressAddon:
flow: http.HTTPFlow, flow: http.HTTPFlow,
request_path: str, request_path: str,
result: ScanResult, result: ScanResult,
slug: str,
env: "typing.Mapping[str, str]",
) -> bool: ) -> bool:
"""Route a token DLP block to the operator's supervisor queue and wait. """Route a token DLP block to the operator's supervisor queue and wait.
`slug` attributes the proposal to the calling bottle (its own queue + Returns True if the operator approved (the matched value is added to
safelist) in the shared gateway this is what keeps one bottle's `self.safe_tokens` and the caller re-scans); False if the request must
approval from unblocking another's request. `env` is the per-bottle env be blocked (a 403 response has been written to `flow`)."""
overlay used to redact secrets from the proposal text. Returns True if
the operator approved (the matched value is added to that bottle's
safelist and the caller re-scans); False if the request must be blocked
(a 403 response has been written to `flow`)."""
host = flow.request.pretty_host host = flow.request.pretty_host
payload = build_token_allow_payload( payload = build_token_allow_payload(
redact_tokens(host, env=env), redact_tokens(host, env=os.environ),
flow.request.method, flow.request.method,
redact_tokens(request_path, env=env), redact_tokens(request_path, env=os.environ),
result, result,
) )
proposal = _sv.Proposal.new( proposal = _sv.Proposal.new(
bottle_slug=slug, bottle_slug=self._supervise_slug,
tool=_sv.TOOL_EGRESS_TOKEN_ALLOW, tool=_sv.TOOL_EGRESS_TOKEN_ALLOW,
proposed_file=payload, proposed_file=payload,
justification=_TOKEN_ALLOW_JUSTIFICATION, justification=_TOKEN_ALLOW_JUSTIFICATION,
@@ -481,13 +449,13 @@ class EgressAddon:
**self._req_ctx(flow), **self._req_ctx(flow),
}) + "\n") }) + "\n")
response = await self._await_token_response(proposal.id, slug) response = await self._await_token_response(proposal.id)
_sv.archive_proposal(slug, proposal.id) _sv.archive_proposal(self._supervise_slug, proposal.id)
if response is not None and response.status in ( if response is not None and response.status in (
_sv.STATUS_APPROVED, _sv.STATUS_MODIFIED, _sv.STATUS_APPROVED, _sv.STATUS_MODIFIED,
): ):
self._safe_tokens_for(slug).add(result.matched) self.safe_tokens.add(result.matched)
if self.config.log >= LOG_BLOCKS: if self.config.log >= LOG_BLOCKS:
sys.stderr.write(json.dumps({ sys.stderr.write(json.dumps({
"event": "egress_token_allowed", "event": "egress_token_allowed",
@@ -510,7 +478,6 @@ class EgressAddon:
async def _await_token_response( async def _await_token_response(
self, self,
proposal_id: str, proposal_id: str,
slug: str,
) -> "_sv.Response | None": ) -> "_sv.Response | None":
"""Poll the DB for the operator's response without blocking the """Poll the DB for the operator's response without blocking the
proxy event loop. Returns the Response, or None on timeout.""" proxy event loop. Returns the Response, or None on timeout."""
@@ -518,7 +485,7 @@ class EgressAddon:
deadline = loop.time() + self._token_allow_timeout deadline = loop.time() + self._token_allow_timeout
while True: while True:
try: try:
return _sv.read_response(slug, proposal_id) return _sv.read_response(self._supervise_slug, proposal_id)
except (OSError, ValueError, KeyError): except (OSError, ValueError, KeyError):
# Not written yet, or a partial/malformed write — retry until # Not written yet, or a partial/malformed write — retry until
# the deadline, then fail closed. # the deadline, then fail closed.
@@ -572,9 +539,6 @@ class EgressAddon:
""" """
if flow.websocket is None: # type: ignore[union-attr] if flow.websocket is None: # type: ignore[union-attr]
return return
# WebSocket DLP runs against the static config only (single-tenant); in
# the consolidated gateway self.config has no routes, so this is inert
# until websocket routing is made source-IP-aware (a separate slice).
route = match_route(self.config.routes, flow.request.pretty_host) route = match_route(self.config.routes, flow.request.pretty_host)
if route is None: if route is None:
return return
@@ -585,7 +549,7 @@ class EgressAddon:
# not an injection vector here — scan only for credential leakage. # not an injection vector here — scan only for credential leakage.
result = scan_outbound( result = scan_outbound(
route, content, os.environ, route, content, os.environ,
safe_tokens=self._safe_tokens_for(self._supervise_slug), crlf_text="", safe_tokens=self.safe_tokens, crlf_text="",
) )
if result is not None and result.severity == "block": if result is not None and result.severity == "block":
sys.stderr.write(f"egress DLP: {result.reason}\n") sys.stderr.write(f"egress DLP: {result.reason}\n")
+12 -50
View File
@@ -3,12 +3,12 @@
Split out of `egress_addon.py` so the host's unit tests can Split out of `egress_addon.py` so the host's unit tests can
exercise the parse + decision functions without depending on the exercise the parse + decision functions without depending on the
`mitmproxy` package. The companion module wraps these with the `mitmproxy` package. The companion module wraps these with the
`mitmproxy.http.HTTPFlow` API and is loaded inside the gateway `mitmproxy.http.HTTPFlow` API and is loaded inside the sidecar
container. container.
Imports: stdlib + `yaml_subset` (which is itself stdlib-only and Imports: stdlib + `yaml_subset` (which is itself stdlib-only and
ships flat into the gateway image alongside this file ships flat into the sidecar bundle image alongside this file
see `Dockerfile.gateway`).""" see `Dockerfile.sidecars`)."""
from __future__ import annotations from __future__ import annotations
@@ -22,7 +22,7 @@ except ImportError: # pragma: no cover - host-side path
from .yaml_subset import YamlSubsetError, parse_yaml_subset from .yaml_subset import YamlSubsetError, parse_yaml_subset
# DLP detector-config parsing lives in a sibling module (also flat-bundled # DLP detector-config parsing lives in a sibling module (also flat-bundled
# into the gateway — see Dockerfile.gateway). Re-exported below so existing # into the sidecar — see Dockerfile.sidecars). Re-exported below so existing
# `from egress_addon_core import ON_MATCH_*` callers keep working. # `from egress_addon_core import ON_MATCH_*` callers keep working.
try: try:
from egress_dlp_config import ( # type: ignore[import-not-found] from egress_dlp_config import ( # type: ignore[import-not-found]
@@ -120,7 +120,7 @@ class ScanResult:
reason: str reason: str
location: str = "" # where the match was found, e.g. "body", "authorization header" location: str = "" # where the match was found, e.g. "body", "authorization header"
context: str = "" # surrounding text with the match replaced by REDACT context: str = "" # surrounding text with the match replaced by REDACT
# Raw substring the detector matched. Used inside the gateway to key the # Raw substring the detector matched. Used inside the sidecar to key the
# supervisor-approved "safe tokens" set (PRD 0062); never logged or written # supervisor-approved "safe tokens" set (PRD 0062); never logged or written
# to a proposal file. Empty for structural detectors (CRLF) that carry no # to a proposal file. Empty for structural detectors (CRLF) that carry no
# safelist-able value. # safelist-able value.
@@ -421,18 +421,6 @@ class PolicyResolverLike(typing.Protocol):
... ...
def _config_from_policy(policy: "str | None") -> "Config":
"""Parse a resolved policy blob into a Config, fail-closed: None / empty /
unparseable all become a deny-all Config (no routes every request
blocked)."""
if not policy:
return Config(routes=()) # unattributed or empty → deny-all
try:
return load_config(policy)
except ValueError:
return Config(routes=()) # unparseable policy → deny
def resolve_client_config( def resolve_client_config(
resolver: PolicyResolverLike, client_ip: str, identity_token: str = "" resolver: PolicyResolverLike, client_ip: str, identity_token: str = ""
) -> "Config": ) -> "Config":
@@ -445,36 +433,12 @@ def resolve_client_config(
policy = resolver.resolve(client_ip, identity_token) policy = resolver.resolve(client_ip, identity_token)
except Exception: # noqa: BLE001 # pylint: disable=broad-exception-caught except Exception: # noqa: BLE001 # pylint: disable=broad-exception-caught
return Config(routes=()) # orchestrator unreachable/errored → deny return Config(routes=()) # orchestrator unreachable/errored → deny
return _config_from_policy(policy) if not policy:
return Config(routes=()) # unattributed or empty → deny-all
class ContextResolverLike(typing.Protocol):
"""The bit of `policy_resolver.PolicyResolver` `resolve_client_context`
needs one round-trip returning policy, bottle id, and auth tokens."""
def resolve_policy_and_bottle_id(
self, source_ip: str, identity_token: str = ...,
) -> "tuple[str | None, str | None, dict[str, str]]":
...
def resolve_client_context(
resolver: ContextResolverLike, client_ip: str, identity_token: str = "",
) -> "tuple[Config, str, dict[str, str]]":
"""The calling client's `(Config, bottle_id, tokens)` in one round-trip —
**fail-closed**. The Config follows `resolve_client_config`'s deny-all
rules; the bottle id is `""` whenever unattributed or the orchestrator
errored (caller treats as "supervise unavailable", never another bottle's
queue); `tokens` are the per-bottle upstream auth values the addon injects.
One `/resolve` keys the egress policy, the supervise queue + safelist, and
auth injection."""
try: try:
policy, bottle_id, tokens = resolver.resolve_policy_and_bottle_id( return load_config(policy)
client_ip, identity_token, except ValueError:
) return Config(routes=()) # unparseable policy → deny
except Exception: # noqa: BLE001 # pylint: disable=broad-exception-caught
return Config(routes=()), "", {} # orchestrator unreachable/errored → deny
return _config_from_policy(policy), (bottle_id or ""), tokens
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@@ -677,7 +641,7 @@ def outbound_scan_headers(
) -> dict[str, str]: ) -> dict[str, str]:
"""Return request headers that should be included in outbound DLP. """Return request headers that should be included in outbound DLP.
Routes that inject gateway-owned auth always strip the agent's Routes that inject sidecar-owned auth always strip the agent's
Authorization header before forwarding. Scanning that header first Authorization header before forwarding. Scanning that header first
creates false positives for provider clients that insist on sending creates false positives for provider clients that insist on sending
their own bearer-shaped placeholder, while still not changing what their own bearer-shaped placeholder, while still not changing what
@@ -728,7 +692,7 @@ def scan_outbound(
crlf_text: str | None = None, crlf_text: str | None = None,
) -> ScanResult | None: ) -> ScanResult | None:
# Lazy import to avoid circular deps and keep dlp_detectors optional # Lazy import to avoid circular deps and keep dlp_detectors optional
# at import time (the gateway copies it flat alongside this file). # at import time (the sidecar copies it flat alongside this file).
try: try:
from dlp_detectors import ( # type: ignore[import-not-found] from dlp_detectors import ( # type: ignore[import-not-found]
scan_crlf_injection, scan_crlf_injection,
@@ -869,9 +833,7 @@ __all__ = [
"is_git_fetch_request", "is_git_fetch_request",
"load_config", "load_config",
"resolve_client_config", "resolve_client_config",
"resolve_client_context",
"PolicyResolverLike", "PolicyResolverLike",
"ContextResolverLike",
"match_route", "match_route",
"outbound_scan_headers", "outbound_scan_headers",
"parse_config", "parse_config",
+2 -2
View File
@@ -6,8 +6,8 @@ and what the proxy does when an outbound detector matches a token
kept apart from the request-time scan/decision flow in `egress_addon_core` kept apart from the request-time scan/decision flow in `egress_addon_core`
so each half reads top-to-bottom without scrolling past the other. so each half reads top-to-bottom without scrolling past the other.
Stdlib-only; ships flat into the gateway image alongside Stdlib-only; ships flat into the sidecar bundle image alongside
`egress_addon_core.py` see `Dockerfile.gateway`.""" `egress_addon_core.py` see `Dockerfile.sidecars`."""
from __future__ import annotations from __future__ import annotations
+3 -3
View File
@@ -1,8 +1,8 @@
#!/bin/sh #!/bin/sh
# Egress daemon entrypoint inside the gateway (PRD 0024). # Egress daemon entrypoint inside the sidecar bundle (PRD 0024).
# #
# Extracted verbatim from Dockerfile.egress's prior inline `sh -c` # Extracted verbatim from Dockerfile.egress's prior inline `sh -c`
# ENTRYPOINT so the supervisor in bot_bottle/gateway_init.py can # ENTRYPOINT so the supervisor in bot_bottle/sidecar_init.py can
# call it as a normal child. Behavior is unchanged: # call it as a normal child. Behavior is unchanged:
# #
# * Upstream proxy: when EGRESS_UPSTREAM_PROXY is set, switch # * Upstream proxy: when EGRESS_UPSTREAM_PROXY is set, switch
@@ -22,7 +22,7 @@ set -e
# Pin mitmproxy's config dir to the bind-mount location of its CA # Pin mitmproxy's config dir to the bind-mount location of its CA
# regardless of which user mitmdump runs as. In the legacy # regardless of which user mitmdump runs as. In the legacy
# four-daemon setup (Dockerfile.egress, USER mitmproxy) this # four-sidecar setup (Dockerfile.egress, USER mitmproxy) this
# resolved naturally to `~mitmproxy/.mitmproxy`. In the PRD 0024 # resolved naturally to `~mitmproxy/.mitmproxy`. In the PRD 0024
# bundle (USER root) `~root/.mitmproxy` is empty, so without this # bundle (USER root) `~root/.mitmproxy` is empty, so without this
# flag mitmdump would generate a fresh CA on the wrong path and # flag mitmdump would generate a fresh CA on the wrong path and
+4 -6
View File
@@ -1,6 +1,6 @@
"""Per-agent git-gate (PRD 0008). """Per-agent git-gate (PRD 0008).
A third per-agent daemon that fronts the bottle's declared git A third per-agent sidecar that fronts the bottle's declared git
upstreams as a transparent mirror. Each `bottle.git` entry maps to upstreams as a transparent mirror. Each `bottle.git` entry maps to
a bare repo on the gate; `git daemon` serves the bare repos over a bare repo on the gate; `git daemon` serves the bare repos over
`git://<gate>/<name>.git`. Two hooks make the mirror bidirectional: `git://<gate>/<name>.git`. Two hooks make the mirror bidirectional:
@@ -15,7 +15,7 @@ a bare repo on the gate; `git daemon` serves the bare repos over
The agent never sees the upstream credential under either path. The agent never sees the upstream credential under either path.
Why a separate daemon (not folded into egress or ssh-gate): the Why a separate sidecar (not folded into egress or ssh-gate): the
gate is the only one of the three that holds upstream push gate is the only one of the three that holds upstream push
credentials. Mixing it with egress would put push creds in the credentials. Mixing it with egress would put push creds in the
same blast radius as internet-facing TLS interception; mixing it same blast radius as internet-facing TLS interception; mixing it
@@ -23,7 +23,7 @@ with ssh-gate would force ssh-gate above L4 and into git-protocol
land. See `docs/prds/0008-git-gate.md`. land. See `docs/prds/0008-git-gate.md`.
This module defines the abstract gate (`GitGate`) and its plan This module defines the abstract gate (`GitGate`) and its plan
dataclass (`GitGatePlan`). The gateway's start/stop lifecycle is dataclass (`GitGatePlan`). The sidecar's start/stop lifecycle is
backend-specific and lives on concrete subclasses (see backend-specific and lives on concrete subclasses (see
`bot_bottle/backend/docker/git_gate.py`).""" `bot_bottle/backend/docker/git_gate.py`)."""
@@ -46,7 +46,6 @@ from .git_gate_render import (
git_gate_known_hosts_line, git_gate_known_hosts_line,
git_gate_render_access_hook, git_gate_render_access_hook,
git_gate_render_entrypoint, git_gate_render_entrypoint,
git_gate_render_provision,
git_gate_render_gitconfig, git_gate_render_gitconfig,
git_gate_render_hook, git_gate_render_hook,
git_gate_upstreams_for_bottle, git_gate_upstreams_for_bottle,
@@ -86,7 +85,7 @@ class GitGatePlan:
class GitGate(ABC): class GitGate(ABC):
"""The per-agent git-gate. Encapsulates the host-side prepare """The per-agent git-gate. Encapsulates the host-side prepare
(upstream lift + entrypoint/hook render); the gateway's (upstream lift + entrypoint/hook render); the sidecar's
start/stop lifecycle is backend-specific and lives on concrete start/stop lifecycle is backend-specific and lives on concrete
subclasses.""" subclasses."""
@@ -156,7 +155,6 @@ __all__ = [
"git_gate_render_gitconfig", "git_gate_render_gitconfig",
"git_gate_known_hosts_line", "git_gate_known_hosts_line",
"git_gate_render_entrypoint", "git_gate_render_entrypoint",
"git_gate_render_provision",
"git_gate_render_hook", "git_gate_render_hook",
"git_gate_render_access_hook", "git_gate_render_access_hook",
"provision_git_gate_dynamic_keys", "provision_git_gate_dynamic_keys",
+24 -60
View File
@@ -1,7 +1,7 @@
"""Pure host-side rendering for the per-agent git-gate (PRD 0008). """Pure host-side rendering for the per-agent git-gate (PRD 0008).
Builds the agent's `.gitconfig` insteadOf rewrites, the known_hosts Builds the agent's `.gitconfig` insteadOf rewrites, the known_hosts
line, and the entrypoint / pre-receive / access-hook scripts the gateway line, and the entrypoint / pre-receive / access-hook scripts the sidecar
runs. No docker or forge calls exposed for tests and reuse across runs. No docker or forge calls exposed for tests and reuse across
backends. Split out of `git_gate.py` so the control surface (`GitGate`) backends. Split out of `git_gate.py` so the control surface (`GitGate`)
and the deploy-key lifecycle (`git_gate_provision`) each read on their and the deploy-key lifecycle (`git_gate_provision`) each read on their
@@ -9,14 +9,13 @@ own; `git_gate` re-exports these names for API stability."""
from __future__ import annotations from __future__ import annotations
import re
import shlex import shlex
from dataclasses import dataclass from dataclasses import dataclass
from pathlib import Path from pathlib import Path
from .manifest import ManifestBottle, ManifestGitEntry from .manifest import ManifestBottle, ManifestGitEntry
# Short network alias for git-gate inside the gateway. The # Short network alias for git-gate inside the sidecar bundle. The
# agent's `.gitconfig` insteadOf rewrites resolve through this name. # agent's `.gitconfig` insteadOf rewrites resolve through this name.
GIT_GATE_HOSTNAME = "git-gate" GIT_GATE_HOSTNAME = "git-gate"
# Shared timeout (seconds) for all git-gate subprocess and CGI calls: # Shared timeout (seconds) for all git-gate subprocess and CGI calls:
@@ -38,7 +37,7 @@ class GitGateUpstream:
KnownHostKey string from the manifest; the gate's start step KnownHostKey string from the manifest; the gate's start step
materialises it into a known_hosts file if non-empty. materialises it into a known_hosts file if non-empty.
the gate credential paths inside the running gateway.""" the gate credential paths inside the running sidecar."""
name: str name: str
upstream_url: str upstream_url: str
@@ -126,18 +125,22 @@ def git_gate_known_hosts_line(host: str, port: str, key: str) -> str:
return f"{target} {key}\n" return f"{target} {key}\n"
def _git_gate_init_repo_fn(repo_root: str, creds_dir: str) -> list[str]: def git_gate_render_entrypoint(upstreams: tuple[GitGateUpstream, ...]) -> str:
"""The `init_repo` shell function, parameterized by the bare-repo root """Posix-sh entrypoint. One `init_repo` call per upstream, then
and the per-bottle creds dir. Single source of the credential-wiring `exec git daemon`. The function reads
logic, shared by the single-tenant daemon entrypoint (`/git`, `/git-gate/creds/<name>-{key,known_hosts}` (bind-mounted into
`/git-gate/creds`) and the consolidated per-bottle provisioning the bundle by the renderer) and wires them into each bare repo's
(`/git/<bottle_id>`, `/git-gate/creds/<bottle_id>`).""" config; the access-hook + pre-receive hook pick those paths up
return [ at fetch / push time."""
lines = [
"#!/bin/sh",
"set -eu",
"",
"init_repo() {", "init_repo() {",
" name=$1", " name=$1",
" upstream_url=$2", " upstream_url=$2",
f" keyfile={creds_dir}/${{name}}-key", " keyfile=/git-gate/creds/${name}-key",
f" hostsfile={creds_dir}/${{name}}-known_hosts", " hostsfile=/git-gate/creds/${name}-known_hosts",
"", "",
# `|| true`: PRD 0018 chunk 3+ bind-mounts these RO from the # `|| true`: PRD 0018 chunk 3+ bind-mounts these RO from the
# host, so chmod-syscalls fail with EROFS. The files already # host, so chmod-syscalls fail with EROFS. The files already
@@ -150,14 +153,14 @@ def _git_gate_init_repo_fn(repo_root: str, creds_dir: str) -> list[str]:
" chmod 600 \"$hostsfile\" 2>/dev/null || true", " chmod 600 \"$hostsfile\" 2>/dev/null || true",
" fi", " fi",
"", "",
f" repo={repo_root}/${{name}}.git", " repo=/git/${name}.git",
" if [ ! -d \"$repo\" ]; then", " if [ ! -d \"$repo\" ]; then",
" git init --bare \"$repo\" >/dev/null", " git init --bare \"$repo\" >/dev/null",
# --mirror=fetch sets remote.origin.fetch = +refs/*:refs/* so a later # --mirror=fetch sets remote.origin.fetch = +refs/*:refs/* so",
# `git fetch origin` mirrors the upstream's full ref graph (heads, # a later `git fetch origin` mirrors the upstream's full ref",
# tags, notes) into the bare repo at canonical paths. It does NOT set # graph (heads, tags, notes) into the bare repo at canonical",
# remote.origin.mirror=true, so an explicit `git push origin # paths. It does NOT set remote.origin.mirror=true, so an",
# <ref>:<ref>` still pushes one ref. # explicit `git push origin <ref>:<ref>` still pushes one ref.",
" git -C \"$repo\" remote add --mirror=fetch origin \"$upstream_url\"", " git -C \"$repo\" remote add --mirror=fetch origin \"$upstream_url\"",
" fi", " fi",
" git -C \"$repo\" config git-gate.identityFile \"$keyfile\"", " git -C \"$repo\" config git-gate.identityFile \"$keyfile\"",
@@ -167,19 +170,9 @@ def _git_gate_init_repo_fn(repo_root: str, creds_dir: str) -> list[str]:
" git -C \"$repo\" config http.receivepack true", " git -C \"$repo\" config http.receivepack true",
" install -m 755 /etc/git-gate/pre-receive \"$repo/hooks/pre-receive\"", " install -m 755 /etc/git-gate/pre-receive \"$repo/hooks/pre-receive\"",
"}", "}",
"",
"mkdir -p /git",
] ]
def git_gate_render_entrypoint(upstreams: tuple[GitGateUpstream, ...]) -> str:
"""Posix-sh entrypoint. One `init_repo` call per upstream, then
`exec git daemon`. The function reads
`/git-gate/creds/<name>-{key,known_hosts}` (bind-mounted into
the bundle by the renderer) and wires them into each bare repo's
config; the access-hook + pre-receive hook pick those paths up
at fetch / push time."""
lines = ["#!/bin/sh", "set -eu", ""]
lines += _git_gate_init_repo_fn("/git", "/git-gate/creds")
lines += ["", "mkdir -p /git"]
for u in upstreams: for u in upstreams:
lines.append(f"init_repo {shlex.quote(u.name)} {shlex.quote(u.upstream_url)}") lines.append(f"init_repo {shlex.quote(u.name)} {shlex.quote(u.upstream_url)}")
lines.extend([ lines.extend([
@@ -197,35 +190,6 @@ def git_gate_render_entrypoint(upstreams: tuple[GitGateUpstream, ...]) -> str:
return "\n".join(lines) + "\n" return "\n".join(lines) + "\n"
# A bottle id namespaces the consolidated gateway's repo + creds dirs; it is
# embedded unquoted in the provisioning script, so restrict it to a shell- and
# path-safe alphabet (registry ids are token_hex — this is defense in depth).
_SAFE_BOTTLE_ID = re.compile(r"[A-Za-z0-9_-]+")
def git_gate_render_provision(
bottle_id: str, upstreams: tuple[GitGateUpstream, ...],
) -> str:
"""Posix-sh script that provisions ONE bottle's bare repos into the
consolidated gateway (PRD 0070), under `/git/<bottle_id>/` with creds
read from `/git-gate/creds/<bottle_id>/`. Init-only no `git daemon`,
since the shared gateway already serves every bottle; run inside the
running gateway when the bottle is registered.
Isolating each bottle's repo root and creds dir by id is what keeps one
bottle's push credentials out of another's repos on the shared gateway."""
if not _SAFE_BOTTLE_ID.fullmatch(bottle_id):
raise ValueError(f"git-gate: unsafe bottle id {bottle_id!r}")
repo_root = f"/git/{bottle_id}"
creds_dir = f"/git-gate/creds/{bottle_id}"
lines = ["#!/bin/sh", "set -eu", ""]
lines += _git_gate_init_repo_fn(repo_root, creds_dir)
lines += ["", f"mkdir -p {shlex.quote(repo_root)}"]
for u in upstreams:
lines.append(f"init_repo {shlex.quote(u.name)} {shlex.quote(u.upstream_url)}")
return "\n".join(lines) + "\n"
def git_gate_render_hook() -> str: def git_gate_render_hook() -> str:
"""The shared pre-receive hook: gitleaks-scan all incoming refs, """The shared pre-receive hook: gitleaks-scan all incoming refs,
then forward each accepted ref to the real upstream (`origin`) then forward each accepted ref to the real upstream (`origin`)
+8 -119
View File
@@ -2,19 +2,10 @@
Used where `git://` push traffic over a host-published Docker port can Used where `git://` push traffic over a host-published Docker port can
hang before receive-pack reaches hooks (e.g. the firecracker backend, hang before receive-pack reaches hooks (e.g. the firecracker backend,
where the guest reaches the gateway over the point-to-point TAP). The where the guest reaches the sidecar over the point-to-point TAP). The
wrapper serves the same `/git/*.git` bare repos through wrapper serves the same `/git/*.git` bare repos through
`git http-backend`, so pre-receive and upstream forwarding remain the `git http-backend`, so pre-receive and upstream forwarding remain the
git-gate enforcement point. git-gate enforcement point.
Consolidated (PRD 0070): when `BOT_BOTTLE_ORCHESTRATOR_URL` is set, one
shared gateway serves every bottle, and each request is served from the
calling bottle's repo namespace (`<root>/<bottle_id>`), attributed from
the unspoofable source IP via the orchestrator. Per-repo credentials +
hooks scope by repo directory, so isolating the *root* per bottle isolates
its creds too. Unattributed clients fail closed (404). Unset the legacy
per-bottle single-tenant flat root, unchanged a transitional path that
gets stripped out once every backend runs the consolidated gateway.
""" """
from __future__ import annotations from __future__ import annotations
@@ -22,89 +13,16 @@ from __future__ import annotations
import os import os
import subprocess import subprocess
import sys import sys
import typing
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path from pathlib import Path
from urllib.parse import urlsplit from urllib.parse import urlsplit
# policy_resolver ships flat alongside this file in the gateway
# image (see Dockerfile.gateway); the bot_bottle.* fallback is the
# host-side / test path. Mirrors egress_addon's import shape.
try:
from policy_resolver import ( # type: ignore[import-not-found]
PolicyResolveError,
PolicyResolver,
)
except ImportError: # pragma: no cover - host-side path
from bot_bottle.policy_resolver import PolicyResolveError, PolicyResolver
DEFAULT_PORT = 9420 DEFAULT_PORT = 9420
# Consolidated (multi-tenant) mode: when this points at the per-host
# orchestrator's control plane, the backend serves each request from the
# *calling* bottle's repo namespace, selected by source IP, instead of a
# single flat repo root. Unset → legacy per-bottle single-tenant mode
# (unchanged). Same env the egress addon reads, so one orchestrator setting
# flips the whole shared gateway multi-tenant.
ORCHESTRATOR_URL_ENV = "BOT_BOTTLE_ORCHESTRATOR_URL"
# App-layer identity token (defense-in-depth over the source-IP invariant);
# the agent injects it, the backend reads it for attribution and never
# forwards it to `git http-backend`. Mirrors egress_addon.IDENTITY_HEADER
# (duplicated, not imported: egress_addon pulls in mitmproxy).
IDENTITY_HEADER = "x-bot-bottle-identity"
# Default flat repo root (single-tenant, and the base under which
# consolidated mode nests each sandbox's namespace).
DEFAULT_REPO_ROOT = "/git"
class ResolverLike(typing.Protocol):
"""Structural type for the resolver `resolve_sandbox_root` needs — just
`resolve_bottle_id`. A Protocol so tests can pass a fake without
importing PolicyResolver (mirrors egress_addon_core.PolicyResolverLike)."""
def resolve_bottle_id(
self, source_ip: str, identity_token: str = ...,
) -> "str | None": ...
def resolve_sandbox_root(
resolver: "ResolverLike | None",
base_root: Path,
source_ip: str,
identity_token: str = "",
) -> Path | None:
"""The per-sandbox repo root to serve this request from, or None to
deny (404).
Single-tenant (`resolver is None`): the flat `base_root`, unchanged.
NOTE: this legacy per-bottle single-tenant path is transitional it
will be stripped out once every backend runs the consolidated gateway
(PRD 0070), leaving only the source-IP-attributed path below.
Consolidated: `base_root/<bottle_id>`, where the sandbox is attributed
from the source IP via the orchestrator. Fail-closed an unattributed
client, a resolver error, or a namespace that would escape `base_root`
all deny, so one sandbox can never reach another's repos."""
if resolver is None:
return base_root
try:
bottle_id = resolver.resolve_bottle_id(source_ip, identity_token)
except PolicyResolveError:
return None # orchestrator unreachable/errored → deny
if not bottle_id:
return None # unattributed → deny
base = base_root.resolve()
namespace = (base / bottle_id).resolve()
if base not in namespace.parents:
return None # bottle_id tried to escape the root → deny
return namespace
# Mirrors git_gate_render.GIT_GATE_TIMEOUT_SECS. Duplicated rather than # Mirrors git_gate_render.GIT_GATE_TIMEOUT_SECS. Duplicated rather than
# imported: this module ships as a flat top-level sibling in the gateway # imported: this module ships as a flat top-level sibling in the sidecar
# bundle image (see Dockerfile.gateway), not as part of the bot_bottle # bundle image (see Dockerfile.sidecars), not as part of the bot_bottle
# package, so `bot_bottle.git_gate` and its dependency chain aren't # package, so `bot_bottle.git_gate` and its dependency chain aren't
# available at runtime. # available at runtime.
GIT_GATE_TIMEOUT_SECS = 15 GIT_GATE_TIMEOUT_SECS = 15
@@ -122,25 +40,10 @@ class GitHttpHandler(BaseHTTPRequestHandler):
def do_POST(self) -> None: def do_POST(self) -> None:
self._run_backend() self._run_backend()
def _sandbox_root(self) -> Path | None:
"""This request's per-sandbox repo root, or None to deny. Single-tenant
unless the server was started with a resolver (consolidated mode), in
which case the root is the calling sandbox's source-IP-selected
namespace. `GIT_PROJECT_ROOT` keeps git's own env-var name."""
base = Path(os.environ.get("GIT_PROJECT_ROOT", DEFAULT_REPO_ROOT))
resolver = getattr(self.server, "policy_resolver", None)
token = self.headers.get(IDENTITY_HEADER, "")
return resolve_sandbox_root(resolver, base, self.client_address[0], token)
def _run_backend(self) -> None: def _run_backend(self) -> None:
sandbox_root = self._sandbox_root()
if sandbox_root is None:
# Unattributed / resolver error: deny before touching any repo.
self.send_error(404)
return
parsed = urlsplit(self.path) parsed = urlsplit(self.path)
if self._is_upload_pack(parsed.path, parsed.query): if self._is_upload_pack(parsed.path, parsed.query):
repo_dir = self._repo_dir(sandbox_root, parsed.path) repo_dir = self._repo_dir(parsed.path)
if repo_dir is None: if repo_dir is None:
self.send_error(404) self.send_error(404)
return return
@@ -174,7 +77,7 @@ class GitHttpHandler(BaseHTTPRequestHandler):
return return
env = os.environ.copy() env = os.environ.copy()
env.update({ env.update({
"GIT_PROJECT_ROOT": str(sandbox_root), "GIT_PROJECT_ROOT": os.environ.get("GIT_PROJECT_ROOT", "/git"),
"GIT_HTTP_EXPORT_ALL": "1", "GIT_HTTP_EXPORT_ALL": "1",
"REQUEST_METHOD": self.command, "REQUEST_METHOD": self.command,
"PATH_INFO": parsed.path, "PATH_INFO": parsed.path,
@@ -188,14 +91,6 @@ class GitHttpHandler(BaseHTTPRequestHandler):
"SERVER_PORT": str(self.server.server_port), # type: ignore "SERVER_PORT": str(self.server.server_port), # type: ignore
"SERVER_PROTOCOL": self.request_version, "SERVER_PROTOCOL": self.request_version,
}) })
# Consolidated mode: attribute the gitleaks-allow supervise proposal
# (written by receive-pack's pre-receive hook, a child of the CGI we
# spawn below) to the calling bottle. The namespaced root is
# `<base>/<bottle_id>`, so its final component is the bottle id — the
# same per-bottle key egress uses. Single-tenant leaves the hook's
# container-stamped SUPERVISE_BOTTLE_SLUG untouched.
if getattr(self.server, "policy_resolver", None) is not None:
env["SUPERVISE_BOTTLE_SLUG"] = sandbox_root.name
for header, variable in ( for header, variable in (
("accept", "HTTP_ACCEPT"), ("accept", "HTTP_ACCEPT"),
("content-encoding", "HTTP_CONTENT_ENCODING"), ("content-encoding", "HTTP_CONTENT_ENCODING"),
@@ -228,8 +123,8 @@ class GitHttpHandler(BaseHTTPRequestHandler):
) )
self._write_cgi_response(proc.stdout) self._write_cgi_response(proc.stdout)
def _repo_dir(self, sandbox_root: Path, path: str) -> Path | None: def _repo_dir(self, path: str) -> Path | None:
root = sandbox_root.resolve() root = Path(os.environ.get("GIT_PROJECT_ROOT", "/git")).resolve()
relative = path.lstrip("/").split(".git", 1)[0] + ".git" relative = path.lstrip("/").split(".git", 1)[0] + ".git"
candidate = (root / relative).resolve() candidate = (root / relative).resolve()
if root not in (candidate, *candidate.parents): if root not in (candidate, *candidate.parents):
@@ -286,13 +181,7 @@ class GitHttpHandler(BaseHTTPRequestHandler):
def main() -> int: def main() -> int:
port = int(os.environ.get("GIT_HTTP_PORT", str(DEFAULT_PORT))) port = int(os.environ.get("GIT_HTTP_PORT", str(DEFAULT_PORT)))
server = ThreadingHTTPServer(("0.0.0.0", port), GitHttpHandler) server = ThreadingHTTPServer(("0.0.0.0", port), GitHttpHandler)
orch_url = os.environ.get(ORCHESTRATOR_URL_ENV, "").strip() sys.stdout.write(f"git-http listening on 0.0.0.0:{port}\n")
# Consolidated mode: resolve each request's sandbox namespace by source
# IP. Absent → single-tenant (flat repo root); behaviour unchanged.
resolver = PolicyResolver(orch_url) if orch_url else None
server.policy_resolver = resolver # type: ignore[attr-defined]
mode = "multi-tenant" if orch_url else "single-tenant"
sys.stdout.write(f"git-http listening on 0.0.0.0:{port} ({mode})\n")
sys.stdout.flush() sys.stdout.flush()
server.serve_forever() server.serve_forever()
return 0 return 0
+2 -2
View File
@@ -17,7 +17,7 @@ class ManifestAgentProvider:
`template` selects a built-in launch/runtime contract. `dockerfile` `template` selects a built-in launch/runtime contract. `dockerfile`
optionally points at a custom agent-image Dockerfile while leaving optionally points at a custom agent-image Dockerfile while leaving
bot-bottle's gateway infrastructure intact. bot-bottle's sidecar infrastructure intact.
`auth_token` names the host env var that holds the provider's OAuth `auth_token` names the host env var that holds the provider's OAuth
token (Claude only). The provisioner injects a provider-owned egress token (Claude only). The provisioner injects a provider-owned egress
@@ -26,7 +26,7 @@ class ManifestAgentProvider:
so the Claude Code CLI starts. so the Claude Code CLI starts.
`forward_host_credentials` forwards the host Codex auth token into `forward_host_credentials` forwards the host Codex auth token into
the egress daemon (Codex only). the egress sidecar (Codex only).
""" """
template: str = "claude" template: str = "claude"
+4 -4
View File
@@ -39,10 +39,10 @@ class ManifestBottle:
# identity without any git-gate.repos upstreams, and vice versa. # identity without any git-gate.repos upstreams, and vice versa.
git_user: ManifestGitUser = field(default_factory=ManifestGitUser) git_user: ManifestGitUser = field(default_factory=ManifestGitUser)
egress: ManifestEgressConfig = field(default_factory=ManifestEgressConfig) egress: ManifestEgressConfig = field(default_factory=ManifestEgressConfig)
# Per-bottle stuck-recovery daemon (PRD 0013). When true (the # Per-bottle stuck-recovery sidecar (PRD 0013). When true (the
# default, issue #249), the launch step brings up a supervise # default, issue #249), the launch step brings up a supervise
# daemon that exposes egress MCP tools to the agent. Set # sidecar that exposes egress MCP tools to the agent. Set
# `supervise: false` to skip the gateway. # `supervise: false` to skip the sidecar.
supervise: bool = True supervise: bool = True
@classmethod @classmethod
@@ -61,7 +61,7 @@ class ManifestBottle:
raise ManifestError( raise ManifestError(
f"bottle '{name}' has an 'ssh' field, which has been removed " f"bottle '{name}' has an 'ssh' field, which has been removed "
f"(PRD 0009). Declare upstreams under 'git-gate.repos' with " f"(PRD 0009). Declare upstreams under 'git-gate.repos' with "
f"url + identity + host_key; the git-gate daemon (PRD 0008) " f"url + identity + host_key; the git-gate sidecar (PRD 0008) "
f"holds the credential and gitleaks-scans pushes." f"holds the credential and gitleaks-scans pushes."
) )
+9 -9
View File
@@ -1,6 +1,6 @@
"""Per-host orchestrator service (PRD 0070). """Per-host orchestrator service (PRD 0070).
A single persistent per-host service that will run the gateway functions A single persistent per-host service that will run the sidecar functions
(egress / git-gate / supervise), coordinate with the console, and broker (egress / git-gate / supervise), coordinate with the console, and broker
agent launches. This package is being built bottom-up, starting with the agent launches. This package is being built bottom-up, starting with the
backend-neutral "consolidation core" that needs no VM packaging: backend-neutral "consolidation core" that needs no VM packaging:
@@ -10,15 +10,15 @@ backend-neutral "consolidation core" that needs no VM packaging:
* `broker` the signed, structured launch-request contract + a * `broker` the signed, structured launch-request contract + a
`LaunchBroker` (stub for the harness) that verifies `LaunchBroker` (stub for the harness) that verifies
provenance before acting. provenance before acting.
* `gateway` the consolidated per-host gateway: a `Gateway` * `sidecar` the consolidated per-host sidecar: a `Sidecar`
lifecycle contract (idempotent singleton) + a lifecycle contract (idempotent singleton) + a
`DockerGateway` impl. One gateway shared by all `DockerSidecar` impl. One sidecar shared by all
bottles instead of one per bottle. bottles instead of one per bottle.
* `service` the `Orchestrator`: owns the registry, brokers the * `service` the `Orchestrator`: owns the registry, brokers the
launch lifecycle (launch/teardown), manages the launch lifecycle (launch/teardown), manages the
shared gateway, attributes. shared sidecar, attributes.
* `control_plane` the HTTP control-plane RPC (launch / teardown / * `control_plane` the HTTP control-plane RPC (launch / teardown /
list / attribute / gateway / health). list / attribute / sidecar / health).
The actual backend-native launch (a real docker/firecracker broker) and The actual backend-native launch (a real docker/firecracker broker) and
the egress/git/supervise data plane land in later slices once this core is the egress/git/supervise data plane land in later slices once this core is
@@ -38,7 +38,7 @@ from .broker import (
verify_request, verify_request,
) )
from .docker_broker import DockerBroker, DockerBrokerError from .docker_broker import DockerBroker, DockerBrokerError
from .gateway import DockerGateway, Gateway, GatewayError from .sidecar import DockerSidecar, Sidecar, SidecarError
from .service import Orchestrator from .service import Orchestrator
from .control_plane import ControlPlaneServer, dispatch, make_server from .control_plane import ControlPlaneServer, dispatch, make_server
@@ -52,9 +52,9 @@ __all__ = [
"StubBroker", "StubBroker",
"DockerBroker", "DockerBroker",
"DockerBrokerError", "DockerBrokerError",
"Gateway", "Sidecar",
"DockerGateway", "DockerSidecar",
"GatewayError", "SidecarError",
"sign_request", "sign_request",
"verify_request", "verify_request",
"Orchestrator", "Orchestrator",
+9 -15
View File
@@ -21,7 +21,7 @@ from .control_plane import make_server
from .docker_broker import DockerBroker from .docker_broker import DockerBroker
from .registry import RegistryStore, default_db_path from .registry import RegistryStore, default_db_path
from .service import Orchestrator from .service import Orchestrator
from .gateway import DockerGateway, Gateway from .sidecar import DockerSidecar, Sidecar
def main(argv: list[str] | None = None) -> int: def main(argv: list[str] | None = None) -> int:
@@ -38,8 +38,8 @@ def main(argv: list[str] | None = None) -> int:
help="launch broker: 'stub' records requests; 'docker' runs containers", help="launch broker: 'stub' records requests; 'docker' runs containers",
) )
parser.add_argument( parser.add_argument(
"--gateway", action="store_true", "--sidecar", action="store_true",
help="run one consolidated per-host gateway (build-if-missing)", help="run one consolidated per-host sidecar bundle (build-if-missing)",
) )
args = parser.parse_args(argv) args = parser.parse_args(argv)
@@ -51,20 +51,14 @@ def main(argv: list[str] | None = None) -> int:
# anything; 'docker' runs real containers (firecracker drops in later). # anything; 'docker' runs real containers (firecracker drops in later).
secret = secrets.token_bytes(32) secret = secrets.token_bytes(32)
broker: LaunchBroker = DockerBroker(secret) if args.broker == "docker" else StubBroker(secret) broker: LaunchBroker = DockerBroker(secret) if args.broker == "docker" else StubBroker(secret)
# Standalone `--gateway` (not the consolidated flow, where the host sidecar: Sidecar | None = DockerSidecar() if args.sidecar else None
# lifecycle runs the gateway). The gateway resolves against this same orchestrator = Orchestrator(registry, broker, secret, sidecar)
# process; the URL is only reachable when they share a docker network.
gateway: Gateway | None = (
DockerGateway(orchestrator_url=f"http://{args.host}:{args.port}")
if args.gateway else None
)
orchestrator = Orchestrator(registry, broker, secret, gateway)
# One persistent per-host gateway, shared by every bottle: build the # One persistent per-host sidecar, shared by every bottle: build the
# bundle image if missing, then bring the singleton up (idempotent). # bundle image if missing, then bring the singleton up (idempotent).
if gateway is not None: if sidecar is not None:
orchestrator.ensure_gateway() orchestrator.ensure_sidecar()
log.info("consolidated gateway ensured", context={"name": gateway.name}) log.info("consolidated sidecar ensured", context={"name": sidecar.name})
server = make_server(orchestrator, host=args.host, port=args.port) server = make_server(orchestrator, host=args.host, port=args.port)
bound_host, bound_port = server.server_address[0], server.server_address[1] bound_host, bound_port = server.server_address[0], server.server_address[1]
+1 -1
View File
@@ -9,7 +9,7 @@ The request it sends is:
request can't be coerced into launching an arbitrary payload; and request can't be coerced into launching an arbitrary payload; and
* **signed** wrapped as a compact JWS/JWT the broker verifies before * **signed** wrapped as a compact JWS/JWT the broker verifies before
acting, so a *compromised co-located component* (an agent-facing acting, so a *compromised co-located component* (an agent-facing
gateway, say) can't forge a launch. This is the concrete form of the sidecar, say) can't forge a launch. This is the concrete form of the
"structured requests only" rule in the PRD security review. "structured requests only" rule in the PRD security review.
Signing is **HS256** over a secret shared by the orchestrator (signer) and Signing is **HS256** over a secret shared by the orchestrator (signer) and
-144
View File
@@ -1,144 +0,0 @@
"""Host-side control-plane client (PRD 0070).
The launch path talks to the orchestrator over its HTTP control plane to
register, re-policy, and tear down bottles the counterpart to the
gateway-side `PolicyResolver` (which only reads `/resolve`). Where
`PolicyResolver` is fail-closed and lives in the untrusted data plane, this
is the trusted control-plane caller: a non-success response is an error the
launch path must surface, not silently swallow.
Stdlib-only, so the CLI can drive the orchestrator without any dependency.
"""
from __future__ import annotations
import json
import urllib.error
import urllib.request
from dataclasses import dataclass
DEFAULT_TIMEOUT_SECONDS = 5.0
class OrchestratorClientError(RuntimeError):
"""A control-plane call failed (unreachable, or an unexpected status)."""
@dataclass(frozen=True)
class RegisteredBottle:
"""What `POST /bottles` returns: the minted bottle id and the per-bottle
identity token the agent presents for app-layer attribution."""
bottle_id: str
identity_token: str
class OrchestratorClient:
"""Trusted host-side client for the orchestrator control plane."""
def __init__(self, base_url: str, *, timeout: float = DEFAULT_TIMEOUT_SECONDS) -> None:
self._base = base_url.rstrip("/")
self._timeout = timeout
def _request(
self, method: str, path: str, body: dict[str, object] | None = None,
) -> tuple[int, dict[str, object]]:
"""Send one request; return `(status, payload)`. Raises
`OrchestratorClientError` only when the orchestrator can't be reached
or returns malformed data HTTP *status* codes are returned so
callers can treat 404 as a meaningful "no such bottle"."""
data = json.dumps(body).encode() if body is not None else None
headers = {"Content-Type": "application/json"} if data is not None else {}
req = urllib.request.Request(
f"{self._base}{path}", data=data, method=method, headers=headers,
)
try:
with urllib.request.urlopen(req, timeout=self._timeout) as resp:
raw = resp.read()
payload = json.loads(raw) if raw else {}
return resp.status, payload if isinstance(payload, dict) else {}
except urllib.error.HTTPError as e:
# A structured error response still carries a usable status.
try:
payload = json.loads(e.read() or b"{}")
except (ValueError, OSError):
payload = {}
return e.code, payload if isinstance(payload, dict) else {}
except (urllib.error.URLError, TimeoutError, OSError, ValueError) as e:
raise OrchestratorClientError(f"{method} {path}: {e}") from e
def _ok(self, method: str, path: str, body: dict[str, object] | None = None) -> dict[str, object]:
"""`_request` that requires a 2xx, raising otherwise."""
status, payload = self._request(method, path, body)
if not 200 <= status < 300:
detail = payload.get("error", "")
raise OrchestratorClientError(f"{method} {path}: HTTP {status} {detail}".rstrip())
return payload
def health(self) -> bool:
"""True iff the control plane answers `GET /health` with 200."""
try:
status, _ = self._request("GET", "/health")
except OrchestratorClientError:
return False
return status == 200
def register_bottle(
self,
source_ip: str,
*,
image_ref: str = "",
metadata: str = "",
policy: str = "",
tokens: dict[str, str] | None = None,
) -> RegisteredBottle:
"""Register a bottle and broker its launch (`POST /bottles`). `tokens`
are the per-bottle egress auth values (env_name -> value) the
orchestrator holds in memory for the gateway to inject. Returns the
minted id + identity token."""
payload = self._ok("POST", "/bottles", {
"source_ip": source_ip,
"image_ref": image_ref,
"metadata": metadata,
"policy": policy,
"tokens": tokens or {},
})
bottle_id = payload.get("bottle_id")
token = payload.get("identity_token")
if not isinstance(bottle_id, str) or not isinstance(token, str):
raise OrchestratorClientError("register: response missing bottle_id/identity_token")
return RegisteredBottle(bottle_id=bottle_id, identity_token=token)
def teardown_bottle(self, bottle_id: str) -> bool:
"""Tear a bottle down (`DELETE /bottles/<id>`). False if the
orchestrator didn't know it (404) — idempotent for cleanup paths."""
status, _ = self._request("DELETE", f"/bottles/{bottle_id}")
if status == 404:
return False
if not 200 <= status < 300:
raise OrchestratorClientError(f"teardown {bottle_id}: HTTP {status}")
return True
def set_policy(self, bottle_id: str, policy: str) -> bool:
"""Live-reload a bottle's policy (`PUT /bottles/<id>/policy`). False on
404 (unknown bottle)."""
status, _ = self._request("PUT", f"/bottles/{bottle_id}/policy", {"policy": policy})
if status == 404:
return False
if not 200 <= status < 300:
raise OrchestratorClientError(f"set_policy {bottle_id}: HTTP {status}")
return True
def list_bottles(self) -> list[dict[str, object]]:
"""Every registered bottle's redacted record (`GET /bottles`)."""
payload = self._ok("GET", "/bottles")
bottles = payload.get("bottles")
return bottles if isinstance(bottles, list) else []
__all__ = [
"OrchestratorClient",
"OrchestratorClientError",
"RegisteredBottle",
"DEFAULT_TIMEOUT_SECONDS",
]
+7 -27
View File
@@ -5,7 +5,7 @@ The backend-agnostic control-plane RPC (CLI / console -> orchestrator) over
vsock / unix-socket portability caveats): vsock / unix-socket portability caveats):
GET /health -> 200 {"status": "ok"} GET /health -> 200 {"status": "ok"}
GET /gateway -> 200 {"configured", ["name","running"]} GET /sidecar -> 200 {"configured", ["name","running"]}
GET /bottles -> 200 {"bottles": [ <redacted record>, ...]} GET /bottles -> 200 {"bottles": [ <redacted record>, ...]}
POST /bottles -> 201 {"bottle_id","identity_token"} (launch) POST /bottles -> 201 {"bottle_id","identity_token"} (launch)
body: {"source_ip", ["image_ref"], body: {"source_ip", ["image_ref"],
@@ -34,7 +34,6 @@ import http.server
import json import json
import os import os
import socketserver import socketserver
import sys
import typing import typing
from urllib.parse import urlsplit from urllib.parse import urlsplit
@@ -64,8 +63,8 @@ def dispatch( # pylint: disable=too-many-return-statements,too-many-branches
if method == "GET" and route == "/health": if method == "GET" and route == "/health":
return 200, {"status": "ok"} return 200, {"status": "ok"}
if method == "GET" and route == "/gateway": if method == "GET" and route == "/sidecar":
return 200, orch.gateway_status() return 200, orch.sidecar_status()
if method == "GET" and route == "/bottles": if method == "GET" and route == "/bottles":
return 200, {"bottles": [r.redacted() for r in orch.registry.all()]} return 200, {"bottles": [r.redacted() for r in orch.registry.all()]}
@@ -81,16 +80,11 @@ def dispatch( # pylint: disable=too-many-return-statements,too-many-branches
image_ref = data.get("image_ref") image_ref = data.get("image_ref")
metadata = data.get("metadata") metadata = data.get("metadata")
policy = data.get("policy") policy = data.get("policy")
raw_tokens = data.get("tokens")
tokens = {
k: v for k, v in raw_tokens.items() if isinstance(k, str) and isinstance(v, str)
} if isinstance(raw_tokens, dict) else {}
rec = orch.launch_bottle( rec = orch.launch_bottle(
source_ip, source_ip,
image_ref=image_ref if isinstance(image_ref, str) else "", image_ref=image_ref if isinstance(image_ref, str) else "",
metadata=metadata if isinstance(metadata, str) else "", metadata=metadata if isinstance(metadata, str) else "",
policy=policy if isinstance(policy, str) else "", policy=policy if isinstance(policy, str) else "",
tokens=tokens,
) )
return 201, {"bottle_id": rec.bottle_id, "identity_token": rec.identity_token} return 201, {"bottle_id": rec.bottle_id, "identity_token": rec.identity_token}
@@ -128,7 +122,7 @@ def dispatch( # pylint: disable=too-many-return-statements,too-many-branches
return 200, {"bottle_id": rec.bottle_id} return 200, {"bottle_id": rec.bottle_id}
if method == "POST" and route == "/resolve": if method == "POST" and route == "/resolve":
# The per-request lookup the multi-tenant gateway makes: returns the # The per-request lookup the multi-tenant sidecar makes: returns the
# bottle's policy. identity_token is OPTIONAL — absent means resolve # bottle's policy. identity_token is OPTIONAL — absent means resolve
# by source IP alone (network-layer attribution). # by source IP alone (network-layer attribution).
try: try:
@@ -142,13 +136,7 @@ def dispatch( # pylint: disable=too-many-return-statements,too-many-branches
rec = orch.resolve(source_ip, token if isinstance(token, str) else "") rec = orch.resolve(source_ip, token if isinstance(token, str) else "")
if rec is None: if rec is None:
return 403, {"error": "unattributed"} return 403, {"error": "unattributed"}
# tokens are the in-memory per-bottle egress auth values the gateway return 200, {"bottle_id": rec.bottle_id, "policy": rec.policy}
# injects; served here, never persisted.
return 200, {
"bottle_id": rec.bottle_id,
"policy": rec.policy,
"tokens": orch.tokens_for(rec.bottle_id),
}
return 404, {"error": "not found"} return 404, {"error": "not found"}
@@ -163,20 +151,12 @@ class Handler(http.server.BaseHTTPRequestHandler):
super().log_message(format, *args) super().log_message(format, *args)
def _serve(self, method: str) -> None: def _serve(self, method: str) -> None:
"""Read the request body, dispatch it, and write the JSON reply. A """Read the request body, dispatch it, and write the JSON reply."""
dispatch failure (e.g. a broker error) returns a 500 rather than
crashing the connection, so one bad request can't take the control
plane down for the caller."""
server = self.server server = self.server
assert isinstance(server, ControlPlaneServer) assert isinstance(server, ControlPlaneServer)
length = int(self.headers.get("Content-Length") or 0) length = int(self.headers.get("Content-Length") or 0)
body = self.rfile.read(length) if length > 0 else b"" body = self.rfile.read(length) if length > 0 else b""
try: status, payload = dispatch(server.orchestrator, method, self.path, body)
status, payload = dispatch(server.orchestrator, method, self.path, body)
except Exception as e: # noqa: BLE001 — the control plane must stay up
sys.stderr.write(f"orchestrator: {method} {self.path} failed: {e!r}\n")
sys.stderr.flush()
status, payload = 500, {"error": f"internal error: {e}"}
data = json.dumps(payload).encode() data = json.dumps(payload).encode()
self.send_response(status) self.send_response(status)
self.send_header("Content-Type", "application/json") self.send_header("Content-Type", "application/json")
+2 -2
View File
@@ -2,12 +2,12 @@
On a verified launch request it starts a Docker container; on teardown it On a verified launch request it starts a Docker container; on teardown it
removes it. This proves the orchestrator -> backend seam on the cheapest removes it. This proves the orchestrator -> backend seam on the cheapest
backend (the gateway is already containers). Only the request's backend (the sidecar bundle is already containers). Only the request's
static ids/flags reach `docker`, so nothing free-form crosses the boundary. static ids/flags reach `docker`, so nothing free-form crosses the boundary.
Slice 3 launches a single container from the request's `image_ref`, named Slice 3 launches a single container from the request's `image_ref`, named
after the bottle id and labelled for cleanup. Wiring the full agent + after the bottle id and labelled for cleanup. Wiring the full agent +
gateway (networks, mounts, the consolidated gateway) is a later sidecar bundle (networks, mounts, the consolidated sidecar) is a later
slice this is the seam, not the finished launcher. slice this is the seam, not the finished launcher.
""" """
-228
View File
@@ -1,228 +0,0 @@
"""The consolidated per-host gateway (PRD 0070).
The core consolidation win: **one** persistent gateway per host, shared by
every bottle, instead of a gateway per bottle. It's safe to share
because the attribution invariant (source IP + identity token, see
`registry`) lets the gateway attribute each request to the right bottle
so per-bottle policy lives in one long-lived process keyed on who's calling.
`Gateway` is the backend-neutral lifecycle contract (mirrors `LaunchBroker`):
ensure the single instance is up, report it, tear it down. `DockerGateway`
is the docker implementation; a firecracker gateway VM slots in later.
The defining behaviour is **idempotent singleton**: `ensure_running` starts
the instance if absent and is a no-op if it's already up, so N bottle
launches never spawn N gateways.
"""
from __future__ import annotations
import abc
import os
import time
from pathlib import Path
from ..docker_cmd import run_docker
# The gateway's mitmproxy writes its CA a beat after the container starts, so
# reads poll for it rather than assuming it's there on a fresh launch.
_CA_POLL_SECONDS = 0.5
DEFAULT_CA_TIMEOUT_SECONDS = 30.0
GATEWAY_NAME = "bot-bottle-orch-gateway"
GATEWAY_LABEL = "bot-bottle-orch-gateway=1"
# The single user-defined network the gateway and every agent bottle share.
# Agents attach here with a pinned IP and reach the gateway's egress /
# git-http / supervise ports by its address — no host port publishing, and
# the source IP the gateway attributes by is the address on this network.
GATEWAY_NETWORK = "bot-bottle-gateway"
# mitmproxy's CA dir in the bundle. A persistent named volume here keeps the
# gateway's self-generated CA STABLE across container recreation — every agent
# installs this one CA to trust the shared gateway's TLS interception, so it
# must not rotate when the gateway restarts.
MITMPROXY_HOME = "/home/mitmproxy/.mitmproxy"
GATEWAY_CA_VOLUME = "bot-bottle-gateway-mitmproxy"
GATEWAY_CA_CERT = f"{MITMPROXY_HOME}/mitmproxy-ca-cert.pem"
# The gateway data-plane image + its Dockerfile. Kept as a local constant
# rather than imported from the backend layer, which would drag
# the whole backend layer into the lean orchestrator (see #359); unify when
# that lands. Env override matches the backend's BOT_BOTTLE_GATEWAY_IMAGE.
GATEWAY_IMAGE = os.environ.get("BOT_BOTTLE_GATEWAY_IMAGE", "bot-bottle-gateway:latest")
GATEWAY_DOCKERFILE = "Dockerfile.gateway"
_REPO_ROOT = Path(__file__).resolve().parents[2]
class GatewayError(Exception):
"""The shared gateway failed to build/start/stop (non-zero `docker` exit)."""
class Gateway(abc.ABC):
"""Lifecycle of the single per-host gateway. Backend-neutral."""
name: str
def ensure_built(self) -> None:
"""Ensure the gateway's image / rootfs exists, building it if needed.
Default: nothing to build (e.g. a stub or a pre-pulled image)."""
return
@abc.abstractmethod
def ensure_running(self) -> None:
"""Start the gateway if it isn't already up. Idempotent: a no-op
when it's already running (that's the whole point one per host).
Assumes the image exists call `ensure_built()` first."""
@abc.abstractmethod
def is_running(self) -> bool:
"""True iff the gateway instance is currently up."""
@abc.abstractmethod
def stop(self) -> None:
"""Remove the gateway. Idempotent — absent is success."""
class DockerGateway(Gateway):
"""The consolidated gateway as a single, fixed-name Docker container.
`image_ref` defaults to the gateway data-plane image; `ensure_built`
builds it from `Dockerfile.gateway` when it's missing. (Note: slice 5
builds + launches the bundle container; wiring its per-bottle,
source-IP-keyed config is a later slice see PRD 0070.)"""
def __init__(
self,
image_ref: str = GATEWAY_IMAGE,
*,
name: str = GATEWAY_NAME,
network: str = GATEWAY_NETWORK,
orchestrator_url: str = "",
build_context: Path | None = None,
dockerfile: str | None = GATEWAY_DOCKERFILE,
) -> None:
self.image_ref = image_ref
self.name = name
self.network = network
# The control-plane URL the gateway's data plane resolves per bottle
# against — reached by container name over docker DNS on the shared
# network (container↔container, no host firewall). Empty → single-tenant.
self._orchestrator_url = orchestrator_url
self._build_context = build_context or _REPO_ROOT
self._dockerfile = dockerfile
def image_exists(self) -> bool:
return run_docker(["docker", "image", "inspect", self.image_ref]).returncode == 0
def ensure_built(self) -> None:
"""Build the bundle image from its Dockerfile, **cache-aware** — cheap
(a cache check) when nothing changed, a real rebuild when the flat
sources (egress addon / git-http / policy_resolver / supervise) moved.
This deliberately builds every time rather than build-if-missing: the
per-bottle model kept the image fresh via compose's `build:` on up, and
a stale image silently runs the OLD single-tenant daemons. No-op only
when no dockerfile is configured (a pre-pulled image). BOT_BOTTLE_NO_CACHE
forces a full rebuild (parity with `start --no-cache`)."""
if self._dockerfile is None:
return
argv = ["docker", "build", "-t", self.image_ref,
"-f", str(self._build_context / self._dockerfile),
str(self._build_context)]
if os.environ.get("BOT_BOTTLE_NO_CACHE"):
argv.insert(2, "--no-cache")
proc = run_docker(argv)
if proc.returncode != 0:
raise GatewayError(f"gateway image build failed: {proc.stderr.strip()}")
def is_running(self) -> bool:
proc = run_docker([
"docker", "ps",
"--filter", f"name=^{self.name}$",
"--filter", "status=running",
"--format", "{{.Names}}",
])
return self.name in proc.stdout.split()
def _running_image_is_current(self) -> bool:
"""True iff the running gateway was created from the *current*
`image_ref`. When `ensure_built` rebuilds the image (a source change),
the running container is still the OLD image running the OLD flat
daemons so this is how a rebuild actually takes effect: a mismatch
means recreate."""
running = run_docker(["docker", "inspect", "--format", "{{.Image}}", self.name])
current = run_docker(["docker", "image", "inspect", "--format", "{{.Id}}", self.image_ref])
if running.returncode != 0 or current.returncode != 0:
return True # can't compare → don't churn a working container
return running.stdout.strip() == current.stdout.strip()
def _ensure_network(self) -> None:
"""Create the shared gateway network if it doesn't exist. Idempotent —
a concurrent create loses harmlessly (the loser sees 'already exists').
Docker picks the subnet; the launcher reads it back to allocate IPs."""
if run_docker(["docker", "network", "inspect", self.network]).returncode == 0:
return
proc = run_docker(["docker", "network", "create", self.network])
if proc.returncode != 0 and "already exists" not in proc.stderr:
raise GatewayError(
f"gateway network {self.network} failed to create: {proc.stderr.strip()}"
)
def ensure_running(self) -> None:
# Recreate when the running container's image is stale (a rebuild),
# so source changes to the gateway's flat daemons take effect — not
# just when the container is absent.
if self.is_running() and self._running_image_is_current():
return
self._ensure_network()
# Clear any stale (stopped OR outdated-image) container holding the
# fixed name, then start fresh. `rm --force` on an absent name is a
# tolerated no-op.
run_docker(["docker", "rm", "--force", self.name])
argv = [
"docker", "run", "--detach",
"--name", self.name,
"--label", GATEWAY_LABEL,
"--network", self.network,
# Persist the self-generated CA so it survives restarts (agents
# trust it) — see GATEWAY_CA_VOLUME.
"--volume", f"{GATEWAY_CA_VOLUME}:{MITMPROXY_HOME}",
]
if self._orchestrator_url:
# Makes the gateway's egress / git / supervise daemons multi-tenant:
# each request resolves source-IP -> policy against the control plane.
argv += ["--env", f"BOT_BOTTLE_ORCHESTRATOR_URL={self._orchestrator_url}"]
argv.append(self.image_ref)
proc = run_docker(argv)
if proc.returncode != 0:
raise GatewayError(f"gateway failed to start: {proc.stderr.strip()}")
def ca_cert_pem(self, *, timeout: float = DEFAULT_CA_TIMEOUT_SECONDS) -> str:
"""The gateway's CA certificate (PEM) that agents install to trust its
TLS interception. mitmproxy generates it a moment after the container
starts, so this **polls** for it (up to `timeout`) rather than assuming
it's already there on a fresh gateway — raising only if it never
appears."""
deadline = time.monotonic() + timeout
while True:
proc = run_docker(["docker", "exec", self.name, "cat", GATEWAY_CA_CERT])
if proc.returncode == 0 and proc.stdout.strip():
return proc.stdout
if time.monotonic() >= deadline:
raise GatewayError(
f"gateway CA cert not available after {timeout:g}s: "
f"{proc.stderr.strip() or 'empty'}"
)
time.sleep(_CA_POLL_SECONDS)
def stop(self) -> None:
proc = run_docker(["docker", "rm", "--force", self.name])
if proc.returncode != 0 and "No such container" not in proc.stderr:
raise GatewayError(f"gateway failed to stop: {proc.stderr.strip()}")
__all__ = [
"Gateway", "DockerGateway", "GatewayError",
"GATEWAY_NAME", "GATEWAY_LABEL", "GATEWAY_IMAGE", "GATEWAY_NETWORK",
"GATEWAY_CA_VOLUME", "GATEWAY_CA_CERT",
]
-248
View File
@@ -1,248 +0,0 @@
"""Orchestrator + gateway lifecycle (PRD 0070, docker slice).
Runs the orchestrator control plane **as a container** on the shared gateway
network, alongside the gateway container. This is the PRD's "virtualize the
orchestrator": container↔container between the gateway and the orchestrator
avoids the host firewall (which drops containerhost traffic), and the gateway
reaches the control plane by container name over docker DNS. The host CLI
reaches it via a published loopback port.
The orchestrator runs with the **register-only broker** the *backend*
launches agent containers (compose), so the orchestrator needs no docker
socket. That keeps this control-plane container unprivileged; the host manages
both containers. `ensure_running` is an idempotent singleton (fixed container
names + the published port).
"""
from __future__ import annotations
import hashlib
import os
import time
import urllib.error
import urllib.request
from pathlib import Path
from .. import log
from ..docker_cmd import run_docker
from ..paths import bot_bottle_root
from .gateway import GATEWAY_IMAGE, GATEWAY_NETWORK, DockerGateway, GatewayError
DEFAULT_PORT = 8099
ORCHESTRATOR_NAME = "bot-bottle-orchestrator"
ORCHESTRATOR_LABEL = "bot-bottle-orchestrator=1"
# The control-plane's own runtime image — lean (python + the stdlib-only
# `bot_bottle` package, bind-mounted at run time), distinct from the heavy
# gateway data-plane image it used to borrow (#384). Env override for
# operators pinning a published build.
ORCHESTRATOR_IMAGE = os.environ.get(
"BOT_BOTTLE_ORCHESTRATOR_IMAGE", "bot-bottle-orchestrator:latest"
)
ORCHESTRATOR_DOCKERFILE = "Dockerfile.orchestrator"
# Baked onto the container as a label so `ensure_running` can tell whether the
# running process is executing the *current* bind-mounted source — see
# `_source_hash`.
ORCHESTRATOR_SOURCE_HASH_LABEL = "bot-bottle-orchestrator-source-hash"
# The repo root is bind-mounted into the control-plane container so
# `python -m bot_bottle.orchestrator` resolves the package (the orchestrator
# is stdlib-only, so the lean orchestrator image's python is enough).
_REPO_ROOT = Path(__file__).resolve().parents[2]
_APP_DIR = "/app"
_ROOT_IN_CONTAINER = "/bot-bottle-root"
_HEALTH_POLL_SECONDS = 0.25
DEFAULT_STARTUP_TIMEOUT_SECONDS = 45.0
_HEALTH_REQUEST_TIMEOUT_SECONDS = 1.0
class OrchestratorStartError(RuntimeError):
"""The orchestrator container did not become healthy within the timeout."""
def _source_hash(repo_root: Path) -> str:
"""Content hash of the orchestrator's bind-mounted Python source (the
`bot_bottle` package the control-plane process imports). This only
changes when the code that would actually run inside the container
changes `ensure_running` recreates the container on a mismatch and
otherwise leaves a healthy one alone, so a bottle launch that isn't
accompanied by a code change doesn't restart the process and drop every
*other* active bottle's in-memory egress tokens (`Orchestrator._tokens`
in `service.py`, never persisted to disk by design)."""
h = hashlib.sha256()
for path in sorted((repo_root / "bot_bottle").rglob("*.py")):
h.update(str(path.relative_to(repo_root)).encode())
h.update(path.read_bytes())
return h.hexdigest()
class OrchestratorService:
"""Manages the orchestrator control-plane container + the shared gateway.
Callers only need `ensure_running()` + `url`."""
def __init__(
self,
*,
port: int = DEFAULT_PORT,
network: str = GATEWAY_NETWORK,
image: str = ORCHESTRATOR_IMAGE,
gateway_image: str = GATEWAY_IMAGE,
repo_root: Path = _REPO_ROOT,
host_root: Path | None = None,
) -> None:
self.port = port
self.network = network
# Two distinct images (#384): `image` is the lean control-plane
# runtime this container runs; `_gateway_image` is the heavy egress /
# git-gate / supervise data plane the gateway container runs. They
# were one conflated image before the split.
self.image = image
self._gateway_image = gateway_image
self._repo_root = repo_root
self._host_root = host_root or bot_bottle_root()
@property
def url(self) -> str:
"""Host-side control-plane URL (published loopback port)."""
return f"http://127.0.0.1:{self.port}"
@property
def internal_url(self) -> str:
"""Control-plane URL as the gateway container reaches it — by name over
docker DNS on the shared network. This is the gateway's
BOT_BOTTLE_ORCHESTRATOR_URL."""
return f"http://{ORCHESTRATOR_NAME}:{self.port}"
def is_healthy(self, *, timeout: float = _HEALTH_REQUEST_TIMEOUT_SECONDS) -> bool:
try:
with urllib.request.urlopen(f"{self.url}/health", timeout=timeout) as resp:
return resp.status == 200
except (urllib.error.URLError, TimeoutError, OSError):
return False
def _container_running(self, name: str) -> bool:
proc = run_docker(["docker", "ps", "--filter", f"name=^/{name}$", "--format", "{{.Names}}"])
return name in proc.stdout.split()
def _run_orchestrator_container(self, source_hash: str) -> None:
"""Start the control-plane container (idempotent: clears a stale
fixed-name container first). Register-only broker no docker socket.
Labels the container with `source_hash` so a later `ensure_running`
can detect a real code change (see `_source_hash`)."""
run_docker(["docker", "rm", "--force", ORCHESTRATOR_NAME])
proc = run_docker([
"docker", "run", "--detach",
"--name", ORCHESTRATOR_NAME,
"--label", ORCHESTRATOR_LABEL,
"--label", f"{ORCHESTRATOR_SOURCE_HASH_LABEL}={source_hash}",
"--network", self.network,
# Host CLI reaches the control plane here; bound to loopback so it
# is not exposed on the host's external interfaces.
"--publish", f"127.0.0.1:{self.port}:{self.port}",
"--volume", f"{self._repo_root}:{_APP_DIR}:ro",
"--workdir", _APP_DIR,
# Persist the registry DB on the host (sole-owner: only the
# orchestrator opens bot-bottle.db).
"--volume", f"{self._host_root}:{_ROOT_IN_CONTAINER}",
"--env", f"BOT_BOTTLE_ROOT={_ROOT_IN_CONTAINER}",
"--entrypoint", "python3",
self.image,
"-m", "bot_bottle.orchestrator",
"--host", "0.0.0.0", "--port", str(self.port), "--broker", "stub",
])
if proc.returncode != 0:
raise OrchestratorStartError(
f"orchestrator container failed to start: {proc.stderr.strip()}"
)
def _gateway(self) -> DockerGateway:
return DockerGateway(
self._gateway_image, network=self.network, orchestrator_url=self.internal_url
)
def _ensure_orchestrator_image(self) -> None:
"""Build the lean control-plane image from `Dockerfile.orchestrator`
when it's missing (#384). Cheap — a `FROM python:*-slim` base with no
deps to install, so the layer cache makes rebuilds a no-op. Unlike the
gateway image this is build-if-missing, not build-every-time: the
control plane bind-mounts its source, so a code change is caught by the
source-hash recreate (below), not by an image rebuild."""
if run_docker(["docker", "image", "inspect", self.image]).returncode == 0:
return
argv = ["docker", "build", "-t", self.image,
"-f", str(self._repo_root / ORCHESTRATOR_DOCKERFILE),
str(self._repo_root)]
if os.environ.get("BOT_BOTTLE_NO_CACHE"):
argv.insert(2, "--no-cache")
proc = run_docker(argv)
if proc.returncode != 0:
raise GatewayError(
f"orchestrator image build failed: {proc.stderr.strip()}"
)
def _orchestrator_source_current(self, current_hash: str) -> bool:
"""True iff the running orchestrator container was created from the
*current* bind-mounted source. Mirrors `DockerGateway`'s
image-staleness check, but by content hash rather than image id since
the orchestrator runs bind-mounted source, not a built image."""
if not self._container_running(ORCHESTRATOR_NAME):
return False
proc = run_docker([
"docker", "inspect", "--format",
"{{ index .Config.Labels \"" + ORCHESTRATOR_SOURCE_HASH_LABEL + "\" }}",
ORCHESTRATOR_NAME,
])
if proc.returncode != 0:
return True # can't compare -> don't churn a working container
return proc.stdout.strip() == current_hash
def ensure_running(
self, *, startup_timeout: float = DEFAULT_STARTUP_TIMEOUT_SECONDS,
) -> str:
"""Ensure the control plane + shared gateway are up; return the host
control-plane URL. Idempotent a healthy control plane running
current code and a running gateway are left untouched. Raises
`OrchestratorStartError` on timeout."""
gateway = self._gateway()
gateway.ensure_built() # rebuild the bundle image on a source change
gateway.ensure_running() # creates the shared network + (re)starts gateway
# Recreate the orchestrator container only when its bind-mounted
# source has actually changed since it started — its Python process
# loaded that code at startup and won't reload, so a stale container
# would keep running OLD control-plane code. Recreating on *every*
# launch (the prior behaviour) would drop every other active
# bottle's in-memory egress tokens each time a new bottle starts,
# since the orchestrator process holds them only in memory (#381).
current_hash = _source_hash(self._repo_root)
if self.is_healthy() and self._orchestrator_source_current(current_hash):
return self.url
self._ensure_orchestrator_image()
log.info("starting orchestrator container", context={"name": ORCHESTRATOR_NAME})
self._run_orchestrator_container(current_hash)
deadline = time.monotonic() + startup_timeout
while time.monotonic() < deadline:
if self.is_healthy():
log.info("orchestrator healthy", context={"url": self.url})
return self.url
time.sleep(_HEALTH_POLL_SECONDS)
raise OrchestratorStartError(
f"orchestrator at {self.url} did not become healthy within {startup_timeout:g}s"
)
def stop(self) -> None:
"""Remove the orchestrator + gateway containers (idempotent)."""
run_docker(["docker", "rm", "--force", ORCHESTRATOR_NAME])
self._gateway().stop()
__all__ = [
"OrchestratorService",
"OrchestratorStartError",
"ORCHESTRATOR_NAME",
"ORCHESTRATOR_IMAGE",
"DEFAULT_PORT",
"DEFAULT_STARTUP_TIMEOUT_SECONDS",
]
-57
View File
@@ -1,57 +0,0 @@
"""Consolidated registration inputs (PRD 0070, docker slice).
Bridges the existing per-bottle `prepare` output to the consolidated
registry: turns a prepared bottle's egress plan into the backend-neutral
inputs `Orchestrator.launch_bottle` takes the egress **policy** blob and
launch **metadata**.
The policy blob is the exact routes YAML the per-bottle egress daemon used
to read from a file; in the consolidated model the multi-tenant gateway's
`PolicyResolver` fetches it from the registry per request (keyed by source
IP) instead. Same render, so consolidated and single-tenant egress apply
byte-identical policy a bottle's allow-list doesn't change when it moves
onto the shared gateway.
Host-side glue (imports `bot_bottle.egress`), used by the launch path not
by the lean orchestrator control-plane process itself.
"""
from __future__ import annotations
import json
from dataclasses import dataclass
from ..egress import EgressPlan, egress_render_routes
@dataclass(frozen=True)
class RegistrationInputs:
"""What `Orchestrator.launch_bottle` needs to register a bottle, derived
from its prepared plan. `policy` is served verbatim by the gateway's
`/resolve`; `metadata` is opaque forward-compat state it carries the
human slug so the console / supervise can show a name, not just the
minted bottle id."""
policy: str
metadata: str
def egress_policy(plan: EgressPlan) -> str:
"""The bottle's egress policy blob: the routes YAML the gateway serves
and the addon parses with `load_config`. Identical to the per-bottle
`routes.yaml` render, so the consolidated path applies the same
allow-list."""
return egress_render_routes(plan.routes, log=plan.log)
def registration_inputs(plan: EgressPlan) -> RegistrationInputs:
"""Assemble the orchestrator registration inputs from a prepared egress
plan. `metadata` records the slug so the shared registry can map a minted
bottle id back to its human name."""
return RegistrationInputs(
policy=egress_policy(plan),
metadata=json.dumps({"slug": plan.slug}),
)
__all__ = ["RegistrationInputs", "egress_policy", "registration_inputs"]
+6 -19
View File
@@ -67,9 +67,9 @@ class BottleRecord:
# Opaque JSON blob for forward-compat (pool slot, ...) — the registry # Opaque JSON blob for forward-compat (pool slot, ...) — the registry
# doesn't interpret it, so new fields need no migration. # doesn't interpret it, so new fields need no migration.
metadata: str = "" metadata: str = ""
# The bottle's gateway policy (opaque JSON — egress allowlist / routes / # The bottle's sidecar policy (opaque JSON — egress allowlist / routes /
# git config). The registry stores and serves it verbatim, keyed by # git config). The registry stores and serves it verbatim, keyed by
# source IP; the multi-tenant gateway interprets it. # source IP; the multi-tenant sidecar interprets it.
policy: str = "" policy: str = ""
def redacted(self) -> dict[str, object]: def redacted(self) -> dict[str, object]:
@@ -102,9 +102,9 @@ _MIGRATIONS = TableMigrations(
# the "one active bottle per source IP" check cheap. # the "one active bottle per source IP" check cheap.
"CREATE INDEX IF NOT EXISTS idx_orchestrator_bottles_source_ip " "CREATE INDEX IF NOT EXISTS idx_orchestrator_bottles_source_ip "
"ON orchestrator_bottles (source_ip)", "ON orchestrator_bottles (source_ip)",
# v3 — per-bottle policy (opaque JSON the gateway interprets): the # v3 — per-bottle policy (opaque JSON the sidecar interprets): the
# egress allowlist / routes / git config selected by source IP. The # egress allowlist / routes / git config selected by source IP. The
# multi-tenant gateway resolves it per request via `attribute`. # multi-tenant sidecar resolves it per request via `attribute`.
"ALTER TABLE orchestrator_bottles ADD COLUMN policy TEXT NOT NULL DEFAULT ''", "ALTER TABLE orchestrator_bottles ADD COLUMN policy TEXT NOT NULL DEFAULT ''",
], ],
) )
@@ -149,15 +149,7 @@ class RegistryStore(DbStore):
policy: str = "", policy: str = "",
) -> BottleRecord: ) -> BottleRecord:
"""Register (or replace) a bottle. Mints a bottle_id / identity token """Register (or replace) a bottle. Mints a bottle_id / identity token
when not supplied. Live this is the control-plane reload path. when not supplied. Live this is the control-plane reload path."""
Supersedes any *other* active bottle at the same source IP first:
`by_source_ip` fail-closes on ambiguity (>1 active row None), so a
leftover row at a reused IP from an untorn-down bottle, crash
recovery, or a persisted DB would otherwise *brick* the new bottle's
attribution. The new registration is authoritative for its IP; the
stale rows go. INSERT OR REPLACE handles a same-`bottle_id` reload in
place (its own row is exempt from the sweep)."""
rec = BottleRecord( rec = BottleRecord(
bottle_id=bottle_id or secrets.token_hex(8), bottle_id=bottle_id or secrets.token_hex(8),
source_ip=source_ip, source_ip=source_ip,
@@ -168,11 +160,6 @@ class RegistryStore(DbStore):
policy=policy, policy=policy,
) )
with self._connect() as conn: with self._connect() as conn:
conn.execute(
"DELETE FROM orchestrator_bottles "
"WHERE source_ip = ? AND state = 'active' AND bottle_id != ?",
(rec.source_ip, rec.bottle_id),
)
conn.execute( conn.execute(
"INSERT OR REPLACE INTO orchestrator_bottles " "INSERT OR REPLACE INTO orchestrator_bottles "
"(bottle_id, source_ip, identity_token, state, created_at, metadata, policy) " "(bottle_id, source_ip, identity_token, state, created_at, metadata, policy) "
@@ -230,7 +217,7 @@ class RegistryStore(DbStore):
IP, or None if unknown or ambiguous (more than one a IP, or None if unknown or ambiguous (more than one a
misconfiguration). Safe as the *sole* attributor only where the misconfiguration). Safe as the *sole* attributor only where the
source IP is unspoofable (Firecracker `/31` + nft) and the control source IP is unspoofable (Firecracker `/31` + nft) and the control
plane is reachable only by the trusted gateway; pair with the plane is reachable only by the trusted sidecar; pair with the
identity token (`attribute`) elsewhere.""" identity token (`attribute`) elsewhere."""
with self._connect() as conn: with self._connect() as conn:
rows = conn.execute( rows = conn.execute(
+22 -39
View File
@@ -19,12 +19,12 @@ from __future__ import annotations
from .broker import LaunchBroker, LaunchRequest, sign_request from .broker import LaunchBroker, LaunchRequest, sign_request
from .registry import BottleRecord, RegistryStore from .registry import BottleRecord, RegistryStore
from .gateway import Gateway from .sidecar import Sidecar
class Orchestrator: class Orchestrator:
"""Owns the registry + brokers launches, and manages the single """Owns the registry + brokers launches, and manages the single
consolidated per-host gateway. Backend-neutral (broker and gateway consolidated per-host sidecar. Backend-neutral (broker and sidecar
abstract the backend-native pieces).""" abstract the backend-native pieces)."""
def __init__( def __init__(
@@ -32,18 +32,12 @@ class Orchestrator:
registry: RegistryStore, registry: RegistryStore,
broker: LaunchBroker, broker: LaunchBroker,
sign_secret: bytes, sign_secret: bytes,
gateway: Gateway | None = None, sidecar: Sidecar | None = None,
) -> None: ) -> None:
self.registry = registry self.registry = registry
self._broker = broker self._broker = broker
self._secret = sign_secret self._secret = sign_secret
self._gateway = gateway self._sidecar = sidecar
# Per-bottle egress auth tokens (env_name -> value), keyed by bottle_id.
# Held **in memory only** — never written to the registry DB — so the
# gateway can inject each bottle's upstream credential without secrets
# at rest. Lost on restart (re-launch re-registers them); the future
# SecretProvider (#355) replaces this with per-request minting.
self._tokens: dict[str, dict[str, str]] = {}
def launch_bottle( def launch_bottle(
self, self,
@@ -53,14 +47,11 @@ class Orchestrator:
slot: int | None = None, slot: int | None = None,
metadata: str = "", metadata: str = "",
policy: str = "", policy: str = "",
tokens: dict[str, str] | None = None,
) -> BottleRecord: ) -> BottleRecord:
"""Register a bottle (with its gateway policy + in-memory egress auth """Register a bottle (with its sidecar policy) and broker its launch.
tokens) and broker its launch. Rolls the registry entry back if the Rolls the registry entry back if the launch doesn't take, so a
launch doesn't take, so a failure leaves no orphan.""" failure leaves no orphan."""
rec = self.registry.register(source_ip, metadata=metadata, policy=policy) rec = self.registry.register(source_ip, metadata=metadata, policy=policy)
if tokens:
self._tokens[rec.bottle_id] = dict(tokens)
req = LaunchRequest( req = LaunchRequest(
op="launch", op="launch",
bottle_id=rec.bottle_id, bottle_id=rec.bottle_id,
@@ -75,7 +66,6 @@ class Orchestrator:
finally: finally:
if not launched: if not launched:
self.registry.deregister(rec.bottle_id) self.registry.deregister(rec.bottle_id)
self._tokens.pop(rec.bottle_id, None)
return rec return rec
def teardown_bottle(self, bottle_id: str) -> bool: def teardown_bottle(self, bottle_id: str) -> bool:
@@ -86,52 +76,45 @@ class Orchestrator:
req = LaunchRequest(op="teardown", bottle_id=bottle_id, source_ip=rec.source_ip) req = LaunchRequest(op="teardown", bottle_id=bottle_id, source_ip=rec.source_ip)
self._broker.submit(sign_request(req, self._secret)) self._broker.submit(sign_request(req, self._secret))
self.registry.deregister(bottle_id) self.registry.deregister(bottle_id)
self._tokens.pop(bottle_id, None)
return True return True
def tokens_for(self, bottle_id: str) -> dict[str, str]:
"""The bottle's in-memory egress auth tokens (env_name -> value), or
empty. The gateway injects these per request; they are never
persisted."""
return dict(self._tokens.get(bottle_id, {}))
def attribute(self, source_ip: str, identity_token: str) -> BottleRecord | None: def attribute(self, source_ip: str, identity_token: str) -> BottleRecord | None:
"""Fail-closed attribution (delegates to the registry).""" """Fail-closed attribution (delegates to the registry)."""
return self.registry.attribute(source_ip, identity_token) return self.registry.attribute(source_ip, identity_token)
def resolve(self, source_ip: str, identity_token: str = "") -> BottleRecord | None: def resolve(self, source_ip: str, identity_token: str = "") -> BottleRecord | None:
"""Resolve the bottle behind a request — the source-IP-keyed lookup """Resolve the bottle behind a request — the source-IP-keyed lookup
the multi-tenant gateway makes per request; the returned record the multi-tenant sidecar makes per request; the returned record
carries its `policy`. With a token, full attribution (source IP + carries its `policy`. With a token, full attribution (source IP +
token); without, network-layer attribution by source IP alone token); without, network-layer attribution by source IP alone
(valid where the IP is unspoofable and the control plane is (valid where the IP is unspoofable and the control plane is
gateway-only).""" sidecar-only)."""
if identity_token: if identity_token:
return self.registry.attribute(source_ip, identity_token) return self.registry.attribute(source_ip, identity_token)
return self.registry.by_source_ip(source_ip) return self.registry.by_source_ip(source_ip)
def set_policy(self, bottle_id: str, policy: str) -> bool: def set_policy(self, bottle_id: str, policy: str) -> bool:
"""Update a bottle's gateway policy in place (live reload). False if """Update a bottle's sidecar policy in place (live reload). False if
the bottle is unknown.""" the bottle is unknown."""
return self.registry.set_policy(bottle_id, policy) return self.registry.set_policy(bottle_id, policy)
# --- consolidated gateway ---------------------------------------------- # --- consolidated sidecar ----------------------------------------------
def ensure_gateway(self) -> None: def ensure_sidecar(self) -> None:
"""Ensure the single per-host gateway is built and up (idempotent). """Ensure the single per-host sidecar is built and up (idempotent).
No-op when no gateway is configured.""" No-op when no sidecar is configured."""
if self._gateway is not None: if self._sidecar is not None:
self._gateway.ensure_built() self._sidecar.ensure_built()
self._gateway.ensure_running() self._sidecar.ensure_running()
def gateway_status(self) -> dict[str, object]: def sidecar_status(self) -> dict[str, object]:
"""Report the shared gateway for the control plane / console.""" """Report the shared sidecar for the control plane / console."""
if self._gateway is None: if self._sidecar is None:
return {"configured": False} return {"configured": False}
return { return {
"configured": True, "configured": True,
"name": self._gateway.name, "name": self._sidecar.name,
"running": self._gateway.is_running(), "running": self._sidecar.is_running(),
} }
+139
View File
@@ -0,0 +1,139 @@
"""The consolidated per-host sidecar (PRD 0070).
The core consolidation win: **one** persistent sidecar per host, shared by
every bottle, instead of a sidecar bundle per bottle. It's safe to share
because the attribution invariant (source IP + identity token, see
`registry`) lets the sidecar attribute each request to the right bottle
so per-bottle policy lives in one long-lived process keyed on who's calling.
`Sidecar` is the backend-neutral lifecycle contract (mirrors `LaunchBroker`):
ensure the single instance is up, report it, tear it down. `DockerSidecar`
is the docker implementation; a firecracker sidecar VM slots in later.
The defining behaviour is **idempotent singleton**: `ensure_running` starts
the instance if absent and is a no-op if it's already up, so N bottle
launches never spawn N sidecars.
"""
from __future__ import annotations
import abc
import os
from pathlib import Path
from ..docker_cmd import run_docker
SIDECAR_NAME = "bot-bottle-orch-sidecar"
SIDECAR_LABEL = "bot-bottle-orch-sidecar=1"
# The real sidecar-bundle image + its Dockerfile. Kept as a local constant
# rather than imported from backend.docker.sidecar_bundle, which would drag
# the whole backend layer into the lean orchestrator (see #359); unify when
# that lands. Env override matches the backend's BOT_BOTTLE_SIDECAR_IMAGE.
SIDECAR_BUNDLE_IMAGE = os.environ.get("BOT_BOTTLE_SIDECAR_IMAGE", "bot-bottle-sidecars:latest")
SIDECAR_DOCKERFILE = "Dockerfile.sidecars"
_REPO_ROOT = Path(__file__).resolve().parents[2]
class SidecarError(Exception):
"""The shared sidecar failed to build/start/stop (non-zero `docker` exit)."""
class Sidecar(abc.ABC):
"""Lifecycle of the single per-host sidecar. Backend-neutral."""
name: str
def ensure_built(self) -> None:
"""Ensure the sidecar's image / rootfs exists, building it if needed.
Default: nothing to build (e.g. a stub or a pre-pulled image)."""
return
@abc.abstractmethod
def ensure_running(self) -> None:
"""Start the sidecar if it isn't already up. Idempotent: a no-op
when it's already running (that's the whole point one per host).
Assumes the image exists call `ensure_built()` first."""
@abc.abstractmethod
def is_running(self) -> bool:
"""True iff the sidecar instance is currently up."""
@abc.abstractmethod
def stop(self) -> None:
"""Remove the sidecar. Idempotent — absent is success."""
class DockerSidecar(Sidecar):
"""The consolidated sidecar as a single, fixed-name Docker container.
`image_ref` defaults to the real sidecar-bundle image; `ensure_built`
builds it from `Dockerfile.sidecars` when it's missing. (Note: slice 5
builds + launches the bundle container; wiring its per-bottle,
source-IP-keyed config is a later slice see PRD 0070.)"""
def __init__(
self,
image_ref: str = SIDECAR_BUNDLE_IMAGE,
*,
name: str = SIDECAR_NAME,
build_context: Path | None = None,
dockerfile: str | None = SIDECAR_DOCKERFILE,
) -> None:
self.image_ref = image_ref
self.name = name
self._build_context = build_context or _REPO_ROOT
self._dockerfile = dockerfile
def image_exists(self) -> bool:
return run_docker(["docker", "image", "inspect", self.image_ref]).returncode == 0
def ensure_built(self) -> None:
"""Build the bundle image from its Dockerfile when it's missing.
No-op when the image is already present, or when no dockerfile is
configured (e.g. a pre-pulled image)."""
if self._dockerfile is None or self.image_exists():
return
proc = run_docker([
"docker", "build",
"-t", self.image_ref,
"-f", str(self._build_context / self._dockerfile),
str(self._build_context),
])
if proc.returncode != 0:
raise SidecarError(f"sidecar image build failed: {proc.stderr.strip()}")
def is_running(self) -> bool:
proc = run_docker([
"docker", "ps",
"--filter", f"name=^{self.name}$",
"--filter", "status=running",
"--format", "{{.Names}}",
])
return self.name in proc.stdout.split()
def ensure_running(self) -> None:
if self.is_running():
return
# Clear any stale (stopped) container holding the fixed name, then
# start fresh. `rm --force` on an absent name is a tolerated no-op.
run_docker(["docker", "rm", "--force", self.name])
proc = run_docker([
"docker", "run", "--detach",
"--name", self.name,
"--label", SIDECAR_LABEL,
self.image_ref,
])
if proc.returncode != 0:
raise SidecarError(f"sidecar failed to start: {proc.stderr.strip()}")
def stop(self) -> None:
proc = run_docker(["docker", "rm", "--force", self.name])
if proc.returncode != 0 and "No such container" not in proc.stderr:
raise SidecarError(f"sidecar failed to stop: {proc.stderr.strip()}")
__all__ = [
"Sidecar", "DockerSidecar", "SidecarError",
"SIDECAR_NAME", "SIDECAR_LABEL", "SIDECAR_BUNDLE_IMAGE",
]
+2 -2
View File
@@ -10,7 +10,7 @@ suite points it at a throwaway dir instead of monkey-patching the function
override covers them all), and operators can relocate the root if needed. override covers them all), and operators can relocate the root if needed.
This module has no bot-bottle imports, so it is safe to import from any This module has no bot-bottle imports, so it is safe to import from any
layer (and to COPY flat into the gateway). layer (and to COPY flat into the sidecar bundle).
""" """
from __future__ import annotations from __future__ import annotations
@@ -35,7 +35,7 @@ def host_db_path() -> Path:
Kept in its own `db/` subdirectory (not directly under the root) so a Kept in its own `db/` subdirectory (not directly under the root) so a
backend that can only bind-mount *directories* can share this one file backend that can only bind-mount *directories* can share this one file
with a gateway without exposing the root's other contents (git-gate with a sidecar without exposing the root's other contents (git-gate
keys, per-bottle state, ...).""" keys, per-bottle state, ...)."""
return bot_bottle_root() / "db" / HOST_DB_FILENAME return bot_bottle_root() / "db" / HOST_DB_FILENAME
+10 -62
View File
@@ -1,6 +1,6 @@
"""Gateway-side per-client policy resolver (PRD 0070). """Sidecar-side per-client policy resolver (PRD 0070).
The consolidated gateway serves every bottle from one process, so for each The consolidated sidecar serves every bottle from one process, so for each
request it must apply the *calling* bottle's policy, selected by the source request it must apply the *calling* bottle's policy, selected by the source
IP the attribution invariant makes unspoofable. This resolves that policy IP the attribution invariant makes unspoofable. This resolves that policy
from the orchestrator's control plane (`POST /resolve`), keyed on the from the orchestrator's control plane (`POST /resolve`), keyed on the
@@ -23,7 +23,7 @@ closed too rather than silently serving stale or empty policy.
The resolved value is the policy blob the orchestrator stores verbatim; the The resolved value is the policy blob the orchestrator stores verbatim; the
consumer parses it (e.g. the egress addon's `load_config`). This module is consumer parses it (e.g. the egress addon's `load_config`). This module is
stdlib-only and free of bot-bottle imports so it can be COPYed flat into stdlib-only and free of bot-bottle imports so it can be COPYed flat into
the gateway. the sidecar bundle.
""" """
from __future__ import annotations from __future__ import annotations
@@ -47,11 +47,12 @@ class PolicyResolver:
self._base = base_url.rstrip("/") self._base = base_url.rstrip("/")
self._timeout = timeout self._timeout = timeout
def _post_resolve(self, source_ip: str, identity_token: str) -> dict[str, object] | None: def resolve(self, source_ip: str, identity_token: str = "") -> str | None:
"""The orchestrator's `/resolve` payload for this client, or None if """The calling bottle's policy blob, or None if unattributed. Always
unattributed (a clean `403`). Raises `PolicyResolveError` on an fetches from the orchestrator so revocations / changes / teardowns
unreachable / unexpected-status / malformed response so every caller are honored immediately. `identity_token` is optional omit it to
can fail closed. Shared by `resolve` and `resolve_bottle_id`.""" resolve by source IP alone (the network-layer attribution).
Raises `PolicyResolveError` if the orchestrator can't be reached."""
body = json.dumps( body = json.dumps(
{"source_ip": source_ip, "identity_token": identity_token} {"source_ip": source_ip, "identity_token": identity_token}
).encode() ).encode()
@@ -68,61 +69,8 @@ class PolicyResolver:
raise PolicyResolveError(f"/resolve returned HTTP {e.code}") from e raise PolicyResolveError(f"/resolve returned HTTP {e.code}") from e
except (urllib.error.URLError, TimeoutError, OSError, ValueError) as e: except (urllib.error.URLError, TimeoutError, OSError, ValueError) as e:
raise PolicyResolveError(f"/resolve unreachable or malformed: {e}") from e raise PolicyResolveError(f"/resolve unreachable or malformed: {e}") from e
return payload if isinstance(payload, dict) else {} policy = payload.get("policy") if isinstance(payload, dict) else None
def resolve(self, source_ip: str, identity_token: str = "") -> str | None:
"""The calling bottle's policy blob, or None if unattributed. Always
fetches from the orchestrator so revocations / changes / teardowns
are honored immediately.
`identity_token` is optional *transitionally* omitting it resolves
by source IP alone (the network-layer attribution, sound by
construction on Firecracker's `/31`+nft, weaker elsewhere). The
consolidated end state makes the token mandatory so the app-layer
defense is always enforced, not silently degraded; that flips at the
`/resolve` boundary once identity-token *delivery* lands (a PRD 0070
open question we can't require what the agent can't yet present).
Raises `PolicyResolveError` if the orchestrator can't be reached."""
payload = self._post_resolve(source_ip, identity_token)
if payload is None:
return None
policy = payload.get("policy")
return policy if isinstance(policy, str) else "" return policy if isinstance(policy, str) else ""
def resolve_bottle_id(self, source_ip: str, identity_token: str = "") -> str | None:
"""The calling bottle's id, or None if unattributed. This is the
source-IP-keyed identity the git-gate uses to select the bottle's
repo namespace (there is no per-bottle policy blob to parse the
bottle *is* the namespace). Same fail-closed contract as `resolve`."""
payload = self._post_resolve(source_ip, identity_token)
if payload is None:
return None
bottle_id = payload.get("bottle_id")
return bottle_id if isinstance(bottle_id, str) and bottle_id else None
def resolve_policy_and_bottle_id(
self, source_ip: str, identity_token: str = "",
) -> tuple[str | None, str | None, dict[str, str]]:
"""The policy blob, bottle id, **and per-bottle egress auth tokens** in
a single `/resolve` so the egress addon gets everything it needs
(policy for routing, bottle id for the supervise queue/safelist, tokens
to inject upstream auth) in one round-trip. Returns `(None, None, {})`
when unattributed (a clean `403`). Raises `PolicyResolveError` if the
orchestrator can't be reached, so the caller still fails closed."""
payload = self._post_resolve(source_ip, identity_token)
if payload is None:
return None, None, {}
policy = payload.get("policy")
bottle_id = payload.get("bottle_id")
raw = payload.get("tokens")
tokens = {
k: v for k, v in raw.items() if isinstance(k, str) and isinstance(v, str)
} if isinstance(raw, dict) else {}
return (
policy if isinstance(policy, str) else "",
bottle_id if isinstance(bottle_id, str) and bottle_id else None,
tokens,
)
__all__ = ["PolicyResolver", "PolicyResolveError"] __all__ = ["PolicyResolver", "PolicyResolveError"]
+1 -1
View File
@@ -26,7 +26,7 @@ class QueueStore(DbStore):
if db_path is not None: if db_path is not None:
resolved = db_path resolved = db_path
else: else:
# In the gateway container SUPERVISE_DB_PATH points at the # In the sidecar container SUPERVISE_DB_PATH points at the
# bind-mounted host DB. On the host this env var is never set, # bind-mounted host DB. On the host this env var is never set,
# so we always fall through to host_db_path(). # so we always fall through to host_db_path().
env_path = os.environ.get("SUPERVISE_DB_PATH", "").strip() env_path = os.environ.get("SUPERVISE_DB_PATH", "").strip()
@@ -1,26 +1,26 @@
"""Gateway data-plane supervisor (PRD 0070; PRD 0024 bundle shape). """Per-bottle sidecar supervisor (PRD 0024 chunk 1).
PID 1 inside the `bot-bottle-gateway` data-plane image. Spawns PID 1 inside the `bot-bottle-sidecars` bundle image. Spawns
the configured daemons (egress, git-gate, supervise), the configured daemons (egress, git-gate, supervise),
forwards SIGTERM/SIGINT to each child, and propagates per-daemon forwards SIGTERM/SIGINT to each child, and propagates per-daemon
stdout+stderr to the container log with a `[name] ` prefix. stdout+stderr to the container log with a `[name] ` prefix.
Failure policy (interim): when a child dies unexpectedly, the Failure policy (interim): when a child dies unexpectedly, the
supervisor logs the death and leaves the surviving children supervisor logs the death and leaves the surviving children
running. The gateway stays up; whatever the dead daemon served running. The bundle stays up; whatever the dead daemon served
will start failing, surfacing in the agent's own error path. will start failing, surfacing in the agent's own error path.
The supervisor itself exits only when (a) the operator sends The supervisor itself exits only when (a) the operator/compose
SIGTERM/SIGINT, or (b) every child has died. sends SIGTERM/SIGINT, or (b) every child has died.
Failure policy (eventual): on unexpected death, the supervisor Failure policy (eventual): on unexpected death, the supervisor
restarts the daemon and emits a notification to the supervise restarts the daemon and emits a notification to the supervise
daemon so the operator sees the event. That lands in a later sidecar so the operator sees the event. That lands in a later
PR; the interim policy is "don't take the gateway down for one PR; the interim policy is "don't take the bundle down for one
sick daemon." sick daemon."
Daemon subset is env-driven via `BOT_BOTTLE_GATEWAY_DAEMONS=egress` Daemon subset is env-driven. The compose renderer narrows it via
for callers that don't use git-gate or supervise. Default: all `BOT_BOTTLE_SIDECAR_DAEMONS=egress` for bottles that
daemons. don't use git-gate or supervise. Default: all daemons.
Stdlib-only by design adding supervisord/s6/runit for four Stdlib-only by design adding supervisord/s6/runit for four
daemons is heavier than this script. daemons is heavier than this script.
@@ -103,8 +103,8 @@ def _selected_daemons(
env: dict[str, str], env: dict[str, str],
all_daemons: Sequence[_DaemonSpec] | None = None, all_daemons: Sequence[_DaemonSpec] | None = None,
) -> tuple[_DaemonSpec, ...]: ) -> tuple[_DaemonSpec, ...]:
"""Filter the daemon set by the BOT_BOTTLE_GATEWAY_DAEMONS env """Filter the daemon set by the BOT_BOTTLE_SIDECAR_DAEMONS env
var. Unknown names in the list are ignored the caller is the var. Unknown names in the list are ignored the renderer is the
source of truth for which daemons are wired. source of truth for which daemons are wired.
`all_daemons` defaults to `_DAEMONS` resolved at call time (not `all_daemons` defaults to `_DAEMONS` resolved at call time (not
@@ -112,7 +112,7 @@ def _selected_daemons(
`_DAEMONS` and have the new value take effect.""" `_DAEMONS` and have the new value take effect."""
if all_daemons is None: if all_daemons is None:
all_daemons = _DAEMONS all_daemons = _DAEMONS
raw = env.get("BOT_BOTTLE_GATEWAY_DAEMONS", "").strip() raw = env.get("BOT_BOTTLE_SIDECAR_DAEMONS", "").strip()
if not raw: if not raw:
return tuple(all_daemons) return tuple(all_daemons)
wanted = {n.strip() for n in raw.split(",") if n.strip()} wanted = {n.strip() for n in raw.split(",") if n.strip()}
@@ -120,7 +120,7 @@ def _selected_daemons(
def _log(msg: str) -> None: def _log(msg: str) -> None:
sys.stdout.write(f"gateway-init: {msg}\n") sys.stdout.write(f"sidecar-init: {msg}\n")
sys.stdout.flush() sys.stdout.flush()
+14 -14
View File
@@ -1,25 +1,25 @@
"""Per-bottle supervise plane (PRD 0013). """Per-bottle supervise plane (PRD 0013).
The supervise plane is the per-bottle MCP daemon plus its host-side The supervise plane is the per-bottle MCP sidecar plus its host-side
queue/audit support. The daemon (bot_bottle.supervise_server) queue/audit support. The sidecar (bot_bottle.supervise_server)
sits on the bottle's internal network and exposes MCP tools the agent sits on the bottle's internal network and exposes MCP tools the agent
calls when it needs an operator-reviewed egress change: calls when it needs an operator-reviewed egress change:
* egress-block / allow agent proposes a new routes.yaml * egress-block / allow agent proposes a new routes.yaml
Each tool call: the agent passes the full proposed file plus a Each tool call: the agent passes the full proposed file plus a
justification text. The gateway validates the proposal syntactically, justification text. The sidecar validates the proposal syntactically,
writes it to the host SQLite queue table, and holds the tool-call writes it to the host SQLite queue table, and holds the tool-call
connection open. The operator's supervise TUI connection open. The operator's supervise TUI
(bot_bottle.cli.supervise) sees the proposal, accepts (bot_bottle.cli.supervise) sees the proposal, accepts
approve / modify / reject, and writes a response row. The gateway sees approve / modify / reject, and writes a response row. The sidecar sees
the response and returns `{status, notes}` to the agent. the response and returns `{status, notes}` to the agent.
This module defines the host-side library: dataclasses for the queue This module defines the host-side library: dataclasses for the queue
record shapes, queue read/write helpers, the audit log writer, and the record shapes, queue read/write helpers, the audit log writer, and the
diff renderer. The in-gateway daemon lives in diff renderer. The in-container sidecar lives in
bot_bottle/supervise_server.py; the supervise daemon's container bot_bottle/supervise_server.py; the supervise daemon's container
lifecycle is owned by the gateway (PRD 0024). lifecycle is owned by the sidecar bundle (PRD 0024).
For 0013 the supervisor's approval handlers are deliberately no-ops: For 0013 the supervisor's approval handlers are deliberately no-ops:
on approval the audit log is written and the response file is on approval the audit log is written and the response file is
@@ -75,18 +75,18 @@ except ImportError:
try: try:
from .paths import bot_bottle_root from .paths import bot_bottle_root
except ImportError: # flat imports inside the gateway except ImportError: # flat imports inside the sidecar bundle
from paths import bot_bottle_root # type: ignore[import-not-found,no-redef] # pylint: disable=import-error,no-name-in-module from paths import bot_bottle_root # type: ignore[import-not-found,no-redef] # pylint: disable=import-error,no-name-in-module
SUPERVISE_HOSTNAME = "supervise" SUPERVISE_HOSTNAME = "supervise"
SUPERVISE_PORT = 9100 SUPERVISE_PORT = 9100
# The supervise daemon uses these to query egress's # The supervise sidecar uses these to query egress's
# introspection endpoint for the `list-egress-routes` MCP # introspection endpoint for the `list-egress-routes` MCP
# tool. The hostname + port match egress's docker network # tool. The hostname + port match egress's docker network
# listen port (see backend.docker.egress.EGRESS_PORT). The supervise # listen port (see backend.docker.egress.EGRESS_PORT). The supervise
# daemon runs inside the gateway alongside egress, so loopback # daemon runs inside the sidecar bundle alongside egress, so loopback
# is the stable address across docker, firecracker, and Apple # is the stable address across docker, firecracker, and Apple
# Container backends. # Container backends.
EGRESS_FORWARD_PROXY = "http://127.0.0.1:9099" EGRESS_FORWARD_PROXY = "http://127.0.0.1:9099"
@@ -117,7 +117,7 @@ try:
from .audit_store import AuditStore from .audit_store import AuditStore
from .store_manager import StoreManager from .store_manager import StoreManager
except ImportError: except ImportError:
# Gateway: files are flat-copied under /app, not a package. # Sidecar bundle: files are flat-copied under /app, not a package.
from queue_store import QueueStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module from queue_store import QueueStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from audit_store import AuditStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module from audit_store import AuditStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from store_manager import StoreManager # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module from store_manager import StoreManager # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
@@ -222,14 +222,14 @@ def sha256_hex(content: str) -> str:
return hashlib.sha256(content.encode("utf-8")).hexdigest() return hashlib.sha256(content.encode("utf-8")).hexdigest()
# --- Gateway plan + abstract lifecycle ------------------------------------- # --- Sidecar plan + abstract lifecycle -------------------------------------
@dataclass(frozen=True) @dataclass(frozen=True)
class SupervisePlan: class SupervisePlan:
"""Output of Supervise.prepare; consumed by .start. """Output of Supervise.prepare; consumed by .start.
`db_path` is the host database bind-mounted into the gateway at `db_path` is the host database bind-mounted into the sidecar at
/run/supervise/bot-bottle.db. `internal_network` is empty at /run/supervise/bot-bottle.db. `internal_network` is empty at
prepare time; the backend's launch step fills it via prepare time; the backend's launch step fills it via
dataclasses.replace before calling .start.""" dataclasses.replace before calling .start."""
@@ -240,8 +240,8 @@ class SupervisePlan:
class Supervise(ABC): class Supervise(ABC):
"""Per-bottle supervise daemon. Encapsulates host-side database """Per-bottle supervise sidecar. Encapsulates host-side database
staging; the gateway's start/stop lifecycle is backend-specific.""" staging; the sidecar's start/stop lifecycle is backend-specific."""
def prepare( def prepare(
self, self,
+7 -52
View File
@@ -1,4 +1,4 @@
"""Supervise daemon HTTP server (PRD 0013). """Supervise sidecar HTTP server (PRD 0013).
Per-bottle MCP server exposing tools the agent calls to propose egress Per-bottle MCP server exposing tools the agent calls to propose egress
config changes when stuck. The tools are `egress-allow`, config changes when stuck. The tools are `egress-allow`,
@@ -15,12 +15,6 @@ The bottle slug arrives via SUPERVISE_BOTTLE_SLUG env (stamped at
container creation by the backend's start step). SUPERVISE_DB_PATH container creation by the backend's start step). SUPERVISE_DB_PATH
points at the bind-mounted host database. points at the bind-mounted host database.
Consolidated (PRD 0070): when BOT_BOTTLE_ORCHESTRATOR_URL is set, one
shared server fronts every bottle and attributes each proposal to the
calling bottle by source IP (resolved from the orchestrator) instead of a
fixed slug an unattributed source fails closed. Unset the legacy
per-bottle single-tenant server, unchanged.
Speaks MCP over HTTP+JSON-RPC. Methods handled: Speaks MCP over HTTP+JSON-RPC. Methods handled:
* `initialize` handshake; returns server info + caps. * `initialize` handshake; returns server info + caps.
@@ -46,18 +40,16 @@ import time
import typing import typing
import urllib.error import urllib.error
import urllib.request import urllib.request
from dataclasses import dataclass, replace from dataclasses import dataclass
try: try:
# Same-directory imports inside the bundle container; these files are # Same-directory imports inside the bundle container; these files are
# COPYed flat under /app by Dockerfile.gateway. # COPYed flat under /app by Dockerfile.sidecars.
from egress_addon_core import LOG_OFF, load_config from egress_addon_core import LOG_OFF, load_config
from policy_resolver import PolicyResolveError, PolicyResolver
import supervise as _sv import supervise as _sv
except ModuleNotFoundError: except ModuleNotFoundError:
# Package imports for host-side tests and tooling. # Package imports for host-side tests and tooling.
from .egress_addon_core import LOG_OFF, load_config from .egress_addon_core import LOG_OFF, load_config
from .policy_resolver import PolicyResolveError, PolicyResolver
from . import supervise as _sv from . import supervise as _sv
@@ -81,12 +73,6 @@ DEFAULT_RESPONSE_TIMEOUT_SECONDS = 30.0
MIN_RESPONSE_POLL_INTERVAL_SECONDS = 0.05 MIN_RESPONSE_POLL_INTERVAL_SECONDS = 0.05
EGRESS_LIST_TIMEOUT_SECONDS = 5.0 EGRESS_LIST_TIMEOUT_SECONDS = 5.0
# Consolidated (multi-tenant) mode: when set, one shared supervise server
# fronts every bottle and attributes each proposal to the calling bottle by
# source IP (resolved from the orchestrator), instead of a single
# SUPERVISE_BOTTLE_SLUG env. Unset → legacy per-bottle single-tenant.
ORCHESTRATOR_URL_ENV = "BOT_BOTTLE_ORCHESTRATOR_URL"
@dataclass(frozen=True) @dataclass(frozen=True)
class JsonRpcRequest: class JsonRpcRequest:
@@ -525,30 +511,9 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
if method == "tools/list": if method == "tools/list":
return handle_tools_list(req.params) return handle_tools_list(req.params)
if method == "tools/call": if method == "tools/call":
# Attribute the proposal to the calling bottle. Single-tenant → the return handle_tools_call(req.params, config)
# env slug on `config`; consolidated → the source-IP-resolved
# bottle id, so one shared server queues each bottle's proposal
# under its own slug.
return handle_tools_call(req.params, self._attributed_config(config))
raise _RpcClientError(ERR_METHOD_NOT_FOUND, f"method not found: {method}") raise _RpcClientError(ERR_METHOD_NOT_FOUND, f"method not found: {method}")
def _attributed_config(self, config: ServerConfig) -> ServerConfig:
"""The ServerConfig with `bottle_slug` bound to *this request's* bottle.
Single-tenant (no resolver): unchanged. Consolidated: the bottle id
attributed from the source IP **fail-closed**, an unattributed or
unreachable source raises so no proposal is queued under the wrong (or
empty) slug."""
resolver = getattr(self.server, "policy_resolver", None)
if resolver is None:
return config
try:
bottle_id = resolver.resolve_bottle_id(self.client_address[0])
except PolicyResolveError as e:
raise _RpcInternalError(f"orchestrator unreachable, cannot attribute: {e}") from e
if not bottle_id:
raise _RpcInternalError("request source is not attributed to a bottle")
return replace(config, bottle_slug=bottle_id)
def _write_jsonrpc(self, body: bytes) -> None: def _write_jsonrpc(self, body: bytes) -> None:
self.send_response(200) self.send_response(200)
self.send_header("Content-Type", "application/json") self.send_header("Content-Type", "application/json")
@@ -572,9 +537,6 @@ class MCPServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
allow_reuse_address = True allow_reuse_address = True
daemon_threads = True daemon_threads = True
config: ServerConfig = ServerConfig(bottle_slug="") config: ServerConfig = ServerConfig(bottle_slug="")
# None → single-tenant (proposals use config.bottle_slug); set → consolidated
# (each proposal attributed to the source-IP-resolved bottle).
policy_resolver: "PolicyResolver | None" = None
# --- Entry point ----------------------------------------------------------- # --- Entry point -----------------------------------------------------------
@@ -586,17 +548,15 @@ def serve(
port: int = _sv.SUPERVISE_PORT, port: int = _sv.SUPERVISE_PORT,
bind: str = "0.0.0.0", bind: str = "0.0.0.0",
response_timeout_seconds: float = DEFAULT_RESPONSE_TIMEOUT_SECONDS, response_timeout_seconds: float = DEFAULT_RESPONSE_TIMEOUT_SECONDS,
resolver: "PolicyResolver | None" = None,
) -> typing.NoReturn: ) -> typing.NoReturn:
server = MCPServer((bind, port), MCPHandler) server = MCPServer((bind, port), MCPHandler)
server.config = ServerConfig( server.config = ServerConfig(
bottle_slug=bottle_slug, bottle_slug=bottle_slug,
response_timeout_seconds=response_timeout_seconds, response_timeout_seconds=response_timeout_seconds,
) )
server.policy_resolver = resolver
mode = "multi-tenant" if resolver else f"slug={bottle_slug!r}"
sys.stderr.write( sys.stderr.write(
f"supervise listening on {bind}:{port}; {mode}; " f"supervise listening on {bind}:{port}; "
f"slug={bottle_slug!r}; "
f"tools: {', '.join(t['name'] for t in TOOL_DEFINITIONS)}\n" # type: ignore[arg-type] f"tools: {', '.join(t['name'] for t in TOOL_DEFINITIONS)}\n" # type: ignore[arg-type]
) )
sys.stderr.flush() sys.stderr.flush()
@@ -611,12 +571,8 @@ def serve(
def main(argv: list[str]) -> int: def main(argv: list[str]) -> int:
del argv # config is env-only, no CLI flags del argv # config is env-only, no CLI flags
orch_url = os.environ.get(ORCHESTRATOR_URL_ENV, "").strip()
resolver = PolicyResolver(orch_url) if orch_url else None
bottle_slug = os.environ.get("SUPERVISE_BOTTLE_SLUG", "") bottle_slug = os.environ.get("SUPERVISE_BOTTLE_SLUG", "")
# Consolidated mode resolves the slug per request, so the env slug is if not bottle_slug:
# optional there; single-tenant still requires it.
if not bottle_slug and resolver is None:
sys.stderr.write("supervise: SUPERVISE_BOTTLE_SLUG env is unset\n") sys.stderr.write("supervise: SUPERVISE_BOTTLE_SLUG env is unset\n")
return 2 return 2
port = int(os.environ.get("SUPERVISE_PORT", str(_sv.SUPERVISE_PORT))) port = int(os.environ.get("SUPERVISE_PORT", str(_sv.SUPERVISE_PORT)))
@@ -631,7 +587,6 @@ def main(argv: list[str]) -> int:
port=port, port=port,
bind=bind, bind=bind,
response_timeout_seconds=response_timeout_seconds, response_timeout_seconds=response_timeout_seconds,
resolver=resolver,
) )
return 0 # serve() does not return return 0 # serve() does not return
+1 -1
View File
@@ -71,7 +71,7 @@ class YamlSubsetError(ValueError):
that want fatal-exit semantics (manifest loader, egress-apply, that want fatal-exit semantics (manifest loader, egress-apply,
etc.) catch this at their own boundary and forward to `die`; etc.) catch this at their own boundary and forward to `die`;
callers running outside the bot-bottle CLI process (the callers running outside the bot-bottle CLI process (the
egress daemon's addon) handle it as a normal exception.""" egress sidecar's addon) handle it as a normal exception."""
+2 -1
View File
@@ -22,7 +22,8 @@ mounted in. That topology breaks two assumptions those tests make:
`http://127.0.0.1:<host_port>` from inside the job time out. `http://127.0.0.1:<host_port>` from inside the job time out.
The affected tests (`test_orphan_cleanup.test_create_and_remove`, The affected tests (`test_orphan_cleanup.test_create_and_remove`,
`test_gateway_image.TestGatewayImage`) still run `test_sidecar_bundle_image.TestSidecarBundleImage`,
`test_sidecar_bundle_compose.TestSidecarBundleCompose`) still run
locally where the test process and Docker daemon share a host. locally where the test process and Docker daemon share a host.
Making them work in CI is a follow-up: either re-write them to Making them work in CI is a follow-up: either re-write them to
discover container IPs via `docker inspect`, or reconfigure the discover container IPs via `docker inspect`, or reconfigure the
+2 -2
View File
@@ -38,7 +38,7 @@ Type "y"
Enter Enter
# Wait for the bottle to launch: networks created, pipelock + git-gate # Wait for the bottle to launch: networks created, pipelock + git-gate
# companion containers started, agent container started, claude boots. # sidecars started, agent container started, claude boots.
Sleep 22s Sleep 22s
# Probe 1 — warm-up. A reply at all proves api.anthropic.com is # Probe 1 — warm-up. A reply at all proves api.anthropic.com is
@@ -72,7 +72,7 @@ Type "init /tmp/r, commit AKIAQRJHK7N5ZPM2VXTL to leak.txt, push to ssh://git@up
Enter Enter
Sleep 30s Sleep 30s
# Leave claude. The launcher tears down the container, companion containers, and # Leave claude. The launcher tears down the container, sidecars, and
# networks on session end. # networks on session end.
Ctrl+D Ctrl+D
Sleep 4s Sleep 4s
+20 -20
View File
@@ -10,8 +10,8 @@
## Summary ## Summary
Replace the **per-bottle gateway bundle** with a single **persistent, Replace the **per-bottle sidecar bundle** with a single **persistent,
per-host orchestrator**: one long-lived service that runs the gateway per-host orchestrator**: one long-lived service that runs the sidecar
functions (egress / git-gate / supervise), coordinates with the console, functions (egress / git-gate / supervise), coordinates with the console,
and brokers agent launches and teardown. It is **virtualized from the and brokers agent launches and teardown. It is **virtualized from the
start** using each backend's native isolation primitive — a Firecracker start** using each backend's native isolation primitive — a Firecracker
@@ -22,13 +22,13 @@ container on the legacy backend — and is fronted by a single
## Motivation ## Motivation
Today each bottle spins up its own gateway bundle (egress mitmproxy + Today each bottle spins up its own sidecar bundle (egress mitmproxy +
git-gate + supervise). That costs: git-gate + supervise). That costs:
- **Resources.** N bottles → N heavy bundles booting and idling. - **Resources.** N bottles → N heavy bundles booting and idling.
- **Operational churn.** Per-launch container/VM lifecycle for the - **Operational churn.** Per-launch container/VM lifecycle for the
gateways, a control path baked at launch and torn down at exit. sidecars, a control path baked at launch and torn down at exit.
- **A blurry contract.** "How a bottle talks to its gateway" is - **A blurry contract.** "How a bottle talks to its sidecar" is
re-implemented per backend instead of being one agreed interface. re-implemented per backend instead of being one agreed interface.
A per-host orchestrator collapses the first two and forces the third to A per-host orchestrator collapses the first two and forces the third to
@@ -52,11 +52,11 @@ weaken, and the weakened ones must be mitigated by design, not hand-waved.
structured, auditable privileged core instead of a fat socket. structured, auditable privileged core instead of a fat socket.
- **Attribution is enforced, not assumed.** Making source-IP identity a - **Attribution is enforced, not assumed.** Making source-IP identity a
first-class contract invariant (below) means each backend must *prove* first-class contract invariant (below) means each backend must *prove*
it, rather than the gateway implicitly trusting network position. it, rather than the sidecar implicitly trusting network position.
### What gets weaker, and the mitigation ### What gets weaker, and the mitigation
1. **Secret concentration.** Per-bottle gateways isolate secrets at the 1. **Secret concentration.** Per-bottle sidecars isolate secrets at the
process boundary — each holds only its bottle's tokens/keys. A host process boundary — each holds only its bottle's tokens/keys. A host
orchestrator concentrates **every bottle's** egress tokens, git deploy orchestrator concentrates **every bottle's** egress tokens, git deploy
keys, and the console credential in one long-lived process. A single keys, and the console credential in one long-lived process. A single
@@ -75,7 +75,7 @@ weaken, and the weakened ones must be mitigated by design, not hand-waved.
fleet, plus launch authority, plus the console token. fleet, plus launch authority, plus the console token.
- *Mitigation:* the orchestrator is itself confined (its own VM/container - *Mitigation:* the orchestrator is itself confined (its own VM/container
with its own fail-closed egress); make it **restartable without killing with its own fail-closed egress); make it **restartable without killing
running agent VMs** (agents keep running; they briefly lose gateway running agent VMs** (agents keep running; they briefly lose sidecar
connectivity until it's back); persist state to a host volume so a connectivity until it's back); persist state to a host volume so a
restart re-adopts live bottles rather than losing them. restart re-adopts live bottles rather than losing them.
@@ -114,13 +114,13 @@ responsibility** and part of the contract:
- **Apple** — the host-only network. - **Apple** — the host-only network.
If a backend can't honor the invariant, source-IP consolidation is not If a backend can't honor the invariant, source-IP consolidation is not
safe there and that backend keeps per-bottle gateways. The invariant is a safe there and that backend keeps per-bottle sidecars. The invariant is a
hard precondition, not an aspiration. hard precondition, not an aspiration.
**Defense-in-depth — a per-bottle identity token.** On top of the **Defense-in-depth — a per-bottle identity token.** On top of the
network-layer invariant, inject a per-bottle secret token into every network-layer invariant, inject a per-bottle secret token into every
request the agent makes to the orchestrator (the agent already egresses request the agent makes to the orchestrator (the agent already egresses
through the gateway proxy, so this is cheap to add). It gives an through the sidecar proxy, so this is cheap to add). It gives an
**application-layer** proof of identity independent of the network layer: **application-layer** proof of identity independent of the network layer:
- On **Firecracker** the `/31` + nft already make source IP unspoofable - On **Firecracker** the `/31` + nft already make source IP unspoofable
@@ -133,7 +133,7 @@ Requirements: the token is **per-bottle, unguessable, and
non-cross-leakable** — a bottle can only ever prove it is *itself* (it non-cross-leakable** — a bottle can only ever prove it is *itself* (it
can't learn another bottle's token, given bottle isolation), so a hostile can't learn another bottle's token, given bottle isolation), so a hostile
agent gains nothing by presenting it. The orchestrator provisions the agent gains nothing by presenting it. The orchestrator provisions the
token at launch and the gateway requires it to attribute + authorize. token at launch and the sidecar requires it to attribute + authorize.
Note this hardens *attribution*, not *secret exposure*: it's app-layer, so Note this hardens *attribution*, not *secret exposure*: it's app-layer, so
a compromised orchestrator still sees every token (that's the a compromised orchestrator still sees every token (that's the
concentration problem, addressed separately under Secret handling). concentration problem, addressed separately under Secret handling).
@@ -178,7 +178,7 @@ manifest anywhere a raw token value is accepted, discovered from
`AgentProvider`s are — because maintaining every forge/cloud/OAuth `AgentProvider`s are — because maintaining every forge/cloud/OAuth
provider in-tree is untenable. The orchestrator's vault mints via these provider in-tree is untenable. The orchestrator's vault mints via these
providers; but the abstraction is shippable independently and today's providers; but the abstraction is shippable independently and today's
per-bottle gateways can use it too. per-bottle sidecars can use it too.
## Design ## Design
@@ -195,7 +195,7 @@ Three surfaces; only one is per-backend.
every host (no vsock/unix-socket portability caveats); a local unix every host (no vsock/unix-socket portability caveats); a local unix
socket is a fine optimization, but HTTP is the wire contract. socket is a fine optimization, but HTTP is the wire contract.
2. **Data plane (agent → orchestrator)** — the egress / git / supervise 2. **Data plane (agent → orchestrator)** — the egress / git / supervise
endpoints. Already agnostic today (agents dial `http://gateway:9099`); endpoints. Already agnostic today (agents dial `http://sidecar:9099`);
only the *address* and *how packets get there* are per-backend. only the *address* and *how packets get there* are per-backend.
3. **Launch / wire (orchestrator → backend)** — the irreducibly 3. **Launch / wire (orchestrator → backend)** — the irreducibly
backend-specific part; lives on `BottleBackend`. backend-specific part; lives on `BottleBackend`.
@@ -249,7 +249,7 @@ forged one from a compromised co-located component. The orchestrator signs;
the broker verifies with the orchestrator's public key (provisioned at the broker verifies with the orchestrator's public key (provisioned at
broker install). This is the concrete form of security #3's "structured broker install). This is the concrete form of security #3's "structured
requests only." Same JSON-with-ids + JWT shape for all requests only." Same JSON-with-ids + JWT shape for all
orchestrator↔broker/gateway communication; cases that don't fit get orchestrator↔broker/sidecar communication; cases that don't fit get
handled as they arise, with this as the default. handled as they arise, with this as the default.
If `BottleBackend` bloats, the pressure valve is composition one level If `BottleBackend` bloats, the pressure valve is composition one level
@@ -299,10 +299,10 @@ and integrate a console against.
the data plane and the console reach state through the control-plane RPC, the data plane and the console reach state through the control-plane RPC,
never a direct file handle.** No agent-facing component gets the file, so never a direct file handle.** No agent-facing component gets the file, so
none can forge attribution. (This supersedes the earlier `ro`-mount idea.) none can forge attribution. (This supersedes the earlier `ro`-mount idea.)
- *Transitional caveat:* today the per-bottle **supervise gateway - *Transitional caveat:* today the per-bottle **supervise sidecar
rw-bind-mounts `bot-bottle.db`** to write proposals — exactly the rw-bind-mounts `bot-bottle.db`** to write proposals — exactly the
pattern the orchestrator removes (supervise consolidates into the pattern the orchestrator removes (supervise consolidates into the
orchestrator; gateway writes become RPC calls). Until that lands, don't orchestrator; sidecar writes become RPC calls). Until that lands, don't
put the attribution registry behind a data-plane-writable mount. put the attribution registry behind a data-plane-writable mount.
Implementation note for the VM slices: SQLite **WAL** over a guest share Implementation note for the VM slices: SQLite **WAL** over a guest share
@@ -327,7 +327,7 @@ orchestrator becomes a VM. Decouple the two risks instead:
Backend order (cheapest proof → hardest → last): Backend order (cheapest proof → hardest → last):
1. **Docker orchestrator** — nearly free (the gateway bundle is already 1. **Docker orchestrator** — nearly free (the sidecar bundle is already
containers; collapse N bundles into one persistent container). Proves containers; collapse N bundles into one persistent container). Proves
consolidation + the `BottleBackend` seam with the least moving parts. consolidation + the `BottleBackend` seam with the least moving parts.
2. **Firecracker orchestrator** — the real work: the shim + VM-to-VM 2. **Firecracker orchestrator** — the real work: the shim + VM-to-VM
@@ -336,13 +336,13 @@ Backend order (cheapest proof → hardest → last):
the dev-harness so the app logic is already proven. the dev-harness so the app logic is already proven.
3. **macOS (Apple container)** — last (container-to-container networking). 3. **macOS (Apple container)** — last (container-to-container networking).
Keep the gateway **service one shared thing** throughout. Keep the sidecar **service one shared thing** throughout.
## Non-goals ## Non-goals
- Removing OCI/Dockerfile support for agent images (0069's concern). - Removing OCI/Dockerfile support for agent images (0069's concern).
- A single database for *all* config (see the three-tier table). - A single database for *all* config (see the three-tier table).
- Changing the per-bottle isolation of agent workloads — only the gateway - Changing the per-bottle isolation of agent workloads — only the sidecar
is consolidated; agents stay one-VM/container-each. is consolidated; agents stay one-VM/container-each.
## Relationship to other work ## Relationship to other work
@@ -393,7 +393,7 @@ Keep the gateway **service one shared thing** throughout.
`BottleBackend.wire()` (DNAT+forward for fc, shared-net for `BottleBackend.wire()` (DNAT+forward for fc, shared-net for
docker/apple), not orchestrator logic. **Not a blocker** — resolvable at docker/apple), not orchestrator logic. **Not a blocker** — resolvable at
implementation time (may require a bit of host modification, as the pool implementation time (may require a bit of host modification, as the pool
setup already does on NixOS); an earlier "gateways in VMs" spike showed setup already does on NixOS); an earlier "sidecars in VMs" spike showed
it's feasible. it's feasible.
- **Live-reload protocol** for per-bottle policy over the HTTP control - **Live-reload protocol** for per-bottle policy over the HTTP control
plane (add/remove routes/keys/proposals without a restart). plane (add/remove routes/keys/proposals without a restart).
@@ -15,25 +15,6 @@ the biggest credential risk).
## Summary ## Summary
> **Status — implemented (2026-07-14).** This note's recommendation
> shipped, so the "today" / "in-flight" tenses below are now historical
> (kept as the reasoning that led here). Current reality: bot-bottle no
> longer injects raw provider tokens into the agent. The agent's environ
> carries only a **placeholder** (e.g.
> `CLAUDE_CODE_OAUTH_TOKEN=egress-placeholder`); the real token is held
> in the egress sidecar and injected as the upstream `Authorization`
> header. Rather than adding a *separate* in-container reverse proxy (as
> proposed below), credential injection was folded into the **existing
> pipelock/mitmproxy egress firewall**: an `EgressRoute` carries
> `auth_scheme` + `token_ref`, the host token is forwarded into the
> sidecar addon under an `EGRESS_TOKEN_N` slot, and the addon rewrites
> `Authorization` on matching routes — **one MITM, not two** (this
> resolves the Infisical-doubling concern noted later). The same
> mechanism now covers Codex and Pi provider tokens and git-host PATs.
> A `printenv` inside the bottle yields the placeholder, not the secret.
> For current wiring see `bot_bottle/egress.py` and
> `bot_bottle/contrib/*/agent_provider.py`.
Today every bot-bottle agent gets `CLAUDE_CODE_OAUTH_TOKEN` (and Today every bot-bottle agent gets `CLAUDE_CODE_OAUTH_TOKEN` (and
any `bottle.env` secrets like a Gitea PAT) injected as env vars, any `bottle.env` secrets like a Gitea PAT) injected as env vars,
which means the agent process can read them with `printenv` or which means the agent process can read them with `printenv` or
@@ -55,24 +36,6 @@ small bot-bottle-specific reverse proxy modeled on the
phantom-token shape is probably the right call. For Gitea / GitHub / phantom-token shape is probably the right call. For Gitea / GitHub /
GitLab, the same proxy generalizes by config. GitLab, the same proxy generalizes by config.
**Updated 2026-07-14:** OneCLI ([onecli.sh](https://onecli.sh/)) —
already listed below — has matured into a GA, YC-backed Rust
credential gateway ("The Identity Gateway for AI Agents", ~2.5k⭐,
300k+ downloads) that implements the **same phantom-token pattern**
this note recommends: the agent holds a placeholder token, the
gateway swaps it for the real (AES-256-GCM-encrypted) credential at
request time. It's now the most mature open-source realization of the
exact design proposed here — a production-ready alternative to
alpha-stage nono — at the cost of being a broader product (built-in
vault + management dashboard + hosted cloud tier + 50+ app
integrations) rather than a minimal proxy. Its managed/cloud tier and
per-agent dashboard also overlap bot-bottle's own planned paid control
plane (bot-bottle-console, issue #327), so it's worth tracking as both
a build-vs-adopt option *and* a product-level competitor. The
build-first recommendation still stands (see synthesis below), but
adopting OneCLI's OSS core is now a credible alternative where nono was
too green.
## The shared problem ## The shared problem
Linux has no per-env-var ACL. Once a var is in a process's Linux has no per-env-var ACL. Once a var is in a process's
@@ -114,9 +77,7 @@ The remaining credible designs reduce to three:
### Anthropic / Claude Code ### Anthropic / Claude Code
**Original wiring** (pre-gateway; **superseded 2026-07-14** — see the **Today's wiring** (`bot_bottle/cli/start.py`): the host's
Status note in the Summary; the token now lives in the egress sidecar
and the agent gets a placeholder): the host's
`BOT_BOTTLE_CLAUDE_OAUTH_TOKEN` is forwarded into the bottle as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN` is forwarded into the bottle as
`CLAUDE_CODE_OAUTH_TOKEN` via `docker run -e CLAUDE_CODE_OAUTH_TOKEN` `CLAUDE_CODE_OAUTH_TOKEN` via `docker run -e CLAUDE_CODE_OAUTH_TOKEN`
(no `=value`, so the value never lands on argv — good). Inside the (no `=value`, so the value never lands on argv — good). Inside the
@@ -262,7 +223,7 @@ Two categories:
| **Infisical Agent Vault** | B | MIT (EE carve-out) | In-process HTTPS_PROXY forward proxy | TLS MITM, dummy-to-real swap | No — HTTPS_PROXY model | Service-level | Active; v0.19.0 May 2026, ~1k⭐ | | **Infisical Agent Vault** | B | MIT (EE carve-out) | In-process HTTPS_PROXY forward proxy | TLS MITM, dummy-to-real swap | No — HTTPS_PROXY model | Service-level | Active; v0.19.0 May 2026, ~1k⭐ |
| **nono** | B | Apache-2.0 | In-process reverse proxy | Phantom token, explicit URL routing | **Yes**`BASE_URL=http://127.0.0.1:PORT/…` | Host + endpoint | Early alpha; v0.53.0 May 2026, 2.4k⭐ | | **nono** | B | Apache-2.0 | In-process reverse proxy | Phantom token, explicit URL routing | **Yes**`BASE_URL=http://127.0.0.1:PORT/…` | Host + endpoint | Early alpha; v0.53.0 May 2026, 2.4k⭐ |
| **Aegis** | B | Apache-2.0 | In-process reverse proxy | Path routing (`localhost:3100/{svc}/…`) | Configurable, undocumented for Anthropic | Method/path/rate/time | Very new, 10⭐ | | **Aegis** | B | Apache-2.0 | In-process reverse proxy | Path routing (`localhost:3100/{svc}/…`) | Configurable, undocumented for Anthropic | Method/path/rate/time | Very new, 10⭐ |
| **OneCLI** | B | Apache-2.0 | Reverse proxy + built-in vault + dashboard | **Phantom token** (placeholder→real swap at request time), AES-256-GCM vault | Yes (host match) | Per-agent + endpoint/method + rate limits | GA; Rust; YC-backed; ~2.5k⭐, 300k+ downloads (Jul 2026) | | **OneCLI** | B | Apache-2.0 | Reverse proxy + management UI | Host/path matching, Bitwarden integration | Configurable | Per-agent scoping | Active; v1.23.0 May 2026, 2.1k⭐ |
| **Aembit** | B | Proprietary | Sidecar + cloud control plane | TLS intercept, SPIFFE, JIT creds | No — intercepts by destination | Policy-based | GA (Apr 2026) | | **Aembit** | B | Proprietary | Sidecar + cloud control plane | TLS intercept, SPIFFE, JIT creds | No — intercepts by destination | Policy-based | GA (Apr 2026) |
| **LiteLLM Proxy** | A | MIT | Reverse proxy | Virtual key → upstream key | Yes — set base URL to LiteLLM | Route-level | 45k⭐; **CVE-2026-42208 exploited Apr 2026**, patch v1.83.7 | | **LiteLLM Proxy** | A | MIT | Reverse proxy | Virtual key → upstream key | Yes — set base URL to LiteLLM | Route-level | 45k⭐; **CVE-2026-42208 exploited Apr 2026**, patch v1.83.7 |
| **Portkey Gateway** | A | MIT (OSS core) | Reverse proxy | Virtual key vault (cloud or Enterprise self-host) | Yes — documented for Claude Code | Config-based | Production; virtual-key vault needs Enterprise for self-host | | **Portkey Gateway** | A | MIT (OSS core) | Reverse proxy | Virtual key vault (cloud or Enterprise self-host) | Yes — documented for Claude Code | Config-based | Production; virtual-key vault needs Enterprise for self-host |
@@ -281,18 +242,6 @@ Two categories:
`ANTHROPIC_BASE_URL`. **Blocker:** nono is explicitly `ANTHROPIC_BASE_URL`. **Blocker:** nono is explicitly
"early alpha, not security audited." "early alpha, not security audited."
**Update (2026-07-14):** OneCLI ([onecli.sh](https://onecli.sh/))
ships the same phantom-token shape but is GA and YC-backed (Rust,
~2.5k⭐, 300k+ downloads) — the maturity nono lacks. Trade-offs: it's
a full identity-gateway *product* (encrypted vault, management
dashboard, 50+ one-click app integrations, hosted cloud tier), much
heavier than the ~100-line proxy proposed below, and its hosted tier
is a third-party credential custodian — precisely the trust
dependency bot-bottle's isolation model exists to avoid. So: its
Apache-2.0 OSS core is a credible *adopt* candidate for the
phantom-token slice; its managed offering is a *competitor*, not a
dependency to lean on.
- **TLS-MITM forward proxies** (Infisical Agent Vault, Cloudflare - **TLS-MITM forward proxies** (Infisical Agent Vault, Cloudflare
Sandbox Auth, Aembit, the existing pipelock) all double up on Sandbox Auth, Aembit, the existing pipelock) all double up on
the CA-trust machinery PRD 0006 already built for pipelock. the CA-trust machinery PRD 0006 already built for pipelock.
@@ -324,15 +273,6 @@ routing matches the design recommended here exactly; zero TLS
work. But "not security audited" + "early alpha" means adopting it work. But "not security audited" + "early alpha" means adopting it
is a bet on the project rather than a buy-vs-build win. is a bet on the project rather than a buy-vs-build win.
**Mature phantom-token option (added 2026-07-14):** OneCLI — same
architecture as nono, but GA, Rust, and YC-backed. Its Apache-2.0 OSS
core is now the strongest *adopt* candidate for the phantom-token slice
if building is undesirable; the caveats are product surface you don't
need (bundled vault + dashboard) and that its hosted tier is a
competitor rather than a dependency. Doesn't change the build-first
recommendation for the narrow Anthropic-token slice, but it does mean
"is there a mature drop-in?" now has a real answer.
**Most mature OSS purpose-built:** Infisical Agent Vault. MIT, **Most mature OSS purpose-built:** Infisical Agent Vault. MIT,
v0.19.0 active, v0.17.0 added a containerized agent mode that v0.19.0 active, v0.17.0 added a containerized agent mode that
maps directly to bot-bottle. Friction is the TLS-MITM topology maps directly to bot-bottle. Friction is the TLS-MITM topology
@@ -353,12 +293,6 @@ exposed.
## Recommended path forward ## Recommended path forward
> **Mostly implemented (2026-07-14).** Steps 13 shipped — the token
> injection moved into the egress sidecar (folded into pipelock rather
> than a standalone reverse proxy) and generalized to Codex/Pi/git-host
> tokens. Steps 45 (issuance-side scope narrowing, per-route allowlists)
> remain good hygiene. See the Status note in the Summary.
In priority order: In priority order:
1. **In-container reverse proxy holding `CLAUDE_CODE_OAUTH_TOKEN`.** 1. **In-container reverse proxy holding `CLAUDE_CODE_OAUTH_TOKEN`.**
@@ -442,9 +376,6 @@ already gives us for upstream push credentials.
- [nono — phantom token blog](https://nono.sh/blog/blog-credential-injection) - [nono — phantom token blog](https://nono.sh/blog/blog-credential-injection)
- [Aegis — GitHub](https://github.com/getaegis/aegis) - [Aegis — GitHub](https://github.com/getaegis/aegis)
- [OneCLI — GitHub](https://github.com/onecli/onecli) - [OneCLI — GitHub](https://github.com/onecli/onecli)
- [OneCLI — homepage](https://onecli.sh/)
- [OneCLI — Y Combinator company page](https://www.ycombinator.com/companies/onecli)
- [Show HN: OneCLI Vault for AI Agents in Rust](https://news.ycombinator.com/item?id=47353558)
- [Sandbox0 — GitHub](https://github.com/sandbox0-ai/sandbox0) - [Sandbox0 — GitHub](https://github.com/sandbox0-ai/sandbox0)
- [Buildkite Cleanroom — GitHub](https://github.com/buildkite/cleanroom) - [Buildkite Cleanroom — GitHub](https://github.com/buildkite/cleanroom)
- [Aembit IAM for Agentic AI — GA](https://aembit.io/blog/aembit-iam-for-agentic-ai-is-now-generally-available/) - [Aembit IAM for Agentic AI — GA](https://aembit.io/blog/aembit-iam-for-agentic-ai-is-now-generally-available/)
@@ -71,56 +71,6 @@ manifest merge.
network egress is logged by pipelock/mitmproxy, and per-run op-log/audit state network egress is logged by pipelock/mitmproxy, and per-run op-log/audit state
is persisted to SQLite. is persisted to SQLite.
- **OneCLI** ([onecli.sh](https://onecli.sh/)) — YC-backed, GA, open-source
(Apache-2.0, Rust) "identity gateway for AI agents": a credential/secret
broker that holds API keys and OAuth tokens out of the agent's reach and
injects them at the network layer (phantom-token — the agent sees a
placeholder, the gateway swaps in the real, AES-256-GCM-encrypted credential
at request time). Framework-agnostic and drop-in for any HTTP-calling agent,
50+ app integrations, plus a hosted cloud tier with a per-agent dashboard and
audit logs. Full technical breakdown in
[`agent-credential-proxy-landscape.md`](agent-credential-proxy-landscape.md).
**How close a competitor:** near-exact on the *single axis of agent secret
custody* — the exact thing bot-bottle sells as "the agent never sees real
credentials, even via `printenv`." OneCLI does that one job well, is mature
and funded, and is *more portable* (it sits in front of anything; bot-bottle
only helps agents launched through bot-bottle). Takeaway: bot-bottle should
stop treating secret custody as a *unique* differentiator. But OneCLI is
**not** a competitor to bot-bottle's actual product — it does no agent
sandboxing (containers/microVMs), no fleet/manifest layer, no named agents /
skills / per-agent system prompts, no multi-provider launching, no egress
firewall.
**Our edge:** (1) *Isolation is the product, not a proxy.* OneCLI keeps the
key out of reach at the network layer, but the agent itself still runs
unsandboxed — a hijacked agent behind OneCLI has full run of its host and can
exfil captured data through any allowed host. bot-bottle runs the agent inside
a kernel/VM-enforced sandbox, injects credentials across that same
out-of-process boundary, *and* clamps egress with pipelock — defense in depth
vs. a single network layer. (2) *Fleet + manifest model* with named agents,
skills, per-agent system prompts, multi-provider and multi-backend — OneCLI
has no equivalent. (3) *Trust posture:* OneCLI's managed tier reintroduces a
third-party credential custodian, whereas bot-bottle's OSS-runtime +
paid-control-plane split keeps custody inside the operator's own boundary —
the stronger story for the security-minded self-hoster. (4) *Runs inside your
network boundary — local/internal reach.* Because bot-bottle executes the
agent on your own host (homelab, corporate LAN, a Tailnet) and egress is a
manifest field, giving an agent *scoped* access to **internal** resources — a
private Gitea, a LAN database, a Tailscale node — is just another egress-route
line, not a networking project (the same move an operator already makes to
reach their Tailscale services). OneCLI's OSS core can self-host too, but it's
a credential *broker* for outbound API calls, not an agent runtime, and its
managed tier + 50+ integrations are oriented at public SaaS — it doesn't put
the agent behind your firewall for you. This is a reach advantage, distinct
from the isolation ones above, and it's a wedge cloud-first agent products
(Devin, Copilot Workspace, OneCLI Cloud) structurally can't match. **Tactical
read:**
adopt OneCLI's OSS core for the credential slice if building is undesirable
(it's mature now); don't build atop its managed tier (competitor, not
dependency); re-position bot-bottle on isolation + fleet + self-hosted custody
rather than "we hide your secrets."
## What no found project does ## What no found project does
None combine: None combine:
+2 -2
View File
@@ -2,7 +2,7 @@
# #
# The one-time privileged setup the Firecracker backend needs: a pool of # The one-time privileged setup the Firecracker backend needs: a pool of
# user- (or group-) owned point-to-point TAP devices plus a fail-closed # user- (or group-) owned point-to-point TAP devices plus a fail-closed
# nftables table that confines every microVM to its own gateway. # nftables table that confines every microVM to its own sidecar.
# #
# NON-INVASIVE BY DESIGN. It does NOT flip `networking.nftables.enable` # NON-INVASIVE BY DESIGN. It does NOT flip `networking.nftables.enable`
# (which would switch your whole host firewall backend) or # (which would switch your whole host firewall backend) or
@@ -150,7 +150,7 @@ in
} }
]; ];
# VM->gateway traffic is DNAT'd and forwarded, so forwarding must be on. # VM->sidecar traffic is DNAT'd and forwarded, so forwarding must be on.
boot.kernel.sysctl."net.ipv4.ip_forward" = 1; boot.kernel.sysctl."net.ipv4.ip_forward" = 1;
# One oneshot brings up the whole pool (TAPs + independent nft table) # One oneshot brings up the whole pool (TAPs + independent nft table)
+7 -7
View File
@@ -4,7 +4,7 @@
# Creates a pool of point-to-point TAP devices (owned by the invoking # Creates a pool of point-to-point TAP devices (owned by the invoking
# user so the backend can open them without root at launch) and a # user so the backend can open them without root at launch) and a
# dedicated nftables table that isolates every VM: a bottle VM can # dedicated nftables table that isolates every VM: a bottle VM can
# reach only its own gateway (published on the host-side TAP IP) and # reach only its own sidecar (published on the host-side TAP IP) and
# nothing else on the host or network. # nothing else on the host or network.
# #
# Why a pool + one-time setup: creating a TAP and assigning it an IP # Why a pool + one-time setup: creating a TAP and assigning it an IP
@@ -72,9 +72,9 @@ for _v in POOL_SIZE IP_BASE PREFIX TABLE; do
[ -n "${!_v}" ] || { echo "error: $_v unresolved (set BOT_BOTTLE_FC_* or fix $_DEFAULTS)" >&2; exit 1; } [ -n "${!_v}" ] || { echo "error: $_v unresolved (set BOT_BOTTLE_FC_* or fix $_DEFAULTS)" >&2; exit 1; }
done done
# Gateway ports (must match the backend). egress=9099, supervise=9100, # Sidecar ports (must match the backend). egress=9099, supervise=9100,
# git-http=9420. Reached by the VM at its host-side TAP IP. # git-http=9420. Reached by the VM at its host-side TAP IP.
GATEWAY_PORTS="9099,9100,9420" SIDECAR_PORTS="9099,9100,9420"
# --- IP math --------------------------------------------------------- # --- IP math ---------------------------------------------------------
# Slot i occupies the /31 {base+2i, base+2i+1}: host = base+2i (the # Slot i occupies the /31 {base+2i, base+2i+1}: host = base+2i (the
@@ -107,7 +107,7 @@ cmd_up() {
fi fi
echo "firecracker net pool: $POOL_SIZE slots, base $IP_BASE, $own_desc" echo "firecracker net pool: $POOL_SIZE slots, base $IP_BASE, $own_desc"
# VM->gateway traffic is DNAT'd to the gateway container and # VM->sidecar traffic is DNAT'd to the sidecar container and
# forwarded, so forwarding must be enabled (Docker also sets this). # forwarded, so forwarding must be enabled (Docker also sets this).
sysctl -qw net.ipv4.ip_forward=1 sysctl -qw net.ipv4.ip_forward=1
@@ -135,11 +135,11 @@ _install_nft() {
# tool's traffic is affected. Priority -10 runs before Docker's # tool's traffic is affected. Priority -10 runs before Docker's
# filter hooks (priority 0); a drop here is terminal for the packet. # filter hooks (priority 0); a drop here is terminal for the packet.
# #
# forward: VM egress is DNAT'd to the gateway (established via # forward: VM egress is DNAT'd to the sidecar (established via
# `ct status dnat`); return traffic via `ct state established`. # `ct status dnat`); return traffic via `ct state established`.
# Anything else from a VM is dropped -> no route to the internet # Anything else from a VM is dropped -> no route to the internet
# or the rest of the host except through the gateway proxy. # or the rest of the host except through the sidecar proxy.
# input: a VM never needs host-local delivery (its gateway is # input: a VM never needs host-local delivery (its sidecar is
# reached via DNAT->forward), so drop all direct input from VMs # reached via DNAT->forward), so drop all direct input from VMs
# -> host services bound on 0.0.0.0 are unreachable from the VM. # -> host services bound on 0.0.0.0 are unreachable from the VM.
nft -f - <<EOF nft -f - <<EOF
+7 -3
View File
@@ -18,7 +18,8 @@ tests/
test_manifest_runtime.py test_manifest_runtime.py
... # many others; see unit/ directory ... # many others; see unit/ directory
integration/ integration/
test_gateway_image.py test_sidecar_bundle_image.py
test_sidecar_bundle_compose.py
test_dry_run_plan.py test_dry_run_plan.py
test_orphan_cleanup.py test_orphan_cleanup.py
... ...
@@ -47,9 +48,12 @@ Discovery is invoked with `-t .` (top-level dir = repo root) so the
the bottle's runtime, and creates zero Docker resources. the bottle's runtime, and creates zero Docker resources.
- `test_orphan_cleanup.py``network_remove` is idempotent against - `test_orphan_cleanup.py``network_remove` is idempotent against
missing resources, so the EXIT trap can call it unconditionally. missing resources, so the EXIT trap can call it unconditionally.
- `test_gateway_image.py` — builds Dockerfile.gateway and - `test_sidecar_bundle_image.py` — builds Dockerfile.sidecars and
probes that gitleaks / mitmdump / supervise are all reachable probes that gitleaks / mitmdump / supervise are all reachable
inside the gateway image. inside the bundle.
- `test_sidecar_bundle_compose.py` — end-to-end compose-up of an
agent + bundle pair; verifies the agent reaches the bundle via
the legacy network aliases.
## Canaries ## Canaries
@@ -0,0 +1,142 @@
"""Integration: Firecracker microVM launch.
End-to-end against a real Firecracker microVM: prepare + launch a bottle
on the firecracker backend and verify the agent execs after provisioning
and that the egress proxy env is wired to the sidecar.
Gated on the `backend status` result for firecracker (0 == the privileged
TAP pool + nft isolation table are provisioned). Skips cleanly with setup
instructions otherwise, so the suite runs on hosts without the pool.
"""
from __future__ import annotations
import contextlib
import io
import os
import shutil
import tempfile
import unittest
from pathlib import Path
from bot_bottle.backend import BottleSpec, get_bottle_backend
from bot_bottle.backend.firecracker import FirecrackerBottleBackend
from bot_bottle.manifest import ManifestIndex
def _firecracker_status_ok() -> bool:
"""Gate on `./cli.py backend status --backend=firecracker`: a 0 exit
means the pool + nft table are ready. Output is captured so the
decorator stays quiet during collection; any error not ready."""
buf = io.StringIO()
try:
with contextlib.redirect_stdout(buf), contextlib.redirect_stderr(buf):
return FirecrackerBottleBackend.status() == 0
except Exception:
return False
_SKIP_MSG = (
"firecracker backend not ready — provision the network pool with "
"`./cli.py backend setup --backend=firecracker`, then confirm with "
"`./cli.py backend status --backend=firecracker`"
)
def _minimal_agent_dockerfile(path: Path) -> None:
path.write_text(
"\n".join((
"FROM node:22-slim",
"RUN apt-get update \\",
" && apt-get install -y --no-install-recommends \\",
" ca-certificates curl git \\",
" && rm -rf /var/lib/apt/lists/*",
"USER node",
"WORKDIR /home/node",
"CMD [\"sleep\", \"infinity\"]",
"",
)),
encoding="utf-8",
)
def _minimal_manifest(dockerfile: Path) -> ManifestIndex:
return ManifestIndex.from_json_obj({
"bottles": {
"dev": {
"agent_provider": {
"template": "pi",
"dockerfile": str(dockerfile),
"settings": {
"provider": "example",
"base_url": "https://example.com/v1",
"models": ["smoke"],
},
},
"egress": {"routes": [{"host": "example.com"}]},
},
},
"agents": {
"demo": {"skills": [], "prompt": "smoke", "bottle": "dev"},
},
})
@unittest.skipIf(
os.environ.get("GITEA_ACTIONS") == "true",
"skipped under act_runner: cannot host Firecracker microVMs",
)
@unittest.skipUnless(_firecracker_status_ok(), _SKIP_MSG)
class TestFirecrackerLaunch(unittest.TestCase):
"""Launch once, reuse the bottle across probes."""
@classmethod
def setUpClass(cls) -> None:
cls.stage = Path(tempfile.mkdtemp(prefix="cb-firecracker-launch."))
cls._launch = None
cls.bottle = None
dockerfile = cls.stage / "Dockerfile.agent-smoke"
_minimal_agent_dockerfile(dockerfile)
os.environ["BOT_BOTTLE_BACKEND"] = "firecracker"
try:
backend = get_bottle_backend()
spec = BottleSpec(
manifest=_minimal_manifest(dockerfile),
agent_name="demo",
copy_cwd=False,
user_cwd=str(cls.stage),
)
cls.plan = backend.prepare(spec, stage_dir=cls.stage)
cls._launch = backend.launch(cls.plan)
cls.bottle = cls._launch.__enter__()
except BaseException:
if cls._launch is not None:
cls._launch.__exit__(None, None, None)
shutil.rmtree(cls.stage, ignore_errors=True)
os.environ.pop("BOT_BOTTLE_BACKEND", None)
raise
@classmethod
def tearDownClass(cls) -> None:
try:
if cls._launch is not None:
cls._launch.__exit__(None, None, None)
finally:
shutil.rmtree(cls.stage, ignore_errors=True)
os.environ.pop("BOT_BOTTLE_BACKEND", None)
def test_smoke_exec_echo(self) -> None:
r = self.bottle.exec("echo hello-from-firecracker") # type: ignore[union-attr]
self.assertEqual(0, r.returncode, msg=r.stderr)
self.assertIn("hello-from-firecracker", r.stdout)
def test_proxy_env_points_at_sidecar(self) -> None:
r = self.bottle.exec( # type: ignore[union-attr]
"printf '%s\\n' \"$HTTPS_PROXY\" \"$HTTP_PROXY\""
)
self.assertEqual(0, r.returncode, msg=r.stderr)
self.assertIn("http", r.stdout.lower())
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,239 @@
"""Integration: macOS Container launch topology.
End-to-end against Apple's real `container` runtime. The smoke launches
a bottle with the experimental macOS Container backend and verifies the
properties that make the explicit-proxy launch acceptable:
- the agent can exec commands after provisioning;
- HTTP(S)_PROXY points at the sidecar's internal-network IP;
- allowlisted HTTPS reaches the egress sidecar;
- direct egress with proxy env removed fails from the internal-only
agent network;
- non-allowlisted proxy traffic is blocked.
Skipped under Gitea Actions and on hosts without Apple's `container`.
"""
from __future__ import annotations
import os
import platform
import shutil
import subprocess
import tempfile
import unittest
from pathlib import Path
from bot_bottle.backend import BottleSpec, get_bottle_backend
from bot_bottle.backend.macos_container.util import (
dns_server as _container_dns_server,
is_available as _container_available,
)
from bot_bottle.manifest import ManifestIndex
_AGENT_PROMPT = "You are a launch smoke-test agent. Be brief."
def _minimal_agent_dockerfile(path: Path) -> None:
path.write_text(
"\n".join((
"FROM node:22-slim",
"RUN apt-get update \\",
" && apt-get install -y --no-install-recommends \\",
" ca-certificates curl git \\",
" && rm -rf /var/lib/apt/lists/*",
"USER node",
"WORKDIR /home/node",
"CMD [\"sleep\", \"infinity\"]",
"",
)),
encoding="utf-8",
)
def _minimal_manifest(dockerfile: Path) -> ManifestIndex:
return ManifestIndex.from_json_obj({
"bottles": {
"dev": {
"agent_provider": {
"template": "pi",
"dockerfile": str(dockerfile),
"settings": {
"provider": "example",
"base_url": "https://example.com/v1",
"models": ["smoke"],
},
},
"egress": {
"routes": [
{"host": "example.com"},
],
},
},
},
"agents": {
"demo": {
"skills": [],
"prompt": _AGENT_PROMPT,
"bottle": "dev",
},
},
})
def _buildkit_dns_available() -> bool:
if platform.system() != "Darwin" or not _container_available():
return False
stage = Path(tempfile.mkdtemp(prefix="cb-container-buildkit-dns."))
image = "bot-bottle-buildkit-dns-check:latest"
try:
dockerfile = stage / "Dockerfile"
dockerfile.write_text(
"FROM debian:bookworm-slim\n"
"RUN getent hosts deb.debian.org\n",
encoding="utf-8",
)
result = subprocess.run(
[
"container", "build",
"--dns", _container_dns_server(),
"-t", image,
"-f", str(dockerfile),
str(stage),
],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
return result.returncode == 0
finally:
subprocess.run(
["container", "image", "delete", image],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
shutil.rmtree(stage, ignore_errors=True)
@unittest.skipIf(
os.environ.get("GITEA_ACTIONS") == "true",
"skipped under act_runner: cannot host Apple Container VMs",
)
@unittest.skipUnless(
platform.system() == "Darwin",
"Apple Container is macOS-only",
)
@unittest.skipUnless(
_container_available(),
"Apple Container not on PATH; install from "
"https://github.com/apple/container/releases",
)
@unittest.skipUnless(
_buildkit_dns_available(),
"Apple Container BuildKit cannot resolve deb.debian.org on this host",
)
class TestMacosContainerLaunch(unittest.TestCase):
"""Launch once and reuse the bottle across probes."""
@classmethod
def setUpClass(cls) -> None:
cls.stage = Path(tempfile.mkdtemp(prefix="cb-macos-container-launch."))
cls._launch = None
cls.bottle = None
dockerfile = cls.stage / "Dockerfile.agent-smoke"
_minimal_agent_dockerfile(dockerfile)
os.environ["BOT_BOTTLE_BACKEND"] = "macos-container"
try:
backend = get_bottle_backend()
spec = BottleSpec(
manifest=_minimal_manifest(dockerfile),
agent_name="demo",
copy_cwd=False,
user_cwd=str(cls.stage),
)
cls.plan = backend.prepare(spec, stage_dir=cls.stage)
cls._launch = backend.launch(cls.plan)
cls.bottle = cls._launch.__enter__()
except BaseException:
if cls._launch is not None:
cls._launch.__exit__(None, None, None)
shutil.rmtree(cls.stage, ignore_errors=True)
os.environ.pop("BOT_BOTTLE_BACKEND", None)
raise
@classmethod
def tearDownClass(cls) -> None:
try:
if cls._launch is not None:
cls._launch.__exit__(None, None, None)
finally:
shutil.rmtree(cls.stage, ignore_errors=True)
os.environ.pop("BOT_BOTTLE_BACKEND", None)
def test_smoke_exec_echo(self):
r = self.bottle.exec( # type: ignore[union-attr]
"echo hello-from-macos-container"
)
self.assertEqual(0, r.returncode, msg=r.stderr)
self.assertIn("hello-from-macos-container", r.stdout)
def test_proxy_env_points_at_sidecar_internal_ip(self):
r = self.bottle.exec( # type: ignore[union-attr]
"printf '%s\n' \"$HTTPS_PROXY\" \"$HTTP_PROXY\" "
"\"$NO_PROXY\" \"$NODE_EXTRA_CA_CERTS\""
)
self.assertEqual(0, r.returncode, msg=r.stderr)
values = [line.strip() for line in r.stdout.splitlines()]
self.assertEqual(4, len(values), values)
self.assertEqual(values[0], values[1], values)
self.assertRegex(values[0], r"^http://[0-9.]+:9099$")
self.assertNotIn("127.0.0.1", values[0])
sidecar_host = values[0].removeprefix("http://").removesuffix(":9099")
self.assertIn(sidecar_host, values[2])
self.assertEqual(
"/usr/local/share/ca-certificates/bot-bottle-mitm-ca.crt",
values[3],
)
def test_allowlisted_https_reaches_egress_proxy(self):
r = self.bottle.exec( # type: ignore[union-attr]
"curl -fsS --max-time 20 https://example.com >/dev/null && echo OK"
)
self.assertEqual(0, r.returncode, msg=r.stderr + r.stdout)
self.assertIn("OK", r.stdout)
def test_direct_egress_bypass_without_proxy_fails(self):
r = self.bottle.exec( # type: ignore[union-attr]
"env -u HTTPS_PROXY -u HTTP_PROXY -u https_proxy -u http_proxy "
"curl -s --show-error --max-time 5 https://example.com 2>&1 || true"
)
self.assertTrue(
"refused" in r.stdout.lower()
or "timed out" in r.stdout.lower()
or "unreachable" in r.stdout.lower()
or "failed" in r.stdout.lower()
or "could not resolve" in r.stdout.lower()
or "connection reset" in r.stdout.lower(),
f"expected direct egress to fail; got: {r.stdout!r}",
)
def test_non_allowlisted_host_fails_through_proxy(self):
r = self.bottle.exec( # type: ignore[union-attr]
"curl -s --show-error --max-time 10 https://iana.org 2>&1 || true"
)
self.assertTrue(
"403" in r.stdout
or "502" in r.stdout
or "blocked" in r.stdout.lower()
or "not allowed" in r.stdout.lower()
or "not in the bottle's egress.routes allowlist" in r.stdout.lower()
or "forbidden" in r.stdout.lower()
or "failed" in r.stdout.lower(),
f"expected non-allowlisted proxy request to fail; got: {r.stdout!r}",
)
if __name__ == "__main__":
unittest.main()
@@ -1,161 +0,0 @@
"""Integration: two bottles share one gateway; source-IP attribution keeps
their egress tokens *and* allowlists isolated (PRD 0070 multi-tenancy).
The whole premise of the consolidation is one shared gateway for every
bottle, isolated only by the unspoofable source IP. A single-bottle test
can't catch cross-bottle token bleed or an allowlist leak — this brings up
the real orchestrator + gateway and drives two bottles through the one
gateway to prove each gets exactly its own token and its own routes.
Gated on a reachable Docker daemon and builds the bundle image, so it runs
with the other docker integration tests, not in unit CI. Uses the real
fixed gateway/orchestrator names (that's what it's testing), so it can't run
concurrently with a live per-host gateway; it points the orchestrator at a
throwaway BOT_BOTTLE_ROOT for a clean registry and tears everything down.
"""
from __future__ import annotations
import os
import subprocess
import tempfile
import unittest
from pathlib import Path
from bot_bottle.backend.docker.consolidated_launch import (
_network_cidr,
_network_container_ips,
)
from bot_bottle.backend.docker.egress import EGRESS_PORT
from bot_bottle.backend.docker.gateway_net import next_free_ip
from bot_bottle.orchestrator.client import OrchestratorClient
from bot_bottle.orchestrator.gateway import GATEWAY_IMAGE, GATEWAY_NAME, GATEWAY_NETWORK
from bot_bottle.orchestrator.lifecycle import OrchestratorService
from tests._docker import skip_unless_docker
# One upstream reachable under two names; reflects the Authorization header so
# the probe can see exactly what the gateway injected (or didn't).
_ECHO_NAME = "bot-bottle-mtitest-echo"
_ECHO_ALIASES = ("echo-shared", "echo-bonly")
_ECHO_SRC = (
"from http.server import BaseHTTPRequestHandler, HTTPServer\n"
"class H(BaseHTTPRequestHandler):\n"
" def do_GET(s):\n"
" a=s.headers.get('Authorization','NONE'); s.send_response(200); s.end_headers()\n"
" s.wfile.write(('AUTH='+a).encode())\n"
" def log_message(s,*a): pass\n"
"HTTPServer(('0.0.0.0',80),H).serve_forever()\n"
)
# A: echo-shared only, authed. B: echo-shared authed + echo-bonly open.
_POLICY_A = (
'routes:\n'
' - host: echo-shared\n'
' auth_scheme: "Bearer"\n'
' token_env: "EGRESS_TOKEN_0"\n'
)
_POLICY_B = _POLICY_A + ' - host: echo-bonly\n'
_TOKEN_A = "sk-AAA-alice"
_TOKEN_B = "sk-BBB-bob"
# Client probe: open http://<host>/ through the gateway proxy from a container
# pinned to the bottle's source IP, printing "<status> <body>". Uses only the
# bundle image's python3 — no dependency on the agent image.
_PROBE_SRC = (
"import sys,urllib.request,urllib.error\n"
"op=urllib.request.build_opener(urllib.request.ProxyHandler({'http':sys.argv[1]}))\n"
"try:\n"
" r=op.open('http://'+sys.argv[2]+'/',timeout=8)\n"
" sys.stdout.write(str(r.status)+' '+r.read().decode())\n"
"except urllib.error.HTTPError as e:\n"
" sys.stdout.write(str(e.code)+' '+e.read().decode())\n"
)
@skip_unless_docker()
@unittest.skipIf(
os.environ.get("GITEA_ACTIONS") == "true",
"skipped under act_runner: the orchestrator container bind-mounts the repo "
"path into a container on the socket-shared host daemon, which can't see the "
"runner's /workspace — same host-bind-mount constraint as the other "
"bottle-bringup integration tests",
)
class TestMultitenantIsolation(unittest.TestCase):
def setUp(self) -> None:
self._tmp = tempfile.TemporaryDirectory()
self.addCleanup(self._tmp.cleanup)
# Throwaway root → a clean registry DB, independent of the host's.
self.svc = OrchestratorService(host_root=Path(self._tmp.name))
self.addCleanup(self._teardown_docker)
# ensure_running builds the bundle image (slow on a cold cache) and
# brings up the shared network + gateway + orchestrator.
url = self.svc.ensure_running(startup_timeout=300)
self.client = OrchestratorClient(url)
self.gw_ip = self._container_ip(GATEWAY_NAME)
self._run_echo()
def _teardown_docker(self) -> None:
subprocess.run(["docker", "rm", "--force", _ECHO_NAME],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False)
self.svc.stop()
subprocess.run(["docker", "network", "rm", GATEWAY_NETWORK],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False)
# The orchestrator container wrote the registry DB as root into the
# throwaway root; chown it back so the (non-root) tempdir cleanup can
# remove it.
subprocess.run(
["docker", "run", "--rm", "-v", f"{self._tmp.name}:/r",
"--entrypoint", "chown", GATEWAY_IMAGE, "-R",
f"{os.getuid()}:{os.getgid()}", "/r"],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False)
@staticmethod
def _container_ip(name: str) -> str:
proc = subprocess.run(
["docker", "inspect", "-f",
'{{(index .NetworkSettings.Networks "' + GATEWAY_NETWORK + '").IPAddress}}', name],
stdout=subprocess.PIPE, text=True, check=True,
)
return proc.stdout.strip()
def _run_echo(self) -> None:
argv = ["docker", "run", "--detach", "--name", _ECHO_NAME,
"--network", GATEWAY_NETWORK]
for alias in _ECHO_ALIASES:
argv += ["--network-alias", alias]
argv += ["--entrypoint", "python3", GATEWAY_IMAGE, "-c", _ECHO_SRC]
proc = subprocess.run(argv, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
text=True, check=False)
self.assertEqual(0, proc.returncode, f"echo upstream failed: {proc.stderr}")
def _free_ip(self, extra_taken: list[str]) -> str:
taken = _network_container_ips(GATEWAY_NETWORK) + extra_taken
return next_free_ip(_network_cidr(GATEWAY_NETWORK), taken)
def _probe(self, source_ip: str, host: str) -> str:
proc = subprocess.run(
["docker", "run", "--rm", "--network", GATEWAY_NETWORK, "--ip", source_ip,
"--entrypoint", "python3", GATEWAY_IMAGE, "-c", _PROBE_SRC,
f"http://{self.gw_ip}:{EGRESS_PORT}", host],
stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, check=False, timeout=90,
)
return proc.stdout.strip()
def test_two_bottles_share_gateway_with_isolated_tokens_and_allowlists(self) -> None:
ip_a = self._free_ip([])
ip_b = self._free_ip([ip_a])
self.client.register_bottle(ip_a, policy=_POLICY_A, tokens={"EGRESS_TOKEN_0": _TOKEN_A})
self.client.register_bottle(ip_b, policy=_POLICY_B, tokens={"EGRESS_TOKEN_0": _TOKEN_B})
# Each bottle gets its OWN token injected on the shared route — no bleed.
self.assertEqual(f"200 AUTH=Bearer {_TOKEN_A}", self._probe(ip_a, "echo-shared"))
self.assertEqual(f"200 AUTH=Bearer {_TOKEN_B}", self._probe(ip_b, "echo-shared"))
# Allowlist is per-bottle: echo-bonly is only in B's policy.
self.assertTrue(self._probe(ip_a, "echo-bonly").startswith("403"), # fail-closed for A
"A reached a host outside its allowlist")
self.assertEqual("200 AUTH=NONE", self._probe(ip_b, "echo-bonly")) # allowed, unauthed for B
if __name__ == "__main__":
unittest.main()
@@ -1,7 +1,7 @@
"""Integration: the consolidated Docker gateway is an idempotent singleton. """Integration: the consolidated Docker sidecar is an idempotent singleton.
Gated on a reachable Docker daemon. Uses a unique name so it never Gated on a reachable Docker daemon. Uses a unique name so it never
collides with a real per-host gateway or a leftover from another run. collides with a real per-host sidecar or a leftover from another run.
""" """
from __future__ import annotations from __future__ import annotations
@@ -10,17 +10,17 @@ import secrets
import subprocess import subprocess
import unittest import unittest
from bot_bottle.orchestrator.gateway import DockerGateway from bot_bottle.orchestrator.sidecar import DockerSidecar
from tests._docker import skip_unless_docker from tests._docker import skip_unless_docker
IMAGE = "busybox" IMAGE = "busybox"
@skip_unless_docker() @skip_unless_docker()
class TestDockerGatewayIntegration(unittest.TestCase): class TestDockerSidecarIntegration(unittest.TestCase):
def setUp(self) -> None: def setUp(self) -> None:
self.name = "bot-bottle-orch-gateway-itest-" + secrets.token_hex(4) self.name = "bot-bottle-orch-sidecar-itest-" + secrets.token_hex(4)
self.sc = DockerGateway(IMAGE, name=self.name) self.sc = DockerSidecar(IMAGE, name=self.name)
self.addCleanup(self.sc.stop) self.addCleanup(self.sc.stop)
def _count(self) -> int: def _count(self) -> int:
@@ -1,7 +1,7 @@
"""Integration: DockerGateway.image_exists reflects real docker state. """Integration: DockerSidecar.image_exists reflects real docker state.
Gated on a reachable Docker daemon. Deliberately does NOT build the full Gated on a reachable Docker daemon. Deliberately does NOT build the full
gateway (a heavy, slow image build) it exercises the `image_exists` sidecar bundle (a heavy, slow image build) it exercises the `image_exists`
primitive that `ensure_built` gates on, against a tiny pulled image and a primitive that `ensure_built` gates on, against a tiny pulled image and a
name that can't exist. name that can't exist.
""" """
@@ -11,24 +11,24 @@ from __future__ import annotations
import subprocess import subprocess
import unittest import unittest
from bot_bottle.orchestrator.gateway import DockerGateway from bot_bottle.orchestrator.sidecar import DockerSidecar
from tests._docker import skip_unless_docker from tests._docker import skip_unless_docker
IMAGE = "busybox" IMAGE = "busybox"
@skip_unless_docker() @skip_unless_docker()
class TestDockerGatewayImageExists(unittest.TestCase): class TestDockerSidecarImageExists(unittest.TestCase):
def test_image_exists_true_for_present_false_for_absent(self) -> None: def test_image_exists_true_for_present_false_for_absent(self) -> None:
# Ensure the tiny image is present (build_if_missing is disabled here # Ensure the tiny image is present (build_if_missing is disabled here
# so this never triggers a Dockerfile.gateway build). # so this never triggers a Dockerfile.sidecars build).
subprocess.run( subprocess.run(
["docker", "pull", IMAGE], ["docker", "pull", IMAGE],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
) )
self.assertTrue(DockerGateway(IMAGE, dockerfile=None).image_exists()) self.assertTrue(DockerSidecar(IMAGE, dockerfile=None).image_exists())
self.assertFalse( self.assertFalse(
DockerGateway("bot-bottle-nonexistent:doesnotexist", dockerfile=None).image_exists() DockerSidecar("bot-bottle-nonexistent:doesnotexist", dockerfile=None).image_exists()
) )
+1 -1
View File
@@ -6,7 +6,7 @@ resources.
The PipelockProxy.stop idempotency case that used to live here was The PipelockProxy.stop idempotency case that used to live here was
removed in PRD 0024 chunk 3 when the per-container .stop method removed in PRD 0024 chunk 3 when the per-container .stop method
went away gateway teardown is now compose's responsibility, and went away sidecar teardown is now compose's responsibility, and
`compose down` already no-ops on missing containers.""" `compose down` already no-ops on missing containers."""
import os import os
+5 -5
View File
@@ -72,7 +72,7 @@ _DUMMY_HOST_KEY = (
os.environ.get("GITEA_ACTIONS") == "true", os.environ.get("GITEA_ACTIONS") == "true",
"skipped under act_runner: egress_tls_init uses a host bind mount " "skipped under act_runner: egress_tls_init uses a host bind mount "
"the runner container can't see, and the network topology hides " "the runner container can't see, and the network topology hides "
"sibling-gateway visibility — same constraint as the other " "sibling-sidecar visibility — same constraint as the other "
"bottle-bringup integration tests", "bottle-bringup integration tests",
) )
class TestSandboxEscape(unittest.TestCase): class TestSandboxEscape(unittest.TestCase):
@@ -90,8 +90,8 @@ class TestSandboxEscape(unittest.TestCase):
@classmethod @classmethod
def setUpClass(cls) -> None: def setUpClass(cls) -> None:
# Docker is always required (the agent + companion containers run under it, # Docker is always required (the agent + sidecars run under it,
# and VM backends still use it for the gateway); the # and VM backends still use it for the sidecar bundle); the
# class-level @skip_unless_docker already covers that. Pin # class-level @skip_unless_docker already covers that. Pin
# Docker when BOT_BOTTLE_BACKEND is unset to preserve the # Docker when BOT_BOTTLE_BACKEND is unset to preserve the
# Docker-backed CI path. # Docker-backed CI path.
@@ -121,7 +121,7 @@ class TestSandboxEscape(unittest.TestCase):
"egress": { "egress": {
"routes": [{"host": "api.anthropic.com"}], "routes": [{"host": "api.anthropic.com"}],
}, },
# git-gate daemon so attack 5 can push. Upstream # git-gate sidecar so attack 5 can push. Upstream
# is intentionally unreachable — the pre-receive # is intentionally unreachable — the pre-receive
# gitleaks hook must reject BEFORE git-gate # gitleaks hook must reject BEFORE git-gate
# attempts the upstream push. A preset `host_key` # attempts the upstream push. A preset `host_key`
@@ -275,7 +275,7 @@ class TestSandboxEscape(unittest.TestCase):
def _assert_sandbox_block(self, label: str, r: object) -> None: # type: ignore def _assert_sandbox_block(self, label: str, r: object) -> None: # type: ignore
"""A real sandbox block produces an HTTP 403 with a """A real sandbox block produces an HTTP 403 with a
recognizable sandbox gateway marker in the body. ANY recognizable sandbox sidecar marker in the body. ANY
other outcome (200 from upstream, 401/404 from upstream, other outcome (200 from upstream, 401/404 from upstream,
non-marker 5xx) means the request escaped the secret non-marker 5xx) means the request escaped the secret
reached the network.""" reached the network."""
@@ -0,0 +1,106 @@
"""Integration: end-to-end smoke for the PRD 0024 bundle shape.
Verifies that flipping `BOT_BOTTLE_SIDECAR_BUNDLE=1` produces a
working bottle: `docker compose up` brings the agent + bundle pair
online, the daemons inside the bundle bind their ports, and the
agent can reach egress + supervise via the bundle's network
aliases (no agent-side config changes between flag positions).
Skipped under GITEA_ACTIONS the bundle image is a multi-stage
build pulling 200+MB of base layers, and the bind-mounts won't
share filesystem with the runner container. Same constraint as
the chunk-1 image-probe test.
"""
from __future__ import annotations
import os
import shutil
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
from bot_bottle.backend import BottleSpec, get_bottle_backend
from bot_bottle.manifest import ManifestIndex
from tests._docker import skip_unless_docker
def _manifest() -> ManifestIndex:
"""Bottle with supervise on so the bundle exercises egress +
supervise. Git is off because a meaningful git-gate test needs
a real upstream and SSH keys out of scope for a bundle smoke."""
return ManifestIndex.from_json_obj({
"bottles": {
"dev": {
"supervise": True,
},
},
"agents": {
"demo": {"skills": [], "prompt": "", "bottle": "dev"},
},
})
@skip_unless_docker()
@unittest.skipIf(
os.environ.get("GITEA_ACTIONS") == "true",
"skipped under act_runner: multi-stage bundle build pulls 200+MB "
"of base layers and bind-mounts don't share fs with the runner",
)
class TestSidecarBundleCompose(unittest.TestCase):
"""One end-to-end pass with the bundle flag on. Skipping under
act_runner; the local docker daemon does the work."""
def test_bottle_up_with_bundle_flag_on(self):
stage_dir = Path(tempfile.mkdtemp(prefix="cb-bundle-smoke."))
try:
with patch.dict(os.environ, {"BOT_BOTTLE_SIDECAR_BUNDLE": "1"}):
backend = get_bottle_backend("docker")
spec = BottleSpec(
manifest=_manifest(),
agent_name="demo",
copy_cwd=False,
user_cwd=str(stage_dir),
)
plan = backend.prepare(spec, stage_dir=stage_dir)
with backend.launch(plan) as bottle:
# The agent's HTTPS_PROXY URL (resolved at
# renderer-time) should reach egress inside
# the bundle. A bare CONNECT with no upstream
# URL gets rejected with 400 or 405 but proves
# the listener is alive at the alias.
probe = bottle.exec(
"set -eu\n"
"echo HTTPS_PROXY=$HTTPS_PROXY\n"
"PORT=$(echo \"$HTTPS_PROXY\" | sed -E 's|.*:([0-9]+).*|\\1|')\n"
"HOST=$(echo \"$HTTPS_PROXY\" | sed -E 's|http://([^:]+):.*|\\1|')\n"
"echo HOST=$HOST PORT=$PORT\n"
"curl -sS --max-time 5 -o /dev/null -w 'http=%{http_code}\\n' "
" \"http://$HOST:$PORT/\" || true\n"
)
# The supervise URL resolves to the same bundle
# via its supervise alias, on a different port.
supervise_probe = bottle.exec(
"set -eu\n"
"curl -sS --max-time 5 -o /dev/null "
" -w 'http=%{http_code}\\n' "
" \"http://supervise:9100/health\" || true\n"
)
finally:
shutil.rmtree(stage_dir, ignore_errors=True)
self.assertEqual(0, probe.returncode, msg=probe.stderr)
# egress answered SOMETHING — any 4xx is fine, just proves
# the egress daemon is listening at the proxy address.
self.assertIn("http=", probe.stdout,
f"no HTTP response from egress: {probe.stdout!r}")
# supervise's /health endpoint exists (PRD 0013); it should
# answer 200 or similar — anything non-empty proves the
# third daemon's alias resolves to the same bundle.
self.assertEqual(0, supervise_probe.returncode, msg=supervise_probe.stderr)
self.assertIn("http=", supervise_probe.stdout)
if __name__ == "__main__":
unittest.main()
@@ -1,4 +1,4 @@
"""Integration: PRD 0024 chunk 1 — the gateway image builds """Integration: PRD 0024 chunk 1 — the sidecar bundle image builds
and the daemon binaries are present + executable inside it. and the daemon binaries are present + executable inside it.
This test does NOT exercise the daemons running against real This test does NOT exercise the daemons running against real
@@ -6,11 +6,11 @@ config (routes.yaml, etc) — that lands in chunk 2 when the
renderer wires the bundle into compose. What we verify here is renderer wires the bundle into compose. What we verify here is
the chunk-1 contract: the chunk-1 contract:
- Dockerfile.gateway builds (multi-stage works, base layers - Dockerfile.sidecars builds (multi-stage works, base layers
pull, COPYs resolve). pull, COPYs resolve).
- gitleaks, mitmdump are at the documented paths and answer - gitleaks, mitmdump are at the documented paths and answer
`--version`. `--version`.
- The Python init at /app/gateway_init.py runs and prints the - The Python init at /app/sidecar_init.py runs and prints the
expected "no daemons selected" line when the supervisor is expected "no daemons selected" line when the supervisor is
pointed at an empty daemon set. pointed at an empty daemon set.
@@ -28,18 +28,18 @@ import unittest
from tests._docker import skip_unless_docker from tests._docker import skip_unless_docker
_IMAGE = "bot-bottle-gateway-test:chunk1" _IMAGE = "bot-bottle-sidecars-test:chunk1"
_DOCKERFILE = "Dockerfile.gateway" _DOCKERFILE = "Dockerfile.sidecars"
@skip_unless_docker() @skip_unless_docker()
@unittest.skipIf( @unittest.skipIf(
os.environ.get("GITEA_ACTIONS") == "true", os.environ.get("GITEA_ACTIONS") == "true",
"skipped under act_runner: multi-stage build pulls a 200+MB " "skipped under act_runner: multi-stage build pulls a 200+MB "
"mitmproxy base + two upstream gateway images; runner storage " "mitmproxy base + two upstream sidecar images; runner storage "
"+ time budget make this an interactive-only test", "+ time budget make this an interactive-only test",
) )
class TestGatewayImage(unittest.TestCase): class TestSidecarBundleImage(unittest.TestCase):
"""Builds the image once for the class, then runs a few """Builds the image once for the class, then runs a few
`docker run` probes against it.""" `docker run` probes against it."""
@@ -103,7 +103,7 @@ class TestGatewayImage(unittest.TestCase):
# ENTRYPOINT wiring works. # ENTRYPOINT wiring works.
proc = subprocess.run( proc = subprocess.run(
["docker", "run", "--rm", ["docker", "run", "--rm",
"-e", "BOT_BOTTLE_GATEWAY_DAEMONS=nothing", "-e", "BOT_BOTTLE_SIDECAR_DAEMONS=nothing",
_IMAGE], _IMAGE],
stdout=subprocess.PIPE, stderr=subprocess.STDOUT, stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
timeout=10.0, timeout=10.0,
+1 -1
View File
@@ -5,7 +5,7 @@ no test ever reads or writes the real ``~/.bot-bottle`` (state, queue,
and audit dirs all derive from ``paths.bot_bottle_root()`` and audit dirs all derive from ``paths.bot_bottle_root()``
``Path.home()``). Without this, a test that takes a ``flock`` on the ``Path.home()``). Without this, a test that takes a ``flock`` on the
real audit log can **block indefinitely** when a live bottle's supervise real audit log can **block indefinitely** when a live bottle's supervise
gateway holds that lock observed as a hung ``coverage run`` at 0% CPU sidecar holds that lock observed as a hung ``coverage run`` at 0% CPU
and unisolated tests otherwise pollute the developer's home dir. and unisolated tests otherwise pollute the developer's home dir.
Individual tests that need their own ``HOME`` still override Individual tests that need their own ``HOME`` still override
-160
View File
@@ -1,160 +0,0 @@
"""Shared in-memory `DockerBottlePlan` fixture for docker-backend tests.
A fully-resolved plan with toggles for the conditional-service matrix
(git-gate / egress / supervise / canary). Consumed by the consolidated
compose tests; kept here (rather than in a test module) so it survives
independent of any one test file.
"""
from __future__ import annotations
from pathlib import Path
from bot_bottle.agent_provider import AgentProvisionPlan
from bot_bottle.backend import BottleSpec
from bot_bottle.backend.docker.bottle_plan import DockerBottlePlan
from bot_bottle.egress import EgressPlan, EgressRoute
from bot_bottle.git_gate import GitGatePlan, GitGateUpstream
from bot_bottle.manifest import ManifestIndex
from bot_bottle.supervise import SupervisePlan
SLUG = "demo-abc12"
STAGE = Path("/tmp/cb-stage")
STATE = Path("/tmp/cb-state")
# Exported to consumers (e.g. test_consolidated_compose); named with a
# leading underscore for the historical fixture convention.
__all__ = ["_plan"]
def _manifest(*, supervise: bool, with_git: bool, with_egress: bool) -> ManifestIndex:
"""Minimal manifest with the toggles the matrix needs. The renderer
only reads from the plan, not the manifest, so this is just here to
back BottleSpec."""
bottle: dict[str, object] = {}
if supervise:
bottle["supervise"] = True
if with_git:
bottle["git-gate"] = {"repos": {
"upstream": {
"url": "ssh://git@example.com:22/x/y.git",
"key": {"provider": "static", "path": "/etc/hostname"},
},
}}
if with_egress:
bottle["egress"] = {
"routes": [{
"host": "api.example",
"auth": {"scheme": "Bearer", "token_ref": "TOK"},
}],
}
return ManifestIndex.from_json_obj({
"bottles": {"dev": bottle},
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
})
def _git_gate_plan(upstreams: tuple[GitGateUpstream, ...] = ()) -> GitGatePlan:
return GitGatePlan(
slug=SLUG,
entrypoint_script=STATE / "git-gate" / "entrypoint.sh",
hook_script=STATE / "git-gate" / "pre-receive",
access_hook_script=STATE / "git-gate" / "access-hook",
upstreams=upstreams,
internal_network=f"bot-bottle-net-{SLUG}",
egress_network=f"bot-bottle-egress-{SLUG}",
)
def _egress_plan(
routes: tuple[EgressRoute, ...] = (),
*,
canary: bool = False,
) -> EgressPlan:
token_env_map = {
r.token_env: r.token_ref
for r in routes
if r.token_env
}
return EgressPlan(
slug=SLUG,
routes_path=STATE / "egress" / "routes.yaml",
routes=routes,
token_env_map=token_env_map,
internal_network=f"bot-bottle-net-{SLUG}",
egress_network=f"bot-bottle-egress-{SLUG}",
mitmproxy_ca_host_path=STATE / "egress-ca" / "mitmproxy-ca.pem",
mitmproxy_ca_cert_only_host_path=STATE / "egress-ca" / "ca.pem",
canary="fake-canary-value" if canary else "",
canary_env="CANON_ALPHA_SECRET" if canary else "",
)
def _supervise_plan() -> SupervisePlan:
return SupervisePlan(
slug=SLUG,
db_path=STATE / "bot-bottle.db",
internal_network=f"bot-bottle-net-{SLUG}",
)
def _plan(
*,
with_git: bool = False,
with_egress: bool = False,
supervise: bool = False,
canary: bool = False,
) -> DockerBottlePlan:
"""Build a fully-resolved DockerBottlePlan. Toggles cover the
matrix the renderer's conditional-service logic branches on."""
upstreams: tuple[GitGateUpstream, ...] = ()
if with_git:
upstreams = (GitGateUpstream(
name="upstream",
upstream_url="ssh://git@example.com:22/x/y.git",
upstream_host="example.com",
upstream_port="22",
identity_file="/etc/hostname",
known_host_key="",
known_hosts_file=STATE / "git-gate" / "upstream-known_hosts",
),)
routes: tuple[EgressRoute, ...] = ()
if with_egress:
routes = (EgressRoute(
host="api.example",
auth_scheme="Bearer",
token_env="EGRESS_TOKEN_0",
token_ref="TOK",
roles=(),
),)
index = _manifest(supervise=supervise, with_git=with_git, with_egress=with_egress)
spec = BottleSpec(
manifest=index,
agent_name="demo",
copy_cwd=False,
user_cwd="/tmp/x",
)
return DockerBottlePlan(
spec=spec,
manifest=index.load_for_agent("demo"),
stage_dir=STAGE,
slug=SLUG,
forwarded_env={"CLAUDE_CODE_OAUTH_TOKEN": "x"},
git_gate_plan=_git_gate_plan(upstreams),
egress_plan=_egress_plan(routes, canary=canary),
supervise_plan=_supervise_plan() if supervise else None,
use_runsc=False,
agent_provision=AgentProvisionPlan(
template="claude",
command="claude",
prompt_mode="append_file",
image="bot-bottle-claude:latest",
dockerfile="",
guest_home="/home/node",
instance_name=f"bot-bottle-{SLUG}",
prompt_file=STAGE / "prompt",
guest_env={},
),
)
+415 -4
View File
@@ -1,23 +1,434 @@
"""Unit: docker compose lifecycle helpers (PRD 0018). """Unit: compose-spec renderer (PRD 0018 chunk 1).
The compose *spec* is built by `consolidated_compose.py`; these Pure-function tests for `bottle_plan_to_compose`. Fixtures build a
tests cover the I/O-side helpers in `compose.py` the slug fully-resolved DockerBottlePlan in memory; the renderer just
project mapping and `docker compose ls` enumeration. translates it to the compose dict. Conditional-service matrix is
covered via parameterized cases (git on/off × egress on/off ×
supervise on/off).
""" """
from __future__ import annotations from __future__ import annotations
import subprocess import subprocess
import unittest import unittest
from pathlib import Path
from typing import Any
from unittest import mock from unittest import mock
from bot_bottle.agent_provider import AgentProvisionPlan
from bot_bottle.backend import BottleSpec
from bot_bottle.backend.docker.bottle_plan import DockerBottlePlan
from bot_bottle.backend.docker.compose import ( from bot_bottle.backend.docker.compose import (
COMPOSE_PROJECT_PREFIX, COMPOSE_PROJECT_PREFIX,
bottle_plan_to_compose,
compose_project_name, compose_project_name,
list_active_slugs, list_active_slugs,
list_compose_projects, list_compose_projects,
slug_from_compose_project, slug_from_compose_project,
) )
from bot_bottle.egress import (
EgressPlan,
EgressRoute,
)
from bot_bottle.git_gate import GitGatePlan, GitGateUpstream
from bot_bottle.manifest import ManifestIndex
from bot_bottle.supervise import SupervisePlan
SLUG = "demo-abc12"
STAGE = Path("/tmp/cb-stage")
STATE = Path("/tmp/cb-state")
def _manifest(*, supervise: bool, with_git: bool, with_egress: bool) -> ManifestIndex:
"""Minimal manifest with the toggles the chunk-1 matrix needs.
The renderer only reads from the plan, not the manifest, so this
is just here to back BottleSpec."""
bottle: dict[str, object] = {}
if supervise:
bottle["supervise"] = True
if with_git:
bottle["git-gate"] = {"repos": {
"upstream": {
"url": "ssh://git@example.com:22/x/y.git",
"key": {"provider": "static", "path": "/etc/hostname"},
},
}}
if with_egress:
bottle["egress"] = {
"routes": [{
"host": "api.example",
"auth": {"scheme": "Bearer", "token_ref": "TOK"},
}],
}
return ManifestIndex.from_json_obj({
"bottles": {"dev": bottle},
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
})
def _git_gate_plan(upstreams: tuple[GitGateUpstream, ...] = ()) -> GitGatePlan:
return GitGatePlan(
slug=SLUG,
entrypoint_script=STATE / "git-gate" / "entrypoint.sh",
hook_script=STATE / "git-gate" / "pre-receive",
access_hook_script=STATE / "git-gate" / "access-hook",
upstreams=upstreams,
internal_network=f"bot-bottle-net-{SLUG}",
egress_network=f"bot-bottle-egress-{SLUG}",
)
def _egress_plan(
routes: tuple[EgressRoute, ...] = (),
*,
canary: bool = False,
) -> EgressPlan:
token_env_map = {
r.token_env: r.token_ref
for r in routes
if r.token_env
}
return EgressPlan(
slug=SLUG,
routes_path=STATE / "egress" / "routes.yaml",
routes=routes,
token_env_map=token_env_map,
internal_network=f"bot-bottle-net-{SLUG}",
egress_network=f"bot-bottle-egress-{SLUG}",
mitmproxy_ca_host_path=STATE / "egress-ca" / "mitmproxy-ca.pem",
mitmproxy_ca_cert_only_host_path=STATE / "egress-ca" / "ca.pem",
canary="fake-canary-value" if canary else "",
canary_env="CANON_ALPHA_SECRET" if canary else "",
)
def _supervise_plan() -> SupervisePlan:
return SupervisePlan(
slug=SLUG,
db_path=STATE / "bot-bottle.db",
internal_network=f"bot-bottle-net-{SLUG}",
)
def _plan(
*,
with_git: bool = False,
with_egress: bool = False,
supervise: bool = False,
canary: bool = False,
) -> DockerBottlePlan:
"""Build a fully-resolved DockerBottlePlan. Toggles cover the
matrix the renderer's conditional-service logic branches on."""
upstreams: tuple[GitGateUpstream, ...] = ()
if with_git:
upstreams = (GitGateUpstream(
name="upstream",
upstream_url="ssh://git@example.com:22/x/y.git",
upstream_host="example.com",
upstream_port="22",
identity_file="/etc/hostname",
known_host_key="",
known_hosts_file=STATE / "git-gate" / "upstream-known_hosts",
),)
routes: tuple[EgressRoute, ...] = ()
if with_egress:
routes = (EgressRoute(
host="api.example",
auth_scheme="Bearer",
token_env="EGRESS_TOKEN_0",
token_ref="TOK",
roles=(),
),)
index = _manifest(supervise=supervise, with_git=with_git, with_egress=with_egress)
spec = BottleSpec(
manifest=index,
agent_name="demo",
copy_cwd=False,
user_cwd="/tmp/x",
)
return DockerBottlePlan(
spec=spec,
manifest=index.load_for_agent("demo"),
stage_dir=STAGE,
slug=SLUG,
forwarded_env={"CLAUDE_CODE_OAUTH_TOKEN": "x"},
git_gate_plan=_git_gate_plan(upstreams),
egress_plan=_egress_plan(routes, canary=canary),
supervise_plan=_supervise_plan() if supervise else None,
use_runsc=False,
agent_provision=AgentProvisionPlan(
template="claude",
command="claude",
prompt_mode="append_file",
image="bot-bottle-claude:latest",
dockerfile="",
guest_home="/home/node",
instance_name=f"bot-bottle-{SLUG}",
prompt_file=STAGE / "prompt",
guest_env={},
),
)
class TestProjectAndNetworks(unittest.TestCase):
def test_project_name(self):
spec = bottle_plan_to_compose(_plan())
self.assertEqual(f"bot-bottle-{SLUG}", spec["name"])
def test_internal_network_is_internal(self):
spec = bottle_plan_to_compose(_plan())
net = spec["networks"]["internal"]
self.assertEqual(f"bot-bottle-net-{SLUG}", net["name"])
self.assertTrue(net["internal"])
def test_egress_network_is_external_bridge(self):
spec = bottle_plan_to_compose(_plan())
net = spec["networks"]["egress"]
self.assertEqual(f"bot-bottle-egress-{SLUG}", net["name"])
# No `internal:` key on the egress network — defaults to a
# normal user-defined bridge.
self.assertNotIn("internal", net)
class TestAgentAlwaysPresent(unittest.TestCase):
def test_agent_in_services(self):
s = bottle_plan_to_compose(_plan())["services"]
self.assertIn("agent", s)
def test_agent_command(self):
s = bottle_plan_to_compose(_plan())["services"]["agent"]
self.assertEqual(["sleep", "infinity"], s["command"])
def test_agent_image_uses_runtime_image(self):
plan = _plan()
s = bottle_plan_to_compose(plan)["services"]["agent"]
self.assertEqual(plan.image, s["image"])
def test_agent_only_on_internal_network(self):
s = bottle_plan_to_compose(_plan())["services"]["agent"]
self.assertEqual({"internal"}, set(s["networks"].keys()))
def test_agent_proxy_always_via_egress(self):
for with_egress in (False, True):
with self.subTest(with_egress=with_egress):
s = bottle_plan_to_compose(
_plan(with_egress=with_egress)
)["services"]["agent"]
proxy_lines = [e for e in s["environment"] if e.startswith("HTTPS_PROXY=")]
self.assertEqual(1, len(proxy_lines))
self.assertEqual("HTTPS_PROXY=http://egress:9099", proxy_lines[0])
def test_agent_proxy_via_egress_when_egress_present(self):
s = bottle_plan_to_compose(_plan(with_egress=True))["services"]["agent"]
proxy = [e for e in s["environment"] if e.startswith("HTTPS_PROXY=")][0]
self.assertEqual("HTTPS_PROXY=http://egress:9099", proxy)
def test_agent_no_proxy_adds_supervise_when_enabled(self):
s = bottle_plan_to_compose(
_plan(supervise=True)
)["services"]["agent"]
no_proxy = [e for e in s["environment"] if e.startswith("NO_PROXY=")][0]
self.assertIn("supervise", no_proxy)
def test_agent_forwarded_env_uses_bare_names(self):
# Bare NAME → compose inherits value from the up-process env,
# so secret token values stay out of the file.
s = bottle_plan_to_compose(_plan())["services"]["agent"]
self.assertIn("CLAUDE_CODE_OAUTH_TOKEN", s["environment"])
def test_agent_provider_env_uses_literal_values(self):
plan = _plan()
provision = AgentProvisionPlan(
template="codex",
command="codex",
prompt_mode="read_prompt_file",
image="bot-bottle-codex:latest",
dockerfile="",
guest_home="/home/node",
instance_name=f"bot-bottle-{SLUG}",
prompt_file=STAGE / "prompt",
guest_env={"CODEX_HOME": "/home/node/.codex"},
)
plan = type(plan)(**{**vars(plan), "agent_provision": provision}) # type: ignore
s = bottle_plan_to_compose(plan)["services"]["agent"]
self.assertIn("CODEX_HOME=/home/node/.codex", s["environment"])
def test_agent_runsc_runtime(self):
plan = _plan()
plan = type(plan)(**{**vars(plan), "use_runsc": True}) # type: ignore
s = bottle_plan_to_compose(plan)["services"]["agent"]
self.assertEqual("runsc", s["runtime"])
def test_agent_depends_only_on_sidecars(self):
# Bundle shape: the init supervisor owns intra-bundle daemon
# ordering, so the agent waits on the bundle container alone.
for kwargs in [{}, {"with_git": True, "with_egress": True, "supervise": True}]:
with self.subTest(**kwargs):
s = bottle_plan_to_compose(_plan(**kwargs))["services"]["agent"]
self.assertEqual(["sidecars"], s["depends_on"])
def test_agent_has_no_current_config_mount_with_supervise(self):
with_sv = bottle_plan_to_compose(_plan(supervise=True))["services"]["agent"]
self.assertNotIn("volumes", with_sv)
without_sv = bottle_plan_to_compose(_plan(supervise=False))["services"]["agent"]
self.assertNotIn("volumes", without_sv)
class TestSidecarBundleShape(unittest.TestCase):
"""The compose renderer emits exactly one `sidecars` service in
place of the daemons it owns (egress + git-gate + supervise).
PRD 0024 chunk 5 dropped the legacy four-sidecar shape entirely,
so the bundle is the only thing exercised here."""
def _render(self, **plan_kwargs: object) -> Any: # type: ignore
return bottle_plan_to_compose(_plan(**plan_kwargs)) # type: ignore
def test_emits_two_services_minimal(self):
spec = self._render()
self.assertEqual({"sidecars", "agent"}, set(spec["services"].keys()))
def test_emits_two_services_full_matrix(self):
spec = self._render(with_git=True, with_egress=True, supervise=True)
# Still two services — the bundle absorbs git-gate/egress/supervise.
self.assertEqual({"sidecars", "agent"}, set(spec["services"].keys()))
def test_bundle_uses_bundle_image_and_dockerfile(self):
sc = self._render()["services"]["sidecars"]
self.assertEqual("bot-bottle-sidecars:latest", sc["image"])
self.assertEqual("Dockerfile.sidecars", sc["build"]["dockerfile"])
def test_bundle_container_name_uses_sidecars_prefix(self):
sc = self._render()["services"]["sidecars"]
self.assertEqual(f"bot-bottle-sidecars-{SLUG}", sc["container_name"])
def test_bundle_joins_both_networks(self):
sc = self._render()["services"]["sidecars"]
self.assertEqual({"internal", "egress"}, set(sc["networks"].keys()))
def test_internal_aliases_include_egress_shortname(self):
sc = self._render()["services"]["sidecars"]
aliases = set(sc["networks"]["internal"]["aliases"])
self.assertIn("egress", aliases)
def test_internal_aliases_omit_inactive_sidecars(self):
# With no git-gate / supervise, those names are NOT aliased
# — keeps the alias list honest about what's actually
# listening inside the bundle.
sc = self._render()["services"]["sidecars"]
aliases = set(sc["networks"]["internal"]["aliases"])
self.assertNotIn("git-gate", aliases)
self.assertNotIn("supervise", aliases)
def test_internal_aliases_include_active_sidecars(self):
sc = self._render(with_git=True, supervise=True)["services"]["sidecars"]
aliases = set(sc["networks"]["internal"]["aliases"])
self.assertIn("git-gate", aliases)
self.assertIn("supervise", aliases)
def test_daemons_csv_lists_only_active(self):
sc = self._render()["services"]["sidecars"]
daemons = {
line.split("=", 1)[1]
for line in sc["environment"]
if line.startswith("BOT_BOTTLE_SIDECAR_DAEMONS=")
}
self.assertEqual({"egress"}, daemons)
def test_daemons_csv_expands_with_optional_sidecars(self):
sc = self._render(with_git=True, supervise=True)["services"]["sidecars"]
for line in sc["environment"]:
if line.startswith("BOT_BOTTLE_SIDECAR_DAEMONS="):
csv = line.split("=", 1)[1]
break
else:
self.fail("BOT_BOTTLE_SIDECAR_DAEMONS not in env")
self.assertEqual(
["egress", "git-gate", "supervise"],
csv.split(","),
)
def test_bundle_env_does_not_set_https_proxy(self):
# HTTPS_PROXY at the container level would route git-gate's
# git fetches through the proxy. Scoping it to mitmdump is
# the job of egress_entrypoint.sh; the bundle env must not
# leak it.
sc = self._render(with_egress=True)["services"]["sidecars"]
for line in sc["environment"]:
self.assertFalse(
line.startswith("HTTPS_PROXY=")
or line.startswith("HTTP_PROXY=")
or line.startswith("NO_PROXY="),
f"bundle env must not set {line!r}",
)
def test_egress_token_env_present_when_routes_declared(self):
sc = self._render(with_egress=True)["services"]["sidecars"]
env_strings = sc["environment"]
self.assertIn("EGRESS_TOKEN_0", env_strings)
def test_egress_token_env_omitted_when_no_routes(self):
sc = self._render()["services"]["sidecars"]
env_strings = sc["environment"]
self.assertNotIn("EGRESS_TOKEN_0", env_strings)
def test_canary_env_registered_as_sensitive_in_sidecar(self):
sc = self._render(canary=True)["services"]["sidecars"]
env_strings = sc["environment"]
self.assertIn("CANON_ALPHA_SECRET=fake-canary-value", env_strings)
self.assertIn(
"BOT_BOTTLE_SENSITIVE_PREFIXES=CANON_ALPHA_SECRET",
env_strings,
)
def test_canary_env_visible_to_agent(self):
agent = self._render(canary=True)["services"]["agent"]
env_strings = agent["environment"]
self.assertIn("CANON_ALPHA_SECRET=fake-canary-value", env_strings)
def test_supervise_env_present_when_active(self):
sc = self._render(supervise=True)["services"]["sidecars"]
env_strings = sc["environment"]
self.assertIn(f"SUPERVISE_BOTTLE_SLUG={SLUG}", env_strings)
self.assertIn("SUPERVISE_DB_PATH=/run/supervise/bot-bottle.db", env_strings)
self.assertTrue(any(e.startswith("SUPERVISE_PORT=") for e in env_strings))
def test_volumes_always_includes_egress_ca(self):
sc = self._render()["services"]["sidecars"]
targets = {v["target"] for v in sc["volumes"]}
self.assertIn("/home/mitmproxy/.mitmproxy/mitmproxy-ca.pem", targets)
def test_volumes_union_full_matrix(self):
sc = self._render(with_git=True, with_egress=True, supervise=True)[
"services"]["sidecars"]
targets = {v["target"] for v in sc["volumes"]}
self.assertIn("/home/mitmproxy/.mitmproxy/mitmproxy-ca.pem", targets)
self.assertIn("/etc/egress", targets)
self.assertIn("/git-gate-entrypoint.sh", targets)
self.assertIn("/git-gate/creds/upstream-known_hosts", targets)
self.assertIn("/run/supervise/bot-bottle.db", targets)
def test_extra_hosts_omitted_for_git_upstreams(self):
sc = self._render(with_git=True)["services"]["sidecars"]
self.assertNotIn("extra_hosts", sc)
def test_agent_depends_on_bundle_only(self):
sc = self._render(with_git=True, with_egress=True, supervise=True)[
"services"]["agent"]
self.assertEqual(["sidecars"], sc["depends_on"])
def test_agent_proxy_url_resolves_via_bundle_alias(self):
# With egress active, the agent's HTTPS_PROXY points at
# `egress` shortname; bundle aliases `egress` to itself so
# the URL keeps working without an agent-side change.
spec = self._render(with_egress=True)
sc = spec["services"]["agent"]
proxy = next(e for e in sc["environment"] if e.startswith("HTTPS_PROXY="))
self.assertIn("egress", proxy)
self.assertIn("egress",
spec["services"]["sidecars"]["networks"]["internal"]["aliases"])
class TestProjectNaming(unittest.TestCase): class TestProjectNaming(unittest.TestCase):
-51
View File
@@ -1,51 +0,0 @@
"""Unit: agent-only consolidated compose render (PRD 0070)."""
from __future__ import annotations
import unittest
from bot_bottle.backend.docker.consolidated_compose import consolidated_agent_compose
from tests.unit._docker_bottle_plan import _plan
_GW = "172.18.0.2"
_IP = "172.18.0.5"
_NET = "bot-bottle-gateway"
class TestConsolidatedAgentCompose(unittest.TestCase):
def _spec(self, *, runsc: bool = False):
plan = _plan(with_egress=True, supervise=True, with_git=True)
if runsc:
plan = type(plan)(**{**vars(plan), "use_runsc": True}) # type: ignore[arg-type]
return consolidated_agent_compose(plan, gateway_ip=_GW, source_ip=_IP, network=_NET)
def test_only_agent_service_no_companion_container(self) -> None:
# The whole point of consolidation: no per-bottle gateway.
self.assertEqual(["agent"], list(self._spec()["services"]))
def test_agent_pinned_on_external_gateway_network(self) -> None:
spec = self._spec()
self.assertEqual({"external": True}, spec["networks"][_NET])
agent_net = spec["services"]["agent"]["networks"][_NET]
self.assertEqual(_IP, agent_net["ipv4_address"])
def test_proxy_and_ca_point_at_gateway(self) -> None:
env = self._spec()["services"]["agent"]["environment"]
self.assertIn(f"HTTPS_PROXY=http://{_GW}:9099", env)
# git-http + supervise on the gateway must bypass the egress proxy.
self.assertTrue(any(e.startswith("NO_PROXY=") and _GW in e for e in env))
def test_no_companion_container_dependency(self) -> None:
self.assertNotIn("depends_on", self._spec()["services"]["agent"])
def test_runsc_runtime_when_enabled(self) -> None:
self.assertEqual("runsc", self._spec(runsc=True)["services"]["agent"]["runtime"])
def test_forwarded_env_stays_bare_names(self) -> None:
env = self._spec()["services"]["agent"]["environment"]
# forwarded secrets are bare names (value inherited from process env).
self.assertIn("CLAUDE_CODE_OAUTH_TOKEN", env)
if __name__ == "__main__":
unittest.main()
-95
View File
@@ -1,95 +0,0 @@
"""Unit: consolidated launch sequence — compose the orchestrator primitives (PRD 0070)."""
from __future__ import annotations
import unittest
from pathlib import Path
from unittest.mock import MagicMock, Mock, patch
from bot_bottle.backend.docker.consolidated_launch import (
launch_consolidated,
teardown_consolidated,
)
from bot_bottle.egress import EgressPlan, EgressRoute
from bot_bottle.git_gate import GitGatePlan
from bot_bottle.orchestrator.client import RegisteredBottle
_MOD = "bot_bottle.backend.docker.consolidated_launch"
def _egress_plan() -> EgressPlan:
return EgressPlan(
slug="demo", routes_path=Path("/x"), routes=(EgressRoute(host="api.example.com"),),
token_env_map={},
)
def _git_plan() -> GitGatePlan:
return GitGatePlan(
slug="demo", entrypoint_script=Path(), hook_script=Path(),
access_hook_script=Path(), upstreams=(),
)
def _client(*, bottles: list[dict[str, object]] | None = None) -> Mock:
c = Mock()
c.list_bottles.return_value = bottles or []
c.register_bottle.return_value = RegisteredBottle("b1", "tok")
return c
class TestLaunchConsolidated(unittest.TestCase):
def _run(
self, client: Mock, provision: Mock | None = None,
*, on_network: tuple[str, ...] = ("172.18.0.2",),
):
service = MagicMock()
service.ensure_running.return_value = "http://orch:8080"
with patch(f"{_MOD}._network_cidr", return_value="172.18.0.0/16"), \
patch(f"{_MOD}._container_ip", return_value="172.18.0.2"), \
patch(f"{_MOD}._network_container_ips", return_value=list(on_network)), \
patch(f"{_MOD}.OrchestratorClient", return_value=client), \
patch(f"{_MOD}.provision_git_gate", provision or Mock()):
return launch_consolidated(_egress_plan(), _git_plan(), service=service)
def test_allocates_ip_registers_and_provisions(self) -> None:
client = _client()
provision = Mock()
ctx = self._run(client, provision)
# .1 is the router, .2 is the gateway → first bottle gets .3.
self.assertEqual("172.18.0.3", ctx.source_ip)
self.assertEqual("172.18.0.2", ctx.gateway_ip)
self.assertEqual("b1", ctx.bottle_id)
self.assertEqual("tok", ctx.identity_token)
self.assertEqual("http://orch:8080", ctx.orchestrator_url)
# Registered with the source IP + the egress policy blob.
kwargs = client.register_bottle.call_args
self.assertEqual("172.18.0.3", kwargs.args[0])
self.assertIn("api.example.com", kwargs.kwargs["policy"])
provision.assert_called_once()
def test_skips_all_addresses_on_the_network(self) -> None:
# Gateway .2 + orchestrator .3 already attached -> agent gets .4.
ctx = self._run(_client(), on_network=("172.18.0.2", "172.18.0.3"))
self.assertEqual("172.18.0.4", ctx.source_ip)
def test_provision_failure_rolls_back_registration(self) -> None:
client = _client()
provision = Mock(side_effect=RuntimeError("provision boom"))
with self.assertRaises(RuntimeError):
self._run(client, provision)
client.teardown_bottle.assert_called_once_with("b1") # no orphan left
class TestTeardownConsolidated(unittest.TestCase):
def test_deregisters_and_deprovisions(self) -> None:
client = Mock()
with patch(f"{_MOD}.OrchestratorClient", return_value=client), \
patch(f"{_MOD}.deprovision_git_gate") as deprov:
teardown_consolidated("b1", orchestrator_url="http://orch:8080")
client.teardown_bottle.assert_called_once_with("b1")
deprov.assert_called_once()
if __name__ == "__main__":
unittest.main()
+113 -64
View File
@@ -1,4 +1,4 @@
"""Unit: Docker launch step uses committed image when available (consolidated).""" """Unit: Docker launch step uses committed image when available."""
from __future__ import annotations from __future__ import annotations
@@ -6,7 +6,6 @@ import contextlib
import io import io
import tempfile import tempfile
import unittest import unittest
from collections.abc import Callable, Generator
from pathlib import Path from pathlib import Path
from typing import Any from typing import Any
from unittest import mock from unittest import mock
@@ -15,11 +14,9 @@ from bot_bottle.agent_provider import AgentProvisionPlan
from bot_bottle.backend import BottleSpec from bot_bottle.backend import BottleSpec
from bot_bottle.backend.docker import launch as launch_mod from bot_bottle.backend.docker import launch as launch_mod
from bot_bottle.backend.docker.bottle_plan import DockerBottlePlan from bot_bottle.backend.docker.bottle_plan import DockerBottlePlan
from bot_bottle.backend.docker.consolidated_launch import LaunchContext
from bot_bottle.egress import EgressPlan from bot_bottle.egress import EgressPlan
from bot_bottle.git_gate import GitGatePlan from bot_bottle.git_gate import GitGatePlan
from bot_bottle.manifest import ManifestIndex from bot_bottle.manifest import ManifestIndex
from tests.unit import use_bottle_root
_SLUG = "dev-abc12" _SLUG = "dev-abc12"
@@ -31,111 +28,163 @@ _IDX = ManifestIndex.from_json_obj({
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}}, "agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
}) })
_CTX = LaunchContext(
bottle_id="b1", identity_token="t", source_ip="172.20.0.4",
network="bot-bottle-gateway", gateway_ip="172.20.0.2",
orchestrator_url="http://orch:8099",
)
def _plan(tmp: str) -> DockerBottlePlan: def _plan(tmp: str) -> DockerBottlePlan:
stage = Path(tmp) stage = Path(tmp)
spec = BottleSpec( spec = BottleSpec(
manifest=_IDX, agent_name="demo", copy_cwd=False, user_cwd=tmp, identity=_SLUG, manifest=_IDX,
agent_name="demo",
copy_cwd=False,
user_cwd=tmp,
identity=_SLUG,
) )
return DockerBottlePlan( return DockerBottlePlan(
spec=spec, spec=spec,
manifest=_IDX.load_for_agent("demo"), manifest=_IDX.load_for_agent("demo"),
stage_dir=stage, stage_dir=stage,
git_gate_plan=GitGatePlan( git_gate_plan=GitGatePlan(
slug=_SLUG, entrypoint_script=stage / "e.sh", hook_script=stage / "h.sh", slug=_SLUG,
access_hook_script=stage / "a.sh", upstreams=(), entrypoint_script=stage / "entrypoint.sh",
hook_script=stage / "hook.sh",
access_hook_script=stage / "access-hook.sh",
upstreams=(),
), ),
egress_plan=EgressPlan( egress_plan=EgressPlan(
slug=_SLUG, routes_path=stage / "egress.yaml", routes=(), token_env_map={}, slug=_SLUG,
routes_path=stage / "egress.yaml",
routes=(),
token_env_map={},
), ),
supervise_plan=None, supervise_plan=None,
agent_provision=AgentProvisionPlan( agent_provision=AgentProvisionPlan(
template="claude", command="claude", prompt_mode="append_file", template="claude",
image=_DEFAULT_IMAGE, dockerfile="", guest_home="/home/node", command="claude",
instance_name=f"bot-bottle-{_SLUG}", prompt_file=stage / "prompt.txt", prompt_mode="append_file",
image=_DEFAULT_IMAGE,
dockerfile="",
guest_home="/home/node",
instance_name=f"bot-bottle-{_SLUG}",
prompt_file=stage / "prompt.txt",
guest_env={}, guest_env={},
), ),
slug=_SLUG, forwarded_env={}, use_runsc=False, slug=_SLUG,
forwarded_env={},
use_runsc=False,
) )
class TestLaunchCommittedImage(unittest.TestCase): class TestLaunchCommittedImage(unittest.TestCase):
def setUp(self) -> None: def setUp(self) -> None:
self._tmp = tempfile.mkdtemp(prefix="launch-committed-test.") self._tmp = tempfile.mkdtemp(prefix="launch-committed-test.")
self.addCleanup(lambda: __import__("shutil").rmtree(self._tmp, ignore_errors=True))
# The launch writes the gateway CA under the bottle root — sandbox it.
self.addCleanup(use_bottle_root(Path(self._tmp)))
@contextlib.contextmanager def tearDown(self) -> None:
def _patched( import shutil
shutil.rmtree(self._tmp, ignore_errors=True)
def _run_launch(
self, self,
plan: DockerBottlePlan,
*, *,
committed_tag: str | None, committed_tag: str | None = None,
image_present: bool, image_present: bool = True,
compose: Callable[..., dict[str, Any]], ) -> list[str]:
) -> Generator[list[str], None, None]: """Drive launch() through its full sequence with the committed-image
behaviour controlled by the arguments. Returns the images that were
passed to `build_image` (empty list if it was never called)."""
built: list[str] = [] built: list[str] = []
gw = mock.Mock()
gw.ca_cert_pem.return_value = "-----BEGIN CERTIFICATE-----\nx\n-----END CERTIFICATE-----\n"
def _build(image: str, ctx: str, *, dockerfile: str = "") -> None: def fake_build(image: str, ctx: str, *, dockerfile: str = "") -> None:
del ctx, dockerfile del ctx, dockerfile
built.append(image) built.append(image)
with mock.patch.object(launch_mod, "read_committed_image", return_value=committed_tag), \ with mock.patch.object(
mock.patch.object(launch_mod.docker_mod, "image_exists", return_value=image_present), \ launch_mod, "read_committed_image", return_value=committed_tag,
mock.patch.object(launch_mod.docker_mod, "build_image", side_effect=_build), \ ), mock.patch.object(
mock.patch.object(launch_mod, "launch_consolidated", return_value=_CTX), \ launch_mod.docker_mod, "image_exists", return_value=image_present,
mock.patch.object(launch_mod, "teardown_consolidated"), \ ), mock.patch.object(
mock.patch.object(launch_mod, "DockerGateway", return_value=gw), \ launch_mod.docker_mod, "build_image", side_effect=fake_build,
mock.patch.object(launch_mod, "consolidated_agent_compose", side_effect=compose), \ ), mock.patch.object(
mock.patch.object(launch_mod, "write_compose_file", return_value=Path("/tmp/c.yml")), \ launch_mod, "egress_tls_init",
mock.patch.object(launch_mod, "compose_up"), \ return_value=(Path("/egress_ca"), Path("/egress_cert")),
mock.patch.object(launch_mod, "compose_dump_logs"), \ ), mock.patch.object(
mock.patch.object(launch_mod, "compose_down"), \ launch_mod.network_mod, "network_name_for_slug",
contextlib.redirect_stderr(io.StringIO()): return_value="bb-internal",
yield built ), mock.patch.object(
launch_mod.network_mod, "network_egress_name_for_slug",
def _run_launch( return_value="bb-egress",
self, plan: DockerBottlePlan, *, ), mock.patch.object(
committed_tag: str | None = None, image_present: bool = True, launch_mod, "bottle_plan_to_compose",
) -> list[str]: return_value={"services": {"agent": {}}},
def compose(_p: DockerBottlePlan, **_kw: Any) -> dict[str, Any]: ), mock.patch.object(
return {"services": {"agent": {}}} launch_mod, "write_compose_file",
with self._patched( return_value=Path("/tmp/compose.yml"),
committed_tag=committed_tag, image_present=image_present, compose=compose, ), mock.patch.object(launch_mod, "compose_up"), \
) as built: mock.patch.object(launch_mod, "compose_dump_logs"), \
with launch_mod.launch(plan, provision=mock.Mock(return_value=None)): mock.patch.object(launch_mod, "compose_down"), \
contextlib.redirect_stderr(io.StringIO()):
provision = mock.Mock(return_value=None)
with launch_mod.launch(plan, provision=provision):
pass pass
return built return built
def test_skips_build_when_committed_image_present(self) -> None: def test_skips_build_when_committed_image_present(self) -> None:
built = self._run_launch(_plan(self._tmp), committed_tag=_COMMITTED_TAG, image_present=True) plan = _plan(self._tmp)
self.assertEqual([], built) # committed image reused, no build built = self._run_launch(plan, committed_tag=_COMMITTED_TAG, image_present=True)
self.assertEqual([], built, "build_image should not be called when committed image exists")
def test_uses_committed_image_in_compose_spec(self) -> None: def test_uses_committed_image_in_compose_spec(self) -> None:
captured: list[DockerBottlePlan] = [] """The compose spec renderer receives the committed image tag via
plan.image captured here by checking what bottle_plan_to_compose
was called with."""
plan = _plan(self._tmp)
captured_plans: list[DockerBottlePlan] = []
def compose(p: DockerBottlePlan, **_kw: Any) -> dict[str, Any]: def fake_compose(p: DockerBottlePlan) -> dict[str, Any]:
captured.append(p) captured_plans.append(p)
return {"services": {"agent": {}}} return {"services": {"agent": {}}}
with self._patched(committed_tag=_COMMITTED_TAG, image_present=True, compose=compose): with mock.patch.object(
with launch_mod.launch(_plan(self._tmp), provision=mock.Mock(return_value=None)): launch_mod, "read_committed_image", return_value=_COMMITTED_TAG,
), mock.patch.object(
launch_mod.docker_mod, "image_exists", return_value=True,
), mock.patch.object(
launch_mod.docker_mod, "build_image",
), mock.patch.object(
launch_mod, "egress_tls_init",
return_value=(Path("/egress_ca"), Path("/egress_cert")),
), mock.patch.object(
launch_mod.network_mod, "network_name_for_slug",
return_value="bb-internal",
), mock.patch.object(
launch_mod.network_mod, "network_egress_name_for_slug",
return_value="bb-egress",
), mock.patch.object(
launch_mod, "bottle_plan_to_compose", side_effect=fake_compose,
), mock.patch.object(
launch_mod, "write_compose_file",
return_value=Path("/tmp/compose.yml"),
), mock.patch.object(launch_mod, "compose_up"), \
mock.patch.object(launch_mod, "compose_dump_logs"), \
mock.patch.object(launch_mod, "compose_down"), \
contextlib.redirect_stderr(io.StringIO()):
provision = mock.Mock(return_value=None)
with launch_mod.launch(plan, provision=provision):
pass pass
self.assertEqual(_COMMITTED_TAG, captured[0].image)
self.assertEqual(1, len(captured_plans))
self.assertEqual(_COMMITTED_TAG, captured_plans[0].image)
def test_falls_back_to_build_when_no_committed_image(self) -> None: def test_falls_back_to_build_when_no_committed_image(self) -> None:
self.assertEqual([_DEFAULT_IMAGE], self._run_launch(_plan(self._tmp), committed_tag=None)) plan = _plan(self._tmp)
built = self._run_launch(plan, committed_tag=None)
self.assertEqual([_DEFAULT_IMAGE], built)
def test_falls_back_to_build_when_committed_image_missing_from_daemon(self) -> None: def test_falls_back_to_build_when_committed_image_missing_from_daemon(self) -> None:
built = self._run_launch(_plan(self._tmp), committed_tag=_COMMITTED_TAG, image_present=False) plan = _plan(self._tmp)
built = self._run_launch(
plan, committed_tag=_COMMITTED_TAG, image_present=False,
)
self.assertEqual([_DEFAULT_IMAGE], built) self.assertEqual([_DEFAULT_IMAGE], built)
+20 -17
View File
@@ -19,11 +19,9 @@ from bot_bottle.agent_provider import AgentProvisionPlan
from bot_bottle.backend import BottleSpec from bot_bottle.backend import BottleSpec
from bot_bottle.backend.docker import launch as launch_mod from bot_bottle.backend.docker import launch as launch_mod
from bot_bottle.backend.docker.bottle_plan import DockerBottlePlan from bot_bottle.backend.docker.bottle_plan import DockerBottlePlan
from bot_bottle.backend.docker.consolidated_launch import LaunchContext
from bot_bottle.egress import EgressPlan from bot_bottle.egress import EgressPlan
from bot_bottle.git_gate import GitGatePlan from bot_bottle.git_gate import GitGatePlan
from bot_bottle.manifest import ManifestIndex from bot_bottle.manifest import ManifestIndex
from tests.unit import use_bottle_root
_INDEX = ManifestIndex.from_json_obj({ _INDEX = ManifestIndex.from_json_obj({
"bottles": {"dev": {}}, "bottles": {"dev": {}},
@@ -79,36 +77,41 @@ def _plan(tmp: str) -> DockerBottlePlan:
class TestTeardownWarning(unittest.TestCase): class TestTeardownWarning(unittest.TestCase):
def setUp(self) -> None: def setUp(self) -> None:
self._tmp = tempfile.mkdtemp(prefix="docker-launch-teardown-test.") self._tmp = tempfile.mkdtemp(prefix="docker-launch-teardown-test.")
self.addCleanup(lambda: __import__("shutil").rmtree(self._tmp, ignore_errors=True))
self.addCleanup(use_bottle_root(Path(self._tmp))) # sandbox the gateway CA write def tearDown(self) -> None:
import shutil
shutil.rmtree(self._tmp, ignore_errors=True)
def test_teardown_failure_emits_warning_with_container_and_operation(self): def test_teardown_failure_emits_warning_with_container_and_operation(self):
plan = _plan(self._tmp) plan = _plan(self._tmp)
buf = io.StringIO() buf = io.StringIO()
gw = mock.Mock()
gw.ca_cert_pem.return_value = "-----BEGIN CERTIFICATE-----\nx\n-----END CERTIFICATE-----\n"
ctx = LaunchContext(
bottle_id="b1", identity_token="t", source_ip="172.20.0.4",
network="bot-bottle-gateway", gateway_ip="172.20.0.2",
orchestrator_url="http://orch:8099",
)
with mock.patch.object(launch_mod.docker_mod, "build_image"), \ with mock.patch.object(launch_mod.docker_mod, "build_image"), \
mock.patch.object(launch_mod, "launch_consolidated", return_value=ctx), \
mock.patch.object(launch_mod, "teardown_consolidated"), \
mock.patch.object(launch_mod, "DockerGateway", return_value=gw), \
mock.patch.object( mock.patch.object(
launch_mod, "consolidated_agent_compose", launch_mod, "egress_tls_init",
return_value=(Path("/egress_ca"), Path("/egress_cert")),
), \
mock.patch.object(
launch_mod.network_mod, "network_name_for_slug",
return_value="bb-internal-test",
), \
mock.patch.object(
launch_mod.network_mod, "network_egress_name_for_slug",
return_value="bb-egress-test",
), \
mock.patch.object(
launch_mod, "bottle_plan_to_compose",
return_value={"services": {"agent": {}}}, return_value={"services": {"agent": {}}},
), \ ), \
mock.patch.object( mock.patch.object(
launch_mod, "write_compose_file", return_value=Path("/tmp/compose.yml"), launch_mod, "write_compose_file",
return_value=Path("/tmp/compose.yml"),
), \ ), \
mock.patch.object(launch_mod, "compose_up"), \ mock.patch.object(launch_mod, "compose_up"), \
mock.patch.object(launch_mod, "compose_dump_logs"), \ mock.patch.object(launch_mod, "compose_dump_logs"), \
mock.patch.object( mock.patch.object(
launch_mod, "compose_down", launch_mod, "compose_down",
side_effect=RuntimeError("compose down failed"), side_effect=RuntimeError("network remove failed"),
), \ ), \
contextlib.redirect_stderr(buf): contextlib.redirect_stderr(buf):
provision = mock.Mock(return_value=None) provision = mock.Mock(return_value=None)
+4 -4
View File
@@ -16,7 +16,7 @@ from bot_bottle.egress import (
egress_render_routes, egress_render_routes,
egress_resolve_token_values, egress_resolve_token_values,
egress_routes_for_bottle, egress_routes_for_bottle,
egress_gateway_env_entries, egress_sidecar_env_entries,
egress_token_env_map, egress_token_env_map,
) )
from bot_bottle.errors import MissingEnvVarError from bot_bottle.errors import MissingEnvVarError
@@ -586,7 +586,7 @@ class TestCanaryGeneration(unittest.TestCase):
class TestEgressEnvEntries(unittest.TestCase): class TestEgressEnvEntries(unittest.TestCase):
def test_gateway_entries_include_route_tokens_and_canary_scan_prefix(self): def test_sidecar_entries_include_route_tokens_and_canary_scan_prefix(self):
plan = EgressPlan( plan = EgressPlan(
slug="s", slug="s",
routes_path=Path("/tmp/r.yaml"), routes_path=Path("/tmp/r.yaml"),
@@ -603,7 +603,7 @@ class TestEgressEnvEntries(unittest.TestCase):
"CANON_ALPHA_SECRET=fake-canary-value", "CANON_ALPHA_SECRET=fake-canary-value",
"BOT_BOTTLE_SENSITIVE_PREFIXES=CANON_ALPHA_SECRET", "BOT_BOTTLE_SENSITIVE_PREFIXES=CANON_ALPHA_SECRET",
), ),
egress_gateway_env_entries(plan), egress_sidecar_env_entries(plan),
) )
def test_agent_entries_include_only_canary_bait(self): def test_agent_entries_include_only_canary_bait(self):
@@ -630,7 +630,7 @@ class TestEgressEnvEntries(unittest.TestCase):
canary="fake-canary-value", canary="fake-canary-value",
) )
self.assertEqual((), egress_gateway_env_entries(plan)) self.assertEqual((), egress_sidecar_env_entries(plan))
self.assertEqual((), egress_agent_env_entries(plan)) self.assertEqual((), egress_agent_env_entries(plan))
+3 -3
View File
@@ -951,7 +951,7 @@ class TestScanOutbound(unittest.TestCase):
body='{"jsonrpc":"2.0","method":"initialize"}', body='{"jsonrpc":"2.0","method":"initialize"}',
) )
self.assertIsNone(scan_outbound(route, text, { self.assertIsNone(scan_outbound(route, text, {
"EGRESS_TOKEN_0": "gateway-owned-secret", "EGRESS_TOKEN_0": "sidecar-owned-secret",
})) }))
def test_token_in_body_blocked(self): def test_token_in_body_blocked(self):
@@ -1302,7 +1302,7 @@ class TestScanOutboundEnhanced(unittest.TestCase):
self.assertEqual("warn", result.severity) self.assertEqual("warn", result.severity)
def test_bot_bottle_sensitive_prefixes_env_var(self): def test_bot_bottle_sensitive_prefixes_env_var(self):
# When the gateway env contains BOT_BOTTLE_SENSITIVE_PREFIXES, # When the sidecar env contains BOT_BOTTLE_SENSITIVE_PREFIXES,
# scan_outbound should scan those additional prefixes. # scan_outbound should scan those additional prefixes.
secret = "extra-sensitive-value-abc" secret = "extra-sensitive-value-abc"
env = { env = {
@@ -1324,7 +1324,7 @@ class TestScanOutboundEnhanced(unittest.TestCase):
self.assertIsNotNone(result) self.assertIsNotNone(result)
def test_canary_detected_via_random_secret_env_name(self): def test_canary_detected_via_random_secret_env_name(self):
# The fake secret uses a randomized env name that the gateway marks # The fake secret uses a randomized env name that the sidecar marks
# as sensitive through BOT_BOTTLE_SENSITIVE_PREFIXES. # as sensitive through BOT_BOTTLE_SENSITIVE_PREFIXES.
canary = "canaryvalue12345abcdef" canary = "canaryvalue12345abcdef"
env = { env = {
@@ -1,6 +1,6 @@
"""Unit: LOG_FULL credential redaction in _log_request / _log_response (issue #257). """Unit: LOG_FULL credential redaction in _log_request / _log_response (issue #257).
egress_addon.py is gateway-only code that depends on mitmproxy, which is egress_addon.py is sidecar-only code that depends on mitmproxy, which is
not installed on the host. This file pre-populates sys.modules with the not installed on the host. This file pre-populates sys.modules with the
minimum mocks needed so EgressAddon can be imported and tested without the minimum mocks needed so EgressAddon can be imported and tested without the
real mitmproxy package.""" real mitmproxy package."""
@@ -17,7 +17,7 @@ from unittest.mock import patch
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Gateway-import shims — must run before importing egress_addon # Sidecar-import shims — must run before importing egress_addon
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
def _ensure_shims() -> None: def _ensure_shims() -> None:
@@ -46,7 +46,7 @@ def _addon() -> EgressAddon:
"""Return a bare EgressAddon with LOG_FULL config and no routes file.""" """Return a bare EgressAddon with LOG_FULL config and no routes file."""
a: EgressAddon = EgressAddon.__new__(EgressAddon) a: EgressAddon = EgressAddon.__new__(EgressAddon)
a.config = Config(routes=(), log=LOG_FULL) a.config = Config(routes=(), log=LOG_FULL)
a._safe_tokens = {} a.safe_tokens = set()
a._supervise_slug = "" a._supervise_slug = ""
a._token_allow_timeout = 300.0 a._token_allow_timeout = 300.0
return a return a
+7 -131
View File
@@ -1,6 +1,6 @@
"""Unit: EgressAddon request/response decision flow (issue #286). """Unit: EgressAddon request/response decision flow (issue #286).
`egress_addon.py` is the gateway-only mitmproxy adapter that wires the `egress_addon.py` is the sidecar-only mitmproxy adapter that wires the
host-importable decision logic in `egress_addon_core` into mitmproxy's host-importable decision logic in `egress_addon_core` into mitmproxy's
request/response hooks. The core logic is exercised directly by request/response hooks. The core logic is exercised directly by
`test_egress_addon_core.py`; the redaction logging by `test_egress_addon_core.py`; the redaction logging by
@@ -13,7 +13,7 @@ from coverage.
mitmproxy is not installed on the host, so we pre-populate `sys.modules` mitmproxy is not installed on the host, so we pre-populate `sys.modules`
with the minimum stubs needed to import the adapter (a `mitmproxy.http` with the minimum stubs needed to import the adapter (a `mitmproxy.http`
module exposing a `Response` with `.make`, plus the flat module exposing a `Response` with `.make`, plus the flat
`egress_addon_core` name the gateway uses).""" `egress_addon_core` name the sidecar uses)."""
from __future__ import annotations from __future__ import annotations
@@ -158,7 +158,7 @@ class _WebSocketData:
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Gateway-import shims — must run before importing egress_addon # Sidecar-import shims — must run before importing egress_addon
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@@ -211,7 +211,7 @@ def _addon(config: Config) -> EgressAddon:
"""Bare EgressAddon with a supplied config and no supervise wiring.""" """Bare EgressAddon with a supplied config and no supervise wiring."""
a: EgressAddon = EgressAddon.__new__(EgressAddon) a: EgressAddon = EgressAddon.__new__(EgressAddon)
a.config = config a.config = config
a._safe_tokens = {} a.safe_tokens = set()
a._supervise_slug = "" a._supervise_slug = ""
a._token_allow_timeout = 300.0 a._token_allow_timeout = 300.0
a.routes_path = "/nonexistent/routes.yaml" a.routes_path = "/nonexistent/routes.yaml"
@@ -222,32 +222,6 @@ def _run_request(addon: EgressAddon, flow: _Flow) -> None:
asyncio.run(addon.request(flow)) # type: ignore[arg-type] asyncio.run(addon.request(flow)) # type: ignore[arg-type]
def _with_client_ip(flow: _Flow, ip: str) -> _Flow:
"""Attach a mitmproxy-style client_conn so consolidated resolution can read
the source IP (`flow.client_conn.peername[0]`)."""
flow.client_conn = types.SimpleNamespace(peername=(ip, 54321)) # type: ignore[attr-defined]
return flow
class _CtxResolver:
"""Fake orchestrator resolver: maps source IP -> bottle id, and grants the
same allow-list to any attributed bottle (unattributed -> deny)."""
def __init__(
self, ip_to_bottle: dict[str, str], tokens: dict[str, str] | None = None,
) -> None:
self._map = ip_to_bottle
self._tokens = tokens or {}
def resolve_policy_and_bottle_id(
self, source_ip: str, identity_token: str = "",
) -> tuple[str | None, str | None, dict[str, str]]:
del identity_token
bottle_id = self._map.get(source_ip)
policy = "routes:\n - host: api.example.com\n" if bottle_id else None
return policy, bottle_id, (self._tokens if bottle_id else {})
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Introspection endpoint # Introspection endpoint
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
@@ -303,9 +277,9 @@ class TestAuthInjection(unittest.TestCase):
route = Route(host="api.example.com", auth_scheme="Bearer", token_env="EGRESS_TOKEN_0") route = Route(host="api.example.com", auth_scheme="Bearer", token_env="EGRESS_TOKEN_0")
addon = _addon(Config(routes=(route,))) addon = _addon(Config(routes=(route,)))
flow = _Flow(_Request(host="api.example.com", headers={"authorization": "Bearer agent-faked"})) flow = _Flow(_Request(host="api.example.com", headers={"authorization": "Bearer agent-faked"}))
with patch.dict("os.environ", {"EGRESS_TOKEN_0": "real-gateway-token"}): with patch.dict("os.environ", {"EGRESS_TOKEN_0": "real-sidecar-token"}):
_run_request(addon, flow) _run_request(addon, flow)
self.assertEqual("Bearer real-gateway-token", flow.request.headers.get("authorization")) self.assertEqual("Bearer real-sidecar-token", flow.request.headers.get("authorization"))
self.assertIsNone(flow.response) self.assertIsNone(flow.response)
def test_auth_route_with_unset_env_blocks(self) -> None: def test_auth_route_with_unset_env_blocks(self) -> None:
@@ -444,8 +418,7 @@ class TestSuperviseBranch(unittest.TestCase):
with patch.object(_ea_mod, "_sv", _fake_sv("approved")): with patch.object(_ea_mod, "_sv", _fake_sv("approved")):
_run_request(addon, flow) _run_request(addon, flow)
self.assertIsNone(flow.response) # forwarded after approval self.assertIsNone(flow.response) # forwarded after approval
# Approval lands in the calling bottle's safelist (keyed by slug). self.assertIn(_OPENAI_KEY, addon.safe_tokens)
self.assertIn(_OPENAI_KEY, addon._safe_tokens_for("test-bottle"))
def test_operator_rejection_blocks(self) -> None: def test_operator_rejection_blocks(self) -> None:
addon = self._supervised_addon() addon = self._supervised_addon()
@@ -762,102 +735,5 @@ class TestLogFullRequest(unittest.TestCase):
self.assertTrue(any(e.get("event") == "egress_request" for e in logged)) self.assertTrue(any(e.get("event") == "egress_request" for e in logged))
class TestSuperviseMultiTenant(unittest.TestCase):
"""Consolidated gateway: supervise proposals + the DLP safelist are keyed
per bottle, resolved by source IP (PRD 0070)."""
def _consolidated_addon(self) -> EgressAddon:
# Static config is empty; the resolver supplies each bottle's config.
addon = _addon(Config(routes=()))
addon._resolver = cast(Any, _CtxResolver({"10.0.0.1": "bottle-a", "10.0.0.2": "bottle-b"}))
addon._token_allow_timeout = 0.05
return addon
def test_approval_is_scoped_to_the_calling_bottle(self) -> None:
addon = self._consolidated_addon()
# bottle-a (10.0.0.1) sends the token; the operator approves.
flow = _with_client_ip(
_Flow(_Request(host="api.example.com", method="POST", body=f"k={_OPENAI_KEY}")),
"10.0.0.1",
)
with patch.object(_ea_mod, "_sv", _fake_sv("approved")):
_run_request(addon, flow)
self.assertIsNone(flow.response) # forwarded after approval
# The approval lands ONLY in bottle-a's safelist — never bottle-b's.
# A global set here would be the cross-tenant leak this slice closes.
self.assertIn(_OPENAI_KEY, addon._safe_tokens_for("bottle-a"))
self.assertNotIn(_OPENAI_KEY, addon._safe_tokens_for("bottle-b"))
def test_proposal_is_attributed_to_the_source_ip_bottle(self) -> None:
addon = self._consolidated_addon()
seen: list[str] = []
fake = _fake_sv("approved")
def _capture(**kw: Any) -> Any:
seen.append(kw["bottle_slug"])
return types.SimpleNamespace(id="p")
fake.Proposal = types.SimpleNamespace(new=_capture)
flow = _with_client_ip(
_Flow(_Request(host="api.example.com", method="POST", body=f"k={_OPENAI_KEY}")),
"10.0.0.2",
)
with patch.object(_ea_mod, "_sv", fake):
_run_request(addon, flow)
self.assertEqual(["bottle-b"], seen) # proposal keyed by the resolved bottle
def test_auth_token_injected_from_resolved_tokens(self) -> None:
# The bottle's upstream token comes from /resolve (in-memory on the
# orchestrator), NOT the gateway's env — the request is forwarded with
# Authorization injected. This is the token-injection cut-over fix.
policy = (
'routes:\n - host: api.example.com\n'
' auth_scheme: "Bearer"\n token_env: "EGRESS_TOKEN_0"\n'
)
class _AuthResolver:
def resolve_policy_and_bottle_id(self, ip: str, identity_token: str = ""):
del ip, identity_token
return policy, "bottle-a", {"EGRESS_TOKEN_0": "sk-secret"}
addon = _addon(Config(routes=()))
addon._resolver = cast(Any, _AuthResolver())
flow = _with_client_ip(_Flow(_Request(host="api.example.com", method="GET")), "10.0.0.1")
_run_request(addon, flow)
self.assertIsNone(flow.response) # forwarded, not blocked
self.assertEqual("Bearer sk-secret", flow.request.headers.get("authorization"))
def test_authed_route_without_token_is_blocked(self) -> None:
# No token resolved for the bottle → the authed route can't inject, so
# the request is blocked (the error the cut-over originally produced).
policy = (
'routes:\n - host: api.example.com\n'
' auth_scheme: "Bearer"\n token_env: "EGRESS_TOKEN_0"\n'
)
class _NoTokenResolver:
def resolve_policy_and_bottle_id(self, ip: str, identity_token: str = ""):
del ip, identity_token
return policy, "bottle-a", {} # no tokens
addon = _addon(Config(routes=()))
addon._resolver = cast(Any, _NoTokenResolver())
flow = _with_client_ip(_Flow(_Request(host="api.example.com", method="GET")), "10.0.0.1")
_run_request(addon, flow)
self.assertIsNotNone(flow.response) # blocked — token unset
def test_unattributed_source_ip_cannot_supervise(self) -> None:
addon = self._consolidated_addon()
# 10.9.9.9 is not in the resolver map -> deny-all config, empty slug.
flow = _with_client_ip(
_Flow(_Request(host="api.example.com", method="POST", body=f"k={_OPENAI_KEY}")),
"10.9.9.9",
)
with patch.object(_ea_mod, "_sv", _fake_sv("approved")):
_run_request(addon, flow)
self.assertIsNotNone(flow.response) # blocked (no route, no supervise)
self.assertNotIn(_OPENAI_KEY, addon._safe_tokens_for(""))
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()
+25 -7
View File
@@ -5,6 +5,8 @@ integration test)."""
import tempfile import tempfile
import unittest import unittest
from pathlib import Path from pathlib import Path
from types import SimpleNamespace
from unittest.mock import patch
from tests.unit import use_bottle_root from tests.unit import use_bottle_root
from bot_bottle.backend.egress_apply import EgressApplyError from bot_bottle.backend.egress_apply import EgressApplyError
@@ -68,16 +70,32 @@ class TestApplyRoutesChange(unittest.TestCase):
self.addCleanup(self._tmp.cleanup) self.addCleanup(self._tmp.cleanup)
self.addCleanup(use_bottle_root(Path(self._tmp.name) / ".bot-bottle")) self.addCleanup(use_bottle_root(Path(self._tmp.name) / ".bot-bottle"))
def test_apply_routes_change_fails_closed_after_companion_removal(self): def test_writes_live_routes_and_signals_reload(self):
# The per-bottle companion container that live route-apply used to calls: list[list[str]] = []
# signal was removed in the companion-container removal (#385); apply now
# fails closed until the gateway-side apply lands. def fake_run(argv: list[str], **kwargs: object) -> SimpleNamespace:
with self.assertRaises(EgressApplyError) as cm: calls.append(list(argv))
applicator.apply_routes_change( return SimpleNamespace(returncode=0, stdout="", stderr="")
with patch(
"bot_bottle.backend.docker.egress_apply.subprocess.run",
side_effect=fake_run,
):
before, after = applicator.apply_routes_change(
"dev", "dev",
"routes:\n - host: google.com\n", "routes:\n - host: google.com\n",
) )
self.assertIn("consolidated gateway", str(cm.exception))
self.assertEqual("", before)
self.assertEqual("routes:\n - host: google.com\n", after)
self.assertEqual(
"routes:\n - host: google.com\n",
(Path(self._tmp.name) / ".bot-bottle/state/dev/egress/routes.yaml").read_text(encoding="utf-8"),
)
self.assertEqual(
["docker", "kill", "--signal", "HUP", "bot-bottle-sidecars-dev"],
calls[0],
)
if __name__ == "__main__": if __name__ == "__main__":
+2 -59
View File
@@ -1,10 +1,10 @@
"""Unit: fail-closed per-client egress resolution — config + context (PRD 0070).""" """Unit: resolve_client_config — fail-closed per-client egress config (PRD 0070)."""
from __future__ import annotations from __future__ import annotations
import unittest import unittest
from bot_bottle.egress_addon_core import resolve_client_config, resolve_client_context from bot_bottle.egress_addon_core import resolve_client_config
from bot_bottle.policy_resolver import PolicyResolveError from bot_bottle.policy_resolver import PolicyResolveError
@@ -49,62 +49,5 @@ class TestResolveClientConfig(unittest.TestCase):
self.assertEqual(("10.243.0.1", "tok"), r.calls[0]) self.assertEqual(("10.243.0.1", "tok"), r.calls[0])
class _FakeContextResolver:
def __init__(
self, policy: str | None = None, bottle_id: str | None = None,
raises: bool = False, tokens: dict[str, str] | None = None,
) -> None:
self._policy = policy
self._bottle_id = bottle_id
self._raises = raises
self._tokens = tokens or {}
def resolve_policy_and_bottle_id(
self, source_ip: str, identity_token: str = "",
) -> tuple[str | None, str | None, dict[str, str]]:
if self._raises:
raise PolicyResolveError("orchestrator down")
return self._policy, self._bottle_id, self._tokens
class TestResolveClientContext(unittest.TestCase):
def test_returns_config_bottle_id_and_tokens(self) -> None:
cfg, slug, tokens = resolve_client_context(
_FakeContextResolver(
policy="routes:\n - host: example.com\n", bottle_id="b1",
tokens={"EGRESS_TOKEN_0": "sekret"},
),
"10.243.0.1",
)
self.assertEqual(("example.com",), tuple(r.host for r in cfg.routes))
self.assertEqual("b1", slug)
self.assertEqual({"EGRESS_TOKEN_0": "sekret"}, tokens) # for auth injection
def test_unattributed_denies_and_empty_slug(self) -> None:
cfg, slug, tokens = resolve_client_context(
_FakeContextResolver(policy=None, bottle_id=None), "10.243.0.9",
)
self.assertEqual((), cfg.routes)
self.assertEqual("", slug) # no bottle → supervise unavailable
self.assertEqual({}, tokens)
def test_resolver_error_denies_empty_slug_no_tokens(self) -> None:
cfg, slug, tokens = resolve_client_context(
_FakeContextResolver(raises=True), "10.243.0.1",
)
self.assertEqual((), cfg.routes)
self.assertEqual("", slug)
self.assertEqual({}, tokens)
def test_unparseable_policy_denies_but_keeps_slug(self) -> None:
# A bad policy denies egress, but the bottle is still attributed (its
# supervise queue is keyed by the id, independent of route parsing).
cfg, slug, _tokens = resolve_client_context(
_FakeContextResolver(policy="routes: notalist\n", bottle_id="b2"), "10.243.0.1",
)
self.assertEqual((), cfg.routes)
self.assertEqual("b2", slug)
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()
+21 -3
View File
@@ -38,6 +38,18 @@ class TestOrphanEnumeration(unittest.TestCase):
with patch.object(fc_cleanup.subprocess, "run", return_value=_proc(returncode=1)): with patch.object(fc_cleanup.subprocess, "run", return_value=_proc(returncode=1)):
self.assertEqual([], fc_cleanup._orphan_vm_pids()) self.assertEqual([], fc_cleanup._orphan_vm_pids())
def test_sidecar_containers_sorted(self):
with patch.object(fc_cleanup.subprocess, "run",
return_value=_proc("bot-bottle-sidecars-b\nbot-bottle-sidecars-a\n")):
self.assertEqual(
["bot-bottle-sidecars-a", "bot-bottle-sidecars-b"],
fc_cleanup._sidecar_containers(),
)
def test_sidecar_containers_empty_on_failure(self):
with patch.object(fc_cleanup.subprocess, "run", return_value=_proc(returncode=1)):
self.assertEqual([], fc_cleanup._sidecar_containers())
def test_run_dirs_empty_when_absent(self): def test_run_dirs_empty_when_absent(self):
with patch.object(fc_cleanup.util, "cache_dir") as cache: with patch.object(fc_cleanup.util, "cache_dir") as cache:
cache.return_value.__truediv__.return_value.is_dir.return_value = False cache.return_value.__truediv__.return_value.is_dir.return_value = False
@@ -45,23 +57,28 @@ class TestOrphanEnumeration(unittest.TestCase):
def test_prepare_cleanup_assembles_plan(self): def test_prepare_cleanup_assembles_plan(self):
with patch.object(fc_cleanup, "_orphan_vm_pids", return_value=[7]), \ with patch.object(fc_cleanup, "_orphan_vm_pids", return_value=[7]), \
patch.object(fc_cleanup, "_sidecar_containers", return_value=["c1"]), \
patch.object(fc_cleanup, "_run_dirs", return_value=["/run/x"]): patch.object(fc_cleanup, "_run_dirs", return_value=["/run/x"]):
plan = fc_cleanup.prepare_cleanup() plan = fc_cleanup.prepare_cleanup()
self.assertEqual((7,), plan.vm_pids) self.assertEqual((7,), plan.vm_pids)
self.assertEqual(("c1",), plan.containers)
self.assertEqual(("/run/x",), plan.run_dirs) self.assertEqual(("/run/x",), plan.run_dirs)
class TestCleanupRemoval(unittest.TestCase): class TestCleanupRemoval(unittest.TestCase):
def test_cleanup_kills_and_rmtrees(self): def test_cleanup_kills_removes_and_rmtrees(self):
plan = FirecrackerBottleCleanupPlan( plan = FirecrackerBottleCleanupPlan(
vm_pids=(101,), vm_pids=(101,), containers=("bot-bottle-sidecars-x",),
run_dirs=("/run/dev-x",), run_dirs=("/run/dev-x",),
) )
with patch.object(fc_cleanup.os, "kill") as kill, \ with patch.object(fc_cleanup.os, "kill") as kill, \
patch.object(fc_cleanup.subprocess, "run") as run, \
patch.object(fc_cleanup.shutil, "rmtree") as rmtree, \ patch.object(fc_cleanup.shutil, "rmtree") as rmtree, \
patch.object(fc_cleanup, "info"): patch.object(fc_cleanup, "info"):
fc_cleanup.cleanup(plan) fc_cleanup.cleanup(plan)
kill.assert_called_once() kill.assert_called_once()
run.assert_called_once()
self.assertIn("bot-bottle-sidecars-x", run.call_args.args[0])
rmtree.assert_called_once_with("/run/dev-x", ignore_errors=True) rmtree.assert_called_once_with("/run/dev-x", ignore_errors=True)
def test_cleanup_tolerates_dead_pid(self): def test_cleanup_tolerates_dead_pid(self):
@@ -84,12 +101,13 @@ class TestCleanupPlan(unittest.TestCase):
def test_print_lists_resources(self): def test_print_lists_resources(self):
plan = FirecrackerBottleCleanupPlan( plan = FirecrackerBottleCleanupPlan(
vm_pids=(5,), run_dirs=("/r",), vm_pids=(5,), containers=("c",), run_dirs=("/r",),
) )
with patch("bot_bottle.backend.firecracker.bottle_cleanup_plan.info") as info: with patch("bot_bottle.backend.firecracker.bottle_cleanup_plan.info") as info:
plan.print() plan.print()
joined = " ".join(c.args[0] for c in info.call_args_list) joined = " ".join(c.args[0] for c in info.call_args_list)
self.assertIn("pid 5", joined) self.assertIn("pid 5", joined)
self.assertIn("container: c", joined)
self.assertIn("run dir: /r", joined) self.assertIn("run dir: /r", joined)
-42
View File
@@ -1,42 +0,0 @@
"""Unit: shared-gateway source-IP allocation (PRD 0070)."""
from __future__ import annotations
import unittest
from bot_bottle.backend.docker.gateway_net import NoFreeAddressError, next_free_ip
class TestNextFreeIp(unittest.TestCase):
def test_skips_router_and_returns_first_host(self) -> None:
# .1 is docker's router; the first assignable address is .2.
self.assertEqual("172.18.0.2", next_free_ip("172.18.0.0/16", []))
def test_skips_taken_addresses(self) -> None:
# Gateway container holds .2, a live bottle holds .3 -> next is .4.
self.assertEqual(
"172.18.0.4", next_free_ip("172.18.0.0/16", ["172.18.0.2", "172.18.0.3"]),
)
def test_taken_order_does_not_matter(self) -> None:
self.assertEqual(
"172.18.0.2", next_free_ip("172.18.0.0/16", ["172.18.0.3", "172.18.0.5"]),
)
def test_allocation_is_deterministic(self) -> None:
taken = ["172.18.0.2"]
self.assertEqual(next_free_ip("172.18.0.0/16", taken),
next_free_ip("172.18.0.0/16", taken))
def test_raises_when_subnet_exhausted(self) -> None:
# /30: hosts are .1 (router, reserved) and .2; taking .2 leaves none.
with self.assertRaises(NoFreeAddressError):
next_free_ip("10.9.9.0/30", ["10.9.9.2"])
def test_accepts_host_bits_set_cidr(self) -> None:
# A container's inspected address arrives as e.g. 172.18.0.2/16.
self.assertEqual("172.18.0.2", next_free_ip("172.18.0.5/16", []))
if __name__ == "__main__":
unittest.main()
-113
View File
@@ -1,113 +0,0 @@
"""Unit: git-gate provisioning into the running shared gateway (PRD 0070)."""
from __future__ import annotations
import unittest
from pathlib import Path
from unittest.mock import Mock, patch
from bot_bottle.backend.docker.gateway_provision import (
GatewayProvisionError,
deprovision_git_gate,
provision_git_gate,
)
from bot_bottle.git_gate import GitGatePlan, GitGateUpstream
_RUN = "bot_bottle.backend.docker.gateway_provision.run_docker"
def _proc(returncode: int = 0, stderr: str = "") -> Mock:
return Mock(returncode=returncode, stderr=stderr)
def _recorder(calls: list[list[str]]):
"""A run_docker side_effect that records argv and returns success."""
def fake(argv: list[str]) -> Mock:
calls.append(argv)
return _proc()
return fake
def _plan(*upstreams: GitGateUpstream) -> GitGatePlan:
return GitGatePlan(
slug="demo",
entrypoint_script=Path("/stage/entrypoint.sh"),
hook_script=Path("/stage/pre-receive"),
access_hook_script=Path("/stage/access-hook"),
upstreams=tuple(upstreams),
)
def _up(name: str, *, key: str = "/host/keys/id", known_hosts: str = "") -> GitGateUpstream:
return GitGateUpstream(
name=name,
upstream_url=f"ssh://git@github.com/x/{name}.git",
upstream_host="github.com",
upstream_port="22",
identity_file=key,
known_host_key="",
known_hosts_file=Path(known_hosts) if known_hosts else Path(),
)
class TestProvisionGitGate(unittest.TestCase):
def test_copies_creds_and_runs_namespaced_init(self) -> None:
calls: list[list[str]] = []
with patch(_RUN, side_effect=_recorder(calls)):
provision_git_gate("gw", "bottle1", _plan(_up("foo", known_hosts="/host/kh")))
cps = [c for c in calls if c[:2] == ["docker", "cp"]]
self.assertIn(["docker", "cp", "/host/keys/id", "gw:/git-gate/creds/bottle1/foo-key"], cps)
self.assertIn(
["docker", "cp", "/host/kh", "gw:/git-gate/creds/bottle1/foo-known_hosts"], cps,
)
# The shared (bottle-agnostic) hooks are installed into the gateway.
self.assertIn(["docker", "cp", "/stage/pre-receive", "gw:/etc/git-gate/pre-receive"], cps)
# The init script runs in the gateway, namespaced under the bottle id.
exec_scripts = [c for c in calls if c[:3] == ["docker", "exec", "gw"] and c[3] == "sh"]
self.assertEqual(1, len(exec_scripts))
self.assertIn("repo=/git/bottle1/${name}.git", exec_scripts[0][-1])
def test_omits_known_hosts_copy_when_absent(self) -> None:
calls: list[list[str]] = []
with patch(_RUN, side_effect=_recorder(calls)):
provision_git_gate("gw", "b1", _plan(_up("foo"))) # no known_hosts
creds_cps = [c for c in calls if c[:2] == ["docker", "cp"] and "/git-gate/creds/" in c[3]]
self.assertEqual(1, len(creds_cps)) # only the key, not known_hosts
self.assertTrue(creds_cps[0][3].endswith("/foo-key"))
def test_no_upstreams_is_noop(self) -> None:
with patch(_RUN) as m:
provision_git_gate("gw", "b1", _plan())
m.assert_not_called()
def test_raises_on_docker_failure(self) -> None:
with patch(_RUN, return_value=_proc(returncode=1, stderr="boom")):
with self.assertRaises(GatewayProvisionError):
provision_git_gate("gw", "b1", _plan(_up("foo")))
def test_rejects_unsafe_bottle_id_before_any_docker(self) -> None:
with patch(_RUN) as m:
with self.assertRaises(GatewayProvisionError):
provision_git_gate("gw", "../etc", _plan(_up("foo")))
m.assert_not_called() # rejected before a single docker call
class TestDeprovision(unittest.TestCase):
def test_removes_repo_and_creds(self) -> None:
with patch(_RUN, return_value=_proc()) as m:
deprovision_git_gate("gw", "b1")
argv = m.call_args.args[0]
self.assertEqual(["docker", "exec", "gw", "rm", "-rf"], argv[:5])
self.assertIn("/git/b1", argv)
self.assertIn("/git-gate/creds/b1", argv)
def test_rejects_unsafe_bottle_id(self) -> None:
with patch(_RUN) as m:
with self.assertRaises(GatewayProvisionError):
deprovision_git_gate("gw", "a/b")
m.assert_not_called()
if __name__ == "__main__":
unittest.main()

Some files were not shown because too many files have changed in this diff Show More