Compare commits

..

27 Commits

Author SHA1 Message Date
didericis 24df322c31 feat(orchestrator): slice 13a — orchestrator process lifecycle (idempotent singleton)
lint / lint (push) Successful in 2m9s
test / unit (pull_request) Successful in 1m4s
test / integration (pull_request) Successful in 20s
test / coverage (pull_request) Successful in 1m14s
First sub-slice of docker launch integration: the foundation the CLI needs
to ensure exactly one orchestrator control plane + shared gateway is up
before registering/launching bottles. Does NOT touch the real launch path
yet — it's the lifecycle primitive the cut-over slices build on.

- OrchestratorProcess.ensure_running(): idempotent singleton — returns the
  control-plane URL if a healthy one already answers /health, else spawns
  `python -m bot_bottle.orchestrator --broker docker --gateway` detached
  (start_new_session, output tee'd to <root>/orchestrator.log) and polls
  /health until healthy or timeout (OrchestratorStartError).
- The control-plane port is the singleton key: a second orchestrator can't
  bind it, so a stray double-start fails fast rather than forking a rival.
- Host-process (not container) per the PRD's dev-harness sequencing — it
  already has the host user's docker access to broker launches; the
  data-plane gateway it manages is the container.

pyright 0 errors; pylint 9.83/10; unit suite green (1714 tests; the 13
test_sidecar_init /bin/sleep errors are pre-existing NixOS-local noise).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 20:33:14 -04:00
didericis 80d75de08a feat(supervise+orchestrator): slice 12 — multi-tenant git-gate hook + supervise server
lint / lint (push) Successful in 2m2s
test / unit (pull_request) Successful in 1m0s
test / integration (pull_request) Successful in 22s
test / coverage (pull_request) Successful in 1m11s
Finishes the supervise data-plane writers for the shared gateway. Both
still keyed off a single SUPERVISE_BOTTLE_SLUG env; now each proposal is
attributed to the calling bottle, keeping slices 10/11's per-bottle model.

- git_http_backend: in consolidated mode stamp SUPERVISE_BOTTLE_SLUG=
  <bottle_id> into the CGI env. The gitleaks-allow pre-receive hook runs as
  a child of the `git http-backend` we spawn, so it inherits the stamp and
  queues under the right bottle — no change to the (fragile) embedded hook.
  The bottle id is the namespaced root's final component (slice 10), the
  same key egress uses. Single-tenant leaves the container-stamped slug.
- supervise_server: resolve the proposal slug per request by source IP when
  BOT_BOTTLE_ORCHESTRATOR_URL is set (`_attributed_config`), fail-closed —
  an unattributed / unreachable source raises rather than queue under the
  wrong or empty slug. `serve`/`main` build + attach the resolver and make
  the env slug optional in consolidated mode. Single-tenant unchanged.

Tests: a real git push proving the slug reaches the hook's env; the
supervise attribution matrix (single-tenant slug kept, source-IP bottle
bound, unattributed + resolver-error fail closed).

Remaining supervise gap (noted): websocket DLP still self.config-only.

pyright 0 errors; pylint 9.83/10; unit suite green (1705 tests; the 13
test_sidecar_init /bin/sleep errors are pre-existing NixOS-local noise).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 20:04:26 -04:00
didericis 8827c64a83 feat(supervise+orchestrator): slice 11 — per-bottle supervise queue + DLP safelist
lint / lint (push) Successful in 2m4s
test / unit (pull_request) Successful in 1m6s
test / integration (pull_request) Successful in 23s
test / coverage (pull_request) Successful in 1m15s
Consolidated egress ran every bottle through one process but keyed the
supervise proposal queue off a single SUPERVISE_BOTTLE_SLUG env and kept
one *global* DLP safelist — so in the shared gateway an operator's token
approval for bottle A would (a) be attributed to the wrong bottle and
(b) leak into bottle B's DLP scan (A's approved secret passes B's egress).
This slice keys both per bottle, resolved by source IP.

- policy_resolver: add `resolve_policy_and_bottle_id` — policy + bottle id
  in one `/resolve`, so egress keys routing *and* the supervise
  queue/safelist from a single round-trip. Fail-closed (403 -> (None,None)).
- egress_addon_core: add `resolve_client_context` (+ `ContextResolverLike`)
  returning `(Config, bottle_id)`, sharing the fail-closed parse with
  `resolve_client_config` via `_config_from_policy`.
- egress_addon: `_active_config` -> `_resolve_flow` returns `(Config, slug)`;
  `safe_tokens` set -> per-bottle `_safe_tokens_for(slug)`; the token-allow
  write/await/archive + the approved-token add all use the resolved slug.
  Single-tenant (no resolver) unchanged — slug = the env SUPERVISE_BOTTLE_SLUG.

New tests cover the resolver, the fail-closed context matrix, and the
cross-tenant isolation (an approval lands only in the calling bottle's
safelist; the proposal is keyed by the source-IP-attributed bottle;
unattributed IPs can't supervise).

Out of scope (noted): the git-gate gitleaks-allow hook + supervise_server
agent-proposal paths, and websocket DLP (still self.config-only, inert in
consolidated mode) — follow-up slices.

pyright 0 errors; pylint 9.83/10; unit suite green (1700 tests; the 13
test_sidecar_init /bin/sleep errors are pre-existing NixOS-local noise).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 19:48:29 -04:00
didericis 115a64c161 fix(git-gate): review — sandbox naming, ResolverLike Protocol, token-required note
test / unit (pull_request) Successful in 1m8s
test / integration (pull_request) Successful in 22s
test / coverage (pull_request) Successful in 1m20s
lint / lint (push) Successful in 2m7s
Addresses PR #365 review:
- Type `resolve_sandbox_root`'s resolver param as a `ResolverLike` Protocol
  (structural, `resolve_bottle_id` only) — fixes the pyright errors from
  passing a duck-typed fake resolver in tests; mirrors
  egress_addon_core.PolicyResolverLike.
- Rename `_project_root`/`project_root`/`resolve_repo_root`/
  `DEFAULT_PROJECT_ROOT` -> `_sandbox_root`/`sandbox_root`/
  `resolve_sandbox_root`/`DEFAULT_REPO_ROOT`: "sandbox" names the per-tenant
  scope; "project" was too amorphous. `GIT_PROJECT_ROOT` keeps git's own
  env-var name.
- Note the single-tenant path is transitional (stripped once every backend
  runs the consolidated gateway).
- policy_resolver: document that the optional identity token is
  transitional — the consolidated end state requires it (flips at the
  /resolve boundary once token *delivery* lands, a PRD 0070 open question).
- Split the long line in main() (was 105 cols).

pyright 0 errors; pylint 9.95/10; unit suite green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 18:47:53 -04:00
didericis 08aef30d87 feat(git-gate): source-IP-keyed multi-tenant repo namespace (PRD 0070)
lint / lint (push) Failing after 2m0s
test / unit (pull_request) Successful in 1m1s
test / integration (pull_request) Successful in 20s
test / coverage (pull_request) Successful in 1m19s
First slice of git-gate consolidation: make the smart-HTTP git backend
serve every bottle from one process, selecting each request's repo root by
the calling bottle's source IP — the same attribution invariant + resolver
the multi-tenant egress addon uses. `git daemon` can't source-IP-route per
connection, so the consolidated gateway serves git-gate over this HTTP
backend (the transport firecracker/macOS already use); wiring the docker
path onto it lands with the launch-integration slice.

- policy_resolver: factor out `_post_resolve`; add `resolve_bottle_id`
  (source IP -> bottle id, same fail-closed 403->None contract as
  `resolve`) — the git-gate has no policy blob to parse, the bottle *is*
  the namespace.
- git_http_backend: `resolve_repo_root(resolver, base, source_ip, token)`
  — single-tenant passthrough when no resolver; else `<base>/<bottle_id>`,
  fail-closed on unattributed / resolver error / namespace escape.
  `BOT_BOTTLE_ORCHESTRATOR_URL` (same env as egress) flips the shared
  gateway multi-tenant; the identity header is read for attribution and
  never forwarded to the CGI. Per-repo creds + hooks scope by repo dir, so
  isolating the root per bottle isolates its creds too.

Single-tenant path unchanged (existing real-git-push tests green). New unit
coverage for the resolver + repo-root selection matrix. Full unit suite
green (1690 tests; the 13 test_sidecar_init /bin/sleep errors are the
pre-existing NixOS-local noise).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 18:31:03 -04:00
didericis e0b0429cd1 refactor(orchestrator): rename Sidecar -> Gateway for the consolidated data plane
test / unit (pull_request) Successful in 1m10s
test / integration (pull_request) Successful in 25s
test / coverage (pull_request) Successful in 1m17s
lint / lint (push) Successful in 2m11s
Retire "sidecar" for the consolidated per-host path (PRD 0070 naming
decision): the orchestrator is the umbrella/control plane, and the
egress/git/supervise data-plane unit it runs is the "gateway".

- git mv sidecar.py -> gateway.py and the two integration + one unit test
  files; DockerSidecar->DockerGateway, Sidecar->Gateway,
  SidecarError->GatewayError, SIDECAR_*->GATEWAY_*, ensure_sidecar->
  ensure_gateway, sidecar_status->gateway_status, container name
  bot-bottle-orch-sidecar->bot-bottle-orch-gateway.
- Prose rename across broker/registry/egress/policy_resolver + PRD 0070.
- Preserved: the image name bot-bottle-sidecars, the
  BOT_BOTTLE_SIDECAR_IMAGE env var, Dockerfile.sidecars, and PRD 0069's
  own stage-name cross-references (that doc still uses "sidecar").

No behavior change. Full unit suite green (1679 tests; the 13
test_sidecar_init /bin/sleep errors are pre-existing NixOS-local noise).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 18:12:17 -04:00
didericis d3e08cf039 feat(orchestrator+egress): slice 8 — multi-tenant egress via the resolver (#352)
lint / lint (push) Successful in 2m8s
test / unit (pull_request) Successful in 1m1s
test / integration (pull_request) Successful in 25s
test / coverage (pull_request) Successful in 1m25s
The egress addon now selects each request's Config by the calling bottle's
source IP, so one shared sidecar serves every bottle. Opt-in and fail-closed;
single-tenant behaviour is unchanged.

Orchestrator side (source-IP-primary attribution, per the PRD invariant):
  * registry: `by_source_ip` (the single active bottle at a source IP —
    network-layer attribution); `attribute` now composes it + the token.
  * service: `resolve(source_ip, token="")` — with a token, strict
    attribution; without, source IP alone.
  * control_plane: `POST /resolve`'s identity_token is now OPTIONAL (absent
    → source-IP-only); split cleanly from the token-required `/attribute`.
  * policy_resolver: `resolve` token now optional.

Egress side:
  * egress_addon_core: `resolve_client_config(resolver, client_ip, token)` —
    fetches + parses the client's Config, **fail-closed**: unattributed, a
    resolver error, or an unparseable policy all yield deny-all (no routes).
    Host-testable; `PolicyResolverLike` Protocol keeps it import-free.
  * egress_addon: consolidated mode when `BOT_BOTTLE_ORCHESTRATOR_URL` is
    set → `_active_config(flow)` resolves per client IP (reads + strips the
    `x-bot-bottle-identity` header); `request()` uses it. Unset → the static
    routes file, exactly as before. `PolicyResolver` added to the bundle.

Security note: source-IP-only resolution is safe where the IP is unspoofable
(Firecracker /31 + nft) AND the control plane is reachable only by the
trusted sidecar; the identity token, when the agent injects it, strengthens
it on weaker backends.

Scope note: the egress data plane is now multi-tenant. Remaining to be fully
live: the network topology routing every bottle's proxy to the one shared
sidecar, git-gate multitenancy, and agent-side identity-token injection.

Tests: registry by_source_ip; orchestrator resolve (with/without token);
control-plane /resolve token-optional; resolver token-optional;
resolve_client_config fail-closed matrix. All 182 egress tests still pass
(single-tenant unchanged). Full suite green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 17:48:46 -04:00
didericis de9da29027 refactor(orchestrator): drop the PolicyResolver cache (#352)
lint / lint (push) Successful in 2m0s
test / unit (pull_request) Successful in 1m1s
test / integration (pull_request) Successful in 20s
test / coverage (pull_request) Successful in 1m12s
Review: the resolver is called rarely enough that a round-trip doesn't
matter, and correctness beats speed — always fetching means a revocation,
policy change, or teardown the orchestrator knows about is honored
immediately instead of lingering for a cache TTL. Remove the TTL cache
(and `invalidate`); `resolve` now hits the orchestrator every call. Noted
in the docstring that any future caching should use orchestrator-driven
invalidation, not a blind TTL.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 17:32:05 -04:00
didericis 72ea500342 feat(orchestrator): slice 7 — sidecar-side PolicyResolver (#352)
lint / lint (push) Successful in 2m2s
test / unit (pull_request) Successful in 1m3s
test / integration (pull_request) Successful in 22s
test / coverage (pull_request) Successful in 1m9s
The data-plane bridge that lets the consolidated sidecar apply each
bottle's policy per request. `PolicyResolver` resolves a client's policy
from the orchestrator's `POST /resolve` keyed on (source_ip, identity
token) and caches it briefly (short TTL) so it isn't a round-trip per
request; `invalidate()` drops an entry on teardown / live reload.

Fail-closed: an unattributed client (orchestrator answers 403) resolves to
None so the caller denies; unreachable / unexpected status raises so the
caller can fail closed too rather than serve stale/empty policy. Stdlib
only and free of bot-bottle imports, so it can be COPYed flat into the
sidecar bundle.

Scope note: this is the sidecar-side *client*. Wiring it into the live
egress mitmproxy addon (select `Config` per client IP in the request path)
and git-gate, plus routing all bottles' egress to the one shared sidecar,
are the remaining data-plane pieces — a heavier change to the sidecar
bundle's adversarial-input code, taken next.

Tests: resolve returns/caches/expires/invalidates; 403 -> None (fail
closed); other HTTP status + unreachable raise; missing policy -> empty;
posts source_ip + identity_token. Full suite green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 17:21:27 -04:00
didericis f754c575d7 feat(orchestrator): slice 6 — source-IP-keyed multi-tenant policy (#352)
lint / lint (push) Successful in 2m1s
test / unit (pull_request) Successful in 1m5s
test / integration (pull_request) Successful in 21s
test / coverage (pull_request) Successful in 1m16s
The orchestrator side of the multi-tenant consolidated sidecar: hold each
bottle's sidecar policy and serve it by verified source IP, with live
reload. One shared sidecar can now get per-bottle config keyed on who's
calling.

  * registry.py — a `policy` column (migration v3, opaque JSON the sidecar
    interprets) on BottleRecord; `register(..., policy=)` stores it,
    `set_policy(bottle_id, policy)` updates it live, and `attribute` returns
    it (the source-IP-keyed resolution).
  * service.py — `launch_bottle(..., policy=)` and `set_policy`.
  * control_plane.py — `POST /bottles` accepts `policy`; `PUT
    /bottles/<id>/policy` live-reloads it; `POST /resolve` returns
    {bottle_id, policy} for a verified (source_ip, token) — the per-request
    call the multi-tenant sidecar makes; `/attribute` stays identity-only.

Scope note: this is the control-plane / state half. The data-plane half —
the egress mitmproxy addon (and git-gate) selecting allowlist / DLP /
token-injection per client IP by calling `/resolve` — is the next slice
(route agent bottles through the shared sidecar). The orchestrator stays
policy-agnostic: it stores and serves the blob verbatim.

Tests: registry policy store/update/persist; Orchestrator launch-with-policy
+ live set_policy; control-plane resolve returns policy (403 on bad token),
PUT policy updates / 404 / 400. Verified live over HTTP (launch -> resolve
-> PUT reload -> resolve reflects). Full suite green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 17:09:32 -04:00
didericis a092d00312 feat(orchestrator): slice 5 — build the consolidated sidecar bundle image (#352)
lint / lint (push) Successful in 2m7s
test / unit (pull_request) Successful in 1m5s
test / integration (pull_request) Successful in 20s
test / coverage (pull_request) Successful in 1m12s
Answers "where do we build the consolidated sidecar": nowhere, until now.

  * sidecar.py — `Sidecar.ensure_built()` (default no-op) + `DockerSidecar`
    now defaults its image to the real bundle (`bot-bottle-sidecars`) and
    `ensure_built()` builds it from `Dockerfile.sidecars` when
    `docker image inspect` shows it's missing (no-op when present or when no
    dockerfile is configured, e.g. a pre-pulled image). `image_exists()`
    added.
  * service.py — `ensure_sidecar()` now builds then runs.
  * __main__.py — `--sidecar` runs the consolidated bundle (build-if-missing).

Scope note: this builds + launches the bundle *container*; making the
running instance functional across bottles needs the per-bottle,
source-IP-keyed multi-tenant config + registration/reload, and routing
agent bottles to it — the next slices (added to PRD 0070's roadmap).

Tests: unit (docker mocked) — image_exists, ensure_built builds when
missing / no-op when present / no-op without a dockerfile / raises on build
failure; ensure_sidecar builds-then-runs; integration (gated, no heavy
build) — image_exists reflects real docker state. Full suite green (only
pre-existing /bin/sleep errors).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 17:00:18 -04:00
didericis 67652899eb refactor(orchestrator): move run_docker to a lean top-level docker_cmd (#352)
lint / lint (push) Successful in 2m2s
test / unit (pull_request) Successful in 1m0s
test / integration (pull_request) Successful in 18s
test / coverage (pull_request) Successful in 1m7s
Review feedback: don't bury a parallel docker util in the orchestrator.
But reusing backend.docker.util (docker_mod) isn't right either — importing
it runs backend/__init__.py, which eagerly loads all three backends
(docker + firecracker + macos) plus the manifest/egress/git-gate/supervise
framework (~76 modules), so every orchestrator import would drag the whole
backend layer in.

Compromise: promote the helper to a top-level, framework-free
bot_bottle/docker_cmd.py (single stdlib import), a proper shared home the
orchestrator's docker components use now and backend.docker.util can adopt
later. Verified `import bot_bottle.orchestrator` stays lean (12 modules, no
firecracker/macos backends).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 16:24:37 -04:00
didericis f85cbdeebf feat(orchestrator): slice 4 — consolidated per-host sidecar (#352)
lint / lint (push) Successful in 1m58s
test / unit (pull_request) Successful in 58s
test / integration (pull_request) Successful in 18s
test / coverage (pull_request) Successful in 1m7s
The core consolidation win: one persistent sidecar per host, shared by
every bottle, instead of a sidecar bundle per bottle. Safe to share
because the attribution invariant (source IP + identity token) lets the
sidecar map each request to the right bottle.

  * orchestrator/sidecar.py — a backend-neutral `Sidecar` lifecycle
    contract (mirrors LaunchBroker) + a `DockerSidecar` impl. The defining
    behaviour is idempotent singleton: `ensure_running` starts the instance
    if absent and is a no-op if it's already up, so N launches never spawn
    N sidecars; `stop` is idempotent.
  * orchestrator/dockerutil.py — a shared `run_docker` helper; DockerBroker
    now uses it too (DRY with slice 3).
  * service.py — the Orchestrator holds an optional `Sidecar`, exposes
    `ensure_sidecar()` + `sidecar_status()`.
  * control_plane.py — `GET /sidecar` reports it; __main__ gains
    `--sidecar-image` and ensures the single sidecar on startup.

Tests: unit (docker mocked) — is_running, ensure idempotent (no-op when up,
starts when absent), failure raises, stop idempotent; Orchestrator sidecar
wiring/status; control-plane /sidecar; integration (gated) — ensure is a
real idempotent singleton (one container after two ensures), stop removes.
Full suite green (only pre-existing /bin/sleep errors); integration
verified locally against real docker.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 15:28:29 -04:00
didericis 44e611d14e fix(orchestrator): pyright — pass explicit LaunchRequest in the docker integration test (#352)
lint / lint (push) Successful in 1m58s
test / unit (pull_request) Successful in 59s
test / integration (pull_request) Successful in 20s
test / coverage (pull_request) Successful in 1m6s
The **kw unpacking put str into slot: int|None. Pass explicit
LaunchRequests instead.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 15:05:55 -04:00
didericis 4fb0f64249 feat(orchestrator): slice 3 — real Docker launch broker (#352)
lint / lint (push) Failing after 1m59s
test / unit (pull_request) Successful in 58s
test / integration (pull_request) Successful in 22s
test / coverage (pull_request) Successful in 1m6s
The first concrete LaunchBroker, proving the orchestrator -> backend seam
on the cheapest backend (the sidecar bundle is already containers):

  * orchestrator/docker_broker.py — DockerBroker runs a container on a
    verified launch (`docker run --detach --name <bottle> --label ...
    <image_ref>`) and removes it on teardown (`docker rm --force`,
    idempotent on an already-absent container). The argv is built only from
    the request's static ids/flags, so nothing free-form reaches docker;
    provenance/schema verification is inherited from LaunchBroker.submit.
  * __main__.py gains `--broker {stub,docker}` so the harness can drive real
    containers.

Slice 3 launches a single container from image_ref (the seam); the full
agent + sidecar bundle is a later slice.

Tests: unit (docker mocked) — argv from static fields, launch/teardown call
the right commands, missing-image and docker-failure raise, teardown
idempotent on missing, forged token never touches docker; integration
(gated on a reachable daemon) — launch creates a real container, teardown
removes it. Full suite green (only pre-existing /bin/sleep errors);
integration verified locally against real docker.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 15:02:11 -04:00
didericis 9226d45041 feat(orchestrator): slice 2 — launch lifecycle + signed launch-broker (#352)
lint / lint (push) Successful in 2m2s
test / unit (pull_request) Successful in 57s
test / integration (pull_request) Successful in 18s
test / coverage (pull_request) Successful in 1m6s
Second slice of PRD 0070, still a backend-neutral dev-harness:

  * orchestrator/broker.py — the launch-broker contract. A LaunchRequest is
    structured (static ids/flags only — bottle id, pool slot, a
    content-addressed image_ref; never a path/argv) and signed as a compact
    HS256 JWT so the broker verifies PROVENANCE before acting: a compromised
    co-located component can't forge a launch without the shared secret.
    verify_request is fail-closed (bad sig / malformed / off-schema -> raise).
    Stdlib only (no runtime deps). Ships a StubBroker that records verified
    requests for the harness/tests.
  * orchestrator/service.py — the Orchestrator: owns the registry and brokers
    the lifecycle. launch_bottle mints the bottle + sends a signed launch,
    rolling the registry entry back if the launch fails (no orphans);
    teardown_bottle brokers teardown then deregisters; attribute delegates.
  * control_plane.py — POST /bottles now launches, DELETE tears down (both go
    through the Orchestrator + broker). dispatch/server take an Orchestrator.
  * __main__.py wires an ephemeral secret + StubBroker for the harness.

Tests: broker sign/verify round-trip, tamper/wrong-secret/malformed/off-schema
rejection, StubBroker fail-closed; Orchestrator launch->registry->attribute,
teardown, rollback-on-broker-failure; control-plane updated for launch/teardown.
Full suite green (only the pre-existing /bin/sleep errors); harness does
launch -> attribute -> teardown over HTTP.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 14:42:35 -04:00
didericis 6847fdf0ab refactor: drop the vestigial bot_bottle_root/host_db_path re-exports (#352)
lint / lint (push) Successful in 1m58s
test / unit (pull_request) Successful in 1m0s
test / integration (pull_request) Successful in 19s
test / coverage (pull_request) Successful in 1m6s
paths is the single home now, so stop re-exporting the path helpers from
supervise: remove host_db_path from supervise's imports + __all__ (it was
re-export-only) and drop bot_bottle_root from __all__ (kept as an import,
still used by audit_dir). supervise_types was already clean. Repoint the
last readers (test_supervise imports host_db_path from paths;
test_supervise_edge calls paths.bot_bottle_root) and refresh the doc
mentions. No supervise.bot_bottle_root / supervise.host_db_path references
remain.

Behavior-preserving: full unit suite unchanged (only the pre-existing
/bin/sleep sidecar-init errors).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 14:30:13 -04:00
didericis 421d31c32f refactor: move bot_bottle_root/host_db_path to paths; kill the monkeypatch (#352)
lint / lint (push) Successful in 1m57s
test / unit (pull_request) Successful in 55s
test / integration (pull_request) Successful in 15s
test / coverage (pull_request) Successful in 1m1s
Create bot_bottle/paths.py as the canonical home for the app-root path
helpers (bot_bottle_root, host_db_path, HOST_DB_FILENAME) — foundational,
not supervise- or db-specific. `bot_bottle_root()` now honours a
BOT_BOTTLE_ROOT env override.

Repoint every consumer (supervise, supervise_types, db_store, queue_store,
audit_store, store_manager, bottle_state, cli/supervise, docker/cleanup,
orchestrator/registry) at paths; remove the definitions (and supervise's
duplicate host_db_path) and the now-dead `import sys`. Add paths.py to the
sidecar bundle (Dockerfile.sidecars) for the flat-import copies.

Tests: replace ~12 files' monkeypatching of supervise.bot_bottle_root (and
the flat/pkg/supervise_types triple-patch dance) with a single
`use_bottle_root()` helper that sets BOT_BOTTLE_ROOT — every module and
flat/package copy reads the same env var, so one override covers them all.
Net -97 lines. Behaviour-preserving: full unit suite unchanged (only the
pre-existing /bin/sleep sidecar-init errors remain).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 14:04:23 -04:00
didericis b4b4e08f62 refactor: move host_db_path/HOST_DB_FILENAME to db_store (#352)
lint / lint (push) Successful in 1m58s
test / unit (pull_request) Successful in 57s
test / integration (pull_request) Successful in 16s
test / coverage (pull_request) Successful in 1m0s
host_db_path is shared DB infrastructure, not supervise-specific, so its
canonical home is db_store (alongside DbStore). It resolves bot_bottle_root
from supervise_types lazily inside the function — no load-time cycle, and a
monkey-patch of supervise_types.bot_bottle_root still propagates.
supervise_types re-exports both names for the historical import path
(queue_store/audit_store unchanged); the orchestrator registry now imports
from db_store. Drops the now-unused `import sys` from supervise_types.

Behavior-preserving: full unit suite unchanged (only the pre-existing
/bin/sleep sidecar-init errors remain); monkeypatch propagation verified.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 13:36:19 -04:00
didericis b174442c60 feat(orchestrator): registry co-tenants the shared bot-bottle.db (#352)
lint / lint (push) Successful in 1m57s
test / unit (pull_request) Successful in 59s
test / integration (pull_request) Successful in 18s
test / coverage (pull_request) Successful in 1m5s
Per review: use one shared bot-bottle.db for all runtime state including
the registry (DbStore namespaces by schema_key), so it's one queryable
file for backup/console. default_db_path() -> host_db_path(). Drop the
unilateral WAL flip — WAL on the shared DB affects supervise/audit and is
finicky over guest shares, so it's a deliberate future change; keep a
busy_timeout for lock contention.

PRD State section updated: integrity now by SOLE ownership (only the
orchestrator opens bot-bottle.db; data plane + console reach state via the
control-plane RPC, never a file handle) rather than ro/rw mount-splitting,
which one shared file can't do. Notes the transitional caveat that the
supervise sidecar currently rw-mounts bot-bottle.db.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 13:26:03 -04:00
didericis ed7307e0e3 docs(prd): 0070 registry DB is host-resident with rw/ro access split (#352)
test / unit (pull_request) Successful in 55s
test / integration (pull_request) Successful in 16s
test / coverage (pull_request) Successful in 1m1s
Per review: the SQLite runtime-state DB lives on the host, not owned
inside the orchestrator unit. Durability (re-adoption must survive an
orchestrator restart) + integrity via access-scoping (control-plane rw,
data-plane ro) rather than location. Note the WAL-over-guest-share wrinkle
for the VM slices (may want a host-side DB owner reached over the RPC).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 13:17:05 -04:00
didericis 1702664b81 feat(orchestrator): slice 1 — registry + attribution + HTTP control plane (#352)
lint / lint (push) Successful in 1m58s
test / unit (pull_request) Successful in 56s
test / integration (pull_request) Successful in 17s
test / coverage (pull_request) Successful in 1m2s
First implementation slice of PRD 0070, the backend-neutral consolidation
core as a plain-process dev-harness (no VM packaging yet):

  * orchestrator/registry.py — SQLite (WAL) runtime-state store on the
    existing DbStore/TableMigrations base. Live bottle registry keyed by
    source IP + per-bottle identity token, with fail-closed attribution:
    a request resolves to a bottle only when its source IP AND identity
    token both match exactly one active record (unknown/ambiguous IP,
    empty token, or token mismatch all deny). Tokens are 256-bit urandom.
  * orchestrator/control_plane.py — the HTTP control plane (the universal
    transport chosen in 0070): register / deregister / list / attribute /
    health. Routing is a pure dispatch() so it is socket-free testable;
    Handler/ControlPlaneServer/make_server are a thin stdlib adapter.
    register/deregister are the live-reload path; listing redacts tokens.
  * orchestrator/__main__.py — `python -m bot_bottle.orchestrator` harness.

Launch/teardown, the launch broker, and the egress/git/supervise data
plane come in later slices. 24 unit tests (attribution matrix, persistence,
dispatch, one real-socket round-trip).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 13:00:04 -04:00
didericis 8d54fc38ea docs(prd): 0070 VM-to-VM routing is not a blocker (#352)
Per review: resolvable at implementation time (host tweak acceptable, as
the pool setup already needs on NixOS); the earlier sidecars-in-VMs spike
showed feasibility. No need to hunt the spike now.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 12:46:08 -04:00
didericis 73a7582abe docs(prd): 0070 fold in review decisions (#352)
Resolve open questions from review: consolidate egress (hardened minimal
surface + identity token + vault mitigations); HTTP control-plane
transport; broker schema = signed-JWT JSON of static flags+ids (provenance
+ un-coercible); state re-adoption procedure (singleton orchestrator →
wait-healthy → adopt via SQLite + process inspection before serving, with
write-ahead intent to close the in-flight-launch race). Add a per-bottle
identity-token defense-in-depth layer on the attribution invariant.
Remaining open: VM-to-VM routing (per-backend wire(), pending spike link),
live-reload protocol, identity-token delivery.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 12:39:06 -04:00
didericis 095896817c docs(prd): 0070 secret-handling as a future pattern (SecretProvider, #355)
lint / lint (push) Successful in 2m3s
Add a "Secret handling — FUTURE pattern (not v1)" subsection: vault as a
separate trust domain holding long-lived roots, deriving short-lived
scoped creds where the upstream allows (with the honest limit that a
compromised proxy can still abuse currently-authorized access). The
mechanism is a generic, user-extensible SecretProvider that generalizes
PRD 0048's DeployKeyProvisioner and drops into the manifest wherever a raw
token is accepted — discovered from ~/.bot-bottle/contrib like user
AgentProviders. Marked explicitly as not required for the initial
orchestrator; tracked as #355.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 12:14:25 -04:00
didericis a30dd49967 docs(prd): 0070 per-host orchestrator service
Fold the per-bottle sidecar bundle into a single persistent per-host
orchestrator: runs the sidecar functions (egress/git-gate/supervise),
coordinates with the console, and brokers agent launches. Virtualized
from the start with backend-native isolation (fc VM / apple ctr / docker
ctr), fronted by a single backend-agnostic contract; per-backend
variation lives on BottleBackend, not a parallel Orchestrator hierarchy.

Leads with the security review (secret concentration, shared fate, the
launch-broker-as-new-privileged-core, and the source-IP attribution
invariant each backend must enforce). Proposes one SQLite DB owned by the
orchestrator for runtime state (slot leases, approvals, registry) —
distinct from build-time constants (flat .env) and user config
(declarative ~/.bot-bottle). Sequences docker -> firecracker -> macos,
developing the service as a plain-process dev-harness before the VM.

Supersedes 0069's Stage-1/4 sidecar framing; depends on 0069's nix-built
fixed images. Tracking issue #351.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 12:14:25 -04:00
didericis 2b970d1170 refactor(firecracker): single-source the network-pool defaults
test / unit (pull_request) Successful in 54s
test / integration (pull_request) Successful in 15s
test / coverage (pull_request) Successful in 56s
lint / lint (push) Successful in 1m59s
test / unit (push) Successful in 53s
test / integration (push) Successful in 20s
test / coverage (push) Successful in 58s
Update Quality Badges / update-badges (push) Successful in 57s
The pool params (size, IP base, iface prefix, nft table) were triplicated
— hardcoded in netpool.py, scripts/firecracker-netpool.sh, and
nix/firecracker-netpool.nix — plus the IP math (3x) and the nft ruleset
(2x). Nothing enforced agreement; changing the base (ce3fad9, off CGNAT)
forced a coordinated three-file edit, and a missed one would silently
provision a range the launcher doesn't expect.

Collapse to one source of truth:

  * netpool.defaults.env — a plain KEY=VALUE file (bash-sourceable,
    systemd EnvironmentFile-compatible, Python- and Nix-parseable) holding
    the four defaults. A BOT_BOTTLE_FC_* env var still overrides any key.
  * netpool.py reads it for the Python defaults (missing file = hard
    error, not confusing empty defaults).
  * the shell script falls back to it (no literal `:-8` / `10.243.0.0`),
    and its `up` is now non-destructive/idempotent (only creates a
    missing TAP), so re-running never cuts a live VM.
  * the Nix module readFile-parses it for its option defaults and
    delegates bring-up to the SAME shell script (dropping its duplicate
    IP math, nft ruleset, and TAP loop) — passing every value as
    Environment= so the store-detached script never needs the file.

Net: defaults 3x -> 1x, nft ruleset 2x -> 1x, TAP loop 2x -> 1x. The one
remaining IP-math dup (Python launch-addressing vs bash bring-up) is
justified — different runtimes. Tests now guard the invariant (Python
reads the shared file; the script/module hold no literals) instead of
pinning duplicated strings.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 04:27:48 -04:00
75 changed files with 4072 additions and 549 deletions
+2
View File
@@ -64,8 +64,10 @@ COPY --from=gitleaks-src /usr/bin/gitleaks /usr/bin/gitleaks
COPY bot_bottle/egress_addon_core.py /app/egress_addon_core.py
COPY bot_bottle/egress_dlp_config.py /app/egress_dlp_config.py
COPY bot_bottle/egress_addon.py /app/egress_addon.py
COPY bot_bottle/policy_resolver.py /app/policy_resolver.py
COPY bot_bottle/dlp_detectors.py /app/dlp_detectors.py
COPY bot_bottle/yaml_subset.py /app/yaml_subset.py
COPY bot_bottle/paths.py /app/paths.py
COPY bot_bottle/migrations.py /app/migrations.py
COPY bot_bottle/db_store.py /app/db_store.py
COPY bot_bottle/supervise_types.py /app/supervise_types.py
-7
View File
@@ -61,13 +61,6 @@ class AgentProviderRuntime:
prompt_mode: PromptMode
bypass_args: tuple[str, ...]
resume_args: tuple[str, ...]
# argv run inside a throwaway container of a freshly built agent
# image, right after `build_image()`, to catch a build that
# exited 0 but produced a broken CLI (e.g. an npm
# optionalDependencies fetch for a platform-native binary that
# silently no-ops on a transient failure). Empty tuple skips the
# check — not every provider has opted in yet.
smoke_test: tuple[str, ...] = ()
@dataclass(frozen=True)
+4 -2
View File
@@ -6,11 +6,13 @@ import sqlite3
from pathlib import Path
try:
from .supervise_types import AuditEntry, host_db_path
from .supervise_types import AuditEntry
from .paths import host_db_path
from .db_store import DbStore
from .migrations import TableMigrations
except ImportError:
from supervise_types import AuditEntry, host_db_path # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from supervise_types import AuditEntry # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from paths import host_db_path # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from db_store import DbStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from migrations import TableMigrations # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
+2 -2
View File
@@ -26,7 +26,7 @@ from __future__ import annotations
import shutil
import subprocess
from ... import supervise as _supervise
from ...paths import bot_bottle_root
from ...log import info, warn
from . import util as docker_mod
from .bottle_cleanup_plan import DockerBottleCleanupPlan
@@ -94,7 +94,7 @@ def _list_orphan_state_dirs(
ANY backend — used so this docker-side check doesn't reap a
running non-docker bottle's state dir (the layout is shared
across backends)."""
state_root = _supervise.bot_bottle_root() / "state"
state_root = bot_bottle_root() / "state"
if not state_root.is_dir():
return []
orphans: list[str] = []
-4
View File
@@ -36,7 +36,6 @@ from contextlib import ExitStack, contextmanager
from pathlib import Path
from typing import Callable, Generator
from ...agent_provider import runtime_for
from ...egress import egress_resolve_token_values
from ...git_gate import (
provision_git_gate_dynamic_keys,
@@ -112,9 +111,6 @@ def launch(
plan.image, _REPO_DIR,
dockerfile=plan.dockerfile_path,
)
docker_mod.verify_agent_image(
plan.image, runtime_for(plan.agent_provider_template).smoke_test,
)
internal_network = network_mod.network_name_for_slug(plan.slug)
egress_network = network_mod.network_egress_name_for_slug(plan.slug)
+1 -57
View File
@@ -4,7 +4,6 @@ existence, and building images."""
from __future__ import annotations
import os
import re
import shutil
import subprocess
@@ -89,31 +88,6 @@ def docker_exec_root(container: str, argv: list[str]) -> None:
)
def docker_exec(container: str, argv: list[str], *, user: str = "") -> None:
"""Run `docker exec` in the named container, dying with the
command's own stderr on failure. Pass `user=\"0\"` to run as root."""
cmd = ["docker", "exec"]
if user:
cmd += ["-u", user]
cmd += [container, *argv]
result = subprocess.run(cmd, capture_output=True, text=True, check=False)
if result.returncode != 0:
die(
f"docker exec in {container} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
def docker_cp(src: str, dest: str) -> None:
"""Run `docker cp`, dying with the command's own stderr on failure."""
result = subprocess.run(
["docker", "cp", src, dest], capture_output=True, text=True, check=False,
)
if result.returncode != 0:
die(f"docker cp {src} -> {dest} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}")
_SLUG_RE = re.compile(r"[^a-z0-9]+")
@@ -134,45 +108,15 @@ def build_image(ref: str, context: str, *, dockerfile: str = "") -> None:
`dockerfile` is an optional path (relative to `context`, or
absolute) for callers that need to build from a non-default
Dockerfile in the same context — e.g. `Dockerfile.git-gate`.
Set `BOT_BOTTLE_NO_CACHE=1` (the `start --no-cache` flag) to force
`--no-cache`. The npm/curl installers some provider Dockerfiles
shell out to can silently no-op on a transient network failure —
e.g. an `optionalDependencies` fetch for a platform-native binary —
and Docker will then cache that broken layer indefinitely."""
Dockerfile in the same context — e.g. `Dockerfile.git-gate`."""
info(f"building image {ref} from {context} (layer cache keeps repeat builds fast)")
args = ["docker", "build", "-t", ref]
if os.environ.get("BOT_BOTTLE_NO_CACHE") == "1":
args.append("--no-cache")
if dockerfile:
args.extend(["-f", dockerfile])
args.append(context)
subprocess.run(args, check=True)
def verify_agent_image(image: str, argv: tuple[str, ...]) -> None:
"""Run `argv` inside a throwaway container of a freshly built agent
image and die loudly if it fails, instead of shipping an image
whose CLI only breaks at first real use. No-op when the provider
hasn't declared a smoke test (`AgentProviderRuntime.smoke_test`)."""
if not argv:
return
result = subprocess.run(
["docker", "run", "--rm", "--entrypoint", argv[0], image, *argv[1:]],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
detail = (result.stderr or result.stdout or "").strip()
die(
f"agent image {image!r} failed its post-build smoke test "
f"({' '.join(argv)}): {detail}\n"
f"Try rebuilding from scratch: bot-bottle start --no-cache"
)
# def build_image_with_cwd(
# derived: str,
# base: str,
+55 -13
View File
@@ -24,7 +24,6 @@ from contextlib import ExitStack, contextmanager
from pathlib import Path
from typing import Callable, Generator
from ...agent_provider import runtime_for
from ...bottle_state import (
egress_state_dir,
git_gate_state_dir,
@@ -43,7 +42,6 @@ from ...git_gate import (
from ...log import die, info, warn
from ...supervise import DB_PATH_IN_CONTAINER, SUPERVISE_PORT
from ...util import expand_tilde
from ..docker import util as docker_mod
from ..docker.egress import (
EGRESS_CA_IN_CONTAINER,
EGRESS_PORT,
@@ -108,9 +106,9 @@ def launch(
plan = _provision_git_gate_keys(plan)
sidecar_name = sidecar_container_name(plan.slug)
docker_mod.force_remove_container(sidecar_name)
_force_remove_container(sidecar_name)
_start_sidecar_bundle(plan, sidecar_name, slot.host_ip)
stack.callback(docker_mod.force_remove_container, sidecar_name)
stack.callback(_force_remove_container, sidecar_name)
_stage_git_gate(plan, sidecar_name)
plan = _stamp_agent_urls(plan, slot.host_ip)
@@ -172,18 +170,15 @@ def _mint_certs(plan: FirecrackerBottlePlan) -> FirecrackerBottlePlan:
def _build_agent_image(plan: FirecrackerBottlePlan) -> FirecrackerBottlePlan:
docker_mod.build_image(SIDECAR_BUNDLE_IMAGE, _REPO_DIR, dockerfile=SIDECAR_BUNDLE_DOCKERFILE)
_docker_build(SIDECAR_BUNDLE_IMAGE, _REPO_DIR, dockerfile=SIDECAR_BUNDLE_DOCKERFILE)
committed = read_committed_image(plan.slug)
if committed and docker_mod.image_exists(committed):
if committed and _image_exists(committed):
info(f"using committed image {committed!r}")
return dataclasses.replace(
plan,
agent_provision=dataclasses.replace(plan.agent_provision, image=committed),
)
docker_mod.build_image(plan.image, _REPO_DIR, dockerfile=plan.dockerfile_path)
docker_mod.verify_agent_image(
plan.image, runtime_for(plan.agent_provider_template).smoke_test,
)
_docker_build(plan.image, _REPO_DIR, dockerfile=plan.dockerfile_path)
return plan
@@ -296,15 +291,15 @@ def _stage_git_gate(plan: FirecrackerBottlePlan, sidecar_name: str) -> None:
gp = plan.git_gate_plan
if not gp.upstreams:
return
docker_mod.docker_exec(sidecar_name, [
_docker_exec(sidecar_name, [
"mkdir", "-p",
str(Path(GIT_GATE_HOOK_IN_CONTAINER).parent),
GIT_GATE_CREDS_DIR_IN_CONTAINER, "/git",
str(Path(_GIT_GATE_READY_FILE).parent),
])
for host_path, container_path in _git_gate_files(plan):
docker_mod.docker_cp(host_path, f"{sidecar_name}:{container_path}")
docker_mod.docker_exec(sidecar_name, [
_docker_cp(host_path, f"{sidecar_name}:{container_path}")
_docker_exec(sidecar_name, [
"sh", "-c",
f"chmod 755 {GIT_GATE_ENTRYPOINT_IN_CONTAINER} "
f"{GIT_GATE_HOOK_IN_CONTAINER} {GIT_GATE_ACCESS_HOOK_IN_CONTAINER} && "
@@ -360,3 +355,50 @@ def _agent_guest_env(plan: FirecrackerBottlePlan, host_ip: str) -> dict[str, str
if value is not None:
env[name] = value
return env
# --- docker helpers --------------------------------------------------
def _docker_build(ref: str, context: str, *, dockerfile: str = "") -> None:
info(f"docker build {ref}")
args = ["docker", "build", "-t", ref]
if dockerfile:
if not os.path.isabs(dockerfile):
dockerfile = os.path.join(context, dockerfile)
args += ["-f", dockerfile]
args.append(context)
result = subprocess.run(args, check=False)
if result.returncode != 0:
die(f"docker build for {ref!r} failed")
def _image_exists(ref: str) -> bool:
return subprocess.run(
["docker", "image", "inspect", ref],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
).returncode == 0
def _force_remove_container(name: str) -> None:
subprocess.run(
["docker", "rm", "-f", name],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
)
def _docker_exec(name: str, argv: list[str]) -> None:
result = subprocess.run(
["docker", "exec", name, *argv], capture_output=True, text=True, check=False,
)
if result.returncode != 0:
die(f"docker exec in {name} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}")
def _docker_cp(host_path: str, dest: str) -> None:
result = subprocess.run(
["docker", "cp", host_path, dest], capture_output=True, text=True, check=False,
)
if result.returncode != 0:
die(f"docker cp {host_path} -> {dest} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}")
@@ -0,0 +1,17 @@
# Firecracker network-pool defaults — the SINGLE source of these values.
#
# Read by every consumer so they can't drift:
# * netpool.py — parses this for the Python defaults (below).
# * scripts/firecracker-netpool.sh — falls back to these when the
# matching BOT_BOTTLE_FC_* env var is unset.
# * nix/firecracker-netpool.nix — readFile-parses this for its option
# defaults, then passes the resolved values back as Environment=.
#
# Plain KEY=VALUE (no quoting, no inline comments, no spaces around `=`)
# so it is bash-sourceable, systemd EnvironmentFile-compatible, and
# trivially parseable from Python and Nix. A real BOT_BOTTLE_FC_* env
# var of the same name always overrides the value here.
BOT_BOTTLE_FC_POOL_SIZE=8
BOT_BOTTLE_FC_IP_BASE=10.243.0.0
BOT_BOTTLE_FC_IFACE_PREFIX=bbfc
BOT_BOTTLE_FC_NFT_TABLE=bot_bottle_fc
+40 -9
View File
@@ -4,11 +4,13 @@ config renderers (shell command + NixOS module) shown to operators.
The Firecracker backend needs a privileged one-time network setup:
a pool of point-to-point TAP devices (owned by the invoking user, so
`./cli.py start` never needs root) and a dedicated nftables table that
isolates every VM. This module is the single source of truth for the
pool parameters — the shell script (`scripts/firecracker-netpool.sh`),
the NixOS module (`nix/firecracker-netpool.nix`), and the backend's
fail-closed preflight all derive from these constants so they can't
drift.
isolates every VM. The pool parameters live in exactly one place —
`netpool.defaults.env`, a plain KEY=VALUE file next to this module —
and every consumer reads *that*: this module (below), the shell script
(`scripts/firecracker-netpool.sh`), and the NixOS module
(`nix/firecracker-netpool.nix`). A `BOT_BOTTLE_FC_*` env var of the
same name always overrides the file, and the backend's fail-closed
preflight derives from these accessors, so nothing can drift.
Topology (per slot i):
* TAP ``bbfc{i}`` — no shared bridge, so no docker0 / virbr0 / cni0
@@ -43,18 +45,47 @@ from typing import IO
from ...log import die
# The pool defaults live in one shared file (see module docstring); the
# shell script and NixOS module read the same file, so the values can't
# drift. This is a packaged data file — a missing/broken install is a
# hard error, surfaced here rather than as confusing empty defaults.
DEFAULTS_FILE = Path(__file__).with_name("netpool.defaults.env")
def _load_defaults() -> dict[str, str]:
out: dict[str, str] = {}
for raw in DEFAULTS_FILE.read_text().splitlines():
line = raw.strip()
if not line or line.startswith("#") or "=" not in line:
continue
key, _, value = line.partition("=")
out[key.strip()] = value.strip()
return out
_DEFAULTS = _load_defaults()
def _cfg(key: str) -> str:
"""A `BOT_BOTTLE_FC_*` env var overrides the shared-file default."""
try:
return os.environ.get(key) or _DEFAULTS[key]
except KeyError:
die(f"{key} is missing from {DEFAULTS_FILE.name} (broken install)")
# Interface names are capped at 15 chars (IFNAMSIZ-1); "bbfc" + a small
# index stays well under that and is distinctive enough to grep for.
IFACE_PREFIX = os.environ.get("BOT_BOTTLE_FC_IFACE_PREFIX", "bbfc")
NFT_TABLE = "bot_bottle_fc"
IFACE_PREFIX = _cfg("BOT_BOTTLE_FC_IFACE_PREFIX")
NFT_TABLE = _cfg("BOT_BOTTLE_FC_NFT_TABLE")
def pool_size() -> int:
return int(os.environ.get("BOT_BOTTLE_FC_POOL_SIZE", "8"))
return int(_cfg("BOT_BOTTLE_FC_POOL_SIZE"))
def ip_base() -> str:
return os.environ.get("BOT_BOTTLE_FC_IP_BASE", "10.243.0.0")
return _cfg("BOT_BOTTLE_FC_IP_BASE")
# Sidecar ports the VM reaches at its host-side TAP IP. Kept in sync
@@ -17,7 +17,6 @@ from contextlib import ExitStack, contextmanager
from pathlib import Path
from typing import Callable, Generator
from ...agent_provider import runtime_for
from ...bottle_state import (
egress_state_dir,
git_gate_state_dir,
@@ -170,9 +169,6 @@ def _build_images(plan: MacosContainerBottlePlan) -> MacosContainerBottlePlan:
_REPO_DIR,
dockerfile=plan.dockerfile_path,
)
container_mod.verify_agent_image(
plan.image, runtime_for(plan.agent_provider_template).smoke_test,
)
return plan
+1 -31
View File
@@ -60,21 +60,13 @@ def dns_server() -> str:
def build_image(ref: str, context: str, *, dockerfile: str = "") -> None:
"""Build an OCI image with Apple's BuildKit-backed `container build`.
Set `BOT_BOTTLE_NO_CACHE=1` (the `start --no-cache` flag) to force
`--no-cache`. The npm/curl installers some provider Dockerfiles
shell out to can silently no-op on a transient network failure —
e.g. an `optionalDependencies` fetch for a platform-native binary —
and the builder will then cache that broken layer indefinitely."""
"""Build an OCI image with Apple's BuildKit-backed `container build`."""
info(
f"building image {ref} from {context} with Apple Container "
"(layer cache keeps repeat builds fast)"
)
_ensure_builder_dns()
args = [_CONTAINER, "build", "-t", ref, "--dns", dns_server()]
if os.environ.get("BOT_BOTTLE_NO_CACHE") == "1":
args.append("--no-cache")
if dockerfile:
# `container build` resolves -f relative to the current working
# directory, not the build context. Anchor a relative Dockerfile to
@@ -86,28 +78,6 @@ def build_image(ref: str, context: str, *, dockerfile: str = "") -> None:
subprocess.run(args, check=True)
def verify_agent_image(image: str, argv: tuple[str, ...]) -> None:
"""Run `argv` inside a throwaway container of a freshly built agent
image and die loudly if it fails, instead of shipping an image
whose CLI only breaks at first real use. No-op when the provider
hasn't declared a smoke test (`AgentProviderRuntime.smoke_test`)."""
if not argv:
return
result = subprocess.run(
[_CONTAINER, "run", "--rm", "--entrypoint", argv[0], image, *argv[1:]],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
detail = (result.stderr or result.stdout or "").strip()
die(
f"agent image {image!r} failed its post-build smoke test "
f"({' '.join(argv)}): {detail}\n"
f"Try rebuilding from scratch: bot-bottle start --no-cache"
)
def commit_container(container_name: str, image_tag: str) -> None:
"""Snapshot a running Apple Container as a local image.
+2 -2
View File
@@ -36,7 +36,7 @@ from dataclasses import dataclass
from pathlib import Path
from typing import cast
from . import supervise as _supervise
from .paths import bot_bottle_root
# Directory layout: ~/.bot-bottle/state/<identity>/...
@@ -163,7 +163,7 @@ def read_metadata(identity: str) -> BottleMetadata | None:
def bottle_state_dir(identity: str) -> Path:
"""Per-bottle state directory on the host. Created lazily by the
write helpers; readers tolerate its absence."""
return _supervise.bot_bottle_root() / _STATE_SUBDIR / identity
return bot_bottle_root() / _STATE_SUBDIR / identity
def per_bottle_dockerfile_path(identity: str) -> Path:
-16
View File
@@ -46,17 +46,6 @@ def cmd_start(argv: list[str]) -> int:
parser = argparse.ArgumentParser(prog=f"{PROG} start", add_help=True)
parser.add_argument("--dry-run", action="store_true")
parser.add_argument("--cwd", action="store_true", help="copy host cwd into the running bottle")
parser.add_argument(
"--no-cache",
action="store_true",
help=(
"rebuild agent/sidecar images from scratch, bypassing the "
"build layer cache. Use when an image looks broken after a "
"dependency bump — e.g. an installer's optionalDependencies "
"fetch silently no-op'd on a transient failure and got baked "
"into a cached layer."
),
)
parser.add_argument(
"--backend",
choices=known_backend_names(),
@@ -108,11 +97,6 @@ def cmd_start(argv: list[str]) -> int:
args = parser.parse_args(argv)
dry_run = args.dry_run or os.environ.get("BOT_BOTTLE_DRY_RUN") == "1"
if args.no_cache or os.environ.get("BOT_BOTTLE_NO_CACHE") == "1":
# Read by build_image() in each backend's util module — set here
# so both the interactive and --headless paths pick it up without
# threading a no_cache field through every backend's plan dataclass.
os.environ["BOT_BOTTLE_NO_CACHE"] = "1"
manifest = ManifestIndex.resolve(USER_CWD)
backend_name: str | None = args.backend
+2 -2
View File
@@ -19,7 +19,7 @@ from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
from .. import supervise as _supervise
from ..paths import bot_bottle_root
from ..bottle_state import read_metadata
from ..backend.docker.egress_apply import (
EgressApplyError,
@@ -276,7 +276,7 @@ def _write_crash_log(exc: BaseException) -> Path:
)
entry = f"=== supervise crash {stamp} ===\n{body}\n"
try:
log_dir = _supervise.bot_bottle_root() / "logs"
log_dir = bot_bottle_root() / "logs"
log_dir.mkdir(parents=True, exist_ok=True)
path = log_dir / "supervise-crash.log"
with path.open("a", encoding="utf-8") as fh:
@@ -91,7 +91,6 @@ _RUNTIME = AgentProviderRuntime(
prompt_mode="append_file",
bypass_args=("--dangerously-skip-permissions",),
resume_args=("--continue",),
smoke_test=("claude", "--version"),
)
@@ -61,7 +61,6 @@ _RUNTIME = AgentProviderRuntime(
prompt_mode="read_prompt_file",
bypass_args=("--dangerously-bypass-approvals-and-sandbox",),
resume_args=("resume", "--last"),
smoke_test=(_CODEX_CLI, "--version"),
)
+27
View File
@@ -0,0 +1,27 @@
"""Lean, framework-free `docker` subprocess primitive.
Deliberately a top-level module with a single stdlib import so it can be
reused from anywhere without cost. It is intentionally *not* placed in
`backend.docker.util`: importing that module runs `backend/__init__.py`,
which eagerly loads all three bottle backends (docker + firecracker +
macos) plus the manifest/egress/git-gate/supervise framework ~76 modules
which would drag the whole backend layer into the deliberately-lean
orchestrator. This primitive stays free of that so both the orchestrator's
docker components and (in time) `backend.docker.util` can share it."""
from __future__ import annotations
import subprocess
def run_docker(argv: list[str]) -> subprocess.CompletedProcess[str]:
"""Run a `docker` command, capturing stdout/stderr as text. Never raises
on a non-zero exit callers inspect `returncode` / `stderr` so they can
stay fail-closed or tolerate idempotent no-ops (e.g. removing an
already-absent container)."""
return subprocess.run(
argv, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, check=False,
)
__all__ = ["run_docker"]
+88 -25
View File
@@ -1,4 +1,4 @@
"""mitmproxy addon entrypoint for the egress sidecar (PRD 0017, PRD 0053).
"""mitmproxy addon entrypoint for the egress gateway (PRD 0017, PRD 0053).
Loaded by `mitmdump -s /app/egress_addon.py` inside the
egress container."""
@@ -32,6 +32,7 @@ from egress_addon_core import ( # type: ignore[import-not-found] # pylint: dis
is_git_push_request,
load_config,
match_route,
resolve_client_context,
outbound_scan_headers,
route_to_yaml_dict,
scan_inbound,
@@ -51,11 +52,26 @@ try:
except ImportError: # pragma: no cover - host-side path
from bot_bottle import supervise as _sv # type: ignore[import-not-found]
try:
from policy_resolver import PolicyResolver # type: ignore[import-not-found]
except ImportError: # pragma: no cover - host-side path
from bot_bottle.policy_resolver import PolicyResolver
DEFAULT_ROUTES_PATH = "/etc/egress/routes.yaml"
INTROSPECT_HOST = "_egress.local"
# Consolidated (multi-tenant) mode: when this points at the per-host
# orchestrator's control plane, the addon resolves each client's Config by
# source IP per request instead of using a single static routes file. Unset
# → legacy per-bottle single-tenant mode (unchanged).
ORCHESTRATOR_URL_ENV = "BOT_BOTTLE_ORCHESTRATOR_URL"
# App-layer identity token (defense-in-depth over the source-IP invariant);
# the agent injects it, the addon strips it so it never leaks upstream.
IDENTITY_HEADER = "x-bot-bottle-identity"
# Seconds the egress proxy holds a token-blocked request open waiting for the
# operator's supervisor decision (PRD 0062), overridable via env.
DEFAULT_TOKEN_ALLOW_TIMEOUT_SECONDS = 300.0
@@ -72,20 +88,40 @@ _TOKEN_ALLOW_JUSTIFICATION = (
class EgressAddon:
# Class default so addons built via __new__ (e.g. in tests) default to
# single-tenant; __init__ sets the instance attribute for real runs.
_resolver: "PolicyResolver | None" = None
def __init__(self) -> None:
self.routes_path = os.environ.get("EGRESS_ROUTES", DEFAULT_ROUTES_PATH)
self.config: Config = Config(routes=())
# Tokens the operator has approved this session (PRD 0062). In-memory
# only — a restart re-prompts. Mutated only from the asyncio loop that
# runs the addon hooks, so no lock is needed.
self.safe_tokens: set[str] = set()
# Consolidated mode: resolve per-client Config from the orchestrator.
# Absent → single-tenant (static routes file); behaviour unchanged.
orch_url = os.environ.get(ORCHESTRATOR_URL_ENV, "").strip()
self._resolver = PolicyResolver(orch_url) if orch_url else None
# Tokens the operator has approved this session (PRD 0062), keyed by
# bottle so the shared gateway keeps each bottle's safelist separate —
# a global set would let bottle A's approved secret pass bottle B's DLP
# scan. In-memory only (a restart re-prompts); mutated only from the
# asyncio loop that runs the addon hooks, so no lock is needed.
self._safe_tokens: dict[str, set[str]] = {}
self._supervise_slug = os.environ.get("SUPERVISE_BOTTLE_SLUG", "").strip()
self._token_allow_timeout = _token_allow_timeout_from_env(os.environ)
self._reload(initial=True)
self._install_sighup()
def _supervise_available(self) -> bool:
return bool(self._supervise_slug)
@staticmethod
def _supervise_available(slug: str) -> bool:
"""Supervise is reachable for this request iff we resolved a bottle to
attribute its proposals to (single-tenant env slug, or a source-IP
-attributed bottle id). Empty fail closed (no queue to write to)."""
return bool(slug)
def _safe_tokens_for(self, slug: str) -> set[str]:
"""This bottle's operator-approved DLP safelist (PRD 0062), created on
first use. Keyed by bottle so the shared gateway never leaks one
bottle's approved token into another's scan."""
return self._safe_tokens.setdefault(slug, set())
def _reload(self, *, initial: bool = False) -> None:
try:
@@ -194,6 +230,22 @@ class EgressAddon:
+ "\n"
)
def _resolve_flow(self, flow: http.HTTPFlow) -> "tuple[Config, str]":
"""The `(Config, supervise slug)` to apply to this request. Single-tenant
the static `self.config` and the env slug. Consolidated the calling
bottle's Config and its bottle id, resolved by source IP in one
round-trip (fail-closed to deny-all + empty slug if unattributed). The
identity token, if the agent injected one, is read then stripped so it
never leaks upstream. The slug keys both the proposal queue and the
per-bottle safelist, so an approval only ever affects its own bottle."""
if self._resolver is None:
return self.config, self._supervise_slug
conn = flow.client_conn
client_ip = conn.peername[0] if conn and conn.peername else ""
token = flow.request.headers.get(IDENTITY_HEADER, "")
flow.request.headers.pop(IDENTITY_HEADER, None)
return resolve_client_context(self._resolver, client_ip, token)
async def request(self, flow: http.HTTPFlow) -> None:
request_path, _, query = flow.request.path.partition("?")
@@ -201,12 +253,14 @@ class EgressAddon:
self._serve_introspection(flow, request_path)
return
config, slug = self._resolve_flow(flow)
# DLP outbound scan BEFORE stripping auth — catches tokens the
# agent tried to smuggle in any header, path, query param, or body.
# Hostname is included to catch DNS-tunnelling exfiltration attempts.
route = match_route(self.config.routes, flow.request.pretty_host)
route = match_route(config.routes, flow.request.pretty_host)
if route is not None:
if not await self._handle_outbound_dlp(flow, route):
if not await self._handle_outbound_dlp(flow, route, slug):
return
# The redact policy may have rewritten the request line; recompute
# the path/query the git checks below rely on.
@@ -224,7 +278,7 @@ class EgressAddon:
if is_git_fetch_request(request_path, query):
git_decision = decide_git_fetch(
self.config.routes, flow.request.pretty_host,
config.routes, flow.request.pretty_host,
)
if git_decision.action == "block":
self._block(
@@ -235,14 +289,14 @@ class EgressAddon:
return
# Strip agent-set Authorization after DLP scan so smuggled tokens
# are caught above; the route may inject sidecar-owned auth below.
# are caught above; the route may inject gateway-owned auth below.
flow.request.headers.pop("authorization", None)
# Build headers mapping for match evaluation
req_headers = {k.lower(): v for k, v in flow.request.headers.items()}
decision = decide(
self.config.routes,
config.routes,
flow.request.pretty_host,
request_path,
os.environ,
@@ -257,7 +311,7 @@ class EgressAddon:
if decision.inject_authorization is not None:
flow.request.headers["authorization"] = decision.inject_authorization
if self.config.log >= LOG_FULL:
if config.log >= LOG_FULL:
self._log_request(flow)
def _block_dlp(self, flow: http.HTTPFlow, result: ScanResult) -> None:
@@ -270,6 +324,7 @@ class EgressAddon:
self,
flow: http.HTTPFlow,
route: Route,
slug: str,
) -> bool:
"""Scan the outbound request and apply the route's on-match policy
(PRD 0062). Returns True if the request may be forwarded, False if a
@@ -291,7 +346,7 @@ class EgressAddon:
)
result = scan_outbound(
route, scan_text, os.environ,
safe_tokens=self.safe_tokens, crlf_text=crlf_text,
safe_tokens=self._safe_tokens_for(slug), crlf_text=crlf_text,
)
if result is None or result.severity != "block":
return True
@@ -326,10 +381,10 @@ class EgressAddon:
# supervise (default): hold the request for operator approval.
# Fall back to a hard 403 when supervise isn't wired for the bottle.
if not self._supervise_available():
if not self._supervise_available(slug):
self._block_dlp(flow, result)
return False
approved = await self._supervise_token_block(flow, request_path, result)
approved = await self._supervise_token_block(flow, request_path, result, slug)
if not approved:
return False # _supervise_token_block wrote the 403 response
# loop: the approved value is now in safe_tokens; re-scan.
@@ -372,12 +427,16 @@ class EgressAddon:
flow: http.HTTPFlow,
request_path: str,
result: ScanResult,
slug: str,
) -> bool:
"""Route a token DLP block to the operator's supervisor queue and wait.
Returns True if the operator approved (the matched value is added to
`self.safe_tokens` and the caller re-scans); False if the request must
be blocked (a 403 response has been written to `flow`)."""
`slug` attributes the proposal to the calling bottle (its own queue +
safelist) in the shared gateway this is what keeps one bottle's
approval from unblocking another's request. Returns True if the operator
approved (the matched value is added to that bottle's safelist and the
caller re-scans); False if the request must be blocked (a 403 response
has been written to `flow`)."""
host = flow.request.pretty_host
payload = build_token_allow_payload(
redact_tokens(host, env=os.environ),
@@ -386,7 +445,7 @@ class EgressAddon:
result,
)
proposal = _sv.Proposal.new(
bottle_slug=self._supervise_slug,
bottle_slug=slug,
tool=_sv.TOOL_EGRESS_TOKEN_ALLOW,
proposed_file=payload,
justification=_TOKEN_ALLOW_JUSTIFICATION,
@@ -409,13 +468,13 @@ class EgressAddon:
**self._req_ctx(flow),
}) + "\n")
response = await self._await_token_response(proposal.id)
_sv.archive_proposal(self._supervise_slug, proposal.id)
response = await self._await_token_response(proposal.id, slug)
_sv.archive_proposal(slug, proposal.id)
if response is not None and response.status in (
_sv.STATUS_APPROVED, _sv.STATUS_MODIFIED,
):
self.safe_tokens.add(result.matched)
self._safe_tokens_for(slug).add(result.matched)
if self.config.log >= LOG_BLOCKS:
sys.stderr.write(json.dumps({
"event": "egress_token_allowed",
@@ -438,6 +497,7 @@ class EgressAddon:
async def _await_token_response(
self,
proposal_id: str,
slug: str,
) -> "_sv.Response | None":
"""Poll the DB for the operator's response without blocking the
proxy event loop. Returns the Response, or None on timeout."""
@@ -445,7 +505,7 @@ class EgressAddon:
deadline = loop.time() + self._token_allow_timeout
while True:
try:
return _sv.read_response(self._supervise_slug, proposal_id)
return _sv.read_response(slug, proposal_id)
except (OSError, ValueError, KeyError):
# Not written yet, or a partial/malformed write — retry until
# the deadline, then fail closed.
@@ -499,6 +559,9 @@ class EgressAddon:
"""
if flow.websocket is None: # type: ignore[union-attr]
return
# WebSocket DLP runs against the static config only (single-tenant); in
# the consolidated gateway self.config has no routes, so this is inert
# until websocket routing is made source-IP-aware (a separate slice).
route = match_route(self.config.routes, flow.request.pretty_host)
if route is None:
return
@@ -509,7 +572,7 @@ class EgressAddon:
# not an injection vector here — scan only for credential leakage.
result = scan_outbound(
route, content, os.environ,
safe_tokens=self.safe_tokens, crlf_text="",
safe_tokens=self._safe_tokens_for(self._supervise_slug), crlf_text="",
)
if result is not None and result.severity == "block":
sys.stderr.write(f"egress DLP: {result.reason}\n")
+70 -5
View File
@@ -3,7 +3,7 @@
Split out of `egress_addon.py` so the host's unit tests can
exercise the parse + decision functions without depending on the
`mitmproxy` package. The companion module wraps these with the
`mitmproxy.http.HTTPFlow` API and is loaded inside the sidecar
`mitmproxy.http.HTTPFlow` API and is loaded inside the gateway
container.
Imports: stdlib + `yaml_subset` (which is itself stdlib-only and
@@ -22,7 +22,7 @@ except ImportError: # pragma: no cover - host-side path
from .yaml_subset import YamlSubsetError, parse_yaml_subset
# DLP detector-config parsing lives in a sibling module (also flat-bundled
# into the sidecar — see Dockerfile.sidecars). Re-exported below so existing
# into the gateway — see Dockerfile.sidecars). Re-exported below so existing
# `from egress_addon_core import ON_MATCH_*` callers keep working.
try:
from egress_dlp_config import ( # type: ignore[import-not-found]
@@ -120,7 +120,7 @@ class ScanResult:
reason: str
location: str = "" # where the match was found, e.g. "body", "authorization header"
context: str = "" # surrounding text with the match replaced by REDACT
# Raw substring the detector matched. Used inside the sidecar to key the
# Raw substring the detector matched. Used inside the gateway to key the
# supervisor-approved "safe tokens" set (PRD 0062); never logged or written
# to a proposal file. Empty for structural detectors (CRLF) that carry no
# safelist-able value.
@@ -413,6 +413,67 @@ def load_config(text: str) -> "Config":
return parse_config(payload)
class PolicyResolverLike(typing.Protocol):
"""The bit of `policy_resolver.PolicyResolver` this module needs — kept a
Protocol so egress_addon_core stays free of that import."""
def resolve(self, source_ip: str, identity_token: str = ...) -> "str | None":
...
def _config_from_policy(policy: "str | None") -> "Config":
"""Parse a resolved policy blob into a Config, fail-closed: None / empty /
unparseable all become a deny-all Config (no routes every request
blocked)."""
if not policy:
return Config(routes=()) # unattributed or empty → deny-all
try:
return load_config(policy)
except ValueError:
return Config(routes=()) # unparseable policy → deny
def resolve_client_config(
resolver: PolicyResolverLike, client_ip: str, identity_token: str = ""
) -> "Config":
"""The calling client's egress Config, resolved from the orchestrator via
`resolver` and parsed **fail-closed**. An unattributed client (None), a
resolver error, or an unparseable policy all yield a deny-all Config (no
routes every request blocked). A compromised, absent, or confused
orchestrator must never *widen* a bottle's egress."""
try:
policy = resolver.resolve(client_ip, identity_token)
except Exception: # noqa: BLE001 # pylint: disable=broad-exception-caught
return Config(routes=()) # orchestrator unreachable/errored → deny
return _config_from_policy(policy)
class ContextResolverLike(typing.Protocol):
"""The bit of `policy_resolver.PolicyResolver` `resolve_client_context`
needs one round-trip returning both policy and bottle id."""
def resolve_policy_and_bottle_id(
self, source_ip: str, identity_token: str = ...,
) -> "tuple[str | None, str | None]":
...
def resolve_client_context(
resolver: ContextResolverLike, client_ip: str, identity_token: str = "",
) -> "tuple[Config, str]":
"""The calling client's `(Config, bottle_id)` in one round-trip —
**fail-closed**. The Config follows `resolve_client_config`'s deny-all
rules; the bottle id is `""` whenever unattributed or the orchestrator
errored, which the caller treats as "supervise unavailable for this
bottle" (never another bottle's queue). One `/resolve` keys both the
per-request egress policy and the per-bottle supervise queue + safelist."""
try:
policy, bottle_id = resolver.resolve_policy_and_bottle_id(client_ip, identity_token)
except Exception: # noqa: BLE001 # pylint: disable=broad-exception-caught
return Config(routes=()), "" # orchestrator unreachable/errored → deny
return _config_from_policy(policy), (bottle_id or "")
# ---------------------------------------------------------------------------
# Match evaluation
# ---------------------------------------------------------------------------
@@ -613,7 +674,7 @@ def outbound_scan_headers(
) -> dict[str, str]:
"""Return request headers that should be included in outbound DLP.
Routes that inject sidecar-owned auth always strip the agent's
Routes that inject gateway-owned auth always strip the agent's
Authorization header before forwarding. Scanning that header first
creates false positives for provider clients that insist on sending
their own bearer-shaped placeholder, while still not changing what
@@ -664,7 +725,7 @@ def scan_outbound(
crlf_text: str | None = None,
) -> ScanResult | None:
# Lazy import to avoid circular deps and keep dlp_detectors optional
# at import time (the sidecar copies it flat alongside this file).
# at import time (the gateway copies it flat alongside this file).
try:
from dlp_detectors import ( # type: ignore[import-not-found]
scan_crlf_injection,
@@ -804,6 +865,10 @@ __all__ = [
"is_git_push_request",
"is_git_fetch_request",
"load_config",
"resolve_client_config",
"resolve_client_context",
"PolicyResolverLike",
"ContextResolverLike",
"match_route",
"outbound_scan_headers",
"parse_config",
+116 -5
View File
@@ -6,6 +6,15 @@ where the guest reaches the sidecar over the point-to-point TAP). The
wrapper serves the same `/git/*.git` bare repos through
`git http-backend`, so pre-receive and upstream forwarding remain the
git-gate enforcement point.
Consolidated (PRD 0070): when `BOT_BOTTLE_ORCHESTRATOR_URL` is set, one
shared gateway serves every bottle, and each request is served from the
calling bottle's repo namespace (`<root>/<bottle_id>`), attributed from
the unspoofable source IP via the orchestrator. Per-repo credentials +
hooks scope by repo directory, so isolating the *root* per bottle isolates
its creds too. Unattributed clients fail closed (404). Unset the legacy
per-bottle single-tenant flat root, unchanged a transitional path that
gets stripped out once every backend runs the consolidated gateway.
"""
from __future__ import annotations
@@ -13,13 +22,86 @@ from __future__ import annotations
import os
import subprocess
import sys
import typing
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from pathlib import Path
from urllib.parse import urlsplit
# policy_resolver ships flat alongside this file in the sidecar bundle
# image (see Dockerfile.sidecars); the bot_bottle.* fallback is the
# host-side / test path. Mirrors egress_addon's import shape.
try:
from policy_resolver import ( # type: ignore[import-not-found]
PolicyResolveError,
PolicyResolver,
)
except ImportError: # pragma: no cover - host-side path
from bot_bottle.policy_resolver import PolicyResolveError, PolicyResolver
DEFAULT_PORT = 9420
# Consolidated (multi-tenant) mode: when this points at the per-host
# orchestrator's control plane, the backend serves each request from the
# *calling* bottle's repo namespace, selected by source IP, instead of a
# single flat repo root. Unset → legacy per-bottle single-tenant mode
# (unchanged). Same env the egress addon reads, so one orchestrator setting
# flips the whole shared gateway multi-tenant.
ORCHESTRATOR_URL_ENV = "BOT_BOTTLE_ORCHESTRATOR_URL"
# App-layer identity token (defense-in-depth over the source-IP invariant);
# the agent injects it, the backend reads it for attribution and never
# forwards it to `git http-backend`. Mirrors egress_addon.IDENTITY_HEADER
# (duplicated, not imported: egress_addon pulls in mitmproxy).
IDENTITY_HEADER = "x-bot-bottle-identity"
# Default flat repo root (single-tenant, and the base under which
# consolidated mode nests each sandbox's namespace).
DEFAULT_REPO_ROOT = "/git"
class ResolverLike(typing.Protocol):
"""Structural type for the resolver `resolve_sandbox_root` needs — just
`resolve_bottle_id`. A Protocol so tests can pass a fake without
importing PolicyResolver (mirrors egress_addon_core.PolicyResolverLike)."""
def resolve_bottle_id(
self, source_ip: str, identity_token: str = ...,
) -> "str | None": ...
def resolve_sandbox_root(
resolver: "ResolverLike | None",
base_root: Path,
source_ip: str,
identity_token: str = "",
) -> Path | None:
"""The per-sandbox repo root to serve this request from, or None to
deny (404).
Single-tenant (`resolver is None`): the flat `base_root`, unchanged.
NOTE: this legacy per-bottle single-tenant path is transitional it
will be stripped out once every backend runs the consolidated gateway
(PRD 0070), leaving only the source-IP-attributed path below.
Consolidated: `base_root/<bottle_id>`, where the sandbox is attributed
from the source IP via the orchestrator. Fail-closed an unattributed
client, a resolver error, or a namespace that would escape `base_root`
all deny, so one sandbox can never reach another's repos."""
if resolver is None:
return base_root
try:
bottle_id = resolver.resolve_bottle_id(source_ip, identity_token)
except PolicyResolveError:
return None # orchestrator unreachable/errored → deny
if not bottle_id:
return None # unattributed → deny
base = base_root.resolve()
namespace = (base / bottle_id).resolve()
if base not in namespace.parents:
return None # bottle_id tried to escape the root → deny
return namespace
# Mirrors git_gate_render.GIT_GATE_TIMEOUT_SECS. Duplicated rather than
# imported: this module ships as a flat top-level sibling in the sidecar
# bundle image (see Dockerfile.sidecars), not as part of the bot_bottle
@@ -40,10 +122,25 @@ class GitHttpHandler(BaseHTTPRequestHandler):
def do_POST(self) -> None:
self._run_backend()
def _sandbox_root(self) -> Path | None:
"""This request's per-sandbox repo root, or None to deny. Single-tenant
unless the server was started with a resolver (consolidated mode), in
which case the root is the calling sandbox's source-IP-selected
namespace. `GIT_PROJECT_ROOT` keeps git's own env-var name."""
base = Path(os.environ.get("GIT_PROJECT_ROOT", DEFAULT_REPO_ROOT))
resolver = getattr(self.server, "policy_resolver", None)
token = self.headers.get(IDENTITY_HEADER, "")
return resolve_sandbox_root(resolver, base, self.client_address[0], token)
def _run_backend(self) -> None:
sandbox_root = self._sandbox_root()
if sandbox_root is None:
# Unattributed / resolver error: deny before touching any repo.
self.send_error(404)
return
parsed = urlsplit(self.path)
if self._is_upload_pack(parsed.path, parsed.query):
repo_dir = self._repo_dir(parsed.path)
repo_dir = self._repo_dir(sandbox_root, parsed.path)
if repo_dir is None:
self.send_error(404)
return
@@ -77,7 +174,7 @@ class GitHttpHandler(BaseHTTPRequestHandler):
return
env = os.environ.copy()
env.update({
"GIT_PROJECT_ROOT": os.environ.get("GIT_PROJECT_ROOT", "/git"),
"GIT_PROJECT_ROOT": str(sandbox_root),
"GIT_HTTP_EXPORT_ALL": "1",
"REQUEST_METHOD": self.command,
"PATH_INFO": parsed.path,
@@ -91,6 +188,14 @@ class GitHttpHandler(BaseHTTPRequestHandler):
"SERVER_PORT": str(self.server.server_port), # type: ignore
"SERVER_PROTOCOL": self.request_version,
})
# Consolidated mode: attribute the gitleaks-allow supervise proposal
# (written by receive-pack's pre-receive hook, a child of the CGI we
# spawn below) to the calling bottle. The namespaced root is
# `<base>/<bottle_id>`, so its final component is the bottle id — the
# same per-bottle key egress uses. Single-tenant leaves the hook's
# container-stamped SUPERVISE_BOTTLE_SLUG untouched.
if getattr(self.server, "policy_resolver", None) is not None:
env["SUPERVISE_BOTTLE_SLUG"] = sandbox_root.name
for header, variable in (
("accept", "HTTP_ACCEPT"),
("content-encoding", "HTTP_CONTENT_ENCODING"),
@@ -123,8 +228,8 @@ class GitHttpHandler(BaseHTTPRequestHandler):
)
self._write_cgi_response(proc.stdout)
def _repo_dir(self, path: str) -> Path | None:
root = Path(os.environ.get("GIT_PROJECT_ROOT", "/git")).resolve()
def _repo_dir(self, sandbox_root: Path, path: str) -> Path | None:
root = sandbox_root.resolve()
relative = path.lstrip("/").split(".git", 1)[0] + ".git"
candidate = (root / relative).resolve()
if root not in (candidate, *candidate.parents):
@@ -181,7 +286,13 @@ class GitHttpHandler(BaseHTTPRequestHandler):
def main() -> int:
port = int(os.environ.get("GIT_HTTP_PORT", str(DEFAULT_PORT)))
server = ThreadingHTTPServer(("0.0.0.0", port), GitHttpHandler)
sys.stdout.write(f"git-http listening on 0.0.0.0:{port}\n")
orch_url = os.environ.get(ORCHESTRATOR_URL_ENV, "").strip()
# Consolidated mode: resolve each request's sandbox namespace by source
# IP. Absent → single-tenant (flat repo root); behaviour unchanged.
resolver = PolicyResolver(orch_url) if orch_url else None
server.policy_resolver = resolver # type: ignore[attr-defined]
mode = "multi-tenant" if orch_url else "single-tenant"
sys.stdout.write(f"git-http listening on 0.0.0.0:{port} ({mode})\n")
sys.stdout.flush()
server.serve_forever()
return 0
+64
View File
@@ -0,0 +1,64 @@
"""Per-host orchestrator service (PRD 0070).
A single persistent per-host service that will run the gateway functions
(egress / git-gate / supervise), coordinate with the console, and broker
agent launches. This package is being built bottom-up, starting with the
backend-neutral "consolidation core" that needs no VM packaging:
* `registry` the SQLite runtime-state store + fail-closed
attribution (source IP + per-bottle identity token).
* `broker` the signed, structured launch-request contract + a
`LaunchBroker` (stub for the harness) that verifies
provenance before acting.
* `gateway` the consolidated per-host gateway: a `Gateway`
lifecycle contract (idempotent singleton) + a
`DockerGateway` impl. One gateway shared by all
bottles instead of one per bottle.
* `service` the `Orchestrator`: owns the registry, brokers the
launch lifecycle (launch/teardown), manages the
shared gateway, attributes.
* `control_plane` the HTTP control-plane RPC (launch / teardown /
list / attribute / gateway / health).
The actual backend-native launch (a real docker/firecracker broker) and
the egress/git/supervise data plane land in later slices once this core is
proven (see the PRD sequencing: plain-process dev-harness -> docker
orchestrator -> firecracker).
"""
from __future__ import annotations
from .registry import BottleRecord, RegistryStore, new_identity_token
from .broker import (
BrokerAuthError,
LaunchBroker,
LaunchRequest,
StubBroker,
sign_request,
verify_request,
)
from .docker_broker import DockerBroker, DockerBrokerError
from .gateway import DockerGateway, Gateway, GatewayError
from .service import Orchestrator
from .control_plane import ControlPlaneServer, dispatch, make_server
__all__ = [
"BottleRecord",
"RegistryStore",
"new_identity_token",
"BrokerAuthError",
"LaunchBroker",
"LaunchRequest",
"StubBroker",
"DockerBroker",
"DockerBrokerError",
"Gateway",
"DockerGateway",
"GatewayError",
"sign_request",
"verify_request",
"Orchestrator",
"ControlPlaneServer",
"dispatch",
"make_server",
]
+79
View File
@@ -0,0 +1,79 @@
"""Run the orchestrator control plane as a plain process (PRD 0070 dev-harness).
python -m bot_bottle.orchestrator [--host H] [--port P] [--db PATH]
The PRD sequences the orchestrator as a plain-process dev-harness first, so
the consolidation core (registry + attribution + HTTP control plane + live
reload) can be exercised with fast iteration, decoupled from any VM /
container packaging. Wrapping this exact service in a backend-native unit
(docker container, then Firecracker VM) comes later.
"""
from __future__ import annotations
import argparse
import secrets
from pathlib import Path
from .. import log
from .broker import LaunchBroker, StubBroker
from .control_plane import make_server
from .docker_broker import DockerBroker
from .registry import RegistryStore, default_db_path
from .service import Orchestrator
from .gateway import DockerGateway, Gateway
def main(argv: list[str] | None = None) -> int:
"""Parse args, migrate the registry, and serve the control plane."""
parser = argparse.ArgumentParser(prog="bot_bottle.orchestrator")
parser.add_argument("--host", default="127.0.0.1", help="bind address")
parser.add_argument("--port", type=int, default=8080, help="bind port (0 = ephemeral)")
parser.add_argument(
"--db", type=Path, default=None,
help=f"registry DB path (default: {default_db_path()})",
)
parser.add_argument(
"--broker", choices=("stub", "docker"), default="stub",
help="launch broker: 'stub' records requests; 'docker' runs containers",
)
parser.add_argument(
"--gateway", action="store_true",
help="run one consolidated per-host sidecar bundle (build-if-missing)",
)
args = parser.parse_args(argv)
registry = RegistryStore(args.db)
registry.migrate()
# An ephemeral signing secret ties the orchestrator (signer) to its
# broker (verifier). 'stub' records launches instead of starting
# anything; 'docker' runs real containers (firecracker drops in later).
secret = secrets.token_bytes(32)
broker: LaunchBroker = DockerBroker(secret) if args.broker == "docker" else StubBroker(secret)
gateway: Gateway | None = DockerGateway() if args.gateway else None
orchestrator = Orchestrator(registry, broker, secret, gateway)
# One persistent per-host gateway, shared by every bottle: build the
# bundle image if missing, then bring the singleton up (idempotent).
if gateway is not None:
orchestrator.ensure_gateway()
log.info("consolidated gateway ensured", context={"name": gateway.name})
server = make_server(orchestrator, host=args.host, port=args.port)
bound_host, bound_port = server.server_address[0], server.server_address[1]
log.info(
"orchestrator control plane listening",
context={"host": bound_host, "port": bound_port, "db": str(registry.db_path)},
)
try:
server.serve_forever()
except KeyboardInterrupt:
log.info("orchestrator shutting down")
finally:
server.server_close()
return 0
if __name__ == "__main__":
raise SystemExit(main())
+176
View File
@@ -0,0 +1,176 @@
"""Launch broker contract (PRD 0070).
A VM/container can't spawn its own host-network siblings, so the
orchestrator brokers agent launches through a small privileged component.
The request it sends is:
* **structured** static flags + ids only (bottle id, pool slot, a
content-addressed image ref), never a free-form path or argv, so a
request can't be coerced into launching an arbitrary payload; and
* **signed** wrapped as a compact JWS/JWT the broker verifies before
acting, so a *compromised co-located component* (an agent-facing
gateway, say) can't forge a launch. This is the concrete form of the
"structured requests only" rule in the PRD security review.
Signing is **HS256** over a secret shared by the orchestrator (signer) and
the broker (verifier) appropriate here because both are trusted host
components provisioned together; the secret is exactly what an
agent-facing component does not have. (Stdlib only the project takes no
runtime deps; a cross-host deployment could swap in an asymmetric alg.)
"""
from __future__ import annotations
import abc
import base64
import hashlib
import hmac
import json
import secrets
import time
from dataclasses import dataclass
_JWT_HEADER = {"alg": "HS256", "typ": "JWT"}
_ALLOWED_OPS = ("launch", "teardown")
class BrokerAuthError(Exception):
"""A broker request failed provenance or schema verification —
bad/absent signature, malformed token, or a payload that doesn't match
the fixed launch-request shape. Fail-closed: the broker must not act."""
@dataclass(frozen=True)
class LaunchRequest:
"""The structured, un-coercible launch/teardown request. Ids + flags
only `image_ref` is a content-addressed id, never a path/argv."""
op: str # one of _ALLOWED_OPS
bottle_id: str
source_ip: str = ""
image_ref: str = ""
slot: int | None = None
# --- minimal JWS/JWT (HS256), stdlib only ----------------------------------
def _b64url(data: bytes) -> str:
return base64.urlsafe_b64encode(data).rstrip(b"=").decode("ascii")
def _b64url_decode(text: str) -> bytes:
return base64.urlsafe_b64decode(text + "=" * (-len(text) % 4))
def _mac(signing_input: str, secret: bytes) -> str:
digest = hmac.new(secret, signing_input.encode("ascii"), hashlib.sha256).digest()
return _b64url(digest)
def sign_request(req: LaunchRequest, secret: bytes) -> str:
"""Serialize `req` to signed-JWT form. Adds a per-signature `jti`
(replay id) and `iat` (issued-at)."""
claims: dict[str, object] = {
"op": req.op,
"bottle_id": req.bottle_id,
"source_ip": req.source_ip,
"image_ref": req.image_ref,
"slot": req.slot,
"jti": secrets.token_hex(8),
"iat": int(time.time()),
}
header = _b64url(json.dumps(_JWT_HEADER, sort_keys=True, separators=(",", ":")).encode())
payload = _b64url(json.dumps(claims, sort_keys=True, separators=(",", ":")).encode())
signing_input = f"{header}.{payload}"
return f"{signing_input}.{_mac(signing_input, secret)}"
def verify_request(token: str, secret: bytes) -> LaunchRequest:
"""Verify the signature and the request shape; return the parsed
request. Raises `BrokerAuthError` on any failure (fail-closed)."""
parts = token.split(".")
if len(parts) != 3:
raise BrokerAuthError("malformed token")
header_b, payload_b, sig = parts
if not hmac.compare_digest(_mac(f"{header_b}.{payload_b}", secret), sig):
raise BrokerAuthError("bad signature")
try:
header = json.loads(_b64url_decode(header_b))
claims = json.loads(_b64url_decode(payload_b))
except (ValueError, json.JSONDecodeError) as e:
raise BrokerAuthError("malformed token payload") from e
if not isinstance(header, dict) or header.get("alg") != "HS256":
raise BrokerAuthError("unexpected header/alg")
if not isinstance(claims, dict):
raise BrokerAuthError("claims are not an object")
op = claims.get("op")
bottle_id = claims.get("bottle_id")
source_ip = claims.get("source_ip", "")
image_ref = claims.get("image_ref", "")
slot = claims.get("slot")
if (
not isinstance(op, str) or op not in _ALLOWED_OPS
or not isinstance(bottle_id, str) or not bottle_id
or not isinstance(source_ip, str) or not isinstance(image_ref, str)
or not (slot is None or isinstance(slot, int))
):
raise BrokerAuthError("request does not match the launch-request schema")
return LaunchRequest(
op=op, bottle_id=bottle_id, source_ip=source_ip, image_ref=image_ref, slot=slot
)
# --- the broker itself ------------------------------------------------------
class LaunchBroker(abc.ABC):
"""Verifies a signed request came from the orchestrator, then performs
the backend-native launch/teardown. Subclasses implement `_launch` /
`_teardown`; verification is shared and fail-closed."""
def __init__(self, secret: bytes) -> None:
self._secret = secret
def submit(self, token: str) -> LaunchRequest:
"""Verify `token` and perform its op. Returns the verified request;
raises `BrokerAuthError` if provenance/shape fails."""
req = verify_request(token, self._secret)
if req.op == "launch":
self._launch(req)
else:
self._teardown(req)
return req
@abc.abstractmethod
def _launch(self, req: LaunchRequest) -> None:
...
@abc.abstractmethod
def _teardown(self, req: LaunchRequest) -> None:
...
class StubBroker(LaunchBroker):
"""Dev-harness broker: records verified requests without launching
anything. Exercises the full sign -> verify -> act contract in-process."""
def __init__(self, secret: bytes) -> None:
super().__init__(secret)
self.launched: list[LaunchRequest] = []
self.torn_down: list[LaunchRequest] = []
def _launch(self, req: LaunchRequest) -> None:
self.launched.append(req)
def _teardown(self, req: LaunchRequest) -> None:
self.torn_down.append(req)
__all__ = [
"BrokerAuthError",
"LaunchRequest",
"LaunchBroker",
"StubBroker",
"sign_request",
"verify_request",
]
+199
View File
@@ -0,0 +1,199 @@
"""Orchestrator HTTP control plane (PRD 0070).
The backend-agnostic control-plane RPC (CLI / console -> orchestrator) over
**HTTP** the universal transport chosen in 0070 (works on every host; no
vsock / unix-socket portability caveats):
GET /health -> 200 {"status": "ok"}
GET /gateway -> 200 {"configured", ["name","running"]}
GET /bottles -> 200 {"bottles": [ <redacted record>, ...]}
POST /bottles -> 201 {"bottle_id","identity_token"} (launch)
body: {"source_ip", ["image_ref"],
["metadata"], ["policy"]}
PUT /bottles/<bottle_id>/policy -> 200 {"updated": true} | 404 (live reload)
body: {"policy"}
DELETE /bottles/<bottle_id> -> 200 {"torn_down": true} | 404 (teardown)
POST /attribute -> 200 {"bottle_id"} | 403
POST /resolve -> 200 {"bottle_id","policy"} | 403
body: {"source_ip","identity_token"}
`POST /bottles` / `DELETE` drive the full launch lifecycle: they mint (or
tear down) the bottle in the registry AND broker the backend-native launch
via the orchestrator. Register/deregister without a launch are internal to
`Orchestrator`, not exposed here.
Routing/handling is the pure function `dispatch()` so it is unit-testable
without a socket; `Handler` / `ControlPlaneServer` / `make_server` are a
thin stdlib adapter around it. Listing redacts identity tokens they are
returned only once, to the caller that launches the bottle.
"""
from __future__ import annotations
import http.server
import json
import os
import socketserver
import typing
from urllib.parse import urlsplit
from .service import Orchestrator
# JSON body payload type (parsed request / rendered response).
Json = dict[str, object]
def _parse_json_object(body: bytes) -> Json:
"""Parse a JSON object body. Raises ValueError for non-objects / bad JSON."""
if not body:
return {}
obj = json.loads(body) # raises json.JSONDecodeError (a ValueError)
if not isinstance(obj, dict):
raise ValueError("request body must be a JSON object")
return obj
def dispatch( # pylint: disable=too-many-return-statements,too-many-branches
orch: Orchestrator, method: str, path: str, body: bytes
) -> tuple[int, Json]:
"""Route one control-plane request to a (status, payload) pair. Pure —
no I/O beyond the orchestrator so it is fully testable without a socket."""
route = urlsplit(path).path.rstrip("/") or "/"
if method == "GET" and route == "/health":
return 200, {"status": "ok"}
if method == "GET" and route == "/gateway":
return 200, orch.gateway_status()
if method == "GET" and route == "/bottles":
return 200, {"bottles": [r.redacted() for r in orch.registry.all()]}
if method == "POST" and route == "/bottles":
try:
data = _parse_json_object(body)
except ValueError as e:
return 400, {"error": f"invalid JSON: {e}"}
source_ip = data.get("source_ip")
if not isinstance(source_ip, str) or not source_ip:
return 400, {"error": "source_ip (string) is required"}
image_ref = data.get("image_ref")
metadata = data.get("metadata")
policy = data.get("policy")
rec = orch.launch_bottle(
source_ip,
image_ref=image_ref if isinstance(image_ref, str) else "",
metadata=metadata if isinstance(metadata, str) else "",
policy=policy if isinstance(policy, str) else "",
)
return 201, {"bottle_id": rec.bottle_id, "identity_token": rec.identity_token}
if method == "PUT" and route.startswith("/bottles/") and route.endswith("/policy"):
bottle_id = route[len("/bottles/"):-len("/policy")]
try:
data = _parse_json_object(body)
except ValueError as e:
return 400, {"error": f"invalid JSON: {e}"}
policy = data.get("policy")
if not isinstance(policy, str):
return 400, {"error": "policy (string) is required"}
if orch.set_policy(bottle_id, policy):
return 200, {"updated": True}
return 404, {"error": "no such bottle"}
if method == "DELETE" and route.startswith("/bottles/"):
bottle_id = route[len("/bottles/"):]
if orch.teardown_bottle(bottle_id):
return 200, {"torn_down": True}
return 404, {"error": "no such bottle"}
if method == "POST" and route == "/attribute":
try:
data = _parse_json_object(body)
except ValueError as e:
return 400, {"error": f"invalid JSON: {e}"}
source_ip = data.get("source_ip")
token = data.get("identity_token")
if not isinstance(source_ip, str) or not isinstance(token, str):
return 400, {"error": "source_ip and identity_token (strings) required"}
rec = orch.attribute(source_ip, token)
if rec is None:
return 403, {"error": "unattributed"}
return 200, {"bottle_id": rec.bottle_id}
if method == "POST" and route == "/resolve":
# The per-request lookup the multi-tenant gateway makes: returns the
# bottle's policy. identity_token is OPTIONAL — absent means resolve
# by source IP alone (network-layer attribution).
try:
data = _parse_json_object(body)
except ValueError as e:
return 400, {"error": f"invalid JSON: {e}"}
source_ip = data.get("source_ip")
token = data.get("identity_token")
if not isinstance(source_ip, str) or not source_ip:
return 400, {"error": "source_ip (string) is required"}
rec = orch.resolve(source_ip, token if isinstance(token, str) else "")
if rec is None:
return 403, {"error": "unattributed"}
return 200, {"bottle_id": rec.bottle_id, "policy": rec.policy}
return 404, {"error": "not found"}
class Handler(http.server.BaseHTTPRequestHandler):
"""Thin stdlib adapter: read the body, call `dispatch`, write JSON."""
# Quiet by default (the orchestrator has its own logging); opt back into
# stdlib access logging with BOT_BOTTLE_ORCHESTRATOR_DEBUG.
def log_message(self, format: str, *args: typing.Any) -> None: # noqa: A002
if os.environ.get("BOT_BOTTLE_ORCHESTRATOR_DEBUG"):
super().log_message(format, *args)
def _serve(self, method: str) -> None:
"""Read the request body, dispatch it, and write the JSON reply."""
server = self.server
assert isinstance(server, ControlPlaneServer)
length = int(self.headers.get("Content-Length") or 0)
body = self.rfile.read(length) if length > 0 else b""
status, payload = dispatch(server.orchestrator, method, self.path, body)
data = json.dumps(payload).encode()
self.send_response(status)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(data)))
self.end_headers()
self.wfile.write(data)
def do_GET(self) -> None:
self._serve("GET")
def do_POST(self) -> None:
self._serve("POST")
def do_PUT(self) -> None:
self._serve("PUT")
def do_DELETE(self) -> None:
self._serve("DELETE")
class ControlPlaneServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
"""Threading HTTP server that carries the orchestrator for its handlers."""
daemon_threads = True
allow_reuse_address = True
def __init__(self, address: tuple[str, int], orchestrator: Orchestrator) -> None:
self.orchestrator = orchestrator
super().__init__(address, Handler)
def make_server(
orchestrator: Orchestrator, host: str = "127.0.0.1", port: int = 0
) -> ControlPlaneServer:
"""Build (but do not start) a control-plane server. `port=0` binds an
ephemeral port read `server.server_address` for the actual one."""
return ControlPlaneServer((host, port), orchestrator)
__all__ = ["dispatch", "Handler", "ControlPlaneServer", "make_server", "Json"]
+83
View File
@@ -0,0 +1,83 @@
"""Docker launch broker (PRD 0070) — the first *real* LaunchBroker.
On a verified launch request it starts a Docker container; on teardown it
removes it. This proves the orchestrator -> backend seam on the cheapest
backend (the sidecar bundle is already containers). Only the request's
static ids/flags reach `docker`, so nothing free-form crosses the boundary.
Slice 3 launches a single container from the request's `image_ref`, named
after the bottle id and labelled for cleanup. Wiring the full agent +
sidecar bundle (networks, mounts, the consolidated sidecar) is a later
slice this is the seam, not the finished launcher.
"""
from __future__ import annotations
import subprocess
from ..docker_cmd import run_docker
from .broker import LaunchBroker, LaunchRequest
CONTAINER_PREFIX = "bot-bottle-orch-"
BOTTLE_ID_LABEL = "bot-bottle-bottle-id"
class DockerBrokerError(Exception):
"""A brokered docker launch/teardown failed (non-zero `docker` exit)."""
def container_name(bottle_id: str) -> str:
"""Deterministic container name for a bottle."""
return f"{CONTAINER_PREFIX}{bottle_id}"
def run_argv(req: LaunchRequest) -> list[str]:
"""`docker run` argv for a launch request — built only from its static
fields, so it can't be coerced into an arbitrary command."""
return [
"docker", "run", "--detach",
"--name", container_name(req.bottle_id),
"--label", f"{BOTTLE_ID_LABEL}={req.bottle_id}",
req.image_ref,
]
def rm_argv(req: LaunchRequest) -> list[str]:
"""`docker rm --force` argv to tear a bottle's container down."""
return ["docker", "rm", "--force", container_name(req.bottle_id)]
class DockerBroker(LaunchBroker):
"""A `LaunchBroker` that runs / removes a Docker container per bottle.
Provenance + schema verification are inherited from `LaunchBroker`."""
def _docker(self, argv: list[str]) -> subprocess.CompletedProcess[str]:
return run_docker(argv)
def _launch(self, req: LaunchRequest) -> None:
if not req.image_ref:
raise DockerBrokerError(f"launch request for {req.bottle_id} has no image_ref")
proc = self._docker(run_argv(req))
if proc.returncode != 0:
raise DockerBrokerError(
f"docker run failed for {req.bottle_id}: {proc.stderr.strip()}"
)
def _teardown(self, req: LaunchRequest) -> None:
proc = self._docker(rm_argv(req))
# Idempotent: an already-absent container is a successful teardown.
if proc.returncode != 0 and "No such container" not in proc.stderr:
raise DockerBrokerError(
f"docker rm failed for {req.bottle_id}: {proc.stderr.strip()}"
)
__all__ = [
"DockerBroker",
"DockerBrokerError",
"container_name",
"run_argv",
"rm_argv",
"CONTAINER_PREFIX",
"BOTTLE_ID_LABEL",
]
+139
View File
@@ -0,0 +1,139 @@
"""The consolidated per-host gateway (PRD 0070).
The core consolidation win: **one** persistent gateway per host, shared by
every bottle, instead of a sidecar bundle per bottle. It's safe to share
because the attribution invariant (source IP + identity token, see
`registry`) lets the gateway attribute each request to the right bottle
so per-bottle policy lives in one long-lived process keyed on who's calling.
`Gateway` is the backend-neutral lifecycle contract (mirrors `LaunchBroker`):
ensure the single instance is up, report it, tear it down. `DockerGateway`
is the docker implementation; a firecracker gateway VM slots in later.
The defining behaviour is **idempotent singleton**: `ensure_running` starts
the instance if absent and is a no-op if it's already up, so N bottle
launches never spawn N gateways.
"""
from __future__ import annotations
import abc
import os
from pathlib import Path
from ..docker_cmd import run_docker
GATEWAY_NAME = "bot-bottle-orch-gateway"
GATEWAY_LABEL = "bot-bottle-orch-gateway=1"
# The real sidecar-bundle image + its Dockerfile. Kept as a local constant
# rather than imported from backend.docker.sidecar_bundle, which would drag
# the whole backend layer into the lean orchestrator (see #359); unify when
# that lands. Env override matches the backend's BOT_BOTTLE_SIDECAR_IMAGE.
GATEWAY_IMAGE = os.environ.get("BOT_BOTTLE_SIDECAR_IMAGE", "bot-bottle-sidecars:latest")
GATEWAY_DOCKERFILE = "Dockerfile.sidecars"
_REPO_ROOT = Path(__file__).resolve().parents[2]
class GatewayError(Exception):
"""The shared gateway failed to build/start/stop (non-zero `docker` exit)."""
class Gateway(abc.ABC):
"""Lifecycle of the single per-host gateway. Backend-neutral."""
name: str
def ensure_built(self) -> None:
"""Ensure the gateway's image / rootfs exists, building it if needed.
Default: nothing to build (e.g. a stub or a pre-pulled image)."""
return
@abc.abstractmethod
def ensure_running(self) -> None:
"""Start the gateway if it isn't already up. Idempotent: a no-op
when it's already running (that's the whole point one per host).
Assumes the image exists call `ensure_built()` first."""
@abc.abstractmethod
def is_running(self) -> bool:
"""True iff the gateway instance is currently up."""
@abc.abstractmethod
def stop(self) -> None:
"""Remove the gateway. Idempotent — absent is success."""
class DockerGateway(Gateway):
"""The consolidated gateway as a single, fixed-name Docker container.
`image_ref` defaults to the real sidecar-bundle image; `ensure_built`
builds it from `Dockerfile.sidecars` when it's missing. (Note: slice 5
builds + launches the bundle container; wiring its per-bottle,
source-IP-keyed config is a later slice see PRD 0070.)"""
def __init__(
self,
image_ref: str = GATEWAY_IMAGE,
*,
name: str = GATEWAY_NAME,
build_context: Path | None = None,
dockerfile: str | None = GATEWAY_DOCKERFILE,
) -> None:
self.image_ref = image_ref
self.name = name
self._build_context = build_context or _REPO_ROOT
self._dockerfile = dockerfile
def image_exists(self) -> bool:
return run_docker(["docker", "image", "inspect", self.image_ref]).returncode == 0
def ensure_built(self) -> None:
"""Build the bundle image from its Dockerfile when it's missing.
No-op when the image is already present, or when no dockerfile is
configured (e.g. a pre-pulled image)."""
if self._dockerfile is None or self.image_exists():
return
proc = run_docker([
"docker", "build",
"-t", self.image_ref,
"-f", str(self._build_context / self._dockerfile),
str(self._build_context),
])
if proc.returncode != 0:
raise GatewayError(f"gateway image build failed: {proc.stderr.strip()}")
def is_running(self) -> bool:
proc = run_docker([
"docker", "ps",
"--filter", f"name=^{self.name}$",
"--filter", "status=running",
"--format", "{{.Names}}",
])
return self.name in proc.stdout.split()
def ensure_running(self) -> None:
if self.is_running():
return
# Clear any stale (stopped) container holding the fixed name, then
# start fresh. `rm --force` on an absent name is a tolerated no-op.
run_docker(["docker", "rm", "--force", self.name])
proc = run_docker([
"docker", "run", "--detach",
"--name", self.name,
"--label", GATEWAY_LABEL,
self.image_ref,
])
if proc.returncode != 0:
raise GatewayError(f"gateway failed to start: {proc.stderr.strip()}")
def stop(self) -> None:
proc = run_docker(["docker", "rm", "--force", self.name])
if proc.returncode != 0 and "No such container" not in proc.stderr:
raise GatewayError(f"gateway failed to stop: {proc.stderr.strip()}")
__all__ = [
"Gateway", "DockerGateway", "GatewayError",
"GATEWAY_NAME", "GATEWAY_LABEL", "GATEWAY_IMAGE",
]
+141
View File
@@ -0,0 +1,141 @@
"""Orchestrator process lifecycle (PRD 0070, docker slice).
Before the CLI can register or launch bottles against the consolidated
model, exactly one orchestrator control plane and the single per-host
gateway it manages must be running. This starts the orchestrator
dev-harness (`python -m bot_bottle.orchestrator`) as a background host
process and health-checks it.
It is an **idempotent singleton**: `ensure_running` returns immediately if a
healthy control plane already answers on the port, and otherwise spawns one
and waits for it to come up. The control-plane port is the singleton key
a second orchestrator can't bind it, so a stray double-start fails fast
rather than forking a rival.
Host-process (not container) on purpose: the PRD sequences the orchestrator
as a plain-process dev-harness first (fast iteration, and it already has the
host user's docker access to broker launches), while the data-plane
*gateway* it manages runs as a container. Wrapping the orchestrator itself
in a backend-native unit is a later step.
"""
from __future__ import annotations
import subprocess
import sys
import time
import urllib.error
import urllib.request
from .. import log
from ..paths import bot_bottle_root
DEFAULT_HOST = "127.0.0.1"
DEFAULT_PORT = 8080
# Poll cadence + default ceiling while waiting for a freshly-spawned control
# plane to answer /health (the first start also builds/boots the gateway).
_HEALTH_POLL_SECONDS = 0.25
DEFAULT_STARTUP_TIMEOUT_SECONDS = 45.0
_HEALTH_REQUEST_TIMEOUT_SECONDS = 1.0
class OrchestratorStartError(RuntimeError):
"""The orchestrator process did not become healthy within the timeout."""
class OrchestratorProcess:
"""Manages the local orchestrator control-plane process for the docker
backend. Backend-neutral callers only need `ensure_running()` + `url`."""
def __init__(
self,
host: str = DEFAULT_HOST,
port: int = DEFAULT_PORT,
*,
broker: str = "docker",
gateway: bool = True,
) -> None:
self.host = host
self.port = port
self._broker = broker
self._gateway = gateway
@property
def url(self) -> str:
"""The control-plane base URL — also what the data plane's
BOT_BOTTLE_ORCHESTRATOR_URL points at."""
return f"http://{self.host}:{self.port}"
def is_healthy(self, *, timeout: float = _HEALTH_REQUEST_TIMEOUT_SECONDS) -> bool:
"""True iff a control plane answers `GET /health` with 200 — the
singleton liveness check."""
try:
with urllib.request.urlopen(f"{self.url}/health", timeout=timeout) as resp:
return resp.status == 200
except (urllib.error.URLError, TimeoutError, OSError):
return False
def ensure_running(
self, *, startup_timeout: float = DEFAULT_STARTUP_TIMEOUT_SECONDS,
) -> str:
"""Return the control-plane URL, starting the orchestrator first if it
isn't already healthy. Idempotent — a healthy control plane is left
untouched. Raises `OrchestratorStartError` if a freshly-spawned one
doesn't answer within `startup_timeout`."""
if self.is_healthy():
return self.url
log.info("starting orchestrator", context={"url": self.url, "broker": self._broker})
self._spawn()
deadline = time.monotonic() + startup_timeout
while time.monotonic() < deadline:
if self.is_healthy():
log.info("orchestrator healthy", context={"url": self.url})
return self.url
time.sleep(_HEALTH_POLL_SECONDS)
raise OrchestratorStartError(
f"orchestrator at {self.url} did not become healthy within "
f"{startup_timeout:g}s"
)
def _argv(self) -> list[str]:
"""`python -m bot_bottle.orchestrator ...` — static flags only."""
argv = [
sys.executable, "-m", "bot_bottle.orchestrator",
"--host", self.host, "--port", str(self.port),
"--broker", self._broker,
]
if self._gateway:
argv.append("--gateway")
return argv
def _log_path(self) -> str:
"""Where the detached orchestrator's stdout/stderr goes so a failed
start is diagnosable after the CLI has moved on."""
return str(bot_bottle_root() / "orchestrator.log")
def _spawn(self) -> None:
"""Launch the orchestrator detached so it outlives this CLI process,
with its output tee'd to a log file under the bot-bottle root."""
root = bot_bottle_root()
root.mkdir(parents=True, exist_ok=True)
logfile = open(self._log_path(), "a", encoding="utf-8") # noqa: SIM115 # pylint: disable=consider-using-with
try:
subprocess.Popen( # noqa: S603 # pylint: disable=consider-using-with
self._argv(),
stdout=logfile,
stderr=subprocess.STDOUT,
stdin=subprocess.DEVNULL,
start_new_session=True,
)
finally:
# The child inherits its own dup'd fd; this handle is ours to drop.
logfile.close()
__all__ = [
"OrchestratorProcess",
"OrchestratorStartError",
"DEFAULT_HOST",
"DEFAULT_PORT",
"DEFAULT_STARTUP_TIMEOUT_SECONDS",
]
+252
View File
@@ -0,0 +1,252 @@
"""Orchestrator bottle registry — the per-host runtime-state store (PRD 0070).
SQLite-backed registry mapping each live bottle to its source IP and
per-bottle identity token, plus the **fail-closed attribution** the data
plane relies on: a request is attributed to a bottle only when its source
IP *and* its identity token both match a single active record.
The registry co-tenants the shared host `bot-bottle.db` (the `DbStore`
framework namespaces each store by `schema_key`), so all bot-bottle
runtime state lives in one queryable file one place to back up, inspect,
and integrate a console against.
This is the "runtime state" tier of PRD 0070 (leases / approvals /
registry) deliberately NOT config (which stays declarative under
`~/.bot-bottle/`) and NOT the build-time constants (a flat file). It is the
source of truth the orchestrator sweeps on restart to re-adopt running
bottles, so it lives in a durable store rather than process memory.
Attribution combines two independent signals, and needs both:
* source IP the network-layer invariant (on Firecracker the `/31` TAP
+ nft make it unspoofable by construction; weaker on other backends).
* identity token an application-layer per-bottle secret, defence in
depth that doesn't lean on network anti-spoof. A bottle can only ever
prove it is *itself* (it can't learn another bottle's token), so a
hostile agent gains nothing by presenting it.
"""
from __future__ import annotations
import hmac
import secrets
import sqlite3
import time
from dataclasses import dataclass
from pathlib import Path
from ..db_store import DbStore
from ..migrations import TableMigrations
from ..paths import host_db_path
# 256 bits of urandom, URL-safe — unguessable per-bottle identity token.
IDENTITY_TOKEN_BYTES = 32
def new_identity_token() -> str:
"""A fresh per-bottle identity token (PRD 0070 attribution defence)."""
return secrets.token_urlsafe(IDENTITY_TOKEN_BYTES)
def default_db_path() -> Path:
"""The shared host state DB (`bot-bottle.db`) — all bot-bottle runtime
state in one file, co-tenanted via schema_key. Host-resident so it
survives orchestrator restarts (re-adoption sweeps it)."""
return host_db_path()
@dataclass(frozen=True)
class BottleRecord:
"""One live bottle in the registry."""
bottle_id: str
source_ip: str
identity_token: str
state: str = "active"
created_at: float = 0.0
# Opaque JSON blob for forward-compat (pool slot, ...) — the registry
# doesn't interpret it, so new fields need no migration.
metadata: str = ""
# The bottle's gateway policy (opaque JSON — egress allowlist / routes /
# git config). The registry stores and serves it verbatim, keyed by
# source IP; the multi-tenant gateway interprets it.
policy: str = ""
def redacted(self) -> dict[str, object]:
"""Public view for listing — never exposes the identity token."""
return {
"bottle_id": self.bottle_id,
"source_ip": self.source_ip,
"state": self.state,
"created_at": self.created_at,
"metadata": self.metadata,
"policy": self.policy,
}
_MIGRATIONS = TableMigrations(
"orchestrator_registry",
[
# v1 — the live bottle registry.
"""
CREATE TABLE IF NOT EXISTS orchestrator_bottles (
bottle_id TEXT PRIMARY KEY,
source_ip TEXT NOT NULL,
identity_token TEXT NOT NULL,
state TEXT NOT NULL DEFAULT 'active',
created_at REAL NOT NULL,
metadata TEXT NOT NULL DEFAULT ''
)
""",
# source_ip is the data-plane attribution key — index it, and make
# the "one active bottle per source IP" check cheap.
"CREATE INDEX IF NOT EXISTS idx_orchestrator_bottles_source_ip "
"ON orchestrator_bottles (source_ip)",
# v3 — per-bottle policy (opaque JSON the gateway interprets): the
# egress allowlist / routes / git config selected by source IP. The
# multi-tenant gateway resolves it per request via `attribute`.
"ALTER TABLE orchestrator_bottles ADD COLUMN policy TEXT NOT NULL DEFAULT ''",
],
)
def _row_to_record(row: sqlite3.Row) -> BottleRecord:
"""Build a BottleRecord from a registry row."""
return BottleRecord(
bottle_id=row["bottle_id"],
source_ip=row["source_ip"],
identity_token=row["identity_token"],
state=row["state"],
created_at=row["created_at"],
metadata=row["metadata"],
policy=row["policy"],
)
class RegistryStore(DbStore):
"""SQLite-backed registry of live bottles + fail-closed attribution."""
def __init__(self, db_path: Path | None = None) -> None:
super().__init__(db_path or default_db_path(), _MIGRATIONS)
def _connect(self) -> sqlite3.Connection:
"""Open a connection with a busy timeout for the shared DB."""
conn = super()._connect()
# The registry co-tenants the shared bot-bottle.db, so a busy_timeout
# rides out brief lock contention with the other stores. WAL for the
# shared DB is a deliberate future change (it affects supervise/audit
# and is finicky over guest shares) — not flipped here.
conn.execute("PRAGMA busy_timeout=5000")
return conn
def register(
self,
source_ip: str,
*,
bottle_id: str | None = None,
identity_token: str | None = None,
metadata: str = "",
policy: str = "",
) -> BottleRecord:
"""Register (or replace) a bottle. Mints a bottle_id / identity token
when not supplied. Live this is the control-plane reload path."""
rec = BottleRecord(
bottle_id=bottle_id or secrets.token_hex(8),
source_ip=source_ip,
identity_token=identity_token or new_identity_token(),
state="active",
created_at=time.time(),
metadata=metadata,
policy=policy,
)
with self._connect() as conn:
conn.execute(
"INSERT OR REPLACE INTO orchestrator_bottles "
"(bottle_id, source_ip, identity_token, state, created_at, metadata, policy) "
"VALUES (?, ?, ?, ?, ?, ?, ?)",
(
rec.bottle_id,
rec.source_ip,
rec.identity_token,
rec.state,
rec.created_at,
rec.metadata,
rec.policy,
),
)
self._chmod()
return rec
def set_policy(self, bottle_id: str, policy: str) -> bool:
"""Update a bottle's policy in place (live reload). Returns True if
the bottle exists."""
with self._connect() as conn:
cur = conn.execute(
"UPDATE orchestrator_bottles SET policy = ? WHERE bottle_id = ?",
(policy, bottle_id),
)
self._chmod()
return cur.rowcount > 0
def deregister(self, bottle_id: str) -> bool:
"""Remove a bottle. Returns True if a row was deleted."""
with self._connect() as conn:
cur = conn.execute(
"DELETE FROM orchestrator_bottles WHERE bottle_id = ?", (bottle_id,)
)
return cur.rowcount > 0
def get(self, bottle_id: str) -> BottleRecord | None:
"""Return the bottle by id, or None if absent."""
with self._connect() as conn:
row = conn.execute(
"SELECT * FROM orchestrator_bottles WHERE bottle_id = ?", (bottle_id,)
).fetchone()
return _row_to_record(row) if row else None
def all(self) -> list[BottleRecord]:
"""Every registered bottle, oldest first."""
with self._connect() as conn:
rows = conn.execute(
"SELECT * FROM orchestrator_bottles ORDER BY created_at"
).fetchall()
return [_row_to_record(r) for r in rows]
def by_source_ip(self, source_ip: str) -> BottleRecord | None:
"""Network-layer attribution: the single active bottle at this source
IP, or None if unknown or ambiguous (more than one a
misconfiguration). Safe as the *sole* attributor only where the
source IP is unspoofable (Firecracker `/31` + nft) and the control
plane is reachable only by the trusted gateway; pair with the
identity token (`attribute`) elsewhere."""
with self._connect() as conn:
rows = conn.execute(
"SELECT * FROM orchestrator_bottles "
"WHERE source_ip = ? AND state = 'active'",
(source_ip,),
).fetchall()
if len(rows) != 1:
return None
return _row_to_record(rows[0])
def attribute(self, source_ip: str, identity_token: str) -> BottleRecord | None:
"""Fail-closed attribution: `by_source_ip` AND a matching identity
token (constant-time). Either signal alone is insufficient here an
unknown/ambiguous IP, an empty token, or a token mismatch all deny."""
if not identity_token:
return None
rec = self.by_source_ip(source_ip)
if rec is None:
return None
if not hmac.compare_digest(rec.identity_token, identity_token):
return None
return rec
__all__ = [
"BottleRecord",
"RegistryStore",
"new_identity_token",
"default_db_path",
"IDENTITY_TOKEN_BYTES",
]
+121
View File
@@ -0,0 +1,121 @@
"""The per-host orchestrator core (PRD 0070).
`Orchestrator` is the single backend-neutral object the control plane talks
to: it owns the registry (runtime state) and brokers agent launches. It
never branches on backend the `LaunchBroker` abstracts the backend-native
launch, so this same object drives docker / firecracker / apple once a real
broker is wired in.
Launch lifecycle:
* `launch_bottle` mints the bottle (registry: source IP + identity
token), sends a *signed, structured* launch request through the broker,
and returns the record. If the broker rejects/fails, the registry entry
is rolled back so a failed launch leaves no orphan.
* `teardown_bottle` sends a signed teardown request, then deregisters.
"""
from __future__ import annotations
from .broker import LaunchBroker, LaunchRequest, sign_request
from .registry import BottleRecord, RegistryStore
from .gateway import Gateway
class Orchestrator:
"""Owns the registry + brokers launches, and manages the single
consolidated per-host gateway. Backend-neutral (broker and gateway
abstract the backend-native pieces)."""
def __init__(
self,
registry: RegistryStore,
broker: LaunchBroker,
sign_secret: bytes,
gateway: Gateway | None = None,
) -> None:
self.registry = registry
self._broker = broker
self._secret = sign_secret
self._gateway = gateway
def launch_bottle(
self,
source_ip: str,
*,
image_ref: str = "",
slot: int | None = None,
metadata: str = "",
policy: str = "",
) -> BottleRecord:
"""Register a bottle (with its gateway policy) and broker its launch.
Rolls the registry entry back if the launch doesn't take, so a
failure leaves no orphan."""
rec = self.registry.register(source_ip, metadata=metadata, policy=policy)
req = LaunchRequest(
op="launch",
bottle_id=rec.bottle_id,
source_ip=source_ip,
image_ref=image_ref,
slot=slot,
)
launched = False
try:
self._broker.submit(sign_request(req, self._secret))
launched = True
finally:
if not launched:
self.registry.deregister(rec.bottle_id)
return rec
def teardown_bottle(self, bottle_id: str) -> bool:
"""Broker teardown then deregister. False if the bottle is unknown."""
rec = self.registry.get(bottle_id)
if rec is None:
return False
req = LaunchRequest(op="teardown", bottle_id=bottle_id, source_ip=rec.source_ip)
self._broker.submit(sign_request(req, self._secret))
self.registry.deregister(bottle_id)
return True
def attribute(self, source_ip: str, identity_token: str) -> BottleRecord | None:
"""Fail-closed attribution (delegates to the registry)."""
return self.registry.attribute(source_ip, identity_token)
def resolve(self, source_ip: str, identity_token: str = "") -> BottleRecord | None:
"""Resolve the bottle behind a request — the source-IP-keyed lookup
the multi-tenant gateway makes per request; the returned record
carries its `policy`. With a token, full attribution (source IP +
token); without, network-layer attribution by source IP alone
(valid where the IP is unspoofable and the control plane is
gateway-only)."""
if identity_token:
return self.registry.attribute(source_ip, identity_token)
return self.registry.by_source_ip(source_ip)
def set_policy(self, bottle_id: str, policy: str) -> bool:
"""Update a bottle's gateway policy in place (live reload). False if
the bottle is unknown."""
return self.registry.set_policy(bottle_id, policy)
# --- consolidated gateway ----------------------------------------------
def ensure_gateway(self) -> None:
"""Ensure the single per-host gateway is built and up (idempotent).
No-op when no gateway is configured."""
if self._gateway is not None:
self._gateway.ensure_built()
self._gateway.ensure_running()
def gateway_status(self) -> dict[str, object]:
"""Report the shared gateway for the control plane / console."""
if self._gateway is None:
return {"configured": False}
return {
"configured": True,
"name": self._gateway.name,
"running": self._gateway.is_running(),
}
__all__ = ["Orchestrator"]
+43
View File
@@ -0,0 +1,43 @@
"""Foundational filesystem paths for bot-bottle.
`bot_bottle_root()` is the app data root state, queue, audit logs,
git-gate keys, and the shared DB all live under it. It defaults to
`~/.bot-bottle` and is overridable with the **`BOT_BOTTLE_ROOT`** env var.
The env override is the single knob for redirecting the root: the test
suite points it at a throwaway dir instead of monkey-patching the function
(every module and every flat/package copy reads the same env var, so one
override covers them all), and operators can relocate the root if needed.
This module has no bot-bottle imports, so it is safe to import from any
layer (and to COPY flat into the sidecar bundle).
"""
from __future__ import annotations
import os
from pathlib import Path
# The single shared host state DB. All bot-bottle SQLite stores (supervise
# queue, audit, the orchestrator registry) co-tenant this one file — the
# TableMigrations schema_key namespaces each store's tables.
HOST_DB_FILENAME = "bot-bottle.db"
def bot_bottle_root() -> Path:
"""The app data root — `$BOT_BOTTLE_ROOT` if set, else `~/.bot-bottle`."""
override = os.environ.get("BOT_BOTTLE_ROOT")
return Path(override) if override else Path.home() / ".bot-bottle"
def host_db_path() -> Path:
"""Path to the shared host state DB, `<root>/db/bot-bottle.db`.
Kept in its own `db/` subdirectory (not directly under the root) so a
backend that can only bind-mount *directories* can share this one file
with a sidecar without exposing the root's other contents (git-gate
keys, per-bottle state, ...)."""
return bot_bottle_root() / "db" / HOST_DB_FILENAME
__all__ = ["HOST_DB_FILENAME", "bot_bottle_root", "host_db_path"]
+123
View File
@@ -0,0 +1,123 @@
"""Gateway-side per-client policy resolver (PRD 0070).
The consolidated gateway serves every bottle from one process, so for each
request it must apply the *calling* bottle's policy, selected by the source
IP the attribution invariant makes unspoofable. This resolves that policy
from the orchestrator's control plane (`POST /resolve`), keyed on the
`(source_ip, identity_token)` pair.
**Always fresh no cache.** The resolver is called rarely enough that a
round-trip doesn't matter, and correctness matters more than speed: every
resolve reflects the orchestrator's *current* view, so a revocation, a
policy change, or a bottle teardown the orchestrator knows about is honored
immediately rather than lingering for a cache TTL. (If this ever becomes a
hot path, add caching with orchestrator-driven invalidation not a blind
TTL.)
**Fail-closed:** an unattributed client (the orchestrator answers `403`)
resolves to `None`, and the caller (the egress addon, git-gate) must then
deny exactly as an unknown bottle should be treated. Orchestrator
*errors* (unreachable / unexpected status) raise, so the caller can fail
closed too rather than silently serving stale or empty policy.
The resolved value is the policy blob the orchestrator stores verbatim; the
consumer parses it (e.g. the egress addon's `load_config`). This module is
stdlib-only and free of bot-bottle imports so it can be COPYed flat into
the sidecar bundle.
"""
from __future__ import annotations
import json
import urllib.error
import urllib.request
DEFAULT_TIMEOUT_SECONDS = 2.0
class PolicyResolveError(RuntimeError):
"""The orchestrator was unreachable or returned an unexpected status —
distinct from a clean `403` (unattributed), which returns None."""
class PolicyResolver:
"""Resolves each client's policy from the orchestrator, fresh per call."""
def __init__(self, base_url: str, *, timeout: float = DEFAULT_TIMEOUT_SECONDS) -> None:
self._base = base_url.rstrip("/")
self._timeout = timeout
def _post_resolve(self, source_ip: str, identity_token: str) -> dict[str, object] | None:
"""The orchestrator's `/resolve` payload for this client, or None if
unattributed (a clean `403`). Raises `PolicyResolveError` on an
unreachable / unexpected-status / malformed response so every caller
can fail closed. Shared by `resolve` and `resolve_bottle_id`."""
body = json.dumps(
{"source_ip": source_ip, "identity_token": identity_token}
).encode()
req = urllib.request.Request(
f"{self._base}/resolve", data=body, method="POST",
headers={"Content-Type": "application/json"},
)
try:
with urllib.request.urlopen(req, timeout=self._timeout) as resp:
payload = json.loads(resp.read())
except urllib.error.HTTPError as e:
if e.code == 403:
return None # unattributed → fail closed (caller denies)
raise PolicyResolveError(f"/resolve returned HTTP {e.code}") from e
except (urllib.error.URLError, TimeoutError, OSError, ValueError) as e:
raise PolicyResolveError(f"/resolve unreachable or malformed: {e}") from e
return payload if isinstance(payload, dict) else {}
def resolve(self, source_ip: str, identity_token: str = "") -> str | None:
"""The calling bottle's policy blob, or None if unattributed. Always
fetches from the orchestrator so revocations / changes / teardowns
are honored immediately.
`identity_token` is optional *transitionally* omitting it resolves
by source IP alone (the network-layer attribution, sound by
construction on Firecracker's `/31`+nft, weaker elsewhere). The
consolidated end state makes the token mandatory so the app-layer
defense is always enforced, not silently degraded; that flips at the
`/resolve` boundary once identity-token *delivery* lands (a PRD 0070
open question we can't require what the agent can't yet present).
Raises `PolicyResolveError` if the orchestrator can't be reached."""
payload = self._post_resolve(source_ip, identity_token)
if payload is None:
return None
policy = payload.get("policy")
return policy if isinstance(policy, str) else ""
def resolve_bottle_id(self, source_ip: str, identity_token: str = "") -> str | None:
"""The calling bottle's id, or None if unattributed. This is the
source-IP-keyed identity the git-gate uses to select the bottle's
repo namespace (there is no per-bottle policy blob to parse the
bottle *is* the namespace). Same fail-closed contract as `resolve`."""
payload = self._post_resolve(source_ip, identity_token)
if payload is None:
return None
bottle_id = payload.get("bottle_id")
return bottle_id if isinstance(bottle_id, str) and bottle_id else None
def resolve_policy_and_bottle_id(
self, source_ip: str, identity_token: str = "",
) -> tuple[str | None, str | None]:
"""Both the policy blob and the bottle id in a single `/resolve` — so
a caller that needs each (the egress addon: policy for routing, bottle
id to key the calling bottle's supervise queue + safelist) makes one
round-trip, not two. Returns `(None, None)` when unattributed (a clean
`403`). Raises `PolicyResolveError` if the orchestrator can't be
reached, so the caller still fails closed."""
payload = self._post_resolve(source_ip, identity_token)
if payload is None:
return None, None
policy = payload.get("policy")
bottle_id = payload.get("bottle_id")
return (
policy if isinstance(policy, str) else "",
bottle_id if isinstance(bottle_id, str) and bottle_id else None,
)
__all__ = ["PolicyResolver", "PolicyResolveError"]
+4 -2
View File
@@ -7,11 +7,13 @@ import sqlite3
from pathlib import Path
try:
from .supervise_types import Proposal, Response, host_db_path
from .supervise_types import Proposal, Response
from .paths import host_db_path
from .db_store import DbStore
from .migrations import TableMigrations
except ImportError:
from supervise_types import Proposal, Response, host_db_path # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from supervise_types import Proposal, Response # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from paths import host_db_path # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from db_store import DbStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from migrations import TableMigrations # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
+2 -5
View File
@@ -23,13 +23,10 @@ class StoreManager:
def __init__(self, db_path: Path | None = None) -> None:
if db_path is None:
# Lazy import to avoid a circular dependency: supervise imports
# StoreManager at module level; StoreManager must not import
# supervise at module level in return.
try:
from .supervise import host_db_path
from .paths import host_db_path
except ImportError:
from supervise import host_db_path # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from paths import host_db_path # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
db_path = host_db_path()
self.db_path = db_path
+6 -16
View File
@@ -41,7 +41,6 @@ try:
from .supervise_types import (
ACTION_OPERATOR_EDIT,
AuditEntry,
HOST_DB_FILENAME,
Proposal,
Response,
STATUSES,
@@ -54,13 +53,11 @@ try:
TOOL_EGRESS_TOKEN_ALLOW,
TOOL_GITLEAKS_ALLOW,
TOOL_LIST_EGRESS_ROUTES,
bot_bottle_root,
)
except ImportError:
from supervise_types import ( # type: ignore[import-not-found,no-redef] # pylint: disable=import-error,no-name-in-module
ACTION_OPERATOR_EDIT,
AuditEntry,
HOST_DB_FILENAME,
Proposal,
Response,
STATUSES,
@@ -73,10 +70,15 @@ except ImportError:
TOOL_EGRESS_TOKEN_ALLOW,
TOOL_GITLEAKS_ALLOW,
TOOL_LIST_EGRESS_ROUTES,
bot_bottle_root,
)
try:
from .paths import bot_bottle_root
except ImportError: # flat imports inside the sidecar bundle
from paths import bot_bottle_root # type: ignore[import-not-found,no-redef] # pylint: disable=import-error,no-name-in-module
SUPERVISE_HOSTNAME = "supervise"
SUPERVISE_PORT = 9100
@@ -106,16 +108,6 @@ def audit_dir() -> Path:
return bot_bottle_root() / "audit"
def host_db_path() -> Path:
# Calls bot_bottle_root() through this module's globals so that patches
# on supervise.bot_bottle_root propagate to callers going through
# supervise.host_db_path().
#
# Kept in its own "db" subdirectory (see supervise_types.host_db_path
# for why) — must stay in sync with that copy.
return bot_bottle_root() / "db" / HOST_DB_FILENAME
def audit_log_path(component: str, slug: str) -> Path:
return audit_dir() / f"{component}-{slug}.log"
@@ -297,8 +289,6 @@ __all__ = [
"archive_proposal",
"audit_dir",
"audit_log_path",
"bot_bottle_root",
"host_db_path",
"list_pending_proposals",
"list_all_pending_proposals",
"read_audit_entries",
+50 -5
View File
@@ -15,6 +15,12 @@ The bottle slug arrives via SUPERVISE_BOTTLE_SLUG env (stamped at
container creation by the backend's start step). SUPERVISE_DB_PATH
points at the bind-mounted host database.
Consolidated (PRD 0070): when BOT_BOTTLE_ORCHESTRATOR_URL is set, one
shared server fronts every bottle and attributes each proposal to the
calling bottle by source IP (resolved from the orchestrator) instead of a
fixed slug an unattributed source fails closed. Unset the legacy
per-bottle single-tenant server, unchanged.
Speaks MCP over HTTP+JSON-RPC. Methods handled:
* `initialize` handshake; returns server info + caps.
@@ -40,16 +46,18 @@ import time
import typing
import urllib.error
import urllib.request
from dataclasses import dataclass
from dataclasses import dataclass, replace
try:
# Same-directory imports inside the bundle container; these files are
# COPYed flat under /app by Dockerfile.sidecars.
from egress_addon_core import LOG_OFF, load_config
from policy_resolver import PolicyResolveError, PolicyResolver
import supervise as _sv
except ModuleNotFoundError:
# Package imports for host-side tests and tooling.
from .egress_addon_core import LOG_OFF, load_config
from .policy_resolver import PolicyResolveError, PolicyResolver
from . import supervise as _sv
@@ -73,6 +81,12 @@ DEFAULT_RESPONSE_TIMEOUT_SECONDS = 30.0
MIN_RESPONSE_POLL_INTERVAL_SECONDS = 0.05
EGRESS_LIST_TIMEOUT_SECONDS = 5.0
# Consolidated (multi-tenant) mode: when set, one shared supervise server
# fronts every bottle and attributes each proposal to the calling bottle by
# source IP (resolved from the orchestrator), instead of a single
# SUPERVISE_BOTTLE_SLUG env. Unset → legacy per-bottle single-tenant.
ORCHESTRATOR_URL_ENV = "BOT_BOTTLE_ORCHESTRATOR_URL"
@dataclass(frozen=True)
class JsonRpcRequest:
@@ -511,9 +525,30 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
if method == "tools/list":
return handle_tools_list(req.params)
if method == "tools/call":
return handle_tools_call(req.params, config)
# Attribute the proposal to the calling bottle. Single-tenant → the
# env slug on `config`; consolidated → the source-IP-resolved
# bottle id, so one shared server queues each bottle's proposal
# under its own slug.
return handle_tools_call(req.params, self._attributed_config(config))
raise _RpcClientError(ERR_METHOD_NOT_FOUND, f"method not found: {method}")
def _attributed_config(self, config: ServerConfig) -> ServerConfig:
"""The ServerConfig with `bottle_slug` bound to *this request's* bottle.
Single-tenant (no resolver): unchanged. Consolidated: the bottle id
attributed from the source IP **fail-closed**, an unattributed or
unreachable source raises so no proposal is queued under the wrong (or
empty) slug."""
resolver = getattr(self.server, "policy_resolver", None)
if resolver is None:
return config
try:
bottle_id = resolver.resolve_bottle_id(self.client_address[0])
except PolicyResolveError as e:
raise _RpcInternalError(f"orchestrator unreachable, cannot attribute: {e}") from e
if not bottle_id:
raise _RpcInternalError("request source is not attributed to a bottle")
return replace(config, bottle_slug=bottle_id)
def _write_jsonrpc(self, body: bytes) -> None:
self.send_response(200)
self.send_header("Content-Type", "application/json")
@@ -537,6 +572,9 @@ class MCPServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
allow_reuse_address = True
daemon_threads = True
config: ServerConfig = ServerConfig(bottle_slug="")
# None → single-tenant (proposals use config.bottle_slug); set → consolidated
# (each proposal attributed to the source-IP-resolved bottle).
policy_resolver: "PolicyResolver | None" = None
# --- Entry point -----------------------------------------------------------
@@ -548,15 +586,17 @@ def serve(
port: int = _sv.SUPERVISE_PORT,
bind: str = "0.0.0.0",
response_timeout_seconds: float = DEFAULT_RESPONSE_TIMEOUT_SECONDS,
resolver: "PolicyResolver | None" = None,
) -> typing.NoReturn:
server = MCPServer((bind, port), MCPHandler)
server.config = ServerConfig(
bottle_slug=bottle_slug,
response_timeout_seconds=response_timeout_seconds,
)
server.policy_resolver = resolver
mode = "multi-tenant" if resolver else f"slug={bottle_slug!r}"
sys.stderr.write(
f"supervise listening on {bind}:{port}; "
f"slug={bottle_slug!r}; "
f"supervise listening on {bind}:{port}; {mode}; "
f"tools: {', '.join(t['name'] for t in TOOL_DEFINITIONS)}\n" # type: ignore[arg-type]
)
sys.stderr.flush()
@@ -571,8 +611,12 @@ def serve(
def main(argv: list[str]) -> int:
del argv # config is env-only, no CLI flags
orch_url = os.environ.get(ORCHESTRATOR_URL_ENV, "").strip()
resolver = PolicyResolver(orch_url) if orch_url else None
bottle_slug = os.environ.get("SUPERVISE_BOTTLE_SLUG", "")
if not bottle_slug:
# Consolidated mode resolves the slug per request, so the env slug is
# optional there; single-tenant still requires it.
if not bottle_slug and resolver is None:
sys.stderr.write("supervise: SUPERVISE_BOTTLE_SLUG env is unset\n")
return 2
port = int(os.environ.get("SUPERVISE_PORT", str(_sv.SUPERVISE_PORT)))
@@ -587,6 +631,7 @@ def main(argv: list[str]) -> int:
port=port,
bind=bind,
response_timeout_seconds=response_timeout_seconds,
resolver=resolver,
)
return 0 # serve() does not return
+5 -32
View File
@@ -1,26 +1,19 @@
"""Shared types and path helpers for supervise stores (PRD 0013).
"""Shared types for supervise stores (PRD 0013).
Extracted from supervise.py so queue_store and audit_store can import
Proposal, Response, AuditEntry, and host_db_path without creating a
circular import (supervise imports from queue_store/audit_store and
vice-versa).
Proposal, Response, and AuditEntry without creating a circular import
(supervise imports from queue_store/audit_store and vice-versa).
Patching bot_bottle_root on this module propagates through host_db_path
because host_db_path looks up bot_bottle_root via sys.modules[__name__]
at call time rather than capturing it at import time.
The path helpers (`bot_bottle_root`, `host_db_path`, `HOST_DB_FILENAME`)
live in `paths` import them from there.
"""
from __future__ import annotations
import dataclasses
import sys
import uuid
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
HOST_DB_FILENAME = "bot-bottle.db"
TOOL_EGRESS_BLOCK = "egress-block"
TOOL_EGRESS_ALLOW = "egress-allow"
@@ -43,23 +36,6 @@ STATUSES: tuple[str, ...] = (STATUS_APPROVED, STATUS_MODIFIED, STATUS_REJECTED)
ACTION_OPERATOR_EDIT = "operator-edit"
def bot_bottle_root() -> Path:
return Path.home() / ".bot-bottle"
def host_db_path() -> Path:
# Look up bot_bottle_root through this module's live namespace so that
# monkey-patches on supervise_types.bot_bottle_root take effect.
#
# Lives in its own "db" subdirectory, not directly under
# bot_bottle_root(), so backends that can only bind-mount
# directories (macos_container's `container run --mount`, which
# rejects file sources with "is not a directory") can share this
# one file with a sidecar without also exposing bot_bottle_root()'s
# other contents (git-gate keys, per-bottle state, etc.).
return sys.modules[__name__].bot_bottle_root() / "db" / HOST_DB_FILENAME
def _require_str(raw: dict[str, object], key: str) -> str:
value = raw.get(key)
if not isinstance(value, str):
@@ -171,7 +147,6 @@ class AuditEntry:
__all__ = [
"ACTION_OPERATOR_EDIT",
"AuditEntry",
"HOST_DB_FILENAME",
"Proposal",
"Response",
"STATUSES",
@@ -184,6 +159,4 @@ __all__ = [
"TOOL_EGRESS_TOKEN_ALLOW",
"TOOL_GITLEAKS_ALLOW",
"TOOL_LIST_EGRESS_ROUTES",
"bot_bottle_root",
"host_db_path",
]
@@ -1,10 +1,16 @@
# PRD 0069: Firecracker-native, Docker-free backend
- **Status:** Draft
- **Status:** Draft (partially superseded)
- **Author:** Claude
- **Created:** 2026-07-12
- **Issue:** #348
> **Superseded in part by [PRD 0070](0070-per-host-orchestrator.md) (#351):**
> the sidecar-consolidation framing here (Stage 1, per-host sidecar; Stage 4,
> sidecar-as-VM) is taken over by 0070's per-host orchestrator. This PRD still
> owns the docker-free **image-building** work — Stage 2 (nix-built fixed
> images, a dependency of 0070) and Stage 3 (in-VM Dockerfile builder).
## Summary
Make the Firecracker backend depend on **firecracker + KVM only**, removing
+401
View File
@@ -0,0 +1,401 @@
# PRD 0070: Per-host orchestrator service
- **Status:** Draft
- **Author:** Claude
- **Created:** 2026-07-12
- **Issue:** #351
- **Supersedes:** the Stage-1 / Stage-4 sidecar-consolidation framing of
PRD 0069 (#348). Depends on 0069's nix-built fixed images (Stage 2) for
bootstrapping; 0069 still owns the docker-free image-building work.
## Summary
Replace the **per-bottle gateway bundle** with a single **persistent,
per-host orchestrator**: one long-lived service that runs the gateway
functions (egress / git-gate / supervise), coordinates with the console,
and brokers agent launches and teardown. It is **virtualized from the
start** using each backend's native isolation primitive — a Firecracker
microVM on the Firecracker backend, an Apple container on macOS, a Docker
container on the legacy backend — and is fronted by a single
**backend-agnostic contract**. Per-backend variation lives on
`BottleBackend`, not in the orchestrator.
## Motivation
Today each bottle spins up its own gateway bundle (egress mitmproxy +
git-gate + supervise). That costs:
- **Resources.** N bottles → N heavy bundles booting and idling.
- **Operational churn.** Per-launch container/VM lifecycle for the
gateways, a control path baked at launch and torn down at exit.
- **A blurry contract.** "How a bottle talks to its gateway" is
re-implemented per backend instead of being one agreed interface.
A per-host orchestrator collapses the first two and forces the third to
be made explicit. It's also the component that will own per-host runtime
**state** (slot leases, the approval queue, the bottle registry) — today
that's ad-hoc `fcntl`-locked files.
## Security review (read this first)
Consolidation is a real change to the trust model. The goal is to **not
significantly weaken** the posture; some properties strengthen, some
weaken, and the weakened ones must be mitigated by design, not hand-waved.
### What gets stronger
- **Build/host isolation of untrusted inputs** (with 0069 Stage 3): user
Dockerfiles build in a disposable VM instead of on the host.
- **One audited privileged surface.** Today the launcher runs as the full
host user and needs the Docker socket (root-equivalent). The orchestrator
model replaces that with a **thin launch broker** (below) — a small,
structured, auditable privileged core instead of a fat socket.
- **Attribution is enforced, not assumed.** Making source-IP identity a
first-class contract invariant (below) means each backend must *prove*
it, rather than the gateway implicitly trusting network position.
### What gets weaker, and the mitigation
1. **Secret concentration.** Per-bottle gateways isolate secrets at the
process boundary — each holds only its bottle's tokens/keys. A host
orchestrator concentrates **every bottle's** egress tokens, git deploy
keys, and the console credential in one long-lived process. A single
attribution bug leaks bottle A's token into bottle B's request — a class
of bug that *cannot exist* per-bottle.
- *Mitigation:* lean on the enforced source-IP invariant for
attribution; keep the most secret-dense, least-shareable service
(**git-gate**, per-repo deploy keys, no natural source-IP scoping)
**per-bottle** unless there's a compelling reason; scope each secret
to the bottle in the state DB so a lookup can't return the wrong
bottle's secret by construction (key every secret access by the
verified source identity, never by ambient state).
2. **Shared fate.** Orchestrator down = no new launches, and running
agents lose egress / git / supervise. Compromise = the whole host's
fleet, plus launch authority, plus the console token.
- *Mitigation:* the orchestrator is itself confined (its own VM/container
with its own fail-closed egress); make it **restartable without killing
running agent VMs** (agents keep running; they briefly lose gateway
connectivity until it's back); persist state to a host volume so a
restart re-adopts live bottles rather than losing them.
3. **The launch broker is the new privileged core.** We don't eliminate
host privilege — we shrink and relocate it. If the broker accepts
arbitrary paths/commands, the orchestrator VM can escape through it.
- *Mitigation:* the broker takes **structured requests only** — "launch
bottle from *this* content-addressed, nix-built rootfs on TAP slot
*k*", never "run this argv". It validates against a fixed image set,
not caller-supplied paths. It is small enough to audit line-by-line.
4. **The egress proxy now parses every bottle's traffic in one process.**
Higher blast radius for a mitmproxy/TLS-bump bug.
- *Mitigation:* this is the argument for virtualizing the orchestrator
from the start (Stage B, not a host daemon) — the code that TLS-bumps
and parses agent traffic and holds every token runs **inside its own
confined VM**, not as a host process. If egress sharing's blast radius
feels too high, egress can stay per-bottle while supervise (near-zero
secrets) goes host-level first.
### The attribution invariant
Source-IP attribution is what makes a shared orchestrator safe: one
process serves every bottle and tells them apart by source address. The
*mechanism* is identical everywhere (read source IP → look up bottle); the
**guarantee that the address can't be forged is a per-backend
responsibility** and part of the contract:
> **Invariant:** a packet's source address, as seen by the orchestrator,
> *provably* identifies the originating bottle.
- **Firecracker** — enforced by the `/31` point-to-point TAP + the
`bot_bottle_fc` nft table (strongest; already built).
- **Docker** — the per-bottle `--internal` network + anti-spoof; weaker,
must be made explicit.
- **Apple** — the host-only network.
If a backend can't honor the invariant, source-IP consolidation is not
safe there and that backend keeps per-bottle gateways. The invariant is a
hard precondition, not an aspiration.
**Defense-in-depth — a per-bottle identity token.** On top of the
network-layer invariant, inject a per-bottle secret token into every
request the agent makes to the orchestrator (the agent already egresses
through the gateway proxy, so this is cheap to add). It gives an
**application-layer** proof of identity independent of the network layer:
- On **Firecracker** the `/31` + nft already make source IP unspoofable
*by construction*, so the token is belt-and-suspenders there — but cheap
insurance against a misconfigured invariant.
- On **weaker backends** (Docker) it is load-bearing, providing attribution
that doesn't lean on network anti-spoof.
Requirements: the token is **per-bottle, unguessable, and
non-cross-leakable** — a bottle can only ever prove it is *itself* (it
can't learn another bottle's token, given bottle isolation), so a hostile
agent gains nothing by presenting it. The orchestrator provisions the
token at launch and the gateway requires it to attribute + authorize.
Note this hardens *attribution*, not *secret exposure*: it's app-layer, so
a compromised orchestrator still sees every token (that's the
concentration problem, addressed separately under Secret handling).
### Secret handling — a FUTURE pattern (not v1)
> **Status: future / not required for the initial orchestrator.** The
> initial cut can inject secrets as today; this section records the
> direction so v1 doesn't paint itself into a corner. Tracked as its own
> work in **#355** (generic `SecretProvider`).
The residual weakness after all of the above is **long-lived credential
concentration** — the data-plane proxies must hold every bottle's upstream
tokens because the agent must never see them. You can't policy-gate the
component whose job is to *use* all the secrets, so a full RCE of a proxy
drains its authorized set regardless. Two moves bound this without
pretending to prevent it:
1. **Vault as a separate trust domain.** The long-lived *roots* live in a
distinct process (ideally its own VM) that the byte-parsing data plane
never shares memory with. The proxies request secrets from it; the
crown jewels are not in the process an agent-facing parser can pop.
2. **Derive short-lived, scoped creds where the upstream allows it.** The
vault holds the root and mints expiring, narrowly-scoped credentials
per bottle-start (or per request) — GitHub App installation tokens,
OAuth/STS token exchange, forge deploy tokens. A compromise then leaks
short-lived material, not permanent keys. For upstreams stuck on static
keys the vault passes the value through (only the at-rest / audit /
revocation benefits apply, not lifetime reduction) — this residue is
accepted and documented, not solved.
Even a plain per-request fetch (no derivation) still buys **at-rest**
reduction (process memory holds only in-flight secrets), a **detection /
rate-limit / revocation chokepoint**, and clean **cross-component
scoping** (an egress RCE can't request git-gate's creds). It does *not*
prevent abuse of currently-authorized access during the compromise window.
**Mechanism (see #355):** generalize the existing `DeployKeyProvisioner`
(PRD 0048) into a user-extensible **`SecretProvider`** droppable into the
manifest anywhere a raw token value is accepted, discovered from
`~/.bot-bottle/contrib/<name>/secret_provider.py` exactly as user
`AgentProvider`s are — because maintaining every forge/cloud/OAuth
provider in-tree is untenable. The orchestrator's vault mints via these
providers; but the abstraction is shippable independently and today's
per-bottle gateways can use it too.
## Design
### The contract (backend-agnostic)
Three surfaces; only one is per-backend.
1. **Control plane (CLI / console → orchestrator)** — an RPC:
`launch_bottle`, `teardown_bottle`, `register_policy`,
`deregister_bottle`, `supervise_queue`. Fully backend-agnostic. Both the
local `cli.py` and the remote console funnel through it, so policy is
uniform and `cli.py` becomes a thin client rather than a parallel
launcher. **Transport: HTTP** — the most universal/reliable choice on
every host (no vsock/unix-socket portability caveats); a local unix
socket is a fine optimization, but HTTP is the wire contract.
2. **Data plane (agent → orchestrator)** — the egress / git / supervise
endpoints. Already agnostic today (agents dial `http://gateway:9099`);
only the *address* and *how packets get there* are per-backend.
3. **Launch / wire (orchestrator → backend)** — the irreducibly
backend-specific part; lives on `BottleBackend`.
### One `Orchestrator`, no subclass tree
The orchestrator is a **single concrete class** holding all the
backend-neutral logic — egress addon, git-gate, supervise, source-IP
attribution, live-reload control plane, console client. It never branches
on backend; it *composes* a `BottleBackend`. That composition is what makes
the contract agnostic: there is nothing backend-specific left in the
orchestrator to leak.
Rejected alternative: an `Orchestrator` ABC with per-backend
implementations. The interesting logic (proxies, attribution, control
plane) is backend-neutral, so three subclasses would triplicate the hard
part; and a second hierarchy paralleling `BottleBackend` reintroduces the
same hand-maintained lockstep coupling we just removed from the netpool
constants (PR #350). Composition over a parallel tree.
### `BottleBackend` absorbs the per-backend variation
A small, cohesive surface — reused for launching agent bottles *and* the
orchestrator's own unit (the orchestrator is just another native unit):
```
launch_unit(spec) -> Handle # agent bottle OR the orchestrator itself
# (fc microVM / apple ctr / docker ctr)
wire(unit, endpoint) -> None # DNAT+forward (fc) | attach shared net (docker/apple)
endpoint_of(unit) -> Endpoint # address resolution
health(unit) -> Status
```
Plus the **launch broker** — the answer to "a VM/container can't spawn its
own host-network siblings." The orchestrator can't directly open host
`/dev/kvm` + a host TAP fd (Firecracker), and a container can't spawn
siblings without a root-equivalent socket (Docker). So every backend
exposes a broker the orchestrator calls to launch an agent:
- **Firecracker** — a thin, structured host shim (see security #3). This
replaces today's implicit "launcher runs as host user."
- **Docker** — the socket today (fat, root-equivalent — the thing 0069's
Stage 3 removes); a narrower broker later.
- **Apple** — the `container` CLI/daemon.
**Broker request schema:** human-readable JSON of **static flags + ids
only** (never free-form paths/argv — that's what keeps it un-coercible
into arbitrary launches), wrapped as a **signed JWT** so the broker
verifies *provenance*: the request came from the real orchestrator, not a
forged one from a compromised co-located component. The orchestrator signs;
the broker verifies with the orchestrator's public key (provisioned at
broker install). This is the concrete form of security #3's "structured
requests only." Same JSON-with-ids + JWT shape for all
orchestrator↔broker/gateway communication; cases that don't fit get
handled as they arise, with this as the default.
If `BottleBackend` bloats, the pressure valve is composition one level
down: vend a `backend.network()` / `Wiring` collaborator rather than
piling methods on — the same discipline, recursed.
### State: one SQLite DB, owned by the orchestrator
The orchestrator is the natural owner of per-host **runtime state**:
- pool **slot leases** (which bottle holds slot *i*) — replaces today's
`fcntl`-locked files with SQLite transactions;
- the **supervise approval queue** + remembered approvals;
- the **live bottle registry** (source IP → bottle → policy/secrets refs),
the lookup table the attribution invariant reads.
This is deliberately **not** a "single source of truth for all config."
Config splits into three tiers with different homes:
| Tier | Example | Home |
|---|---|---|
| Build-time constants | pool size, IP base, nft table | flat `.env` (PR #350) — must be readable by Nix eval + root bash, zero runtime |
| User-authored config | bottle manifests, egress routes, secret refs | declarative files under `~/.bot-bottle/` — trust boundary at `$HOME`, git-trackable, "unknown keys die at load" |
| Runtime state | slot leases, approvals, registry | one shared **`bot-bottle.db`**, solely owned by the orchestrator |
SQLite is right for the runtime tier (mutable, concurrent, queried) and
wrong for the other two (Nix can't read it at eval time; it fights the
declarative manifest trust model). Keep the tiers separate.
**One shared `bot-bottle.db` for all runtime state** (decided in review).
The registry co-tenants the existing host `bot-bottle.db` — the `DbStore`
framework already namespaces each store by `schema_key`, so slot
leases / approvals / registry share one file. One place to query, back up,
and integrate a console against.
- **Host-resident, for durability.** Re-adoption sweeps the registry after
an orchestrator restart, so state *must* outlive the orchestrator
instance. The file lives on the host (`bot_bottle_root()/db/bot-bottle.db`);
the orchestrator unit reaches it, it doesn't carry it.
- **Integrity by sole ownership, not mount permissions.** Agents can't
touch the DB directly wherever it lives (network-isolated in their
bottles). The risk is a *compromised agent-facing data-plane service*
(egress/git-gate, which parse hostile bytes) writing the registry and
forging attribution. Because it's now one shared file, coarse `ro`/`rw`
mount-splitting no longer isolates the registry — so the rule is stronger
and simpler: **only the orchestrator (control plane) opens `bot-bottle.db`;
the data plane and the console reach state through the control-plane RPC,
never a direct file handle.** No agent-facing component gets the file, so
none can forge attribution. (This supersedes the earlier `ro`-mount idea.)
- *Transitional caveat:* today the per-bottle **supervise gateway
rw-bind-mounts `bot-bottle.db`** to write proposals — exactly the
pattern the orchestrator removes (supervise consolidates into the
orchestrator; gateway writes become RPC calls). Until that lands, don't
put the attribution registry behind a data-plane-writable mount.
Implementation note for the VM slices: SQLite **WAL** over a guest share
(virtiofs/9p) is finicky (the `-shm`/`-wal` files need real mmap/locking),
which is a second reason the DB wants a **host-side owner** the orchestrator
reaches over the RPC rather than a shared mount into the VM. WAL on the
shared DB is therefore a deliberate, tested future change — not enabled ad
hoc. `sqlite3` itself is stdlib, so "the host needs SQLite" is a non-cost.
## Sequencing
Jump straight to the **virtualized** end state (not a host-daemon stepping
stone): a host daemon's agent→`localhost` transport is throwaway once the
orchestrator becomes a VM. Decouple the two risks instead:
- **Consolidation risk** (one process, all secrets, attribution, reload)
and **packaging/transport risk** (VM-to-VM wiring, the shim) are
independent. Develop the orchestrator **service as a plain process
dev-harness** first, so the consolidation logic (attribution, reload,
secret handling) is proven with fast iteration — *then* wrap that exact
service in the VM and solve wiring separately.
Backend order (cheapest proof → hardest → last):
1. **Docker orchestrator** — nearly free (the gateway bundle is already
containers; collapse N bundles into one persistent container). Proves
consolidation + the `BottleBackend` seam with the least moving parts.
2. **Firecracker orchestrator** — the real work: the shim + VM-to-VM
routing (host forwards `bbfcN` → orchestrator TAP; the nft table grows
forward rules where today it drops all non-DNAT egress). Built against
the dev-harness so the app logic is already proven.
3. **macOS (Apple container)** — last (container-to-container networking).
Keep the gateway **service one shared thing** throughout.
## Non-goals
- Removing OCI/Dockerfile support for agent images (0069's concern).
- A single database for *all* config (see the three-tier table).
- Changing the per-bottle isolation of agent workloads — only the gateway
is consolidated; agents stay one-VM/container-each.
## Relationship to other work
- **PRD 0069 (#348):** 0070 subsumes its Stage 1 (per-host sidecar) and
Stage 4 (sidecar-as-VM). 0069 retains Stage 2 (nix-built fixed images —
a **dependency** here: the orchestrator and agent base must be
nix-built so the broker launches from a fixed image set and bootstrapping
has no chicken-and-egg) and Stage 3 (in-VM Dockerfile builder).
- **Minimal CI runner (paused):** the Firecracker broker + no host Docker
is what lets a dedicated `gitea` runner user drop the root-equivalent
`docker` group — it only needs broker-socket access + `kvm`/pool group
membership. This work unblocks it.
- **Generic `SecretProvider` (#355):** the future secret-handling
mechanism (see "Secret handling") — generalizes PRD 0048's
`DeployKeyProvisioner` into a user-extensible provider that mints
short-lived creds. Shippable independently; the orchestrator's vault
mints through it.
- **PR #350 (netpool single-source):** the same "one source per fact,
composition over parallel hierarchies" discipline the contract follows.
## Decisions (review 2026-07-13)
- **Egress sharing:** **consolidate egress** (worth it). Treat it as a
minimal, hardened attack surface for a malicious agent rather than
keeping it per-bottle; pair it with the identity token above and the
short-lived-vault-token mitigations (Secret handling / #355).
- **Control-plane transport:** **HTTP** — most universal/reliable on every
host; unix socket is an optional local optimization (see the contract).
- **Broker request schema:** **signed-JWT JSON, static flags + ids only**
(see the launch broker) — provenance + un-coercible by construction.
- **State re-adoption:** the restart procedure is:
1. **Singleton:** a new orchestrator launch requires no pre-existing
orchestrator; any found (healthy or not) is fully shut down first.
2. Re-adoption **waits for the new orchestrator to be healthy**.
3. Once healthy, it discovers all agents needing adoption via **both the
SQLite registry and live process/VM inspection** *before serving any
other request*. The two-source sweep is what closes the *in-flight
launch* race — a launch that started (VM booting / slot claimed) but
hadn't committed to SQLite when the old orchestrator died would be
invisible to a SQLite-only sweep; process inspection catches it.
Launches should also write an **intent record ahead of committing
resources** so the sweep can reconcile intent vs. actual.
## Open questions
- **VM-to-VM routing:** per-backend, and in the design it *is*
`BottleBackend.wire()` (DNAT+forward for fc, shared-net for
docker/apple), not orchestrator logic. **Not a blocker** — resolvable at
implementation time (may require a bit of host modification, as the pool
setup already does on NixOS); an earlier "gateways in VMs" spike showed
it's feasible.
- **Live-reload protocol** for per-bottle policy over the HTTP control
plane (add/remove routes/keys/proposals without a restart).
- **Identity-token delivery:** exactly how the per-bottle token is placed
where the agent can present it but not swap in another bottle's.
+67 -93
View File
@@ -1,104 +1,70 @@
# bot-bottle Firecracker network pool — declarative NixOS module.
#
# The one-time privileged setup the Firecracker backend needs: a pool of
# user-owned point-to-point TAP devices plus a fail-closed nftables table
# that confines every microVM to its own sidecar.
# user- (or group-) owned point-to-point TAP devices plus a fail-closed
# nftables table that confines every microVM to its own sidecar.
#
# NON-INVASIVE BY DESIGN. It does NOT flip `networking.nftables.enable`
# (which would switch your whole host firewall backend) or
# `systemd.network.enable` (which would hand your interfaces to
# systemd-networkd). Instead a single systemd oneshot service brings the
# pool up on boot — the declarative equivalent of
# `sudo ./scripts/firecracker-netpool.sh up`. The `inet <tableName>` table
# is independent (its own hooks at priority -10), so it coexists with an
# iptables `networking.firewall`, Docker, ufw, firewalld, etc.
# systemd-networkd). Instead a single systemd oneshot brings the pool up
# on boot by running the SAME bring-up script as every other install
# path (`scripts/firecracker-netpool.sh`) — so the TAP/nft logic lives in
# exactly one place. The `inet <tableName>` table is independent (its own
# hooks at priority -10), so it coexists with an iptables
# `networking.firewall`, Docker, ufw, firewalld, etc.
#
# The oneshot is only restarted when this config changes, so a
# `nixos-rebuild switch` doesn't tear down TAPs out from under running
# VMs unless you actually changed the pool.
# The oneshot is only restarted when this config changes, and the shared
# script is non-destructive (it never tears down an existing TAP), so a
# `nixos-rebuild switch` won't cut TAPs out from under running VMs.
#
# The option defaults MUST match the backend constants in
# bot_bottle/backend/firecracker/netpool.py (pool size, IP base, iface
# prefix, table name). The backend reads the same values at launch from
# the BOT_BOTTLE_FC_* env vars; if you override an option here, override
# the matching env var for the CLI too (or set writeEnvFile = true below
# to have this module emit them). Keep the two sides in lockstep.
# Single source of the pool defaults: bot_bottle/backend/firecracker/
# netpool.defaults.env. This module readFile-parses it for the option
# defaults below, then passes the resolved values back to the script as
# Environment=, so the host pool and the CLI launcher can't drift.
{ config, lib, pkgs, ... }:
let
cfg = config.services.bot-bottle-firecracker;
# --- IPv4 <-> int (full 32-bit carry math) -------------------------
toOctets = s: map lib.toInt (lib.splitString "." s);
ipToInt = s:
let o = toOctets s; in
(lib.elemAt o 0) * 16777216
+ (lib.elemAt o 1) * 65536
+ (lib.elemAt o 2) * 256
+ (lib.elemAt o 3);
intToIp = n:
# --- shared single-source defaults ---------------------------------
# Parse the KEY=VALUE defaults file (the same one netpool.py and the
# shell script read). Pure eval — just readFile, no import-from-
# derivation — so it works for both flake and channel consumers.
readDefaults = file:
let
b0 = n / 16777216; r0 = n - b0 * 16777216;
b1 = r0 / 65536; r1 = r0 - b1 * 65536;
b2 = r1 / 256;
b3 = r1 - b2 * 256;
in "${toString b0}.${toString b1}.${toString b2}.${toString b3}";
lines = lib.splitString "\n" (builtins.readFile file);
keep = l: l != "" && !(lib.hasPrefix "#" l) && lib.hasInfix "=" l;
toPair = l:
let parts = lib.splitString "=" l;
in lib.nameValuePair (lib.head parts)
(lib.concatStringsSep "=" (lib.tail parts));
in lib.listToAttrs (map toPair (lib.filter keep lines));
baseInt = ipToInt cfg.ipBase;
defaults = readDefaults ../bot_bottle/backend/firecracker/netpool.defaults.env;
# Slot i: host = base + 2i (the VM's gateway), on its own /31.
slots = lib.genList (i: {
iface = "${cfg.ifacePrefix}${toString i}";
hostIp = intToIp (baseInt + 2 * i);
}) cfg.poolSize;
# The one bring-up implementation, shared with the sudo/systemd paths.
netpoolScript = ../scripts/firecracker-netpool.sh;
ip = "${pkgs.iproute2}/bin/ip";
nft = "${pkgs.nftables}/bin/nft";
# /31 alignment == an even final octet (only bit 0 matters for base+2i).
lastOctet = lib.toInt (lib.last (lib.splitString "." cfg.ipBase));
# Idempotent ruleset: (create-if-absent → delete → recreate) is atomic
# within one `nft -f`, so re-running `up` always lands a clean table.
nftFile = pkgs.writeText "bot-bottle-fc.nft" ''
table inet ${cfg.tableName}
delete table inet ${cfg.tableName}
table inet ${cfg.tableName} {
chain forward {
type filter hook forward priority -10; policy accept;
iifname != "${cfg.ifacePrefix}*" return
ct state established,related accept
ct status dnat accept
drop
}
chain input {
type filter hook input priority -10; policy accept;
iifname != "${cfg.ifacePrefix}*" return
ct state established,related accept
drop
}
}
'';
# The script needs ip/nft/sysctl + the usual coreutils. It gets every
# pool value via the unit's Environment=, so it never reads the shared
# defaults file (which isn't beside it once copied to the store).
runtimePath = with pkgs; [ iproute2 nftables procps coreutils gnused ];
# A non-owner user can't open a user-owned TAP, but any member of a
# TAP's owning GROUP can (the kernel checks owner-uid OR owning-group).
# So `group` lets an interactive user and a CI-runner user share one
# pool; `owner` is the single-user default.
tapOwn = if cfg.group != null then "group ${cfg.group}" else "user ${cfg.owner}";
ownEnv =
if cfg.group != null
then { BOT_BOTTLE_FC_GROUP = cfg.group; }
else { BOT_BOTTLE_FC_OWNER = cfg.owner; };
upScript = pkgs.writeShellScript "bot-bottle-fc-netpool-up" ''
set -eu
${lib.concatMapStringsSep "\n" (s: ''
${ip} link show ${s.iface} >/dev/null 2>&1 || ${ip} tuntap add dev ${s.iface} mode tap ${tapOwn}
${ip} addr replace ${s.hostIp}/31 dev ${s.iface}
${ip} link set ${s.iface} up
'') slots}
${nft} -f ${nftFile}
'';
downScript = pkgs.writeShellScript "bot-bottle-fc-netpool-down" ''
${nft} delete table inet ${cfg.tableName} 2>/dev/null || true
${lib.concatMapStringsSep "\n" (s: ''
${ip} link show ${s.iface} >/dev/null 2>&1 && ${ip} tuntap del dev ${s.iface} mode tap || true
'') slots}
'';
unitEnv = {
BOT_BOTTLE_FC_POOL_SIZE = toString cfg.poolSize;
BOT_BOTTLE_FC_IP_BASE = cfg.ipBase;
BOT_BOTTLE_FC_IFACE_PREFIX = cfg.ifacePrefix;
BOT_BOTTLE_FC_NFT_TABLE = cfg.tableName;
} // ownEnv;
in
{
options.services.bot-bottle-firecracker = {
@@ -106,25 +72,28 @@ in
poolSize = lib.mkOption {
type = lib.types.ints.positive;
default = 8;
default = lib.toInt defaults.BOT_BOTTLE_FC_POOL_SIZE;
defaultText = lib.literalMD "the shared `netpool.defaults.env` value";
description = "Number of pool slots (concurrent bottles). Must match BOT_BOTTLE_FC_POOL_SIZE.";
};
ipBase = lib.mkOption {
type = lib.types.str;
default = "10.243.0.0";
default = defaults.BOT_BOTTLE_FC_IP_BASE;
defaultText = lib.literalMD "the shared `netpool.defaults.env` value";
description = ''
Base IPv4 of the /31 pool; slot i uses host = base + 2i. Must
be /31-aligned (even final address) and must match
BOT_BOTTLE_FC_IP_BASE. Default is an obscure RFC-1918 /16 that
dodges docker/libvirt/k8s/LAN and, deliberately, Tailscale's
100.64.0.0/10 CGNAT range.
be /31-aligned (even final octet) and must match
BOT_BOTTLE_FC_IP_BASE. The shared default is an obscure RFC-1918
/16 that dodges docker/libvirt/k8s/LAN and, deliberately,
Tailscale's 100.64.0.0/10 CGNAT range.
'';
};
ifacePrefix = lib.mkOption {
type = lib.types.str;
default = "bbfc";
default = defaults.BOT_BOTTLE_FC_IFACE_PREFIX;
defaultText = lib.literalMD "the shared `netpool.defaults.env` value";
description = "TAP interface name prefix. Must match BOT_BOTTLE_FC_IFACE_PREFIX.";
};
@@ -153,7 +122,8 @@ in
tableName = lib.mkOption {
type = lib.types.str;
default = "bot_bottle_fc";
default = defaults.BOT_BOTTLE_FC_NFT_TABLE;
defaultText = lib.literalMD "the shared `netpool.defaults.env` value";
description = "nftables table name for the isolation boundary. Must match netpool.NFT_TABLE.";
};
@@ -171,7 +141,7 @@ in
config = lib.mkIf cfg.enable {
assertions = [
{
assertion = lib.mod baseInt 2 == 0;
assertion = lib.mod lastOctet 2 == 0;
message = "services.bot-bottle-firecracker.ipBase must be /31-aligned (even final octet); got ${cfg.ipBase}.";
}
{
@@ -183,17 +153,20 @@ in
# VM->sidecar traffic is DNAT'd and forwarded, so forwarding must be on.
boot.kernel.sysctl."net.ipv4.ip_forward" = 1;
# One oneshot brings up the whole pool (TAPs + independent nft table).
# No networking.nftables.enable / systemd.network.enable — see header.
# One oneshot brings up the whole pool (TAPs + independent nft table)
# by running the shared bring-up script. No networking.nftables.enable
# / systemd.network.enable — see header.
systemd.services."bot-bottle-firecracker-netpool" = {
description = "bot-bottle Firecracker TAP pool + nft isolation table";
wantedBy = [ "multi-user.target" ];
after = [ "network-pre.target" ];
path = runtimePath;
environment = unitEnv;
serviceConfig = {
Type = "oneshot";
RemainAfterExit = true;
ExecStart = upScript;
ExecStop = downScript;
ExecStart = "${pkgs.bash}/bin/bash ${netpoolScript} up";
ExecStop = "${pkgs.bash}/bin/bash ${netpoolScript} down";
};
};
@@ -202,6 +175,7 @@ in
BOT_BOTTLE_FC_POOL_SIZE=${toString cfg.poolSize}
BOT_BOTTLE_FC_IP_BASE=${cfg.ipBase}
BOT_BOTTLE_FC_IFACE_PREFIX=${cfg.ifacePrefix}
BOT_BOTTLE_FC_NFT_TABLE=${cfg.tableName}
'';
};
};
+40 -19
View File
@@ -33,24 +33,44 @@
# sudo ./scripts/firecracker-netpool.sh down
# ./scripts/firecracker-netpool.sh status
#
# Env overrides (must match the backend's util.py constants):
# BOT_BOTTLE_FC_POOL_SIZE number of slots (default 8)
# BOT_BOTTLE_FC_IP_BASE base IPv4 of the /31 pool (default 10.243.0.0)
# BOT_BOTTLE_FC_IFACE_PREFIX TAP name prefix (default bbfc)
# BOT_BOTTLE_FC_OWNER owning user (default $SUDO_USER or $USER)
# BOT_BOTTLE_FC_GROUP owning group; if set, TAPs are group-owned
# instead of user-owned, so any group member
# (e.g. an interactive user + a CI runner
# user) can open the pool. Overrides OWNER.
# Pool params default to the shared single-source file
# (bot_bottle/backend/firecracker/netpool.defaults.env); a matching
# env var overrides its key:
# BOT_BOTTLE_FC_POOL_SIZE number of slots
# BOT_BOTTLE_FC_IP_BASE base IPv4 of the /31 pool
# BOT_BOTTLE_FC_IFACE_PREFIX TAP name prefix
# BOT_BOTTLE_FC_NFT_TABLE isolation table name
# BOT_BOTTLE_FC_OWNER owning user (default $SUDO_USER or $USER)
# BOT_BOTTLE_FC_GROUP owning group; if set, TAPs are group-owned
# instead of user-owned, so any group member
# (e.g. an interactive user + a CI runner
# user) can open the pool. Overrides OWNER.
set -euo pipefail
POOL_SIZE="${BOT_BOTTLE_FC_POOL_SIZE:-8}"
IP_BASE="${BOT_BOTTLE_FC_IP_BASE:-10.243.0.0}"
PREFIX="${BOT_BOTTLE_FC_IFACE_PREFIX:-bbfc}"
# The single source of the pool defaults, shared with netpool.py and the
# Nix module. The Nix path passes every value as env (the script runs
# from the store, detached from this file), so this lookup only matters
# on the direct/sudo path where the script sits in the repo tree.
_SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
_DEFAULTS="$_SCRIPT_DIR/../bot_bottle/backend/firecracker/netpool.defaults.env"
_default() { # value of KEY=... from the shared file; empty if unavailable
[ -f "$_DEFAULTS" ] || return 0
sed -n "s/^$1=//p" "$_DEFAULTS" | tail -1
}
POOL_SIZE="${BOT_BOTTLE_FC_POOL_SIZE:-$(_default BOT_BOTTLE_FC_POOL_SIZE)}"
IP_BASE="${BOT_BOTTLE_FC_IP_BASE:-$(_default BOT_BOTTLE_FC_IP_BASE)}"
PREFIX="${BOT_BOTTLE_FC_IFACE_PREFIX:-$(_default BOT_BOTTLE_FC_IFACE_PREFIX)}"
TABLE="${BOT_BOTTLE_FC_NFT_TABLE:-$(_default BOT_BOTTLE_FC_NFT_TABLE)}"
OWNER="${BOT_BOTTLE_FC_OWNER:-${SUDO_USER:-$USER}}"
GROUP="${BOT_BOTTLE_FC_GROUP:-}"
TABLE="bot_bottle_fc"
# Fail loudly rather than provisioning a half/empty range if a value
# resolved to nothing (env unset AND the shared file unreadable).
for _v in POOL_SIZE IP_BASE PREFIX TABLE; do
[ -n "${!_v}" ] || { echo "error: $_v unresolved (set BOT_BOTTLE_FC_* or fix $_DEFAULTS)" >&2; exit 1; }
done
# Sidecar ports (must match the backend). egress=9099, supervise=9100,
# git-http=9420. Reached by the VM at its host-side TAP IP.
@@ -94,12 +114,13 @@ cmd_up() {
for i in $(seq 0 $((POOL_SIZE-1))); do
local dev host
dev="$(iface "$i")" ; host="$(host_ip "$i")"
if ip link show "$dev" >/dev/null 2>&1; then
ip link set "$dev" down 2>/dev/null || true
ip tuntap del dev "$dev" mode tap 2>/dev/null || true
fi
ip tuntap add dev "$dev" mode tap "${own_args[@]}"
ip addr add "$host/31" dev "$dev"
# Non-destructive + idempotent: only create a missing TAP (tearing
# an existing one down would cut a running VM), and `addr replace`
# is safe to re-run. Ownership is fixed at creation, so to change
# owner/group run `down` then `up`.
ip link show "$dev" >/dev/null 2>&1 \
|| ip tuntap add dev "$dev" mode tap "${own_args[@]}"
ip addr replace "$host/31" dev "$dev"
ip link set "$dev" up
echo " $dev host=$host guest=$(guest_ip "$i") $own_desc"
done
@@ -0,0 +1,56 @@
"""Integration: the Docker launch broker starts and removes a real container.
Gated on a reachable Docker daemon (skips cleanly otherwise). Uses a tiny
image; the container may exit immediately we only assert it exists after
launch and is gone after teardown.
"""
from __future__ import annotations
import secrets
import subprocess
import unittest
from bot_bottle.orchestrator.broker import LaunchRequest, sign_request
from bot_bottle.orchestrator.docker_broker import DockerBroker, container_name
from tests._docker import skip_unless_docker
IMAGE = "busybox"
@skip_unless_docker()
class TestDockerBrokerIntegration(unittest.TestCase):
def setUp(self) -> None:
self.secret = secrets.token_bytes(16)
self.broker = DockerBroker(self.secret)
self.bottle_id = "itest" + secrets.token_hex(4)
self.name = container_name(self.bottle_id)
self.addCleanup(
lambda: subprocess.run(
["docker", "rm", "--force", self.name],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
)
)
def _exists(self) -> bool:
proc = subprocess.run(
["docker", "ps", "-a", "--filter", f"name=^{self.name}$",
"--format", "{{.Names}}"],
stdout=subprocess.PIPE, text=True, check=False,
)
return self.name in proc.stdout.split()
def _submit(self, req: LaunchRequest) -> None:
self.broker.submit(sign_request(req, self.secret))
def test_launch_creates_then_teardown_removes(self) -> None:
self._submit(
LaunchRequest(op="launch", bottle_id=self.bottle_id, image_ref=IMAGE)
)
self.assertTrue(self._exists(), "container should exist after launch")
self._submit(LaunchRequest(op="teardown", bottle_id=self.bottle_id))
self.assertFalse(self._exists(), "container should be gone after teardown")
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,45 @@
"""Integration: the consolidated Docker gateway is an idempotent singleton.
Gated on a reachable Docker daemon. Uses a unique name so it never
collides with a real per-host gateway or a leftover from another run.
"""
from __future__ import annotations
import secrets
import subprocess
import unittest
from bot_bottle.orchestrator.gateway import DockerGateway
from tests._docker import skip_unless_docker
IMAGE = "busybox"
@skip_unless_docker()
class TestDockerGatewayIntegration(unittest.TestCase):
def setUp(self) -> None:
self.name = "bot-bottle-orch-gateway-itest-" + secrets.token_hex(4)
self.sc = DockerGateway(IMAGE, name=self.name)
self.addCleanup(self.sc.stop)
def _count(self) -> int:
proc = subprocess.run(
["docker", "ps", "-a", "--filter", f"name=^{self.name}$",
"--format", "{{.Names}}"],
stdout=subprocess.PIPE, text=True, check=False,
)
return sum(1 for n in proc.stdout.split() if n == self.name)
def test_ensure_is_idempotent_singleton(self) -> None:
self.sc.ensure_running()
self.assertEqual(1, self._count())
# A second ensure must not spawn a second container — one per host.
self.sc.ensure_running()
self.assertEqual(1, self._count())
self.sc.stop()
self.assertEqual(0, self._count())
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,36 @@
"""Integration: DockerGateway.image_exists reflects real docker state.
Gated on a reachable Docker daemon. Deliberately does NOT build the full
sidecar bundle (a heavy, slow image build) it exercises the `image_exists`
primitive that `ensure_built` gates on, against a tiny pulled image and a
name that can't exist.
"""
from __future__ import annotations
import subprocess
import unittest
from bot_bottle.orchestrator.gateway import DockerGateway
from tests._docker import skip_unless_docker
IMAGE = "busybox"
@skip_unless_docker()
class TestDockerGatewayImageExists(unittest.TestCase):
def test_image_exists_true_for_present_false_for_absent(self) -> None:
# Ensure the tiny image is present (build_if_missing is disabled here
# so this never triggers a Dockerfile.sidecars build).
subprocess.run(
["docker", "pull", IMAGE],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
)
self.assertTrue(DockerGateway(IMAGE, dockerfile=None).image_exists())
self.assertFalse(
DockerGateway("bot-bottle-nonexistent:doesnotexist", dockerfile=None).image_exists()
)
if __name__ == "__main__":
unittest.main()
+22 -3
View File
@@ -2,7 +2,7 @@
Isolates ``HOME`` to a throwaway directory for the entire unit suite so
no test ever reads or writes the real ``~/.bot-bottle`` (state, queue,
and audit dirs all derive from ``supervise.bot_bottle_root()``
and audit dirs all derive from ``paths.bot_bottle_root()``
``Path.home()``). Without this, a test that takes a ``flock`` on the
real audit log can **block indefinitely** when a live bottle's supervise
sidecar holds that lock observed as a hung ``coverage run`` at 0% CPU
@@ -10,8 +10,12 @@ and unisolated tests otherwise pollute the developer's home dir.
Individual tests that need their own ``HOME`` still override
``os.environ['HOME']`` and restore it; they now restore to this isolated
dir rather than the real one, so isolation holds either way. Tests that
patch ``supervise.bot_bottle_root`` directly are unaffected.
dir rather than the real one, so isolation holds either way.
Tests that need their own app-data root call ``use_bottle_root`` (below),
which sets ``BOT_BOTTLE_ROOT`` the supported override read by
``paths.bot_bottle_root`` instead of monkey-patching. One env setting
covers every module and every flat/package copy of the helper.
"""
from __future__ import annotations
@@ -20,6 +24,9 @@ import atexit
import os
import shutil
import tempfile
from collections.abc import Callable
from pathlib import Path
from unittest import mock
_real_home = os.environ.get("HOME")
_tmp_home = tempfile.mkdtemp(prefix="bot-bottle-unit-home.")
@@ -35,3 +42,15 @@ def _restore_home() -> None:
atexit.register(_restore_home)
def use_bottle_root(root: Path) -> Callable[[], None]:
"""Redirect ``paths.bot_bottle_root()`` to ``root`` by setting
``BOT_BOTTLE_ROOT`` the supported override. Returns a callable that
undoes it (store it as the test's restore hook, or pass to addCleanup).
Replaces monkey-patching ``paths.bot_bottle_root``; one env var covers
every module and the flat/package copies alike (they all read it)."""
patcher = mock.patch.dict(os.environ, {"BOT_BOTTLE_ROOT": str(root)})
patcher.start()
return patcher.stop
+3 -8
View File
@@ -7,7 +7,8 @@ import unittest
from pathlib import Path
from unittest.mock import patch
from bot_bottle import supervise, bottle_state
from bot_bottle import bottle_state
from tests.unit import use_bottle_root
from bot_bottle.backend import ActiveAgent
from bot_bottle.backend.freeze import get_freezer
from bot_bottle.backend.docker.freezer import DockerFreezer
@@ -18,13 +19,7 @@ from bot_bottle.backend.firecracker.freezer import FirecrackerFreezer
class _FakeHomeMixin:
def _setup_fake_home(self):
self._tmp = tempfile.TemporaryDirectory(prefix="freezer-test.")
original = supervise.bot_bottle_root
def fake_root() -> Path:
return Path(self._tmp.name) / ".bot-bottle"
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
self._restore = lambda: setattr(supervise, "bot_bottle_root", original)
self._restore = use_bottle_root(Path(self._tmp.name) / ".bot-bottle")
def _teardown_fake_home(self):
self._restore()
+3 -4
View File
@@ -13,7 +13,7 @@ from pathlib import Path
from unittest.mock import patch
from bot_bottle import bottle_state
from bot_bottle import supervise
from tests.unit import use_bottle_root
from bot_bottle.backend import BottleSpec
from bot_bottle.backend.docker import DockerBottleBackend
from bot_bottle.backend.resolve_common import mint_slug
@@ -55,11 +55,10 @@ class _FakeStateMixin:
def setUp(self) -> None:
self.tmp = tempfile.TemporaryDirectory(prefix="backend-prepare.")
self.root = Path(self.tmp.name) / ".bot-bottle"
self.original_root = supervise.bot_bottle_root
supervise.bot_bottle_root = lambda: self.root # type: ignore[assignment]
self._restore = use_bottle_root(self.root)
def tearDown(self) -> None:
supervise.bot_bottle_root = self.original_root # type: ignore[assignment]
self._restore()
self.tmp.cleanup()
+3 -4
View File
@@ -13,7 +13,7 @@ from pathlib import Path
from unittest.mock import MagicMock, call, patch
from bot_bottle import bottle_state
from bot_bottle import supervise
from tests.unit import use_bottle_root
from bot_bottle.backend import Bottle, BottleSpec, ExecResult
from bot_bottle.backend.docker import DockerBottleBackend
from bot_bottle.backend.firecracker import FirecrackerBottleBackend
@@ -55,11 +55,10 @@ class _FakeStateMixin:
self.tmp_dir = tempfile.TemporaryDirectory(prefix="backend-workspace.")
self.tmp = Path(self.tmp_dir.name)
self.root = self.tmp / ".bot-bottle"
self.original_root = supervise.bot_bottle_root
supervise.bot_bottle_root = lambda: self.root # type: ignore[assignment]
self._restore = use_bottle_root(self.root)
def tearDown(self) -> None:
supervise.bot_bottle_root = self.original_root # type: ignore[assignment]
self._restore()
self.tmp_dir.cleanup()
+2 -8
View File
@@ -6,7 +6,7 @@ import tempfile
import unittest
from pathlib import Path
from bot_bottle import supervise
from tests.unit import use_bottle_root
from bot_bottle import bottle_state
from bot_bottle.bottle_state import (
BottleMetadata,
@@ -18,13 +18,7 @@ from bot_bottle.bottle_state import (
class _FakeHomeMixin:
def _setup_fake_home(self):
self._tmp = tempfile.TemporaryDirectory(prefix="bottle-state-test.")
original = supervise.bot_bottle_root
def fake_root() -> Path:
return Path(self._tmp.name) / ".bot-bottle"
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
self._restore = lambda: setattr(supervise, "bot_bottle_root", original)
self._restore = use_bottle_root(Path(self._tmp.name) / ".bot-bottle")
def _teardown_fake_home(self):
self._restore()
+2 -8
View File
@@ -8,7 +8,7 @@ from pathlib import Path
from unittest.mock import MagicMock, patch
from bot_bottle.cli.commit import cmd_commit
from bot_bottle import supervise
from tests.unit import use_bottle_root
from bot_bottle import bottle_state
from bot_bottle.backend.freeze import CommitCancelled
@@ -16,13 +16,7 @@ from bot_bottle.backend.freeze import CommitCancelled
class _FakeHomeMixin:
def _setup_fake_home(self):
self._tmp = tempfile.TemporaryDirectory(prefix="cli-commit-test.")
original = supervise.bot_bottle_root
def fake_root() -> Path:
return Path(self._tmp.name) / ".bot-bottle"
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
self._restore = lambda: setattr(supervise, "bot_bottle_root", original)
self._restore = use_bottle_root(Path(self._tmp.name) / ".bot-bottle")
def _teardown_fake_home(self):
self._restore()
+3 -8
View File
@@ -8,7 +8,7 @@ import tempfile
import unittest
from pathlib import Path
from bot_bottle import supervise
from tests.unit import use_bottle_root
from bot_bottle import bottle_state
from bot_bottle.cli import start as start_mod
@@ -16,15 +16,10 @@ from bot_bottle.cli import start as start_mod
class _FakeHomeMixin:
def _setup_fake_home(self):
self._tmp = tempfile.TemporaryDirectory(prefix="cli-start-settle.")
self._original = supervise.bot_bottle_root
def fake_root() -> Path:
return Path(self._tmp.name) / ".bot-bottle"
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
self._restore = use_bottle_root(Path(self._tmp.name) / ".bot-bottle")
def _teardown_fake_home(self):
supervise.bot_bottle_root = self._original # type: ignore[assignment]
self._restore()
self._tmp.cleanup()
+2 -8
View File
@@ -15,7 +15,7 @@ import tempfile
import unittest
from pathlib import Path
from bot_bottle import supervise
from tests.unit import use_bottle_root
from bot_bottle import bottle_state
from bot_bottle.backend.docker.cleanup import _list_orphan_state_dirs
@@ -23,13 +23,7 @@ from bot_bottle.backend.docker.cleanup import _list_orphan_state_dirs
class _FakeHomeMixin:
def _setup_fake_home(self) -> None:
self._tmp = tempfile.TemporaryDirectory(prefix="docker-cleanup-test.")
original = supervise.bot_bottle_root
def fake_root() -> Path:
return Path(self._tmp.name) / ".bot-bottle"
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
self._restore = lambda: setattr(supervise, "bot_bottle_root", original)
self._restore = use_bottle_root(Path(self._tmp.name) / ".bot-bottle")
def _teardown_fake_home(self) -> None:
self._restore()
+2 -8
View File
@@ -23,7 +23,7 @@ import tempfile
import unittest
from pathlib import Path
from bot_bottle import supervise
from tests.unit import use_bottle_root
from bot_bottle import bottle_state
from bot_bottle.backend.docker import enumerate as _enumerate
@@ -75,13 +75,7 @@ class TestParseServicesByProject(unittest.TestCase):
class _FakeHomeMixin:
def _setup_fake_home(self) -> None:
self._tmp = tempfile.TemporaryDirectory(prefix="enum-active.")
original = supervise.bot_bottle_root
def fake_root() -> Path:
return Path(self._tmp.name) / ".bot-bottle"
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
self._restore_home = lambda: setattr(supervise, "bot_bottle_root", original)
self._restore_home = use_bottle_root(Path(self._tmp.name) / ".bot-bottle")
def _teardown_fake_home(self) -> None:
self._restore_home()
@@ -103,8 +103,6 @@ class TestLaunchCommittedImage(unittest.TestCase):
launch_mod.docker_mod, "image_exists", return_value=image_present,
), mock.patch.object(
launch_mod.docker_mod, "build_image", side_effect=fake_build,
), mock.patch.object(
launch_mod.docker_mod, "verify_agent_image",
), mock.patch.object(
launch_mod, "egress_tls_init",
return_value=(Path("/egress_ca"), Path("/egress_cert")),
@@ -87,7 +87,6 @@ class TestTeardownWarning(unittest.TestCase):
buf = io.StringIO()
with mock.patch.object(launch_mod.docker_mod, "build_image"), \
mock.patch.object(launch_mod.docker_mod, "verify_agent_image"), \
mock.patch.object(
launch_mod, "egress_tls_init",
return_value=(Path("/egress_ca"), Path("/egress_cert")),
@@ -46,7 +46,7 @@ def _addon() -> EgressAddon:
"""Return a bare EgressAddon with LOG_FULL config and no routes file."""
a: EgressAddon = EgressAddon.__new__(EgressAddon)
a.config = Config(routes=(), log=LOG_FULL)
a.safe_tokens = set()
a._safe_tokens = {}
a._supervise_slug = ""
a._token_allow_timeout = 300.0
return a
+83 -2
View File
@@ -211,7 +211,7 @@ def _addon(config: Config) -> EgressAddon:
"""Bare EgressAddon with a supplied config and no supervise wiring."""
a: EgressAddon = EgressAddon.__new__(EgressAddon)
a.config = config
a.safe_tokens = set()
a._safe_tokens = {}
a._supervise_slug = ""
a._token_allow_timeout = 300.0
a.routes_path = "/nonexistent/routes.yaml"
@@ -222,6 +222,29 @@ def _run_request(addon: EgressAddon, flow: _Flow) -> None:
asyncio.run(addon.request(flow)) # type: ignore[arg-type]
def _with_client_ip(flow: _Flow, ip: str) -> _Flow:
"""Attach a mitmproxy-style client_conn so consolidated resolution can read
the source IP (`flow.client_conn.peername[0]`)."""
flow.client_conn = types.SimpleNamespace(peername=(ip, 54321)) # type: ignore[attr-defined]
return flow
class _CtxResolver:
"""Fake orchestrator resolver: maps source IP -> bottle id, and grants the
same allow-list to any attributed bottle (unattributed -> deny)."""
def __init__(self, ip_to_bottle: dict[str, str]) -> None:
self._map = ip_to_bottle
def resolve_policy_and_bottle_id(
self, source_ip: str, identity_token: str = "",
) -> tuple[str | None, str | None]:
del identity_token
bottle_id = self._map.get(source_ip)
policy = "routes:\n - host: api.example.com\n" if bottle_id else None
return policy, bottle_id
# ---------------------------------------------------------------------------
# Introspection endpoint
# ---------------------------------------------------------------------------
@@ -418,7 +441,8 @@ class TestSuperviseBranch(unittest.TestCase):
with patch.object(_ea_mod, "_sv", _fake_sv("approved")):
_run_request(addon, flow)
self.assertIsNone(flow.response) # forwarded after approval
self.assertIn(_OPENAI_KEY, addon.safe_tokens)
# Approval lands in the calling bottle's safelist (keyed by slug).
self.assertIn(_OPENAI_KEY, addon._safe_tokens_for("test-bottle"))
def test_operator_rejection_blocks(self) -> None:
addon = self._supervised_addon()
@@ -735,5 +759,62 @@ class TestLogFullRequest(unittest.TestCase):
self.assertTrue(any(e.get("event") == "egress_request" for e in logged))
class TestSuperviseMultiTenant(unittest.TestCase):
"""Consolidated gateway: supervise proposals + the DLP safelist are keyed
per bottle, resolved by source IP (PRD 0070)."""
def _consolidated_addon(self) -> EgressAddon:
# Static config is empty; the resolver supplies each bottle's config.
addon = _addon(Config(routes=()))
addon._resolver = cast(Any, _CtxResolver({"10.0.0.1": "bottle-a", "10.0.0.2": "bottle-b"}))
addon._token_allow_timeout = 0.05
return addon
def test_approval_is_scoped_to_the_calling_bottle(self) -> None:
addon = self._consolidated_addon()
# bottle-a (10.0.0.1) sends the token; the operator approves.
flow = _with_client_ip(
_Flow(_Request(host="api.example.com", method="POST", body=f"k={_OPENAI_KEY}")),
"10.0.0.1",
)
with patch.object(_ea_mod, "_sv", _fake_sv("approved")):
_run_request(addon, flow)
self.assertIsNone(flow.response) # forwarded after approval
# The approval lands ONLY in bottle-a's safelist — never bottle-b's.
# A global set here would be the cross-tenant leak this slice closes.
self.assertIn(_OPENAI_KEY, addon._safe_tokens_for("bottle-a"))
self.assertNotIn(_OPENAI_KEY, addon._safe_tokens_for("bottle-b"))
def test_proposal_is_attributed_to_the_source_ip_bottle(self) -> None:
addon = self._consolidated_addon()
seen: list[str] = []
fake = _fake_sv("approved")
def _capture(**kw: Any) -> Any:
seen.append(kw["bottle_slug"])
return types.SimpleNamespace(id="p")
fake.Proposal = types.SimpleNamespace(new=_capture)
flow = _with_client_ip(
_Flow(_Request(host="api.example.com", method="POST", body=f"k={_OPENAI_KEY}")),
"10.0.0.2",
)
with patch.object(_ea_mod, "_sv", fake):
_run_request(addon, flow)
self.assertEqual(["bottle-b"], seen) # proposal keyed by the resolved bottle
def test_unattributed_source_ip_cannot_supervise(self) -> None:
addon = self._consolidated_addon()
# 10.9.9.9 is not in the resolver map -> deny-all config, empty slug.
flow = _with_client_ip(
_Flow(_Request(host="api.example.com", method="POST", body=f"k={_OPENAI_KEY}")),
"10.9.9.9",
)
with patch.object(_ea_mod, "_sv", _fake_sv("approved")):
_run_request(addon, flow)
self.assertIsNotNone(flow.response) # blocked (no route, no supervise)
self.assertNotIn(_OPENAI_KEY, addon._safe_tokens_for(""))
if __name__ == "__main__":
unittest.main()
+2 -8
View File
@@ -8,7 +8,7 @@ from pathlib import Path
from types import SimpleNamespace
from unittest.mock import patch
from bot_bottle import supervise
from tests.unit import use_bottle_root
from bot_bottle.backend.egress_apply import EgressApplyError
from bot_bottle.backend.docker.egress_apply import applicator
@@ -67,14 +67,8 @@ class TestValidateRoutesContent(unittest.TestCase):
class TestApplyRoutesChange(unittest.TestCase):
def setUp(self):
self._tmp = tempfile.TemporaryDirectory(prefix="egress-apply-test.")
original = supervise.bot_bottle_root
def fake_root() -> Path:
return Path(self._tmp.name) / ".bot-bottle"
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
self.addCleanup(lambda: setattr(supervise, "bot_bottle_root", original))
self.addCleanup(self._tmp.cleanup)
self.addCleanup(use_bottle_root(Path(self._tmp.name) / ".bot-bottle"))
def test_writes_live_routes_and_signals_reload(self):
calls: list[list[str]] = []
+100
View File
@@ -0,0 +1,100 @@
"""Unit: fail-closed per-client egress resolution — config + context (PRD 0070)."""
from __future__ import annotations
import unittest
from bot_bottle.egress_addon_core import resolve_client_config, resolve_client_context
from bot_bottle.policy_resolver import PolicyResolveError
class _FakeResolver:
def __init__(self, result: str | None = None, raises: bool = False) -> None:
self._result = result
self._raises = raises
self.calls: list[tuple[str, str]] = []
def resolve(self, source_ip: str, identity_token: str = "") -> str | None:
self.calls.append((source_ip, identity_token))
if self._raises:
raise PolicyResolveError("orchestrator down")
return self._result
class TestResolveClientConfig(unittest.TestCase):
def test_valid_policy_is_parsed(self) -> None:
cfg = resolve_client_config(
_FakeResolver(result="routes:\n - host: example.com\n"), "10.243.0.1"
)
self.assertEqual(("example.com",), tuple(r.host for r in cfg.routes))
def test_unattributed_none_denies_all(self) -> None:
cfg = resolve_client_config(_FakeResolver(result=None), "10.243.0.9")
self.assertEqual((), cfg.routes) # no routes → default-deny
def test_empty_policy_denies_all(self) -> None:
self.assertEqual((), resolve_client_config(_FakeResolver(result=""), "10.243.0.1").routes)
def test_resolver_error_denies_all(self) -> None:
# Orchestrator unreachable/errored must never widen egress.
self.assertEqual((), resolve_client_config(_FakeResolver(raises=True), "10.243.0.1").routes)
def test_unparseable_policy_denies_all(self) -> None:
cfg = resolve_client_config(_FakeResolver(result="routes: notalist\n"), "10.243.0.1")
self.assertEqual((), cfg.routes)
def test_forwards_source_ip_and_token(self) -> None:
r = _FakeResolver(result=None)
resolve_client_config(r, "10.243.0.1", "tok")
self.assertEqual(("10.243.0.1", "tok"), r.calls[0])
class _FakeContextResolver:
def __init__(
self, policy: str | None = None, bottle_id: str | None = None, raises: bool = False,
) -> None:
self._policy = policy
self._bottle_id = bottle_id
self._raises = raises
def resolve_policy_and_bottle_id(
self, source_ip: str, identity_token: str = "",
) -> tuple[str | None, str | None]:
if self._raises:
raise PolicyResolveError("orchestrator down")
return self._policy, self._bottle_id
class TestResolveClientContext(unittest.TestCase):
def test_returns_config_and_bottle_id(self) -> None:
cfg, slug = resolve_client_context(
_FakeContextResolver(policy="routes:\n - host: example.com\n", bottle_id="b1"),
"10.243.0.1",
)
self.assertEqual(("example.com",), tuple(r.host for r in cfg.routes))
self.assertEqual("b1", slug)
def test_unattributed_denies_and_empty_slug(self) -> None:
cfg, slug = resolve_client_context(
_FakeContextResolver(policy=None, bottle_id=None), "10.243.0.9",
)
self.assertEqual((), cfg.routes)
self.assertEqual("", slug) # no bottle → supervise unavailable
def test_resolver_error_denies_and_empty_slug(self) -> None:
cfg, slug = resolve_client_context(_FakeContextResolver(raises=True), "10.243.0.1")
self.assertEqual((), cfg.routes)
self.assertEqual("", slug)
def test_unparseable_policy_denies_but_keeps_slug(self) -> None:
# A bad policy denies egress, but the bottle is still attributed (its
# supervise queue is keyed by the id, independent of route parsing).
cfg, slug = resolve_client_context(
_FakeContextResolver(policy="routes: notalist\n", bottle_id="b2"), "10.243.0.1",
)
self.assertEqual((), cfg.routes)
self.assertEqual("b2", slug)
if __name__ == "__main__":
unittest.main()
+50 -19
View File
@@ -67,10 +67,24 @@ class TestNetpoolRenderers(unittest.TestCase):
self.assertNotIn("networking.nftables.enable = true", mod)
self.assertNotIn("systemd.network.enable = true", mod)
self.assertIn("bot-bottle-firecracker-netpool", mod) # the oneshot
self.assertIn(netpool.NFT_TABLE, mod)
self.assertIn('iifname != "${cfg.ifacePrefix}*" return', mod)
# Supports group-owned TAPs (shared pool for a multi-user host).
self.assertIn("group ${cfg.group}", mod)
self.assertIn("BOT_BOTTLE_FC_GROUP", mod)
def test_nixos_module_delegates_and_holds_no_literals(self):
# Single-source discipline: the module must NOT re-implement the
# bring-up (it delegates to the shared script) and must NOT hard-
# code the pool defaults (it readFile-parses the shared .env), so
# nothing can drift from netpool.py / the shell script.
root = Path(__file__).resolve().parents[2]
mod = (root / "nix" / "firecracker-netpool.nix").read_text()
# Delegation to the one bring-up implementation + shared defaults.
self.assertIn("scripts/firecracker-netpool.sh", mod)
self.assertIn("netpool.defaults.env", mod)
self.assertIn("BOT_BOTTLE_FC_NFT_TABLE", mod)
# No duplicated literals or nft ruleset.
self.assertNotIn(netpool.ip_base(), mod) # 10.243.0.0
self.assertNotIn(netpool.NFT_TABLE, mod) # bot_bottle_fc
self.assertNotIn("ct status dnat", mod) # the nft ruleset
def test_shell_setup_reflects_overrides(self):
with patch.dict(os.environ, {"BOT_BOTTLE_FC_POOL_SIZE": "3"}):
@@ -262,26 +276,43 @@ class TestBottleAgentArgv(unittest.TestCase):
self.assertIn("USER=node", argv)
class TestSetupScriptConsistency(unittest.TestCase):
"""The shell setup script duplicates the pool defaults; keep them in
lockstep with the Python constants so the two setup paths agree."""
class TestNetpoolDefaultsSingleSource(unittest.TestCase):
"""The pool defaults live in one shared file (netpool.defaults.env);
Python parses it and the shell script + Nix module read the same file,
so the three setup paths can't drift."""
def _script(self) -> str:
root = Path(__file__).resolve().parent.parent.parent
return (root / "scripts" / "firecracker-netpool.sh").read_text()
def _root(self) -> Path:
return Path(__file__).resolve().parents[2]
def test_defaults_match_python(self):
script = self._script()
def _shared_defaults(self) -> dict[str, str]:
text = netpool.DEFAULTS_FILE.read_text()
out: dict[str, str] = {}
for raw in text.splitlines():
line = raw.strip()
if line and not line.startswith("#") and "=" in line:
k, _, v = line.partition("=")
out[k.strip()] = v.strip()
return out
def test_python_reads_the_shared_file(self):
d = self._shared_defaults()
with patch.dict(os.environ, {}, clear=True):
self.assertIn(f'POOL_SIZE:-{netpool.pool_size()}', script)
self.assertIn(f'BOT_BOTTLE_FC_IP_BASE:-{netpool.ip_base()}', script)
self.assertIn(f'IFACE_PREFIX:-{netpool.IFACE_PREFIX}', script)
self.assertEqual(int(d["BOT_BOTTLE_FC_POOL_SIZE"]), netpool.pool_size())
self.assertEqual(d["BOT_BOTTLE_FC_IP_BASE"], netpool.ip_base())
# Module constants resolve through the same shared file.
self.assertEqual(d["BOT_BOTTLE_FC_IFACE_PREFIX"], netpool.IFACE_PREFIX)
self.assertEqual(d["BOT_BOTTLE_FC_NFT_TABLE"], netpool.NFT_TABLE)
def test_table_name_and_ports_match(self):
script = self._script()
self.assertIn(f'TABLE="{netpool.NFT_TABLE}"', script)
for port in netpool.SIDECAR_PORTS:
self.assertIn(str(port), script)
def test_env_var_overrides_the_shared_default(self):
with patch.dict(os.environ, {"BOT_BOTTLE_FC_IP_BASE": "10.99.0.0"}):
self.assertEqual("10.99.0.0", netpool.ip_base())
def test_script_defers_to_shared_file_without_literals(self):
script = (self._root() / "scripts" / "firecracker-netpool.sh").read_text()
# Delegates its defaults to the shared file, not hardcoded values.
self.assertIn("netpool.defaults.env", script)
self.assertIn("BOT_BOTTLE_FC_POOL_SIZE:-$(_default BOT_BOTTLE_FC_POOL_SIZE)", script)
self.assertIn('TABLE="${BOT_BOTTLE_FC_NFT_TABLE:-$(_default BOT_BOTTLE_FC_NFT_TABLE)}"', script)
class TestBootArgs(unittest.TestCase):
+67
View File
@@ -13,6 +13,17 @@ from bot_bottle.git_gate import GIT_GATE_TIMEOUT_SECS
from bot_bottle.git_http_backend import GitHttpHandler, MAX_BODY_BYTES
class _FixedResolver:
"""Maps every source IP to one bottle id (consolidated-mode stub)."""
def __init__(self, bottle_id: str) -> None:
self._bottle_id = bottle_id
def resolve_bottle_id(self, source_ip: str, identity_token: str = "") -> str:
del source_ip, identity_token
return self._bottle_id
class TestGitHttpBackend(unittest.TestCase):
def test_real_git_push_reaches_bare_repo(self):
from http.server import ThreadingHTTPServer
@@ -94,6 +105,62 @@ class TestGitHttpBackend(unittest.TestCase):
).strip()
self.assertEqual(head, cloned)
def test_consolidated_push_stamps_bottle_slug_for_the_hook(self):
# In consolidated mode the backend attributes the push by source IP and
# stamps SUPERVISE_BOTTLE_SLUG=<bottle_id> into the CGI env, so the
# gitleaks-allow pre-receive hook queues its proposal under the right
# bottle. The hook here just records what it received.
from http.server import ThreadingHTTPServer
bottle_id = "bottleab12"
with tempfile.TemporaryDirectory() as tmp:
root = Path(tmp)
bare = root / bottle_id / "repo.git" # namespaced per bottle (slice 10)
bare.parent.mkdir(parents=True)
subprocess.run(["git", "init", "--bare", str(bare)],
check=True, capture_output=True, text=True)
subprocess.run(
["git", "-C", str(bare), "config", "http.receivepack", "true"],
check=True,
)
capture = root / "slug-capture"
hook = bare / "hooks" / "pre-receive"
hook.write_text(
f"#!/bin/sh\nprintf '%s' \"${{SUPERVISE_BOTTLE_SLUG:-UNSET}}\" > "
f"{capture}\ncat >/dev/null\nexit 0\n"
)
hook.chmod(0o755)
old_root = os.environ.get("GIT_PROJECT_ROOT")
os.environ["GIT_PROJECT_ROOT"] = str(root) # base; backend nests per bottle
self.addCleanup(self._restore_env, old_root)
server = ThreadingHTTPServer(("127.0.0.1", 0), GitHttpHandler)
server.policy_resolver = _FixedResolver(bottle_id) # type: ignore[attr-defined]
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
self.addCleanup(server.shutdown)
self.addCleanup(server.server_close)
work = root / "work"
work.mkdir()
subprocess.run(["git", "init"], cwd=work, check=True,
capture_output=True, text=True)
subprocess.run(["git", "config", "user.name", "test"], cwd=work, check=True)
subprocess.run(["git", "config", "user.email", "t@example.invalid"],
cwd=work, check=True)
(work / "README.md").write_text("test\n")
subprocess.run(["git", "add", "README.md"], cwd=work, check=True)
subprocess.run(["git", "commit", "-m", "init"], cwd=work,
check=True, capture_output=True, text=True)
url = f"http://127.0.0.1:{server.server_port}/repo.git"
subprocess.run(
["git", "push", url, "HEAD:refs/heads/main"],
cwd=work, check=True, capture_output=True, text=True, timeout=5,
)
self.assertEqual(bottle_id, capture.read_text())
def test_post_forwards_git_cgi_headers(self):
from http.server import ThreadingHTTPServer
+65
View File
@@ -0,0 +1,65 @@
"""Unit: resolve_sandbox_root — source-IP-keyed git-gate repo namespace (PRD 0070).
The consolidated git-http backend serves every bottle from one process and
selects each request's repo root by the calling bottle's source IP. This is
the fail-closed selection logic, tested without a live server.
"""
from __future__ import annotations
import unittest
from pathlib import Path
from bot_bottle.git_http_backend import resolve_sandbox_root
from bot_bottle.policy_resolver import PolicyResolveError
_BASE = Path("/git")
class _FakeResolver:
def __init__(self, bottle_id: str | None = None, raises: bool = False) -> None:
self._bottle_id = bottle_id
self._raises = raises
self.calls: list[tuple[str, str]] = []
def resolve_bottle_id(self, source_ip: str, identity_token: str = "") -> str | None:
self.calls.append((source_ip, identity_token))
if self._raises:
raise PolicyResolveError("orchestrator down")
return self._bottle_id
class TestResolveRepoRoot(unittest.TestCase):
def test_single_tenant_passthrough(self) -> None:
# No resolver → the flat base root, unchanged (legacy per-bottle mode).
self.assertEqual(_BASE, resolve_sandbox_root(None, _BASE, "10.243.0.1"))
def test_attributed_bottle_gets_namespaced_root(self) -> None:
root = resolve_sandbox_root(_FakeResolver(bottle_id="ab12cd34"), _BASE, "10.243.0.1")
self.assertEqual(Path("/git/ab12cd34"), root)
def test_unattributed_denies(self) -> None:
self.assertIsNone(resolve_sandbox_root(_FakeResolver(bottle_id=None), _BASE, "10.243.0.9"))
def test_empty_bottle_id_denies(self) -> None:
self.assertIsNone(resolve_sandbox_root(_FakeResolver(bottle_id=""), _BASE, "10.243.0.1"))
def test_resolver_error_denies(self) -> None:
# Orchestrator unreachable/errored must never serve repos.
self.assertIsNone(resolve_sandbox_root(_FakeResolver(raises=True), _BASE, "10.243.0.1"))
def test_namespace_escape_denies(self) -> None:
# A bottle_id that would climb out of the root is rejected (defense in
# depth — registry ids are token_hex, but never trust the namespace).
self.assertIsNone(
resolve_sandbox_root(_FakeResolver(bottle_id="../etc"), _BASE, "10.243.0.1")
)
def test_forwards_source_ip_and_token(self) -> None:
r = _FakeResolver(bottle_id="b1")
resolve_sandbox_root(r, _BASE, "10.243.0.1", "tok")
self.assertEqual(("10.243.0.1", "tok"), r.calls[0])
if __name__ == "__main__":
unittest.main()
+1 -6
View File
@@ -357,17 +357,12 @@ class TestMacosContainerLaunchCommittedImage(unittest.TestCase):
launch.container_mod, "image_exists", return_value=False,
), patch.object(
launch.container_mod, "build_image", side_effect=fake_build,
), patch.object(
launch.container_mod, "verify_agent_image",
) as verify:
):
updated = launch._build_images(plan)
self.assertEqual("bot-bottle-agent:latest", updated.image)
self.assertEqual(2, len(calls))
self.assertEqual("bot-bottle-agent:latest", calls[1][0])
verify.assert_called_once_with(
"bot-bottle-agent:latest", ("claude", "--version"),
)
if __name__ == "__main__":
+86
View File
@@ -0,0 +1,86 @@
"""Unit tests for the orchestrator launch broker (PRD 0070)."""
from __future__ import annotations
import secrets
import unittest
from bot_bottle.orchestrator.broker import (
BrokerAuthError,
LaunchRequest,
StubBroker,
sign_request,
verify_request,
)
class TestSignVerify(unittest.TestCase):
def setUp(self) -> None:
self.secret = secrets.token_bytes(16)
def test_launch_round_trip(self) -> None:
req = LaunchRequest(
op="launch", bottle_id="b1", source_ip="10.243.0.1",
image_ref="sha256:abc", slot=3,
)
self.assertEqual(req, verify_request(sign_request(req, self.secret), self.secret))
def test_teardown_round_trip(self) -> None:
req = LaunchRequest(op="teardown", bottle_id="b1")
got = verify_request(sign_request(req, self.secret), self.secret)
self.assertEqual(("teardown", "b1"), (got.op, got.bottle_id))
def test_wrong_secret_rejected(self) -> None:
token = sign_request(LaunchRequest(op="launch", bottle_id="b1"), self.secret)
with self.assertRaises(BrokerAuthError):
verify_request(token, secrets.token_bytes(16))
def test_tampered_payload_rejected(self) -> None:
token = sign_request(LaunchRequest(op="launch", bottle_id="b1"), self.secret)
header, payload, sig = token.split(".")
flipped = payload[:-1] + ("A" if payload[-1] != "A" else "B")
with self.assertRaises(BrokerAuthError):
verify_request(f"{header}.{flipped}.{sig}", self.secret)
def test_malformed_tokens_rejected(self) -> None:
for bad in ("", "a.b", "a.b.c.d", "not-a-token"):
with self.assertRaises(BrokerAuthError):
verify_request(bad, self.secret)
def test_unknown_op_rejected_despite_valid_signature(self) -> None:
# A correctly-signed token whose op isn't in the allow-list must
# still be refused — the schema guard is independent of provenance.
token = sign_request(LaunchRequest(op="rm-rf", bottle_id="b1"), self.secret)
with self.assertRaises(BrokerAuthError):
verify_request(token, self.secret)
class TestStubBroker(unittest.TestCase):
def setUp(self) -> None:
self.secret = secrets.token_bytes(16)
self.broker = StubBroker(self.secret)
def test_submit_launch_records_verified_request(self) -> None:
req = LaunchRequest(op="launch", bottle_id="b1", source_ip="10.243.0.1")
got = self.broker.submit(sign_request(req, self.secret))
self.assertEqual(req, got)
self.assertEqual(["b1"], [r.bottle_id for r in self.broker.launched])
self.assertEqual([], self.broker.torn_down)
def test_submit_teardown_records(self) -> None:
self.broker.submit(
sign_request(LaunchRequest(op="teardown", bottle_id="b1"), self.secret)
)
self.assertEqual(["b1"], [r.bottle_id for r in self.broker.torn_down])
def test_submit_forged_token_is_fail_closed(self) -> None:
forged = sign_request(
LaunchRequest(op="launch", bottle_id="b1"), secrets.token_bytes(16)
)
with self.assertRaises(BrokerAuthError):
self.broker.submit(forged)
self.assertEqual([], self.broker.launched) # nothing acted on
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,227 @@
"""Unit tests for the orchestrator HTTP control plane (PRD 0070).
Mostly exercises the pure `dispatch()` (socket-free, like the supervise
server tests), plus one real-socket round-trip to prove the handler wiring.
"""
from __future__ import annotations
import json
import secrets
import tempfile
import threading
import unittest
import urllib.request
from pathlib import Path
from bot_bottle.orchestrator.broker import StubBroker
from bot_bottle.orchestrator.control_plane import dispatch, make_server
from bot_bottle.orchestrator.registry import RegistryStore
from bot_bottle.orchestrator.service import Orchestrator
def _body(obj: object) -> bytes:
return json.dumps(obj).encode()
def _orchestrator(db_path: Path) -> Orchestrator:
store = RegistryStore(db_path)
store.migrate()
secret = secrets.token_bytes(16)
return Orchestrator(store, StubBroker(secret), secret)
class TestDispatch(unittest.TestCase):
def setUp(self) -> None:
self._tmp = tempfile.TemporaryDirectory()
self.orch = _orchestrator(Path(self._tmp.name) / "r.db")
def tearDown(self) -> None:
self._tmp.cleanup()
def test_health(self) -> None:
status, payload = dispatch(self.orch, "GET", "/health", b"")
self.assertEqual(200, status)
self.assertEqual("ok", payload["status"])
def test_register_returns_id_and_token(self) -> None:
status, payload = dispatch(
self.orch, "POST", "/bottles", _body({"source_ip": "10.243.0.1"})
)
self.assertEqual(201, status)
self.assertTrue(payload["bottle_id"])
self.assertTrue(payload["identity_token"])
def test_register_requires_source_ip(self) -> None:
status, _ = dispatch(self.orch, "POST", "/bottles", _body({}))
self.assertEqual(400, status)
def test_register_rejects_bad_json(self) -> None:
status, _ = dispatch(self.orch, "POST", "/bottles", b"{not json")
self.assertEqual(400, status)
def test_list_redacts_identity_token(self) -> None:
dispatch(self.orch, "POST", "/bottles", _body({"source_ip": "10.243.0.1"}))
status, payload = dispatch(self.orch, "GET", "/bottles", b"")
self.assertEqual(200, status)
bottles = payload["bottles"]
assert isinstance(bottles, list)
self.assertEqual(1, len(bottles))
first = bottles[0]
assert isinstance(first, dict)
self.assertNotIn("identity_token", first)
self.assertEqual("10.243.0.1", first["source_ip"])
def test_attribute_ok_and_forbidden(self) -> None:
_, reg = dispatch(
self.orch, "POST", "/bottles", _body({"source_ip": "10.243.0.5"})
)
token = reg["identity_token"]
ok_status, ok = dispatch(
self.orch, "POST", "/attribute",
_body({"source_ip": "10.243.0.5", "identity_token": token}),
)
self.assertEqual(200, ok_status)
self.assertEqual(reg["bottle_id"], ok["bottle_id"])
bad_status, _ = dispatch(
self.orch, "POST", "/attribute",
_body({"source_ip": "10.243.0.5", "identity_token": "nope"}),
)
self.assertEqual(403, bad_status)
def test_attribute_requires_both_fields(self) -> None:
status, _ = dispatch(
self.orch, "POST", "/attribute", _body({"source_ip": "10.243.0.5"})
)
self.assertEqual(400, status)
def test_delete(self) -> None:
_, reg = dispatch(
self.orch, "POST", "/bottles", _body({"source_ip": "10.243.0.1"})
)
bid = reg["bottle_id"]
status, payload = dispatch(self.orch, "DELETE", f"/bottles/{bid}", b"")
self.assertEqual(200, status)
self.assertEqual(True, payload["torn_down"])
self.assertEqual([], self.orch.registry.all())
def test_delete_missing_404(self) -> None:
status, _ = dispatch(self.orch, "DELETE", "/bottles/ghost", b"")
self.assertEqual(404, status)
def test_unknown_route_404(self) -> None:
status, _ = dispatch(self.orch, "GET", "/nope", b"")
self.assertEqual(404, status)
def test_trailing_slash_normalized(self) -> None:
status, _ = dispatch(self.orch, "GET", "/health/", b"")
self.assertEqual(200, status)
def test_gateway_status_unconfigured(self) -> None:
status, payload = dispatch(self.orch, "GET", "/gateway", b"")
self.assertEqual(200, status)
self.assertEqual(False, payload["configured"])
def test_launch_stores_policy_and_resolve_returns_it(self) -> None:
_, reg = dispatch(
self.orch, "POST", "/bottles",
_body({"source_ip": "10.243.0.5", "policy": '{"a":1}'}),
)
status, payload = dispatch(
self.orch, "POST", "/resolve",
_body({"source_ip": "10.243.0.5", "identity_token": reg["identity_token"]}),
)
self.assertEqual(200, status)
self.assertEqual(reg["bottle_id"], payload["bottle_id"])
self.assertEqual('{"a":1}', payload["policy"])
def test_resolve_forbidden_on_bad_token(self) -> None:
dispatch(self.orch, "POST", "/bottles", _body({"source_ip": "10.243.0.5"}))
status, _ = dispatch(
self.orch, "POST", "/resolve",
_body({"source_ip": "10.243.0.5", "identity_token": "nope"}),
)
self.assertEqual(403, status)
def test_put_policy_updates_live(self) -> None:
_, reg = dispatch(self.orch, "POST", "/bottles", _body({"source_ip": "10.243.0.1"}))
bid = reg["bottle_id"]
status, payload = dispatch(
self.orch, "PUT", f"/bottles/{bid}/policy", _body({"policy": '{"v":9}'})
)
self.assertEqual(200, status)
self.assertEqual(True, payload["updated"])
_, resolved = dispatch(
self.orch, "POST", "/resolve",
_body({"source_ip": "10.243.0.1", "identity_token": reg["identity_token"]}),
)
self.assertEqual('{"v":9}', resolved["policy"])
def test_put_policy_missing_bottle_404(self) -> None:
status, _ = dispatch(
self.orch, "PUT", "/bottles/ghost/policy", _body({"policy": "{}"})
)
self.assertEqual(404, status)
def test_put_policy_requires_string(self) -> None:
_, reg = dispatch(self.orch, "POST", "/bottles", _body({"source_ip": "10.243.0.1"}))
status, _ = dispatch(
self.orch, "PUT", f"/bottles/{reg['bottle_id']}/policy", _body({})
)
self.assertEqual(400, status)
def test_resolve_without_token_by_source_ip(self) -> None:
_, reg = dispatch(
self.orch, "POST", "/bottles",
_body({"source_ip": "10.243.0.5", "policy": "P"}),
)
status, payload = dispatch(
self.orch, "POST", "/resolve", _body({"source_ip": "10.243.0.5"})
)
self.assertEqual(200, status)
self.assertEqual(reg["bottle_id"], payload["bottle_id"])
self.assertEqual("P", payload["policy"])
def test_resolve_requires_source_ip(self) -> None:
status, _ = dispatch(self.orch, "POST", "/resolve", _body({}))
self.assertEqual(400, status)
class TestServerRoundTrip(unittest.TestCase):
def test_http_register_health_attribute(self) -> None:
tmp = tempfile.TemporaryDirectory()
self.addCleanup(tmp.cleanup)
orch = _orchestrator(Path(tmp.name) / "r.db")
server = make_server(orch, "127.0.0.1", 0)
self.addCleanup(server.server_close)
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
self.addCleanup(server.shutdown)
host, port = server.server_address[0], server.server_address[1]
base = f"http://{host}:{port}"
reg = json.load(urllib.request.urlopen(
urllib.request.Request(
f"{base}/bottles", data=_body({"source_ip": "10.243.0.7"}),
method="POST", headers={"Content-Type": "application/json"},
), timeout=5,
))
self.assertTrue(reg["bottle_id"])
health = json.load(urllib.request.urlopen(f"{base}/health", timeout=5))
self.assertEqual("ok", health["status"])
attr = json.load(urllib.request.urlopen(
urllib.request.Request(
f"{base}/attribute",
data=_body({"source_ip": "10.243.0.7", "identity_token": reg["identity_token"]}),
method="POST", headers={"Content-Type": "application/json"},
), timeout=5,
))
self.assertEqual(reg["bottle_id"], attr["bottle_id"])
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,100 @@
"""Unit tests for the Docker launch broker (PRD 0070). Docker is mocked."""
from __future__ import annotations
import secrets
import unittest
from unittest.mock import Mock, patch
from bot_bottle.orchestrator.broker import (
BrokerAuthError,
LaunchRequest,
sign_request,
)
from bot_bottle.orchestrator.docker_broker import (
BOTTLE_ID_LABEL,
DockerBroker,
DockerBrokerError,
container_name,
rm_argv,
run_argv,
)
class TestArgv(unittest.TestCase):
def test_run_argv_uses_only_static_fields(self) -> None:
req = LaunchRequest(op="launch", bottle_id="b1", image_ref="busybox")
argv = run_argv(req)
self.assertEqual(["docker", "run"], argv[:2])
self.assertIn("--name", argv)
self.assertIn(container_name("b1"), argv)
self.assertIn(f"{BOTTLE_ID_LABEL}=b1", argv)
self.assertEqual("busybox", argv[-1]) # image is the terminal arg
def test_rm_argv(self) -> None:
req = LaunchRequest(op="teardown", bottle_id="b1")
self.assertEqual(["docker", "rm", "--force", container_name("b1")], rm_argv(req))
def test_container_name_is_prefixed(self) -> None:
name = container_name("b1")
self.assertTrue(name.startswith("bot-bottle-orch-"))
self.assertTrue(name.endswith("b1"))
class TestDockerBroker(unittest.TestCase):
def setUp(self) -> None:
self.secret = secrets.token_bytes(16)
self.broker = DockerBroker(self.secret)
def _submit(self, req: LaunchRequest) -> None:
self.broker.submit(sign_request(req, self.secret))
def test_launch_invokes_docker_run(self) -> None:
req = LaunchRequest(op="launch", bottle_id="b1", image_ref="busybox")
with patch.object(self.broker, "_docker", return_value=Mock(returncode=0, stderr="")) as m:
self._submit(req)
m.assert_called_once()
self.assertEqual(run_argv(req), m.call_args.args[0])
def test_launch_without_image_raises_and_skips_docker(self) -> None:
req = LaunchRequest(op="launch", bottle_id="b1", image_ref="")
with patch.object(self.broker, "_docker") as m:
with self.assertRaises(DockerBrokerError):
self._submit(req)
m.assert_not_called()
def test_launch_docker_failure_raises(self) -> None:
req = LaunchRequest(op="launch", bottle_id="b1", image_ref="busybox")
with patch.object(self.broker, "_docker", return_value=Mock(returncode=1, stderr="boom")):
with self.assertRaises(DockerBrokerError):
self._submit(req)
def test_teardown_invokes_docker_rm(self) -> None:
req = LaunchRequest(op="teardown", bottle_id="b1")
with patch.object(self.broker, "_docker", return_value=Mock(returncode=0, stderr="")) as m:
self._submit(req)
self.assertEqual(rm_argv(req), m.call_args.args[0])
def test_teardown_is_idempotent_on_missing_container(self) -> None:
req = LaunchRequest(op="teardown", bottle_id="b1")
absent = Mock(returncode=1, stderr="Error: No such container: bot-bottle-orch-b1")
with patch.object(self.broker, "_docker", return_value=absent):
self._submit(req) # must not raise
def test_teardown_other_failure_raises(self) -> None:
req = LaunchRequest(op="teardown", bottle_id="b1")
with patch.object(self.broker, "_docker", return_value=Mock(returncode=1, stderr="daemon down")):
with self.assertRaises(DockerBrokerError):
self._submit(req)
def test_forged_token_never_touches_docker(self) -> None:
req = LaunchRequest(op="launch", bottle_id="b1", image_ref="busybox")
forged = sign_request(req, secrets.token_bytes(16))
with patch.object(self.broker, "_docker") as m:
with self.assertRaises(BrokerAuthError):
self.broker.submit(forged)
m.assert_not_called()
if __name__ == "__main__":
unittest.main()
+129
View File
@@ -0,0 +1,129 @@
"""Unit tests for the consolidated Docker gateway (PRD 0070). Docker mocked."""
from __future__ import annotations
import unittest
from unittest.mock import Mock, patch
from bot_bottle.orchestrator.gateway import (
GATEWAY_NAME,
DockerGateway,
GatewayError,
)
_RUN_DOCKER = "bot_bottle.orchestrator.gateway.run_docker"
def _proc(returncode: int = 0, stdout: str = "", stderr: str = "") -> Mock:
return Mock(returncode=returncode, stdout=stdout, stderr=stderr)
class TestDockerGateway(unittest.TestCase):
def setUp(self) -> None:
self.sc = DockerGateway("bot-bottle-sidecars:latest")
def test_default_name(self) -> None:
self.assertEqual(GATEWAY_NAME, self.sc.name)
def test_is_running_reads_docker_ps(self) -> None:
with patch(_RUN_DOCKER, return_value=_proc(stdout=self.sc.name + "\n")):
self.assertTrue(self.sc.is_running())
with patch(_RUN_DOCKER, return_value=_proc(stdout="")):
self.assertFalse(self.sc.is_running())
def test_ensure_running_is_noop_when_already_up(self) -> None:
with patch(_RUN_DOCKER, return_value=_proc(stdout=self.sc.name)) as m:
self.sc.ensure_running()
# Only the is_running() ps probe — no rm / run.
self.assertEqual(1, m.call_count)
def test_ensure_running_starts_the_singleton_when_absent(self) -> None:
calls: list[list[str]] = []
def fake(argv: list[str]) -> Mock:
calls.append(argv)
return _proc(stdout="") if argv[:2] == ["docker", "ps"] else _proc()
with patch(_RUN_DOCKER, side_effect=fake):
self.sc.ensure_running()
runs = [c for c in calls if c[:2] == ["docker", "run"]]
self.assertEqual(1, len(runs))
self.assertIn(self.sc.name, runs[0])
self.assertIn("bot-bottle-sidecars:latest", runs[0])
def test_ensure_running_raises_on_docker_failure(self) -> None:
def fake(argv: list[str]) -> Mock:
if argv[:2] == ["docker", "ps"]:
return _proc(stdout="")
if argv[:2] == ["docker", "run"]:
return _proc(returncode=1, stderr="boom")
return _proc()
with patch(_RUN_DOCKER, side_effect=fake):
with self.assertRaises(GatewayError):
self.sc.ensure_running()
def test_stop_is_idempotent_on_missing(self) -> None:
absent = _proc(returncode=1, stderr="Error: No such container: x")
with patch(_RUN_DOCKER, return_value=absent):
self.sc.stop() # must not raise
def test_stop_raises_on_other_failure(self) -> None:
with patch(_RUN_DOCKER, return_value=_proc(returncode=1, stderr="daemon down")):
with self.assertRaises(GatewayError):
self.sc.stop()
class TestDockerGatewayBuild(unittest.TestCase):
def setUp(self) -> None:
self.sc = DockerGateway() # defaults to the real bundle image + dockerfile
def test_image_exists_reads_docker_inspect(self) -> None:
with patch(_RUN_DOCKER, return_value=_proc(returncode=0)):
self.assertTrue(self.sc.image_exists())
with patch(_RUN_DOCKER, return_value=_proc(returncode=1)):
self.assertFalse(self.sc.image_exists())
def test_ensure_built_is_noop_when_image_present(self) -> None:
with patch(_RUN_DOCKER, return_value=_proc(returncode=0)) as m:
self.sc.ensure_built()
self.assertEqual(1, m.call_count) # only the image-inspect probe
def test_ensure_built_builds_from_dockerfile_when_missing(self) -> None:
calls: list[list[str]] = []
def fake(argv: list[str]) -> Mock:
calls.append(argv)
if argv[:3] == ["docker", "image", "inspect"]:
return _proc(returncode=1) # missing
return _proc()
with patch(_RUN_DOCKER, side_effect=fake):
self.sc.ensure_built()
builds = [c for c in calls if c[:2] == ["docker", "build"]]
self.assertEqual(1, len(builds))
self.assertIn(self.sc.image_ref, builds[0])
self.assertIn("-f", builds[0])
self.assertTrue(any(a.endswith("Dockerfile.sidecars") for a in builds[0]))
def test_ensure_built_noop_when_no_dockerfile(self) -> None:
sc = DockerGateway("busybox", dockerfile=None)
with patch(_RUN_DOCKER) as m:
sc.ensure_built()
m.assert_not_called()
def test_ensure_built_raises_on_build_failure(self) -> None:
def fake(argv: list[str]) -> Mock:
if argv[:3] == ["docker", "image", "inspect"]:
return _proc(returncode=1)
return _proc(returncode=1, stderr="build boom")
with patch(_RUN_DOCKER, side_effect=fake):
with self.assertRaises(GatewayError):
self.sc.ensure_built()
if __name__ == "__main__":
unittest.main()
+87
View File
@@ -0,0 +1,87 @@
"""Unit: orchestrator process lifecycle — idempotent singleton (PRD 0070)."""
from __future__ import annotations
import tempfile
import unittest
import urllib.error
from pathlib import Path
from unittest.mock import MagicMock, patch
from bot_bottle.orchestrator.lifecycle import (
OrchestratorProcess,
OrchestratorStartError,
)
from tests.unit import use_bottle_root
_URLOPEN = "bot_bottle.orchestrator.lifecycle.urllib.request.urlopen"
_POPEN = "bot_bottle.orchestrator.lifecycle.subprocess.Popen"
_SLEEP = "bot_bottle.orchestrator.lifecycle.time.sleep"
_MONOTONIC = "bot_bottle.orchestrator.lifecycle.time.monotonic"
def _health(status: int) -> MagicMock:
"""A urlopen() context-manager whose `.status` is `status`."""
m = MagicMock()
m.__enter__.return_value.status = status
return m
class TestOrchestratorProcess(unittest.TestCase):
def setUp(self) -> None:
self._tmp = tempfile.TemporaryDirectory()
self.addCleanup(self._tmp.cleanup)
self.addCleanup(use_bottle_root(Path(self._tmp.name)))
self.p = OrchestratorProcess(port=8099)
def test_url(self) -> None:
self.assertEqual("http://127.0.0.1:8099", self.p.url)
def test_is_healthy_true_on_200(self) -> None:
with patch(_URLOPEN, return_value=_health(200)):
self.assertTrue(self.p.is_healthy())
def test_is_healthy_false_on_error(self) -> None:
with patch(_URLOPEN, side_effect=urllib.error.URLError("refused")):
self.assertFalse(self.p.is_healthy())
def test_ensure_running_noop_when_already_healthy(self) -> None:
with patch(_URLOPEN, return_value=_health(200)), patch(_POPEN) as popen:
self.assertEqual(self.p.url, self.p.ensure_running())
popen.assert_not_called() # a live control plane is left untouched
def test_ensure_running_spawns_then_waits_for_health(self) -> None:
# First check (before spawn) fails; after spawn the poll succeeds.
with patch(_URLOPEN, side_effect=[urllib.error.URLError("down"), _health(200)]), \
patch(_POPEN) as popen, patch(_SLEEP):
url = self.p.ensure_running()
self.assertEqual(self.p.url, url)
popen.assert_called_once()
def test_ensure_running_raises_on_startup_timeout(self) -> None:
with patch(_URLOPEN, side_effect=urllib.error.URLError("down")), \
patch(_POPEN), patch(_SLEEP), \
patch(_MONOTONIC, side_effect=[0.0, 0.5, 2.0]):
with self.assertRaises(OrchestratorStartError):
self.p.ensure_running(startup_timeout=1.0)
def test_argv_includes_gateway_and_broker(self) -> None:
argv = OrchestratorProcess(port=8099, broker="docker", gateway=True)._argv()
self.assertIn("--gateway", argv)
self.assertIn("bot_bottle.orchestrator", argv)
self.assertEqual("docker", argv[argv.index("--broker") + 1])
def test_argv_omits_gateway_when_disabled(self) -> None:
self.assertNotIn("--gateway", OrchestratorProcess(gateway=False)._argv())
def test_spawn_launches_detached_and_logs(self) -> None:
with patch(_POPEN) as popen:
self.p._spawn()
popen.assert_called_once()
kwargs = popen.call_args.kwargs
self.assertTrue(kwargs["start_new_session"]) # outlives the CLI
self.assertTrue((Path(self._tmp.name) / "orchestrator.log").exists())
if __name__ == "__main__":
unittest.main()
+138
View File
@@ -0,0 +1,138 @@
"""Unit tests for the orchestrator bottle registry + attribution (PRD 0070)."""
from __future__ import annotations
import tempfile
import unittest
from pathlib import Path
from bot_bottle.orchestrator.registry import (
BottleRecord,
RegistryStore,
new_identity_token,
)
class TestRegistryStore(unittest.TestCase):
def setUp(self) -> None:
self._tmp = tempfile.TemporaryDirectory()
self.db = Path(self._tmp.name) / "registry.db"
self.store = RegistryStore(self.db)
self.store.migrate()
def tearDown(self) -> None:
self._tmp.cleanup()
def test_register_mints_id_and_token(self) -> None:
rec = self.store.register("10.243.0.1")
self.assertTrue(rec.bottle_id)
self.assertTrue(rec.identity_token)
self.assertEqual("active", rec.state)
self.assertEqual(rec, self.store.get(rec.bottle_id))
def test_identity_tokens_are_unique(self) -> None:
tokens = {new_identity_token() for _ in range(200)}
self.assertEqual(200, len(tokens))
a = self.store.register("10.243.0.1")
b = self.store.register("10.243.0.3")
self.assertNotEqual(a.identity_token, b.identity_token)
def test_all_and_deregister(self) -> None:
a = self.store.register("10.243.0.1")
b = self.store.register("10.243.0.3")
self.assertEqual({a.bottle_id, b.bottle_id}, {r.bottle_id for r in self.store.all()})
self.assertTrue(self.store.deregister(a.bottle_id))
self.assertEqual([b.bottle_id], [r.bottle_id for r in self.store.all()])
def test_deregister_missing_is_false(self) -> None:
self.assertFalse(self.store.deregister("nope"))
def test_explicit_id_replaces(self) -> None:
self.store.register("10.243.0.1", bottle_id="fixed", metadata="a")
self.store.register("10.243.0.9", bottle_id="fixed", metadata="b")
rec = self.store.get("fixed")
assert rec is not None
self.assertEqual("10.243.0.9", rec.source_ip)
self.assertEqual("b", rec.metadata)
self.assertEqual(1, len(self.store.all()))
def test_attribute_success_requires_ip_and_token(self) -> None:
rec = self.store.register("10.243.0.1")
got = self.store.attribute("10.243.0.1", rec.identity_token)
assert got is not None
self.assertEqual(rec.bottle_id, got.bottle_id)
def test_attribute_wrong_token_denied(self) -> None:
self.store.register("10.243.0.1")
self.assertIsNone(self.store.attribute("10.243.0.1", "wrong-token"))
def test_attribute_unknown_ip_denied(self) -> None:
rec = self.store.register("10.243.0.1")
self.assertIsNone(self.store.attribute("10.243.9.9", rec.identity_token))
def test_attribute_empty_token_denied(self) -> None:
self.store.register("10.243.0.1")
self.assertIsNone(self.store.attribute("10.243.0.1", ""))
def test_attribute_ambiguous_ip_denied(self) -> None:
# Two active bottles on one source IP is a misconfiguration — deny
# rather than guess (fail-closed), even with a valid token.
a = self.store.register("10.243.0.1", bottle_id="a")
self.store.register("10.243.0.1", bottle_id="b")
self.assertIsNone(self.store.attribute("10.243.0.1", a.identity_token))
def test_state_persists_across_reopen(self) -> None:
rec = self.store.register("10.243.0.1")
reopened = RegistryStore(self.db)
got = reopened.get(rec.bottle_id)
assert got is not None
self.assertEqual(rec, got)
# Attribution works against the reopened (durable) store too.
self.assertIsNotNone(reopened.attribute("10.243.0.1", rec.identity_token))
def test_redacted_hides_token(self) -> None:
rec = BottleRecord(
bottle_id="x", source_ip="10.243.0.1", identity_token="secret"
)
self.assertNotIn("identity_token", rec.redacted())
self.assertEqual("10.243.0.1", rec.redacted()["source_ip"])
def test_register_with_policy_resolves_via_attribution(self) -> None:
rec = self.store.register("10.243.0.1", policy='{"allow":["x"]}')
self.assertEqual('{"allow":["x"]}', rec.policy)
got = self.store.attribute("10.243.0.1", rec.identity_token)
assert got is not None
self.assertEqual('{"allow":["x"]}', got.policy)
def test_set_policy_updates_live(self) -> None:
rec = self.store.register("10.243.0.1")
self.assertEqual("", rec.policy)
self.assertTrue(self.store.set_policy(rec.bottle_id, '{"v":2}'))
got = self.store.get(rec.bottle_id)
assert got is not None
self.assertEqual('{"v":2}', got.policy)
def test_set_policy_missing_is_false(self) -> None:
self.assertFalse(self.store.set_policy("ghost", "{}"))
def test_policy_defaults_empty_and_persists(self) -> None:
rec = self.store.register("10.243.0.1")
got = RegistryStore(self.db).get(rec.bottle_id)
assert got is not None
self.assertEqual("", got.policy)
def test_by_source_ip_returns_single_active(self) -> None:
rec = self.store.register("10.243.0.1")
got = self.store.by_source_ip("10.243.0.1")
assert got is not None
self.assertEqual(rec.bottle_id, got.bottle_id)
def test_by_source_ip_unknown_or_ambiguous_denied(self) -> None:
self.assertIsNone(self.store.by_source_ip("10.243.9.9")) # unknown
self.store.register("10.243.0.1", bottle_id="a")
self.store.register("10.243.0.1", bottle_id="b")
self.assertIsNone(self.store.by_source_ip("10.243.0.1")) # ambiguous
if __name__ == "__main__":
unittest.main()
+144
View File
@@ -0,0 +1,144 @@
"""Unit tests for the Orchestrator launch lifecycle (PRD 0070)."""
from __future__ import annotations
import secrets
import tempfile
import unittest
from pathlib import Path
from bot_bottle.orchestrator.broker import LaunchBroker, LaunchRequest, StubBroker
from bot_bottle.orchestrator.registry import RegistryStore
from bot_bottle.orchestrator.service import Orchestrator
from bot_bottle.orchestrator.gateway import Gateway
class _FailingBroker(LaunchBroker):
"""Verifies the token like any broker, then fails the launch — to
exercise the orchestrator's registry rollback."""
def _launch(self, req: LaunchRequest) -> None:
raise RuntimeError("launch failed")
def _teardown(self, req: LaunchRequest) -> None:
pass
class _FakeGateway(Gateway):
"""In-memory gateway for wiring tests."""
def __init__(self) -> None:
self.name = "fake-gateway"
self.ensured = 0
self.built = 0
self._running = False
def ensure_built(self) -> None:
self.built += 1
def ensure_running(self) -> None:
self.ensured += 1
self._running = True
def is_running(self) -> bool:
return self._running
def stop(self) -> None:
self._running = False
class TestOrchestrator(unittest.TestCase):
def setUp(self) -> None:
self._tmp = tempfile.TemporaryDirectory()
self.secret = secrets.token_bytes(16)
self.store = RegistryStore(Path(self._tmp.name) / "r.db")
self.store.migrate()
self.broker = StubBroker(self.secret)
self.orch = Orchestrator(self.store, self.broker, self.secret)
def tearDown(self) -> None:
self._tmp.cleanup()
def test_launch_registers_and_brokers_signed_request(self) -> None:
rec = self.orch.launch_bottle("10.243.0.1", image_ref="sha256:abc", slot=2)
self.assertIsNotNone(self.store.get(rec.bottle_id))
self.assertEqual(1, len(self.broker.launched))
req = self.broker.launched[0]
self.assertEqual("launch", req.op)
self.assertEqual(rec.bottle_id, req.bottle_id)
self.assertEqual("10.243.0.1", req.source_ip)
self.assertEqual("sha256:abc", req.image_ref)
self.assertEqual(2, req.slot)
def test_launch_then_attribute(self) -> None:
rec = self.orch.launch_bottle("10.243.0.3")
got = self.orch.attribute("10.243.0.3", rec.identity_token)
assert got is not None
self.assertEqual(rec.bottle_id, got.bottle_id)
self.assertIsNone(self.orch.attribute("10.243.0.3", "wrong-token"))
def test_teardown_brokers_and_deregisters(self) -> None:
rec = self.orch.launch_bottle("10.243.0.1")
self.assertTrue(self.orch.teardown_bottle(rec.bottle_id))
self.assertIsNone(self.store.get(rec.bottle_id))
self.assertEqual(["teardown"], [r.op for r in self.broker.torn_down])
def test_teardown_unknown_is_false(self) -> None:
self.assertFalse(self.orch.teardown_bottle("ghost"))
def test_launch_with_policy_is_resolvable(self) -> None:
rec = self.orch.launch_bottle("10.243.0.1", policy='{"routes":[]}')
got = self.orch.attribute("10.243.0.1", rec.identity_token)
assert got is not None
self.assertEqual('{"routes":[]}', got.policy)
def test_set_policy_live_reload(self) -> None:
rec = self.orch.launch_bottle("10.243.0.3")
self.assertTrue(self.orch.set_policy(rec.bottle_id, '{"x":1}'))
got = self.orch.attribute("10.243.0.3", rec.identity_token)
assert got is not None
self.assertEqual('{"x":1}', got.policy)
def test_set_policy_unknown_is_false(self) -> None:
self.assertFalse(self.orch.set_policy("ghost", "{}"))
def test_resolve_by_source_ip_without_token(self) -> None:
rec = self.orch.launch_bottle("10.243.0.1", policy="P")
got = self.orch.resolve("10.243.0.1") # network-layer, no token
assert got is not None
self.assertEqual(rec.bottle_id, got.bottle_id)
self.assertEqual("P", got.policy)
def test_resolve_with_token_stays_strict(self) -> None:
rec = self.orch.launch_bottle("10.243.0.3")
self.assertIsNotNone(self.orch.resolve("10.243.0.3", rec.identity_token))
self.assertIsNone(self.orch.resolve("10.243.0.3", "wrong-token"))
def test_launch_rolls_back_registry_on_broker_failure(self) -> None:
orch = Orchestrator(self.store, _FailingBroker(self.secret), self.secret)
with self.assertRaises(RuntimeError):
orch.launch_bottle("10.243.0.9")
self.assertEqual([], self.store.all()) # no orphan
def test_gateway_unconfigured_by_default(self) -> None:
self.assertEqual({"configured": False}, self.orch.gateway_status())
self.orch.ensure_gateway() # no-op, must not raise
def test_gateway_wired_and_ensured(self) -> None:
sc = _FakeGateway()
orch = Orchestrator(self.store, self.broker, self.secret, gateway=sc)
self.assertEqual(
{"configured": True, "name": "fake-gateway", "running": False},
orch.gateway_status(),
)
orch.ensure_gateway()
self.assertEqual(1, sc.built) # ensure_gateway builds first,
self.assertEqual(1, sc.ensured) # then runs
self.assertEqual(
{"configured": True, "name": "fake-gateway", "running": True},
orch.gateway_status(),
)
if __name__ == "__main__":
unittest.main()
+107
View File
@@ -0,0 +1,107 @@
"""Unit tests for the gateway-side PolicyResolver (PRD 0070). HTTP mocked."""
from __future__ import annotations
import json
import unittest
import urllib.error
from unittest.mock import MagicMock, patch
from bot_bottle.policy_resolver import PolicyResolveError, PolicyResolver
_URLOPEN = "bot_bottle.policy_resolver.urllib.request.urlopen"
def _resp(payload: object) -> MagicMock:
"""A urlopen() return value: a context manager whose read() yields JSON."""
m = MagicMock()
m.__enter__.return_value.read.return_value = json.dumps(payload).encode()
return m
def _http_error(code: int) -> urllib.error.HTTPError:
return urllib.error.HTTPError("http://x/resolve", code, "err", {}, None) # type: ignore[arg-type]
class TestPolicyResolver(unittest.TestCase):
def setUp(self) -> None:
self.r = PolicyResolver("http://orch:8080")
def test_resolve_returns_policy(self) -> None:
with patch(_URLOPEN, return_value=_resp({"bottle_id": "b1", "policy": "P"})):
self.assertEqual("P", self.r.resolve("10.243.0.1", "tok"))
def test_resolve_always_fetches_fresh(self) -> None:
# No cache — every resolve hits the orchestrator so revocations /
# policy changes are honored immediately.
with patch(_URLOPEN, return_value=_resp({"policy": "P"})) as m:
self.r.resolve("10.243.0.1", "tok")
self.r.resolve("10.243.0.1", "tok")
self.assertEqual(2, m.call_count)
def test_unattributed_403_is_none_fail_closed(self) -> None:
with patch(_URLOPEN, side_effect=_http_error(403)):
self.assertIsNone(self.r.resolve("10.243.0.9", "tok"))
def test_other_http_error_raises(self) -> None:
with patch(_URLOPEN, side_effect=_http_error(500)):
with self.assertRaises(PolicyResolveError):
self.r.resolve("10.243.0.1", "tok")
def test_unreachable_raises(self) -> None:
with patch(_URLOPEN, side_effect=urllib.error.URLError("refused")):
with self.assertRaises(PolicyResolveError):
self.r.resolve("10.243.0.1", "tok")
def test_missing_policy_field_is_empty(self) -> None:
with patch(_URLOPEN, return_value=_resp({"bottle_id": "b1"})):
self.assertEqual("", self.r.resolve("10.243.0.1", "tok"))
def test_posts_source_ip_and_token(self) -> None:
with patch(_URLOPEN, return_value=_resp({"policy": "P"})) as m:
self.r.resolve("10.243.0.7", "the-token")
req = m.call_args.args[0]
self.assertTrue(req.full_url.endswith("/resolve"))
sent = json.loads(req.data)
self.assertEqual("10.243.0.7", sent["source_ip"])
self.assertEqual("the-token", sent["identity_token"])
def test_resolve_without_token_sends_empty(self) -> None:
with patch(_URLOPEN, return_value=_resp({"policy": "P"})) as m:
self.r.resolve("10.243.0.7") # token optional
self.assertEqual("", json.loads(m.call_args.args[0].data)["identity_token"])
def test_resolve_bottle_id_returns_id(self) -> None:
with patch(_URLOPEN, return_value=_resp({"bottle_id": "b1", "policy": "P"})):
self.assertEqual("b1", self.r.resolve_bottle_id("10.243.0.1", "tok"))
def test_resolve_bottle_id_403_is_none_fail_closed(self) -> None:
with patch(_URLOPEN, side_effect=_http_error(403)):
self.assertIsNone(self.r.resolve_bottle_id("10.243.0.9", "tok"))
def test_resolve_bottle_id_missing_field_is_none(self) -> None:
with patch(_URLOPEN, return_value=_resp({"policy": "P"})):
self.assertIsNone(self.r.resolve_bottle_id("10.243.0.1", "tok"))
def test_resolve_bottle_id_error_raises(self) -> None:
with patch(_URLOPEN, side_effect=_http_error(500)):
with self.assertRaises(PolicyResolveError):
self.r.resolve_bottle_id("10.243.0.1", "tok")
def test_resolve_policy_and_bottle_id_one_call(self) -> None:
with patch(_URLOPEN, return_value=_resp({"bottle_id": "b1", "policy": "P"})) as m:
self.assertEqual(("P", "b1"), self.r.resolve_policy_and_bottle_id("10.243.0.1", "t"))
self.assertEqual(1, m.call_count) # both from a single /resolve
def test_resolve_policy_and_bottle_id_403_is_none_none(self) -> None:
with patch(_URLOPEN, side_effect=_http_error(403)):
self.assertEqual((None, None), self.r.resolve_policy_and_bottle_id("10.243.0.9"))
def test_resolve_policy_and_bottle_id_error_raises(self) -> None:
with patch(_URLOPEN, side_effect=urllib.error.URLError("refused")):
with self.assertRaises(PolicyResolveError):
self.r.resolve_policy_and_bottle_id("10.243.0.1")
if __name__ == "__main__":
unittest.main()
+5 -44
View File
@@ -8,7 +8,8 @@ from datetime import datetime, timezone
from pathlib import Path
from bot_bottle import supervise
from bot_bottle import supervise_types as sv_types
from bot_bottle.paths import host_db_path
from tests.unit import use_bottle_root
from bot_bottle.audit_store import AuditStore
from bot_bottle.queue_store import QueueStore
from bot_bottle.supervise import (
@@ -21,7 +22,6 @@ from bot_bottle.supervise import (
TOOL_EGRESS_ALLOW,
TOOL_GITLEAKS_ALLOW,
archive_proposal,
host_db_path,
list_pending_proposals,
read_audit_entries,
read_proposal,
@@ -123,20 +123,7 @@ class TestQueueIO(unittest.TestCase):
self._tmp.cleanup()
def _patch_home(self, fake_home: Path):
original_sv = supervise.bot_bottle_root
original_svt = sv_types.bot_bottle_root
def fake_root() -> Path:
return fake_home / ".bot-bottle"
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
sv_types.bot_bottle_root = fake_root # type: ignore[assignment]
def restore() -> None:
supervise.bot_bottle_root = original_sv # type: ignore[assignment]
sv_types.bot_bottle_root = original_svt # type: ignore[assignment]
return restore
return use_bottle_root(fake_home / ".bot-bottle")
def test_write_and_read_proposal(self):
p = _proposal()
@@ -240,20 +227,7 @@ class TestAuditLog(unittest.TestCase):
self._tmp.cleanup()
def _patch_home(self, fake_home: Path):
original_sv = supervise.bot_bottle_root
original_svt = sv_types.bot_bottle_root
def fake_root() -> Path:
return fake_home / ".bot-bottle"
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
sv_types.bot_bottle_root = fake_root # type: ignore[assignment]
def restore() -> None:
supervise.bot_bottle_root = original_sv # type: ignore[assignment]
sv_types.bot_bottle_root = original_svt # type: ignore[assignment]
return restore
return use_bottle_root(fake_home / ".bot-bottle")
def test_write_then_read_single_entry(self):
e = AuditEntry(
@@ -400,20 +374,7 @@ class TestSupervisePrepare(unittest.TestCase):
self._tmp.cleanup()
def _patch_home(self, fake_home: Path):
original_sv = supervise.bot_bottle_root
original_svt = sv_types.bot_bottle_root
def fake_root() -> Path:
return fake_home / ".bot-bottle"
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
sv_types.bot_bottle_root = fake_root # type: ignore[assignment]
def restore() -> None:
supervise.bot_bottle_root = original_sv # type: ignore[assignment]
sv_types.bot_bottle_root = original_svt # type: ignore[assignment]
return restore
return use_bottle_root(fake_home / ".bot-bottle")
def test_prepare_creates_queue(self):
plan = _StubSupervise().prepare("dev", self.stage_dir)
+3 -16
View File
@@ -12,7 +12,7 @@ from pathlib import Path
from unittest.mock import patch
from bot_bottle import supervise
from bot_bottle import supervise_types as sv_types
from tests.unit import use_bottle_root
from bot_bottle.audit_store import AuditStore
from bot_bottle.cli import supervise as supervise_cli
from bot_bottle.queue_store import QueueStore
@@ -51,24 +51,11 @@ def _proposal(slug: str = "dev", tool: str = TOOL_EGRESS_ALLOW) -> Proposal:
class _FakeHomeMixin:
"""Patch supervise.bot_bottle_root to a temp dir for the test."""
"""Point bot_bottle_root at a temp dir (via BOT_BOTTLE_ROOT) for the test."""
def _setup_fake_home(self):
self._tmp = tempfile.TemporaryDirectory(prefix="supervise-test.")
original_sv = supervise.bot_bottle_root
original_svt = sv_types.bot_bottle_root
def fake_root() -> Path:
return Path(self._tmp.name) / ".bot-bottle"
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
sv_types.bot_bottle_root = fake_root # type: ignore[assignment]
def restore() -> None:
supervise.bot_bottle_root = original_sv # type: ignore[assignment]
sv_types.bot_bottle_root = original_svt # type: ignore[assignment]
self._restore_home = restore
self._restore_home = use_bottle_root(Path(self._tmp.name) / ".bot-bottle")
QueueStore("").migrate()
AuditStore().migrate()
+11 -12
View File
@@ -11,12 +11,13 @@ from __future__ import annotations
import contextlib
import io
import os
import tempfile
import unittest
from pathlib import Path
from unittest import mock
from bot_bottle import supervise
from tests.unit import use_bottle_root
from bot_bottle.cli import supervise as supervise_cli
from bot_bottle.log import Die, die
@@ -39,18 +40,16 @@ class TestDieCarriesMessage(unittest.TestCase):
class _FakeHomeMixin:
"""Point supervise.bot_bottle_root (what _write_crash_log resolves
through) at a temp dir so the crash log doesn't touch the real
~/.bot-bottle."""
"""Point bot_bottle_root (what _write_crash_log resolves through) at a
temp dir so the crash log doesn't touch the real ~/.bot-bottle."""
def _setup_fake_home(self):
self._tmp = tempfile.TemporaryDirectory(prefix="supervise-crash-test.")
self._orig_root = supervise.bot_bottle_root
self._root = Path(self._tmp.name) / ".bot-bottle"
supervise.bot_bottle_root = lambda: self._root # type: ignore[assignment]
self._restore_root = use_bottle_root(self._root)
def _teardown_fake_home(self):
supervise.bot_bottle_root = self._orig_root # type: ignore[assignment]
self._restore_root()
self._tmp.cleanup()
@@ -127,11 +126,11 @@ class TestWriteCrashLog(_FakeHomeMixin, unittest.TestCase):
# OSError and the helper must fall back to a tempfile.
bad = Path(self._tmp.name) / "not-a-dir"
bad.write_text("x")
supervise.bot_bottle_root = lambda: bad # type: ignore[assignment]
try:
raise RuntimeError("explode2")
except RuntimeError as e:
path = supervise_cli._write_crash_log(e)
with mock.patch.dict(os.environ, {"BOT_BOTTLE_ROOT": str(bad)}):
try:
raise RuntimeError("explode2")
except RuntimeError as e:
path = supervise_cli._write_crash_log(e)
self.assertTrue(path.exists())
self.assertIn("explode2", path.read_text())
+2 -1
View File
@@ -12,6 +12,7 @@ from unittest.mock import patch
from bot_bottle import supervise
from bot_bottle.audit_store import AuditStore
from bot_bottle.paths import bot_bottle_root
from bot_bottle.queue_store import QueueStore
from bot_bottle.supervise import (
AuditEntry,
@@ -39,7 +40,7 @@ def _proposal() -> Proposal:
class TestPathHelpers(unittest.TestCase):
def test_bot_bottle_root(self) -> None:
self.assertTrue(str(supervise.bot_bottle_root()).endswith(".bot-bottle"))
self.assertTrue(str(bot_bottle_root()).endswith(".bot-bottle"))
class TestReadMalformed(unittest.TestCase):
+56 -19
View File
@@ -6,10 +6,13 @@ import sys
import tempfile
import threading
import time
import types
import unittest
from pathlib import Path
from unittest.mock import patch
from tests.unit import use_bottle_root
# The server module loads `supervise` via same-directory import inside
# the container (Dockerfile.supervise WORKDIRs into /app). For tests
@@ -19,10 +22,8 @@ sys.path.insert(0, str(Path(__file__).resolve().parent.parent.parent / "bot_bott
import supervise as _sv # noqa: E402 # type: ignore
import queue_store as _qs # noqa: E402 # type: ignore
import audit_store as _as # noqa: E402 # type: ignore
import supervise_types as svt_flat # noqa: E402 # type: ignore
from bot_bottle import supervise_server # noqa: E402
from bot_bottle import supervise_types as svt_pkg # noqa: E402
from bot_bottle.supervise_server import (
ERR_INTERNAL,
ERR_INVALID_PARAMS,
@@ -279,23 +280,7 @@ class TestHandleToolsCall(unittest.TestCase):
self._tmp.cleanup()
def _patch_home(self, fake_home: Path):
original_sv = _sv.bot_bottle_root
original_flat = svt_flat.bot_bottle_root
original_pkg = svt_pkg.bot_bottle_root
def fake_root() -> Path:
return fake_home / ".bot-bottle"
_sv.bot_bottle_root = fake_root # type: ignore[assignment]
svt_flat.bot_bottle_root = fake_root # type: ignore[assignment]
svt_pkg.bot_bottle_root = fake_root # type: ignore[assignment]
def restore() -> None:
_sv.bot_bottle_root = original_sv # type: ignore[assignment]
svt_flat.bot_bottle_root = original_flat # type: ignore[assignment]
svt_pkg.bot_bottle_root = original_pkg # type: ignore[assignment]
return restore
return use_bottle_root(fake_home / ".bot-bottle")
def _respond_when_proposal_appears(self, status: str, notes: str = "") -> threading.Thread:
"""Background thread: poll the queue for a fresh proposal, write a
@@ -645,5 +630,57 @@ class TestHttpEndToEnd(unittest.TestCase):
conn.close()
class _FakeResolver:
def __init__(self, bottle_id: str | None = None, raises: bool = False) -> None:
self._bottle_id = bottle_id
self._raises = raises
self.calls: list[str] = []
def resolve_bottle_id(self, source_ip: str, identity_token: str = "") -> str | None:
del identity_token
self.calls.append(source_ip)
if self._raises:
# Raise the exact class supervise_server catches (it imports
# policy_resolver flat inside the bundle, package-side in tests).
raise supervise_server.PolicyResolveError("orchestrator down")
return self._bottle_id
def _handler(resolver: object) -> MCPHandler:
"""A bare MCPHandler wired with a server (carrying the resolver) and a
client address, enough to exercise `_attributed_config` off-socket."""
h: MCPHandler = MCPHandler.__new__(MCPHandler)
h.server = types.SimpleNamespace(policy_resolver=resolver) # type: ignore[assignment]
h.client_address = ("10.0.0.7", 4321)
return h
class TestAttributedConfig(unittest.TestCase):
"""Consolidated supervise: each proposal is attributed to the calling
bottle by source IP; single-tenant keeps the env slug (PRD 0070)."""
def test_single_tenant_keeps_env_slug(self) -> None:
cfg = _handler(None)._attributed_config(ServerConfig(bottle_slug="dev"))
self.assertEqual("dev", cfg.bottle_slug)
def test_consolidated_binds_source_ip_bottle(self) -> None:
r = _FakeResolver(bottle_id="bottle-x")
cfg = _handler(r)._attributed_config(ServerConfig(bottle_slug="ignored"))
self.assertEqual("bottle-x", cfg.bottle_slug) # resolved slug wins
self.assertEqual(["10.0.0.7"], r.calls)
def test_unattributed_source_fails_closed(self) -> None:
with self.assertRaises(_RpcInternalError):
_handler(_FakeResolver(bottle_id=None))._attributed_config(
ServerConfig(bottle_slug="x")
)
def test_resolver_error_fails_closed(self) -> None:
with self.assertRaises(_RpcInternalError):
_handler(_FakeResolver(raises=True))._attributed_config(
ServerConfig(bottle_slug="x")
)
if __name__ == "__main__":
unittest.main()