Compare commits

..

24 Commits

Author SHA1 Message Date
didericis e0b0429cd1 refactor(orchestrator): rename Sidecar -> Gateway for the consolidated data plane
test / unit (pull_request) Successful in 1m10s
test / integration (pull_request) Successful in 25s
test / coverage (pull_request) Successful in 1m17s
lint / lint (push) Successful in 2m11s
Retire "sidecar" for the consolidated per-host path (PRD 0070 naming
decision): the orchestrator is the umbrella/control plane, and the
egress/git/supervise data-plane unit it runs is the "gateway".

- git mv sidecar.py -> gateway.py and the two integration + one unit test
  files; DockerSidecar->DockerGateway, Sidecar->Gateway,
  SidecarError->GatewayError, SIDECAR_*->GATEWAY_*, ensure_sidecar->
  ensure_gateway, sidecar_status->gateway_status, container name
  bot-bottle-orch-sidecar->bot-bottle-orch-gateway.
- Prose rename across broker/registry/egress/policy_resolver + PRD 0070.
- Preserved: the image name bot-bottle-sidecars, the
  BOT_BOTTLE_SIDECAR_IMAGE env var, Dockerfile.sidecars, and PRD 0069's
  own stage-name cross-references (that doc still uses "sidecar").

No behavior change. Full unit suite green (1679 tests; the 13
test_sidecar_init /bin/sleep errors are pre-existing NixOS-local noise).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 18:12:17 -04:00
didericis d3e08cf039 feat(orchestrator+egress): slice 8 — multi-tenant egress via the resolver (#352)
lint / lint (push) Successful in 2m8s
test / unit (pull_request) Successful in 1m1s
test / integration (pull_request) Successful in 25s
test / coverage (pull_request) Successful in 1m25s
The egress addon now selects each request's Config by the calling bottle's
source IP, so one shared sidecar serves every bottle. Opt-in and fail-closed;
single-tenant behaviour is unchanged.

Orchestrator side (source-IP-primary attribution, per the PRD invariant):
  * registry: `by_source_ip` (the single active bottle at a source IP —
    network-layer attribution); `attribute` now composes it + the token.
  * service: `resolve(source_ip, token="")` — with a token, strict
    attribution; without, source IP alone.
  * control_plane: `POST /resolve`'s identity_token is now OPTIONAL (absent
    → source-IP-only); split cleanly from the token-required `/attribute`.
  * policy_resolver: `resolve` token now optional.

Egress side:
  * egress_addon_core: `resolve_client_config(resolver, client_ip, token)` —
    fetches + parses the client's Config, **fail-closed**: unattributed, a
    resolver error, or an unparseable policy all yield deny-all (no routes).
    Host-testable; `PolicyResolverLike` Protocol keeps it import-free.
  * egress_addon: consolidated mode when `BOT_BOTTLE_ORCHESTRATOR_URL` is
    set → `_active_config(flow)` resolves per client IP (reads + strips the
    `x-bot-bottle-identity` header); `request()` uses it. Unset → the static
    routes file, exactly as before. `PolicyResolver` added to the bundle.

Security note: source-IP-only resolution is safe where the IP is unspoofable
(Firecracker /31 + nft) AND the control plane is reachable only by the
trusted sidecar; the identity token, when the agent injects it, strengthens
it on weaker backends.

Scope note: the egress data plane is now multi-tenant. Remaining to be fully
live: the network topology routing every bottle's proxy to the one shared
sidecar, git-gate multitenancy, and agent-side identity-token injection.

Tests: registry by_source_ip; orchestrator resolve (with/without token);
control-plane /resolve token-optional; resolver token-optional;
resolve_client_config fail-closed matrix. All 182 egress tests still pass
(single-tenant unchanged). Full suite green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 17:48:46 -04:00
didericis de9da29027 refactor(orchestrator): drop the PolicyResolver cache (#352)
lint / lint (push) Successful in 2m0s
test / unit (pull_request) Successful in 1m1s
test / integration (pull_request) Successful in 20s
test / coverage (pull_request) Successful in 1m12s
Review: the resolver is called rarely enough that a round-trip doesn't
matter, and correctness beats speed — always fetching means a revocation,
policy change, or teardown the orchestrator knows about is honored
immediately instead of lingering for a cache TTL. Remove the TTL cache
(and `invalidate`); `resolve` now hits the orchestrator every call. Noted
in the docstring that any future caching should use orchestrator-driven
invalidation, not a blind TTL.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 17:32:05 -04:00
didericis 72ea500342 feat(orchestrator): slice 7 — sidecar-side PolicyResolver (#352)
lint / lint (push) Successful in 2m2s
test / unit (pull_request) Successful in 1m3s
test / integration (pull_request) Successful in 22s
test / coverage (pull_request) Successful in 1m9s
The data-plane bridge that lets the consolidated sidecar apply each
bottle's policy per request. `PolicyResolver` resolves a client's policy
from the orchestrator's `POST /resolve` keyed on (source_ip, identity
token) and caches it briefly (short TTL) so it isn't a round-trip per
request; `invalidate()` drops an entry on teardown / live reload.

Fail-closed: an unattributed client (orchestrator answers 403) resolves to
None so the caller denies; unreachable / unexpected status raises so the
caller can fail closed too rather than serve stale/empty policy. Stdlib
only and free of bot-bottle imports, so it can be COPYed flat into the
sidecar bundle.

Scope note: this is the sidecar-side *client*. Wiring it into the live
egress mitmproxy addon (select `Config` per client IP in the request path)
and git-gate, plus routing all bottles' egress to the one shared sidecar,
are the remaining data-plane pieces — a heavier change to the sidecar
bundle's adversarial-input code, taken next.

Tests: resolve returns/caches/expires/invalidates; 403 -> None (fail
closed); other HTTP status + unreachable raise; missing policy -> empty;
posts source_ip + identity_token. Full suite green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 17:21:27 -04:00
didericis f754c575d7 feat(orchestrator): slice 6 — source-IP-keyed multi-tenant policy (#352)
lint / lint (push) Successful in 2m1s
test / unit (pull_request) Successful in 1m5s
test / integration (pull_request) Successful in 21s
test / coverage (pull_request) Successful in 1m16s
The orchestrator side of the multi-tenant consolidated sidecar: hold each
bottle's sidecar policy and serve it by verified source IP, with live
reload. One shared sidecar can now get per-bottle config keyed on who's
calling.

  * registry.py — a `policy` column (migration v3, opaque JSON the sidecar
    interprets) on BottleRecord; `register(..., policy=)` stores it,
    `set_policy(bottle_id, policy)` updates it live, and `attribute` returns
    it (the source-IP-keyed resolution).
  * service.py — `launch_bottle(..., policy=)` and `set_policy`.
  * control_plane.py — `POST /bottles` accepts `policy`; `PUT
    /bottles/<id>/policy` live-reloads it; `POST /resolve` returns
    {bottle_id, policy} for a verified (source_ip, token) — the per-request
    call the multi-tenant sidecar makes; `/attribute` stays identity-only.

Scope note: this is the control-plane / state half. The data-plane half —
the egress mitmproxy addon (and git-gate) selecting allowlist / DLP /
token-injection per client IP by calling `/resolve` — is the next slice
(route agent bottles through the shared sidecar). The orchestrator stays
policy-agnostic: it stores and serves the blob verbatim.

Tests: registry policy store/update/persist; Orchestrator launch-with-policy
+ live set_policy; control-plane resolve returns policy (403 on bad token),
PUT policy updates / 404 / 400. Verified live over HTTP (launch -> resolve
-> PUT reload -> resolve reflects). Full suite green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 17:09:32 -04:00
didericis a092d00312 feat(orchestrator): slice 5 — build the consolidated sidecar bundle image (#352)
lint / lint (push) Successful in 2m7s
test / unit (pull_request) Successful in 1m5s
test / integration (pull_request) Successful in 20s
test / coverage (pull_request) Successful in 1m12s
Answers "where do we build the consolidated sidecar": nowhere, until now.

  * sidecar.py — `Sidecar.ensure_built()` (default no-op) + `DockerSidecar`
    now defaults its image to the real bundle (`bot-bottle-sidecars`) and
    `ensure_built()` builds it from `Dockerfile.sidecars` when
    `docker image inspect` shows it's missing (no-op when present or when no
    dockerfile is configured, e.g. a pre-pulled image). `image_exists()`
    added.
  * service.py — `ensure_sidecar()` now builds then runs.
  * __main__.py — `--sidecar` runs the consolidated bundle (build-if-missing).

Scope note: this builds + launches the bundle *container*; making the
running instance functional across bottles needs the per-bottle,
source-IP-keyed multi-tenant config + registration/reload, and routing
agent bottles to it — the next slices (added to PRD 0070's roadmap).

Tests: unit (docker mocked) — image_exists, ensure_built builds when
missing / no-op when present / no-op without a dockerfile / raises on build
failure; ensure_sidecar builds-then-runs; integration (gated, no heavy
build) — image_exists reflects real docker state. Full suite green (only
pre-existing /bin/sleep errors).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 17:00:18 -04:00
didericis 67652899eb refactor(orchestrator): move run_docker to a lean top-level docker_cmd (#352)
lint / lint (push) Successful in 2m2s
test / unit (pull_request) Successful in 1m0s
test / integration (pull_request) Successful in 18s
test / coverage (pull_request) Successful in 1m7s
Review feedback: don't bury a parallel docker util in the orchestrator.
But reusing backend.docker.util (docker_mod) isn't right either — importing
it runs backend/__init__.py, which eagerly loads all three backends
(docker + firecracker + macos) plus the manifest/egress/git-gate/supervise
framework (~76 modules), so every orchestrator import would drag the whole
backend layer in.

Compromise: promote the helper to a top-level, framework-free
bot_bottle/docker_cmd.py (single stdlib import), a proper shared home the
orchestrator's docker components use now and backend.docker.util can adopt
later. Verified `import bot_bottle.orchestrator` stays lean (12 modules, no
firecracker/macos backends).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 16:24:37 -04:00
didericis f85cbdeebf feat(orchestrator): slice 4 — consolidated per-host sidecar (#352)
lint / lint (push) Successful in 1m58s
test / unit (pull_request) Successful in 58s
test / integration (pull_request) Successful in 18s
test / coverage (pull_request) Successful in 1m7s
The core consolidation win: one persistent sidecar per host, shared by
every bottle, instead of a sidecar bundle per bottle. Safe to share
because the attribution invariant (source IP + identity token) lets the
sidecar map each request to the right bottle.

  * orchestrator/sidecar.py — a backend-neutral `Sidecar` lifecycle
    contract (mirrors LaunchBroker) + a `DockerSidecar` impl. The defining
    behaviour is idempotent singleton: `ensure_running` starts the instance
    if absent and is a no-op if it's already up, so N launches never spawn
    N sidecars; `stop` is idempotent.
  * orchestrator/dockerutil.py — a shared `run_docker` helper; DockerBroker
    now uses it too (DRY with slice 3).
  * service.py — the Orchestrator holds an optional `Sidecar`, exposes
    `ensure_sidecar()` + `sidecar_status()`.
  * control_plane.py — `GET /sidecar` reports it; __main__ gains
    `--sidecar-image` and ensures the single sidecar on startup.

Tests: unit (docker mocked) — is_running, ensure idempotent (no-op when up,
starts when absent), failure raises, stop idempotent; Orchestrator sidecar
wiring/status; control-plane /sidecar; integration (gated) — ensure is a
real idempotent singleton (one container after two ensures), stop removes.
Full suite green (only pre-existing /bin/sleep errors); integration
verified locally against real docker.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 15:28:29 -04:00
didericis 44e611d14e fix(orchestrator): pyright — pass explicit LaunchRequest in the docker integration test (#352)
lint / lint (push) Successful in 1m58s
test / unit (pull_request) Successful in 59s
test / integration (pull_request) Successful in 20s
test / coverage (pull_request) Successful in 1m6s
The **kw unpacking put str into slot: int|None. Pass explicit
LaunchRequests instead.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 15:05:55 -04:00
didericis 4fb0f64249 feat(orchestrator): slice 3 — real Docker launch broker (#352)
lint / lint (push) Failing after 1m59s
test / unit (pull_request) Successful in 58s
test / integration (pull_request) Successful in 22s
test / coverage (pull_request) Successful in 1m6s
The first concrete LaunchBroker, proving the orchestrator -> backend seam
on the cheapest backend (the sidecar bundle is already containers):

  * orchestrator/docker_broker.py — DockerBroker runs a container on a
    verified launch (`docker run --detach --name <bottle> --label ...
    <image_ref>`) and removes it on teardown (`docker rm --force`,
    idempotent on an already-absent container). The argv is built only from
    the request's static ids/flags, so nothing free-form reaches docker;
    provenance/schema verification is inherited from LaunchBroker.submit.
  * __main__.py gains `--broker {stub,docker}` so the harness can drive real
    containers.

Slice 3 launches a single container from image_ref (the seam); the full
agent + sidecar bundle is a later slice.

Tests: unit (docker mocked) — argv from static fields, launch/teardown call
the right commands, missing-image and docker-failure raise, teardown
idempotent on missing, forged token never touches docker; integration
(gated on a reachable daemon) — launch creates a real container, teardown
removes it. Full suite green (only pre-existing /bin/sleep errors);
integration verified locally against real docker.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 15:02:11 -04:00
didericis 9226d45041 feat(orchestrator): slice 2 — launch lifecycle + signed launch-broker (#352)
lint / lint (push) Successful in 2m2s
test / unit (pull_request) Successful in 57s
test / integration (pull_request) Successful in 18s
test / coverage (pull_request) Successful in 1m6s
Second slice of PRD 0070, still a backend-neutral dev-harness:

  * orchestrator/broker.py — the launch-broker contract. A LaunchRequest is
    structured (static ids/flags only — bottle id, pool slot, a
    content-addressed image_ref; never a path/argv) and signed as a compact
    HS256 JWT so the broker verifies PROVENANCE before acting: a compromised
    co-located component can't forge a launch without the shared secret.
    verify_request is fail-closed (bad sig / malformed / off-schema -> raise).
    Stdlib only (no runtime deps). Ships a StubBroker that records verified
    requests for the harness/tests.
  * orchestrator/service.py — the Orchestrator: owns the registry and brokers
    the lifecycle. launch_bottle mints the bottle + sends a signed launch,
    rolling the registry entry back if the launch fails (no orphans);
    teardown_bottle brokers teardown then deregisters; attribute delegates.
  * control_plane.py — POST /bottles now launches, DELETE tears down (both go
    through the Orchestrator + broker). dispatch/server take an Orchestrator.
  * __main__.py wires an ephemeral secret + StubBroker for the harness.

Tests: broker sign/verify round-trip, tamper/wrong-secret/malformed/off-schema
rejection, StubBroker fail-closed; Orchestrator launch->registry->attribute,
teardown, rollback-on-broker-failure; control-plane updated for launch/teardown.
Full suite green (only the pre-existing /bin/sleep errors); harness does
launch -> attribute -> teardown over HTTP.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 14:42:35 -04:00
didericis 6847fdf0ab refactor: drop the vestigial bot_bottle_root/host_db_path re-exports (#352)
lint / lint (push) Successful in 1m58s
test / unit (pull_request) Successful in 1m0s
test / integration (pull_request) Successful in 19s
test / coverage (pull_request) Successful in 1m6s
paths is the single home now, so stop re-exporting the path helpers from
supervise: remove host_db_path from supervise's imports + __all__ (it was
re-export-only) and drop bot_bottle_root from __all__ (kept as an import,
still used by audit_dir). supervise_types was already clean. Repoint the
last readers (test_supervise imports host_db_path from paths;
test_supervise_edge calls paths.bot_bottle_root) and refresh the doc
mentions. No supervise.bot_bottle_root / supervise.host_db_path references
remain.

Behavior-preserving: full unit suite unchanged (only the pre-existing
/bin/sleep sidecar-init errors).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 14:30:13 -04:00
didericis 421d31c32f refactor: move bot_bottle_root/host_db_path to paths; kill the monkeypatch (#352)
lint / lint (push) Successful in 1m57s
test / unit (pull_request) Successful in 55s
test / integration (pull_request) Successful in 15s
test / coverage (pull_request) Successful in 1m1s
Create bot_bottle/paths.py as the canonical home for the app-root path
helpers (bot_bottle_root, host_db_path, HOST_DB_FILENAME) — foundational,
not supervise- or db-specific. `bot_bottle_root()` now honours a
BOT_BOTTLE_ROOT env override.

Repoint every consumer (supervise, supervise_types, db_store, queue_store,
audit_store, store_manager, bottle_state, cli/supervise, docker/cleanup,
orchestrator/registry) at paths; remove the definitions (and supervise's
duplicate host_db_path) and the now-dead `import sys`. Add paths.py to the
sidecar bundle (Dockerfile.sidecars) for the flat-import copies.

Tests: replace ~12 files' monkeypatching of supervise.bot_bottle_root (and
the flat/pkg/supervise_types triple-patch dance) with a single
`use_bottle_root()` helper that sets BOT_BOTTLE_ROOT — every module and
flat/package copy reads the same env var, so one override covers them all.
Net -97 lines. Behaviour-preserving: full unit suite unchanged (only the
pre-existing /bin/sleep sidecar-init errors remain).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 14:04:23 -04:00
didericis b4b4e08f62 refactor: move host_db_path/HOST_DB_FILENAME to db_store (#352)
lint / lint (push) Successful in 1m58s
test / unit (pull_request) Successful in 57s
test / integration (pull_request) Successful in 16s
test / coverage (pull_request) Successful in 1m0s
host_db_path is shared DB infrastructure, not supervise-specific, so its
canonical home is db_store (alongside DbStore). It resolves bot_bottle_root
from supervise_types lazily inside the function — no load-time cycle, and a
monkey-patch of supervise_types.bot_bottle_root still propagates.
supervise_types re-exports both names for the historical import path
(queue_store/audit_store unchanged); the orchestrator registry now imports
from db_store. Drops the now-unused `import sys` from supervise_types.

Behavior-preserving: full unit suite unchanged (only the pre-existing
/bin/sleep sidecar-init errors remain); monkeypatch propagation verified.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 13:36:19 -04:00
didericis b174442c60 feat(orchestrator): registry co-tenants the shared bot-bottle.db (#352)
lint / lint (push) Successful in 1m57s
test / unit (pull_request) Successful in 59s
test / integration (pull_request) Successful in 18s
test / coverage (pull_request) Successful in 1m5s
Per review: use one shared bot-bottle.db for all runtime state including
the registry (DbStore namespaces by schema_key), so it's one queryable
file for backup/console. default_db_path() -> host_db_path(). Drop the
unilateral WAL flip — WAL on the shared DB affects supervise/audit and is
finicky over guest shares, so it's a deliberate future change; keep a
busy_timeout for lock contention.

PRD State section updated: integrity now by SOLE ownership (only the
orchestrator opens bot-bottle.db; data plane + console reach state via the
control-plane RPC, never a file handle) rather than ro/rw mount-splitting,
which one shared file can't do. Notes the transitional caveat that the
supervise sidecar currently rw-mounts bot-bottle.db.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 13:26:03 -04:00
didericis ed7307e0e3 docs(prd): 0070 registry DB is host-resident with rw/ro access split (#352)
test / unit (pull_request) Successful in 55s
test / integration (pull_request) Successful in 16s
test / coverage (pull_request) Successful in 1m1s
Per review: the SQLite runtime-state DB lives on the host, not owned
inside the orchestrator unit. Durability (re-adoption must survive an
orchestrator restart) + integrity via access-scoping (control-plane rw,
data-plane ro) rather than location. Note the WAL-over-guest-share wrinkle
for the VM slices (may want a host-side DB owner reached over the RPC).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 13:17:05 -04:00
didericis 1702664b81 feat(orchestrator): slice 1 — registry + attribution + HTTP control plane (#352)
lint / lint (push) Successful in 1m58s
test / unit (pull_request) Successful in 56s
test / integration (pull_request) Successful in 17s
test / coverage (pull_request) Successful in 1m2s
First implementation slice of PRD 0070, the backend-neutral consolidation
core as a plain-process dev-harness (no VM packaging yet):

  * orchestrator/registry.py — SQLite (WAL) runtime-state store on the
    existing DbStore/TableMigrations base. Live bottle registry keyed by
    source IP + per-bottle identity token, with fail-closed attribution:
    a request resolves to a bottle only when its source IP AND identity
    token both match exactly one active record (unknown/ambiguous IP,
    empty token, or token mismatch all deny). Tokens are 256-bit urandom.
  * orchestrator/control_plane.py — the HTTP control plane (the universal
    transport chosen in 0070): register / deregister / list / attribute /
    health. Routing is a pure dispatch() so it is socket-free testable;
    Handler/ControlPlaneServer/make_server are a thin stdlib adapter.
    register/deregister are the live-reload path; listing redacts tokens.
  * orchestrator/__main__.py — `python -m bot_bottle.orchestrator` harness.

Launch/teardown, the launch broker, and the egress/git/supervise data
plane come in later slices. 24 unit tests (attribution matrix, persistence,
dispatch, one real-socket round-trip).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 13:00:04 -04:00
didericis 8d54fc38ea docs(prd): 0070 VM-to-VM routing is not a blocker (#352)
Per review: resolvable at implementation time (host tweak acceptable, as
the pool setup already needs on NixOS); the earlier sidecars-in-VMs spike
showed feasibility. No need to hunt the spike now.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 12:46:08 -04:00
didericis 73a7582abe docs(prd): 0070 fold in review decisions (#352)
Resolve open questions from review: consolidate egress (hardened minimal
surface + identity token + vault mitigations); HTTP control-plane
transport; broker schema = signed-JWT JSON of static flags+ids (provenance
+ un-coercible); state re-adoption procedure (singleton orchestrator →
wait-healthy → adopt via SQLite + process inspection before serving, with
write-ahead intent to close the in-flight-launch race). Add a per-bottle
identity-token defense-in-depth layer on the attribution invariant.
Remaining open: VM-to-VM routing (per-backend wire(), pending spike link),
live-reload protocol, identity-token delivery.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 12:39:06 -04:00
didericis 095896817c docs(prd): 0070 secret-handling as a future pattern (SecretProvider, #355)
lint / lint (push) Successful in 2m3s
Add a "Secret handling — FUTURE pattern (not v1)" subsection: vault as a
separate trust domain holding long-lived roots, deriving short-lived
scoped creds where the upstream allows (with the honest limit that a
compromised proxy can still abuse currently-authorized access). The
mechanism is a generic, user-extensible SecretProvider that generalizes
PRD 0048's DeployKeyProvisioner and drops into the manifest wherever a raw
token is accepted — discovered from ~/.bot-bottle/contrib like user
AgentProviders. Marked explicitly as not required for the initial
orchestrator; tracked as #355.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 12:14:25 -04:00
didericis a30dd49967 docs(prd): 0070 per-host orchestrator service
Fold the per-bottle sidecar bundle into a single persistent per-host
orchestrator: runs the sidecar functions (egress/git-gate/supervise),
coordinates with the console, and brokers agent launches. Virtualized
from the start with backend-native isolation (fc VM / apple ctr / docker
ctr), fronted by a single backend-agnostic contract; per-backend
variation lives on BottleBackend, not a parallel Orchestrator hierarchy.

Leads with the security review (secret concentration, shared fate, the
launch-broker-as-new-privileged-core, and the source-IP attribution
invariant each backend must enforce). Proposes one SQLite DB owned by the
orchestrator for runtime state (slot leases, approvals, registry) —
distinct from build-time constants (flat .env) and user config
(declarative ~/.bot-bottle). Sequences docker -> firecracker -> macos,
developing the service as a plain-process dev-harness before the VM.

Supersedes 0069's Stage-1/4 sidecar framing; depends on 0069's nix-built
fixed images. Tracking issue #351.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 12:14:25 -04:00
didericis 2b970d1170 refactor(firecracker): single-source the network-pool defaults
test / unit (pull_request) Successful in 54s
test / integration (pull_request) Successful in 15s
test / coverage (pull_request) Successful in 56s
lint / lint (push) Successful in 1m59s
test / unit (push) Successful in 53s
test / integration (push) Successful in 20s
test / coverage (push) Successful in 58s
Update Quality Badges / update-badges (push) Successful in 57s
The pool params (size, IP base, iface prefix, nft table) were triplicated
— hardcoded in netpool.py, scripts/firecracker-netpool.sh, and
nix/firecracker-netpool.nix — plus the IP math (3x) and the nft ruleset
(2x). Nothing enforced agreement; changing the base (ce3fad9, off CGNAT)
forced a coordinated three-file edit, and a missed one would silently
provision a range the launcher doesn't expect.

Collapse to one source of truth:

  * netpool.defaults.env — a plain KEY=VALUE file (bash-sourceable,
    systemd EnvironmentFile-compatible, Python- and Nix-parseable) holding
    the four defaults. A BOT_BOTTLE_FC_* env var still overrides any key.
  * netpool.py reads it for the Python defaults (missing file = hard
    error, not confusing empty defaults).
  * the shell script falls back to it (no literal `:-8` / `10.243.0.0`),
    and its `up` is now non-destructive/idempotent (only creates a
    missing TAP), so re-running never cuts a live VM.
  * the Nix module readFile-parses it for its option defaults and
    delegates bring-up to the SAME shell script (dropping its duplicate
    IP math, nft ruleset, and TAP loop) — passing every value as
    Environment= so the store-detached script never needs the file.

Net: defaults 3x -> 1x, nft ruleset 2x -> 1x, TAP loop 2x -> 1x. The one
remaining IP-math dup (Python launch-addressing vs bash bring-up) is
justified — different runtimes. Tests now guard the invariant (Python
reads the shared file; the script/module hold no literals) instead of
pinning duplicated strings.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-13 04:27:48 -04:00
Quality Badge Bot 1bfc6c5d16 chore: update quality badges
- Coverage: 82%
- Core coverage: 95%

[skip ci]
2026-07-12 20:54:05 +00:00
didericis-claude ed9adfb678 Merge pull request 'feat(firecracker): Linux microVM backend to replace smolmachines' (#343) from firecracker-backend into main
lint / lint (push) Successful in 1m58s
test / unit (push) Successful in 55s
test / integration (push) Successful in 16s
test / coverage (push) Successful in 56s
Update Quality Badges / update-badges (push) Successful in 56s
2026-07-12 16:49:01 -04:00
56 changed files with 3150 additions and 375 deletions
+2
View File
@@ -64,8 +64,10 @@ COPY --from=gitleaks-src /usr/bin/gitleaks /usr/bin/gitleaks
COPY bot_bottle/egress_addon_core.py /app/egress_addon_core.py
COPY bot_bottle/egress_dlp_config.py /app/egress_dlp_config.py
COPY bot_bottle/egress_addon.py /app/egress_addon.py
COPY bot_bottle/policy_resolver.py /app/policy_resolver.py
COPY bot_bottle/dlp_detectors.py /app/dlp_detectors.py
COPY bot_bottle/yaml_subset.py /app/yaml_subset.py
COPY bot_bottle/paths.py /app/paths.py
COPY bot_bottle/migrations.py /app/migrations.py
COPY bot_bottle/db_store.py /app/db_store.py
COPY bot_bottle/supervise_types.py /app/supervise_types.py
+1 -1
View File
@@ -5,7 +5,7 @@
# bot-bottle
[![test](https://gitea.dideric.is/didericis/bot-bottle/actions/workflows/test.yml/badge.svg?branch=main)](https://gitea.dideric.is/didericis/bot-bottle/actions?workflow=test.yml)
[![coverage](https://img.shields.io/badge/coverage-84%25-brightgreen)](https://coverage.readthedocs.io/)
[![coverage](https://img.shields.io/badge/coverage-82%25-brightgreen)](https://coverage.readthedocs.io/)
[![core coverage](https://img.shields.io/badge/core%20coverage-95%25-brightgreen)](https://gitea.dideric.is/didericis/bot-bottle/src/branch/main/docs/decisions/0004-coverage-policy.md)
**Problem:** Developer wants to run a coding agent without supervision, but they don't want a prompt injected or misbehaving agent wrecking their environment or exfiltrating sensitive data.
+4 -2
View File
@@ -6,11 +6,13 @@ import sqlite3
from pathlib import Path
try:
from .supervise_types import AuditEntry, host_db_path
from .supervise_types import AuditEntry
from .paths import host_db_path
from .db_store import DbStore
from .migrations import TableMigrations
except ImportError:
from supervise_types import AuditEntry, host_db_path # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from supervise_types import AuditEntry # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from paths import host_db_path # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from db_store import DbStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from migrations import TableMigrations # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
+2 -2
View File
@@ -26,7 +26,7 @@ from __future__ import annotations
import shutil
import subprocess
from ... import supervise as _supervise
from ...paths import bot_bottle_root
from ...log import info, warn
from . import util as docker_mod
from .bottle_cleanup_plan import DockerBottleCleanupPlan
@@ -94,7 +94,7 @@ def _list_orphan_state_dirs(
ANY backend — used so this docker-side check doesn't reap a
running non-docker bottle's state dir (the layout is shared
across backends)."""
state_root = _supervise.bot_bottle_root() / "state"
state_root = bot_bottle_root() / "state"
if not state_root.is_dir():
return []
orphans: list[str] = []
@@ -0,0 +1,17 @@
# Firecracker network-pool defaults — the SINGLE source of these values.
#
# Read by every consumer so they can't drift:
# * netpool.py — parses this for the Python defaults (below).
# * scripts/firecracker-netpool.sh — falls back to these when the
# matching BOT_BOTTLE_FC_* env var is unset.
# * nix/firecracker-netpool.nix — readFile-parses this for its option
# defaults, then passes the resolved values back as Environment=.
#
# Plain KEY=VALUE (no quoting, no inline comments, no spaces around `=`)
# so it is bash-sourceable, systemd EnvironmentFile-compatible, and
# trivially parseable from Python and Nix. A real BOT_BOTTLE_FC_* env
# var of the same name always overrides the value here.
BOT_BOTTLE_FC_POOL_SIZE=8
BOT_BOTTLE_FC_IP_BASE=10.243.0.0
BOT_BOTTLE_FC_IFACE_PREFIX=bbfc
BOT_BOTTLE_FC_NFT_TABLE=bot_bottle_fc
+40 -9
View File
@@ -4,11 +4,13 @@ config renderers (shell command + NixOS module) shown to operators.
The Firecracker backend needs a privileged one-time network setup:
a pool of point-to-point TAP devices (owned by the invoking user, so
`./cli.py start` never needs root) and a dedicated nftables table that
isolates every VM. This module is the single source of truth for the
pool parameters — the shell script (`scripts/firecracker-netpool.sh`),
the NixOS module (`nix/firecracker-netpool.nix`), and the backend's
fail-closed preflight all derive from these constants so they can't
drift.
isolates every VM. The pool parameters live in exactly one place —
`netpool.defaults.env`, a plain KEY=VALUE file next to this module —
and every consumer reads *that*: this module (below), the shell script
(`scripts/firecracker-netpool.sh`), and the NixOS module
(`nix/firecracker-netpool.nix`). A `BOT_BOTTLE_FC_*` env var of the
same name always overrides the file, and the backend's fail-closed
preflight derives from these accessors, so nothing can drift.
Topology (per slot i):
* TAP ``bbfc{i}`` — no shared bridge, so no docker0 / virbr0 / cni0
@@ -43,18 +45,47 @@ from typing import IO
from ...log import die
# The pool defaults live in one shared file (see module docstring); the
# shell script and NixOS module read the same file, so the values can't
# drift. This is a packaged data file — a missing/broken install is a
# hard error, surfaced here rather than as confusing empty defaults.
DEFAULTS_FILE = Path(__file__).with_name("netpool.defaults.env")
def _load_defaults() -> dict[str, str]:
out: dict[str, str] = {}
for raw in DEFAULTS_FILE.read_text().splitlines():
line = raw.strip()
if not line or line.startswith("#") or "=" not in line:
continue
key, _, value = line.partition("=")
out[key.strip()] = value.strip()
return out
_DEFAULTS = _load_defaults()
def _cfg(key: str) -> str:
"""A `BOT_BOTTLE_FC_*` env var overrides the shared-file default."""
try:
return os.environ.get(key) or _DEFAULTS[key]
except KeyError:
die(f"{key} is missing from {DEFAULTS_FILE.name} (broken install)")
# Interface names are capped at 15 chars (IFNAMSIZ-1); "bbfc" + a small
# index stays well under that and is distinctive enough to grep for.
IFACE_PREFIX = os.environ.get("BOT_BOTTLE_FC_IFACE_PREFIX", "bbfc")
NFT_TABLE = "bot_bottle_fc"
IFACE_PREFIX = _cfg("BOT_BOTTLE_FC_IFACE_PREFIX")
NFT_TABLE = _cfg("BOT_BOTTLE_FC_NFT_TABLE")
def pool_size() -> int:
return int(os.environ.get("BOT_BOTTLE_FC_POOL_SIZE", "8"))
return int(_cfg("BOT_BOTTLE_FC_POOL_SIZE"))
def ip_base() -> str:
return os.environ.get("BOT_BOTTLE_FC_IP_BASE", "10.243.0.0")
return _cfg("BOT_BOTTLE_FC_IP_BASE")
# Sidecar ports the VM reaches at its host-side TAP IP. Kept in sync
+2 -2
View File
@@ -36,7 +36,7 @@ from dataclasses import dataclass
from pathlib import Path
from typing import cast
from . import supervise as _supervise
from .paths import bot_bottle_root
# Directory layout: ~/.bot-bottle/state/<identity>/...
@@ -163,7 +163,7 @@ def read_metadata(identity: str) -> BottleMetadata | None:
def bottle_state_dir(identity: str) -> Path:
"""Per-bottle state directory on the host. Created lazily by the
write helpers; readers tolerate its absence."""
return _supervise.bot_bottle_root() / _STATE_SUBDIR / identity
return bot_bottle_root() / _STATE_SUBDIR / identity
def per_bottle_dockerfile_path(identity: str) -> Path:
+2 -2
View File
@@ -19,7 +19,7 @@ from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
from .. import supervise as _supervise
from ..paths import bot_bottle_root
from ..bottle_state import read_metadata
from ..backend.docker.egress_apply import (
EgressApplyError,
@@ -276,7 +276,7 @@ def _write_crash_log(exc: BaseException) -> Path:
)
entry = f"=== supervise crash {stamp} ===\n{body}\n"
try:
log_dir = _supervise.bot_bottle_root() / "logs"
log_dir = bot_bottle_root() / "logs"
log_dir.mkdir(parents=True, exist_ok=True)
path = log_dir / "supervise-crash.log"
with path.open("a", encoding="utf-8") as fh:
+27
View File
@@ -0,0 +1,27 @@
"""Lean, framework-free `docker` subprocess primitive.
Deliberately a top-level module with a single stdlib import so it can be
reused from anywhere without cost. It is intentionally *not* placed in
`backend.docker.util`: importing that module runs `backend/__init__.py`,
which eagerly loads all three bottle backends (docker + firecracker +
macos) plus the manifest/egress/git-gate/supervise framework — ~76 modules
— which would drag the whole backend layer into the deliberately-lean
orchestrator. This primitive stays free of that so both the orchestrator's
docker components and (in time) `backend.docker.util` can share it."""
from __future__ import annotations
import subprocess
def run_docker(argv: list[str]) -> subprocess.CompletedProcess[str]:
"""Run a `docker` command, capturing stdout/stderr as text. Never raises
on a non-zero exit — callers inspect `returncode` / `stderr` so they can
stay fail-closed or tolerate idempotent no-ops (e.g. removing an
already-absent container)."""
return subprocess.run(
argv, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, check=False,
)
__all__ = ["run_docker"]
+46 -6
View File
@@ -1,4 +1,4 @@
"""mitmproxy addon entrypoint for the egress sidecar (PRD 0017, PRD 0053).
"""mitmproxy addon entrypoint for the egress gateway (PRD 0017, PRD 0053).
Loaded by `mitmdump -s /app/egress_addon.py` inside the
egress container."""
@@ -32,6 +32,7 @@ from egress_addon_core import ( # type: ignore[import-not-found] # pylint: dis
is_git_push_request,
load_config,
match_route,
resolve_client_config,
outbound_scan_headers,
route_to_yaml_dict,
scan_inbound,
@@ -51,11 +52,26 @@ try:
except ImportError: # pragma: no cover - host-side path
from bot_bottle import supervise as _sv # type: ignore[import-not-found]
try:
from policy_resolver import PolicyResolver # type: ignore[import-not-found]
except ImportError: # pragma: no cover - host-side path
from bot_bottle.policy_resolver import PolicyResolver
DEFAULT_ROUTES_PATH = "/etc/egress/routes.yaml"
INTROSPECT_HOST = "_egress.local"
# Consolidated (multi-tenant) mode: when this points at the per-host
# orchestrator's control plane, the addon resolves each client's Config by
# source IP per request instead of using a single static routes file. Unset
# → legacy per-bottle single-tenant mode (unchanged).
ORCHESTRATOR_URL_ENV = "BOT_BOTTLE_ORCHESTRATOR_URL"
# App-layer identity token (defense-in-depth over the source-IP invariant);
# the agent injects it, the addon strips it so it never leaks upstream.
IDENTITY_HEADER = "x-bot-bottle-identity"
# Seconds the egress proxy holds a token-blocked request open waiting for the
# operator's supervisor decision (PRD 0062), overridable via env.
DEFAULT_TOKEN_ALLOW_TIMEOUT_SECONDS = 300.0
@@ -72,9 +88,17 @@ _TOKEN_ALLOW_JUSTIFICATION = (
class EgressAddon:
# Class default so addons built via __new__ (e.g. in tests) default to
# single-tenant; __init__ sets the instance attribute for real runs.
_resolver: "PolicyResolver | None" = None
def __init__(self) -> None:
self.routes_path = os.environ.get("EGRESS_ROUTES", DEFAULT_ROUTES_PATH)
self.config: Config = Config(routes=())
# Consolidated mode: resolve per-client Config from the orchestrator.
# Absent → single-tenant (static routes file); behaviour unchanged.
orch_url = os.environ.get(ORCHESTRATOR_URL_ENV, "").strip()
self._resolver = PolicyResolver(orch_url) if orch_url else None
# Tokens the operator has approved this session (PRD 0062). In-memory
# only — a restart re-prompts. Mutated only from the asyncio loop that
# runs the addon hooks, so no lock is needed.
@@ -194,6 +218,20 @@ class EgressAddon:
+ "\n"
)
def _active_config(self, flow: http.HTTPFlow) -> Config:
"""The Config to apply to this request. Single-tenant → the static
`self.config`. Consolidated → the calling bottle's Config, resolved
by source IP (fail-closed to deny-all if unattributed). The identity
token, if the agent injected one, is read then stripped so it never
leaks upstream."""
if self._resolver is None:
return self.config
conn = flow.client_conn
client_ip = conn.peername[0] if conn and conn.peername else ""
token = flow.request.headers.get(IDENTITY_HEADER, "")
flow.request.headers.pop(IDENTITY_HEADER, None)
return resolve_client_config(self._resolver, client_ip, token)
async def request(self, flow: http.HTTPFlow) -> None:
request_path, _, query = flow.request.path.partition("?")
@@ -201,10 +239,12 @@ class EgressAddon:
self._serve_introspection(flow, request_path)
return
config = self._active_config(flow)
# DLP outbound scan BEFORE stripping auth — catches tokens the
# agent tried to smuggle in any header, path, query param, or body.
# Hostname is included to catch DNS-tunnelling exfiltration attempts.
route = match_route(self.config.routes, flow.request.pretty_host)
route = match_route(config.routes, flow.request.pretty_host)
if route is not None:
if not await self._handle_outbound_dlp(flow, route):
return
@@ -224,7 +264,7 @@ class EgressAddon:
if is_git_fetch_request(request_path, query):
git_decision = decide_git_fetch(
self.config.routes, flow.request.pretty_host,
config.routes, flow.request.pretty_host,
)
if git_decision.action == "block":
self._block(
@@ -235,14 +275,14 @@ class EgressAddon:
return
# Strip agent-set Authorization after DLP scan so smuggled tokens
# are caught above; the route may inject sidecar-owned auth below.
# are caught above; the route may inject gateway-owned auth below.
flow.request.headers.pop("authorization", None)
# Build headers mapping for match evaluation
req_headers = {k.lower(): v for k, v in flow.request.headers.items()}
decision = decide(
self.config.routes,
config.routes,
flow.request.pretty_host,
request_path,
os.environ,
@@ -257,7 +297,7 @@ class EgressAddon:
if decision.inject_authorization is not None:
flow.request.headers["authorization"] = decision.inject_authorization
if self.config.log >= LOG_FULL:
if config.log >= LOG_FULL:
self._log_request(flow)
def _block_dlp(self, flow: http.HTTPFlow, result: ScanResult) -> None:
+35 -5
View File
@@ -3,7 +3,7 @@
Split out of `egress_addon.py` so the host's unit tests can
exercise the parse + decision functions without depending on the
`mitmproxy` package. The companion module wraps these with the
`mitmproxy.http.HTTPFlow` API and is loaded inside the sidecar
`mitmproxy.http.HTTPFlow` API and is loaded inside the gateway
container.
Imports: stdlib + `yaml_subset` (which is itself stdlib-only and
@@ -22,7 +22,7 @@ except ImportError: # pragma: no cover - host-side path
from .yaml_subset import YamlSubsetError, parse_yaml_subset
# DLP detector-config parsing lives in a sibling module (also flat-bundled
# into the sidecar — see Dockerfile.sidecars). Re-exported below so existing
# into the gateway — see Dockerfile.sidecars). Re-exported below so existing
# `from egress_addon_core import ON_MATCH_*` callers keep working.
try:
from egress_dlp_config import ( # type: ignore[import-not-found]
@@ -120,7 +120,7 @@ class ScanResult:
reason: str
location: str = "" # where the match was found, e.g. "body", "authorization header"
context: str = "" # surrounding text with the match replaced by REDACT
# Raw substring the detector matched. Used inside the sidecar to key the
# Raw substring the detector matched. Used inside the gateway to key the
# supervisor-approved "safe tokens" set (PRD 0062); never logged or written
# to a proposal file. Empty for structural detectors (CRLF) that carry no
# safelist-able value.
@@ -413,6 +413,34 @@ def load_config(text: str) -> "Config":
return parse_config(payload)
class PolicyResolverLike(typing.Protocol):
"""The bit of `policy_resolver.PolicyResolver` this module needs — kept a
Protocol so egress_addon_core stays free of that import."""
def resolve(self, source_ip: str, identity_token: str = ...) -> "str | None":
...
def resolve_client_config(
resolver: PolicyResolverLike, client_ip: str, identity_token: str = ""
) -> "Config":
"""The calling client's egress Config, resolved from the orchestrator via
`resolver` and parsed — **fail-closed**. An unattributed client (None), a
resolver error, or an unparseable policy all yield a deny-all Config (no
routes → every request blocked). A compromised, absent, or confused
orchestrator must never *widen* a bottle's egress."""
try:
policy = resolver.resolve(client_ip, identity_token)
except Exception: # noqa: BLE001 # pylint: disable=broad-exception-caught
return Config(routes=()) # orchestrator unreachable/errored → deny
if not policy:
return Config(routes=()) # unattributed or empty → deny-all
try:
return load_config(policy)
except ValueError:
return Config(routes=()) # unparseable policy → deny
# ---------------------------------------------------------------------------
# Match evaluation
# ---------------------------------------------------------------------------
@@ -613,7 +641,7 @@ def outbound_scan_headers(
) -> dict[str, str]:
"""Return request headers that should be included in outbound DLP.
Routes that inject sidecar-owned auth always strip the agent's
Routes that inject gateway-owned auth always strip the agent's
Authorization header before forwarding. Scanning that header first
creates false positives for provider clients that insist on sending
their own bearer-shaped placeholder, while still not changing what
@@ -664,7 +692,7 @@ def scan_outbound(
crlf_text: str | None = None,
) -> ScanResult | None:
# Lazy import to avoid circular deps and keep dlp_detectors optional
# at import time (the sidecar copies it flat alongside this file).
# at import time (the gateway copies it flat alongside this file).
try:
from dlp_detectors import ( # type: ignore[import-not-found]
scan_crlf_injection,
@@ -804,6 +832,8 @@ __all__ = [
"is_git_push_request",
"is_git_fetch_request",
"load_config",
"resolve_client_config",
"PolicyResolverLike",
"match_route",
"outbound_scan_headers",
"parse_config",
+64
View File
@@ -0,0 +1,64 @@
"""Per-host orchestrator service (PRD 0070).
A single persistent per-host service that will run the gateway functions
(egress / git-gate / supervise), coordinate with the console, and broker
agent launches. This package is being built bottom-up, starting with the
backend-neutral "consolidation core" that needs no VM packaging:
* `registry` — the SQLite runtime-state store + fail-closed
attribution (source IP + per-bottle identity token).
* `broker` — the signed, structured launch-request contract + a
`LaunchBroker` (stub for the harness) that verifies
provenance before acting.
* `gateway` — the consolidated per-host gateway: a `Gateway`
lifecycle contract (idempotent singleton) + a
`DockerGateway` impl. One gateway shared by all
bottles instead of one per bottle.
* `service` — the `Orchestrator`: owns the registry, brokers the
launch lifecycle (launch/teardown), manages the
shared gateway, attributes.
* `control_plane` — the HTTP control-plane RPC (launch / teardown /
list / attribute / gateway / health).
The actual backend-native launch (a real docker/firecracker broker) and
the egress/git/supervise data plane land in later slices once this core is
proven (see the PRD sequencing: plain-process dev-harness -> docker
orchestrator -> firecracker).
"""
from __future__ import annotations
from .registry import BottleRecord, RegistryStore, new_identity_token
from .broker import (
BrokerAuthError,
LaunchBroker,
LaunchRequest,
StubBroker,
sign_request,
verify_request,
)
from .docker_broker import DockerBroker, DockerBrokerError
from .gateway import DockerGateway, Gateway, GatewayError
from .service import Orchestrator
from .control_plane import ControlPlaneServer, dispatch, make_server
__all__ = [
"BottleRecord",
"RegistryStore",
"new_identity_token",
"BrokerAuthError",
"LaunchBroker",
"LaunchRequest",
"StubBroker",
"DockerBroker",
"DockerBrokerError",
"Gateway",
"DockerGateway",
"GatewayError",
"sign_request",
"verify_request",
"Orchestrator",
"ControlPlaneServer",
"dispatch",
"make_server",
]
+79
View File
@@ -0,0 +1,79 @@
"""Run the orchestrator control plane as a plain process (PRD 0070 dev-harness).
python -m bot_bottle.orchestrator [--host H] [--port P] [--db PATH]
The PRD sequences the orchestrator as a plain-process dev-harness first, so
the consolidation core (registry + attribution + HTTP control plane + live
reload) can be exercised with fast iteration, decoupled from any VM /
container packaging. Wrapping this exact service in a backend-native unit
(docker container, then Firecracker VM) comes later.
"""
from __future__ import annotations
import argparse
import secrets
from pathlib import Path
from .. import log
from .broker import LaunchBroker, StubBroker
from .control_plane import make_server
from .docker_broker import DockerBroker
from .registry import RegistryStore, default_db_path
from .service import Orchestrator
from .gateway import DockerGateway, Gateway
def main(argv: list[str] | None = None) -> int:
"""Parse args, migrate the registry, and serve the control plane."""
parser = argparse.ArgumentParser(prog="bot_bottle.orchestrator")
parser.add_argument("--host", default="127.0.0.1", help="bind address")
parser.add_argument("--port", type=int, default=8080, help="bind port (0 = ephemeral)")
parser.add_argument(
"--db", type=Path, default=None,
help=f"registry DB path (default: {default_db_path()})",
)
parser.add_argument(
"--broker", choices=("stub", "docker"), default="stub",
help="launch broker: 'stub' records requests; 'docker' runs containers",
)
parser.add_argument(
"--gateway", action="store_true",
help="run one consolidated per-host sidecar bundle (build-if-missing)",
)
args = parser.parse_args(argv)
registry = RegistryStore(args.db)
registry.migrate()
# An ephemeral signing secret ties the orchestrator (signer) to its
# broker (verifier). 'stub' records launches instead of starting
# anything; 'docker' runs real containers (firecracker drops in later).
secret = secrets.token_bytes(32)
broker: LaunchBroker = DockerBroker(secret) if args.broker == "docker" else StubBroker(secret)
gateway: Gateway | None = DockerGateway() if args.gateway else None
orchestrator = Orchestrator(registry, broker, secret, gateway)
# One persistent per-host gateway, shared by every bottle: build the
# bundle image if missing, then bring the singleton up (idempotent).
if gateway is not None:
orchestrator.ensure_gateway()
log.info("consolidated gateway ensured", context={"name": gateway.name})
server = make_server(orchestrator, host=args.host, port=args.port)
bound_host, bound_port = server.server_address[0], server.server_address[1]
log.info(
"orchestrator control plane listening",
context={"host": bound_host, "port": bound_port, "db": str(registry.db_path)},
)
try:
server.serve_forever()
except KeyboardInterrupt:
log.info("orchestrator shutting down")
finally:
server.server_close()
return 0
if __name__ == "__main__":
raise SystemExit(main())
+176
View File
@@ -0,0 +1,176 @@
"""Launch broker contract (PRD 0070).
A VM/container can't spawn its own host-network siblings, so the
orchestrator brokers agent launches through a small privileged component.
The request it sends is:
* **structured** — static flags + ids only (bottle id, pool slot, a
content-addressed image ref), never a free-form path or argv, so a
request can't be coerced into launching an arbitrary payload; and
* **signed** — wrapped as a compact JWS/JWT the broker verifies before
acting, so a *compromised co-located component* (an agent-facing
gateway, say) can't forge a launch. This is the concrete form of the
"structured requests only" rule in the PRD security review.
Signing is **HS256** over a secret shared by the orchestrator (signer) and
the broker (verifier) — appropriate here because both are trusted host
components provisioned together; the secret is exactly what an
agent-facing component does not have. (Stdlib only — the project takes no
runtime deps; a cross-host deployment could swap in an asymmetric alg.)
"""
from __future__ import annotations
import abc
import base64
import hashlib
import hmac
import json
import secrets
import time
from dataclasses import dataclass
_JWT_HEADER = {"alg": "HS256", "typ": "JWT"}
_ALLOWED_OPS = ("launch", "teardown")
class BrokerAuthError(Exception):
"""A broker request failed provenance or schema verification —
bad/absent signature, malformed token, or a payload that doesn't match
the fixed launch-request shape. Fail-closed: the broker must not act."""
@dataclass(frozen=True)
class LaunchRequest:
"""The structured, un-coercible launch/teardown request. Ids + flags
only — `image_ref` is a content-addressed id, never a path/argv."""
op: str # one of _ALLOWED_OPS
bottle_id: str
source_ip: str = ""
image_ref: str = ""
slot: int | None = None
# --- minimal JWS/JWT (HS256), stdlib only ----------------------------------
def _b64url(data: bytes) -> str:
return base64.urlsafe_b64encode(data).rstrip(b"=").decode("ascii")
def _b64url_decode(text: str) -> bytes:
return base64.urlsafe_b64decode(text + "=" * (-len(text) % 4))
def _mac(signing_input: str, secret: bytes) -> str:
digest = hmac.new(secret, signing_input.encode("ascii"), hashlib.sha256).digest()
return _b64url(digest)
def sign_request(req: LaunchRequest, secret: bytes) -> str:
"""Serialize `req` to signed-JWT form. Adds a per-signature `jti`
(replay id) and `iat` (issued-at)."""
claims: dict[str, object] = {
"op": req.op,
"bottle_id": req.bottle_id,
"source_ip": req.source_ip,
"image_ref": req.image_ref,
"slot": req.slot,
"jti": secrets.token_hex(8),
"iat": int(time.time()),
}
header = _b64url(json.dumps(_JWT_HEADER, sort_keys=True, separators=(",", ":")).encode())
payload = _b64url(json.dumps(claims, sort_keys=True, separators=(",", ":")).encode())
signing_input = f"{header}.{payload}"
return f"{signing_input}.{_mac(signing_input, secret)}"
def verify_request(token: str, secret: bytes) -> LaunchRequest:
"""Verify the signature and the request shape; return the parsed
request. Raises `BrokerAuthError` on any failure (fail-closed)."""
parts = token.split(".")
if len(parts) != 3:
raise BrokerAuthError("malformed token")
header_b, payload_b, sig = parts
if not hmac.compare_digest(_mac(f"{header_b}.{payload_b}", secret), sig):
raise BrokerAuthError("bad signature")
try:
header = json.loads(_b64url_decode(header_b))
claims = json.loads(_b64url_decode(payload_b))
except (ValueError, json.JSONDecodeError) as e:
raise BrokerAuthError("malformed token payload") from e
if not isinstance(header, dict) or header.get("alg") != "HS256":
raise BrokerAuthError("unexpected header/alg")
if not isinstance(claims, dict):
raise BrokerAuthError("claims are not an object")
op = claims.get("op")
bottle_id = claims.get("bottle_id")
source_ip = claims.get("source_ip", "")
image_ref = claims.get("image_ref", "")
slot = claims.get("slot")
if (
not isinstance(op, str) or op not in _ALLOWED_OPS
or not isinstance(bottle_id, str) or not bottle_id
or not isinstance(source_ip, str) or not isinstance(image_ref, str)
or not (slot is None or isinstance(slot, int))
):
raise BrokerAuthError("request does not match the launch-request schema")
return LaunchRequest(
op=op, bottle_id=bottle_id, source_ip=source_ip, image_ref=image_ref, slot=slot
)
# --- the broker itself ------------------------------------------------------
class LaunchBroker(abc.ABC):
"""Verifies a signed request came from the orchestrator, then performs
the backend-native launch/teardown. Subclasses implement `_launch` /
`_teardown`; verification is shared and fail-closed."""
def __init__(self, secret: bytes) -> None:
self._secret = secret
def submit(self, token: str) -> LaunchRequest:
"""Verify `token` and perform its op. Returns the verified request;
raises `BrokerAuthError` if provenance/shape fails."""
req = verify_request(token, self._secret)
if req.op == "launch":
self._launch(req)
else:
self._teardown(req)
return req
@abc.abstractmethod
def _launch(self, req: LaunchRequest) -> None:
...
@abc.abstractmethod
def _teardown(self, req: LaunchRequest) -> None:
...
class StubBroker(LaunchBroker):
"""Dev-harness broker: records verified requests without launching
anything. Exercises the full sign -> verify -> act contract in-process."""
def __init__(self, secret: bytes) -> None:
super().__init__(secret)
self.launched: list[LaunchRequest] = []
self.torn_down: list[LaunchRequest] = []
def _launch(self, req: LaunchRequest) -> None:
self.launched.append(req)
def _teardown(self, req: LaunchRequest) -> None:
self.torn_down.append(req)
__all__ = [
"BrokerAuthError",
"LaunchRequest",
"LaunchBroker",
"StubBroker",
"sign_request",
"verify_request",
]
+199
View File
@@ -0,0 +1,199 @@
"""Orchestrator HTTP control plane (PRD 0070).
The backend-agnostic control-plane RPC (CLI / console -> orchestrator) over
**HTTP** — the universal transport chosen in 0070 (works on every host; no
vsock / unix-socket portability caveats):
GET /health -> 200 {"status": "ok"}
GET /gateway -> 200 {"configured", ["name","running"]}
GET /bottles -> 200 {"bottles": [ <redacted record>, ...]}
POST /bottles -> 201 {"bottle_id","identity_token"} (launch)
body: {"source_ip", ["image_ref"],
["metadata"], ["policy"]}
PUT /bottles/<bottle_id>/policy -> 200 {"updated": true} | 404 (live reload)
body: {"policy"}
DELETE /bottles/<bottle_id> -> 200 {"torn_down": true} | 404 (teardown)
POST /attribute -> 200 {"bottle_id"} | 403
POST /resolve -> 200 {"bottle_id","policy"} | 403
body: {"source_ip","identity_token"}
`POST /bottles` / `DELETE` drive the full launch lifecycle: they mint (or
tear down) the bottle in the registry AND broker the backend-native launch
via the orchestrator. Register/deregister without a launch are internal to
`Orchestrator`, not exposed here.
Routing/handling is the pure function `dispatch()` so it is unit-testable
without a socket; `Handler` / `ControlPlaneServer` / `make_server` are a
thin stdlib adapter around it. Listing redacts identity tokens — they are
returned only once, to the caller that launches the bottle.
"""
from __future__ import annotations
import http.server
import json
import os
import socketserver
import typing
from urllib.parse import urlsplit
from .service import Orchestrator
# JSON body payload type (parsed request / rendered response).
Json = dict[str, object]
def _parse_json_object(body: bytes) -> Json:
"""Parse a JSON object body. Raises ValueError for non-objects / bad JSON."""
if not body:
return {}
obj = json.loads(body) # raises json.JSONDecodeError (a ValueError)
if not isinstance(obj, dict):
raise ValueError("request body must be a JSON object")
return obj
def dispatch( # pylint: disable=too-many-return-statements,too-many-branches
orch: Orchestrator, method: str, path: str, body: bytes
) -> tuple[int, Json]:
"""Route one control-plane request to a (status, payload) pair. Pure —
no I/O beyond the orchestrator — so it is fully testable without a socket."""
route = urlsplit(path).path.rstrip("/") or "/"
if method == "GET" and route == "/health":
return 200, {"status": "ok"}
if method == "GET" and route == "/gateway":
return 200, orch.gateway_status()
if method == "GET" and route == "/bottles":
return 200, {"bottles": [r.redacted() for r in orch.registry.all()]}
if method == "POST" and route == "/bottles":
try:
data = _parse_json_object(body)
except ValueError as e:
return 400, {"error": f"invalid JSON: {e}"}
source_ip = data.get("source_ip")
if not isinstance(source_ip, str) or not source_ip:
return 400, {"error": "source_ip (string) is required"}
image_ref = data.get("image_ref")
metadata = data.get("metadata")
policy = data.get("policy")
rec = orch.launch_bottle(
source_ip,
image_ref=image_ref if isinstance(image_ref, str) else "",
metadata=metadata if isinstance(metadata, str) else "",
policy=policy if isinstance(policy, str) else "",
)
return 201, {"bottle_id": rec.bottle_id, "identity_token": rec.identity_token}
if method == "PUT" and route.startswith("/bottles/") and route.endswith("/policy"):
bottle_id = route[len("/bottles/"):-len("/policy")]
try:
data = _parse_json_object(body)
except ValueError as e:
return 400, {"error": f"invalid JSON: {e}"}
policy = data.get("policy")
if not isinstance(policy, str):
return 400, {"error": "policy (string) is required"}
if orch.set_policy(bottle_id, policy):
return 200, {"updated": True}
return 404, {"error": "no such bottle"}
if method == "DELETE" and route.startswith("/bottles/"):
bottle_id = route[len("/bottles/"):]
if orch.teardown_bottle(bottle_id):
return 200, {"torn_down": True}
return 404, {"error": "no such bottle"}
if method == "POST" and route == "/attribute":
try:
data = _parse_json_object(body)
except ValueError as e:
return 400, {"error": f"invalid JSON: {e}"}
source_ip = data.get("source_ip")
token = data.get("identity_token")
if not isinstance(source_ip, str) or not isinstance(token, str):
return 400, {"error": "source_ip and identity_token (strings) required"}
rec = orch.attribute(source_ip, token)
if rec is None:
return 403, {"error": "unattributed"}
return 200, {"bottle_id": rec.bottle_id}
if method == "POST" and route == "/resolve":
# The per-request lookup the multi-tenant gateway makes: returns the
# bottle's policy. identity_token is OPTIONAL — absent means resolve
# by source IP alone (network-layer attribution).
try:
data = _parse_json_object(body)
except ValueError as e:
return 400, {"error": f"invalid JSON: {e}"}
source_ip = data.get("source_ip")
token = data.get("identity_token")
if not isinstance(source_ip, str) or not source_ip:
return 400, {"error": "source_ip (string) is required"}
rec = orch.resolve(source_ip, token if isinstance(token, str) else "")
if rec is None:
return 403, {"error": "unattributed"}
return 200, {"bottle_id": rec.bottle_id, "policy": rec.policy}
return 404, {"error": "not found"}
class Handler(http.server.BaseHTTPRequestHandler):
"""Thin stdlib adapter: read the body, call `dispatch`, write JSON."""
# Quiet by default (the orchestrator has its own logging); opt back into
# stdlib access logging with BOT_BOTTLE_ORCHESTRATOR_DEBUG.
def log_message(self, format: str, *args: typing.Any) -> None: # noqa: A002
if os.environ.get("BOT_BOTTLE_ORCHESTRATOR_DEBUG"):
super().log_message(format, *args)
def _serve(self, method: str) -> None:
"""Read the request body, dispatch it, and write the JSON reply."""
server = self.server
assert isinstance(server, ControlPlaneServer)
length = int(self.headers.get("Content-Length") or 0)
body = self.rfile.read(length) if length > 0 else b""
status, payload = dispatch(server.orchestrator, method, self.path, body)
data = json.dumps(payload).encode()
self.send_response(status)
self.send_header("Content-Type", "application/json")
self.send_header("Content-Length", str(len(data)))
self.end_headers()
self.wfile.write(data)
def do_GET(self) -> None:
self._serve("GET")
def do_POST(self) -> None:
self._serve("POST")
def do_PUT(self) -> None:
self._serve("PUT")
def do_DELETE(self) -> None:
self._serve("DELETE")
class ControlPlaneServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
"""Threading HTTP server that carries the orchestrator for its handlers."""
daemon_threads = True
allow_reuse_address = True
def __init__(self, address: tuple[str, int], orchestrator: Orchestrator) -> None:
self.orchestrator = orchestrator
super().__init__(address, Handler)
def make_server(
orchestrator: Orchestrator, host: str = "127.0.0.1", port: int = 0
) -> ControlPlaneServer:
"""Build (but do not start) a control-plane server. `port=0` binds an
ephemeral port — read `server.server_address` for the actual one."""
return ControlPlaneServer((host, port), orchestrator)
__all__ = ["dispatch", "Handler", "ControlPlaneServer", "make_server", "Json"]
+83
View File
@@ -0,0 +1,83 @@
"""Docker launch broker (PRD 0070) — the first *real* LaunchBroker.
On a verified launch request it starts a Docker container; on teardown it
removes it. This proves the orchestrator -> backend seam on the cheapest
backend (the sidecar bundle is already containers). Only the request's
static ids/flags reach `docker`, so nothing free-form crosses the boundary.
Slice 3 launches a single container from the request's `image_ref`, named
after the bottle id and labelled for cleanup. Wiring the full agent +
sidecar bundle (networks, mounts, the consolidated sidecar) is a later
slice — this is the seam, not the finished launcher.
"""
from __future__ import annotations
import subprocess
from ..docker_cmd import run_docker
from .broker import LaunchBroker, LaunchRequest
CONTAINER_PREFIX = "bot-bottle-orch-"
BOTTLE_ID_LABEL = "bot-bottle-bottle-id"
class DockerBrokerError(Exception):
"""A brokered docker launch/teardown failed (non-zero `docker` exit)."""
def container_name(bottle_id: str) -> str:
"""Deterministic container name for a bottle."""
return f"{CONTAINER_PREFIX}{bottle_id}"
def run_argv(req: LaunchRequest) -> list[str]:
"""`docker run` argv for a launch request — built only from its static
fields, so it can't be coerced into an arbitrary command."""
return [
"docker", "run", "--detach",
"--name", container_name(req.bottle_id),
"--label", f"{BOTTLE_ID_LABEL}={req.bottle_id}",
req.image_ref,
]
def rm_argv(req: LaunchRequest) -> list[str]:
"""`docker rm --force` argv to tear a bottle's container down."""
return ["docker", "rm", "--force", container_name(req.bottle_id)]
class DockerBroker(LaunchBroker):
"""A `LaunchBroker` that runs / removes a Docker container per bottle.
Provenance + schema verification are inherited from `LaunchBroker`."""
def _docker(self, argv: list[str]) -> subprocess.CompletedProcess[str]:
return run_docker(argv)
def _launch(self, req: LaunchRequest) -> None:
if not req.image_ref:
raise DockerBrokerError(f"launch request for {req.bottle_id} has no image_ref")
proc = self._docker(run_argv(req))
if proc.returncode != 0:
raise DockerBrokerError(
f"docker run failed for {req.bottle_id}: {proc.stderr.strip()}"
)
def _teardown(self, req: LaunchRequest) -> None:
proc = self._docker(rm_argv(req))
# Idempotent: an already-absent container is a successful teardown.
if proc.returncode != 0 and "No such container" not in proc.stderr:
raise DockerBrokerError(
f"docker rm failed for {req.bottle_id}: {proc.stderr.strip()}"
)
__all__ = [
"DockerBroker",
"DockerBrokerError",
"container_name",
"run_argv",
"rm_argv",
"CONTAINER_PREFIX",
"BOTTLE_ID_LABEL",
]
+139
View File
@@ -0,0 +1,139 @@
"""The consolidated per-host gateway (PRD 0070).
The core consolidation win: **one** persistent gateway per host, shared by
every bottle, instead of a sidecar bundle per bottle. It's safe to share
because the attribution invariant (source IP + identity token, see
`registry`) lets the gateway attribute each request to the right bottle —
so per-bottle policy lives in one long-lived process keyed on who's calling.
`Gateway` is the backend-neutral lifecycle contract (mirrors `LaunchBroker`):
ensure the single instance is up, report it, tear it down. `DockerGateway`
is the docker implementation; a firecracker gateway VM slots in later.
The defining behaviour is **idempotent singleton**: `ensure_running` starts
the instance if absent and is a no-op if it's already up, so N bottle
launches never spawn N gateways.
"""
from __future__ import annotations
import abc
import os
from pathlib import Path
from ..docker_cmd import run_docker
GATEWAY_NAME = "bot-bottle-orch-gateway"
GATEWAY_LABEL = "bot-bottle-orch-gateway=1"
# The real sidecar-bundle image + its Dockerfile. Kept as a local constant
# rather than imported from backend.docker.sidecar_bundle, which would drag
# the whole backend layer into the lean orchestrator (see #359); unify when
# that lands. Env override matches the backend's BOT_BOTTLE_SIDECAR_IMAGE.
GATEWAY_IMAGE = os.environ.get("BOT_BOTTLE_SIDECAR_IMAGE", "bot-bottle-sidecars:latest")
GATEWAY_DOCKERFILE = "Dockerfile.sidecars"
_REPO_ROOT = Path(__file__).resolve().parents[2]
class GatewayError(Exception):
"""The shared gateway failed to build/start/stop (non-zero `docker` exit)."""
class Gateway(abc.ABC):
"""Lifecycle of the single per-host gateway. Backend-neutral."""
name: str
def ensure_built(self) -> None:
"""Ensure the gateway's image / rootfs exists, building it if needed.
Default: nothing to build (e.g. a stub or a pre-pulled image)."""
return
@abc.abstractmethod
def ensure_running(self) -> None:
"""Start the gateway if it isn't already up. Idempotent: a no-op
when it's already running (that's the whole point — one per host).
Assumes the image exists — call `ensure_built()` first."""
@abc.abstractmethod
def is_running(self) -> bool:
"""True iff the gateway instance is currently up."""
@abc.abstractmethod
def stop(self) -> None:
"""Remove the gateway. Idempotent — absent is success."""
class DockerGateway(Gateway):
"""The consolidated gateway as a single, fixed-name Docker container.
`image_ref` defaults to the real sidecar-bundle image; `ensure_built`
builds it from `Dockerfile.sidecars` when it's missing. (Note: slice 5
builds + launches the bundle container; wiring its per-bottle,
source-IP-keyed config is a later slice — see PRD 0070.)"""
def __init__(
self,
image_ref: str = GATEWAY_IMAGE,
*,
name: str = GATEWAY_NAME,
build_context: Path | None = None,
dockerfile: str | None = GATEWAY_DOCKERFILE,
) -> None:
self.image_ref = image_ref
self.name = name
self._build_context = build_context or _REPO_ROOT
self._dockerfile = dockerfile
def image_exists(self) -> bool:
return run_docker(["docker", "image", "inspect", self.image_ref]).returncode == 0
def ensure_built(self) -> None:
"""Build the bundle image from its Dockerfile when it's missing.
No-op when the image is already present, or when no dockerfile is
configured (e.g. a pre-pulled image)."""
if self._dockerfile is None or self.image_exists():
return
proc = run_docker([
"docker", "build",
"-t", self.image_ref,
"-f", str(self._build_context / self._dockerfile),
str(self._build_context),
])
if proc.returncode != 0:
raise GatewayError(f"gateway image build failed: {proc.stderr.strip()}")
def is_running(self) -> bool:
proc = run_docker([
"docker", "ps",
"--filter", f"name=^{self.name}$",
"--filter", "status=running",
"--format", "{{.Names}}",
])
return self.name in proc.stdout.split()
def ensure_running(self) -> None:
if self.is_running():
return
# Clear any stale (stopped) container holding the fixed name, then
# start fresh. `rm --force` on an absent name is a tolerated no-op.
run_docker(["docker", "rm", "--force", self.name])
proc = run_docker([
"docker", "run", "--detach",
"--name", self.name,
"--label", GATEWAY_LABEL,
self.image_ref,
])
if proc.returncode != 0:
raise GatewayError(f"gateway failed to start: {proc.stderr.strip()}")
def stop(self) -> None:
proc = run_docker(["docker", "rm", "--force", self.name])
if proc.returncode != 0 and "No such container" not in proc.stderr:
raise GatewayError(f"gateway failed to stop: {proc.stderr.strip()}")
__all__ = [
"Gateway", "DockerGateway", "GatewayError",
"GATEWAY_NAME", "GATEWAY_LABEL", "GATEWAY_IMAGE",
]
+252
View File
@@ -0,0 +1,252 @@
"""Orchestrator bottle registry — the per-host runtime-state store (PRD 0070).
SQLite-backed registry mapping each live bottle to its source IP and
per-bottle identity token, plus the **fail-closed attribution** the data
plane relies on: a request is attributed to a bottle only when its source
IP *and* its identity token both match a single active record.
The registry co-tenants the shared host `bot-bottle.db` (the `DbStore`
framework namespaces each store by `schema_key`), so all bot-bottle
runtime state lives in one queryable file — one place to back up, inspect,
and integrate a console against.
This is the "runtime state" tier of PRD 0070 (leases / approvals /
registry) — deliberately NOT config (which stays declarative under
`~/.bot-bottle/`) and NOT the build-time constants (a flat file). It is the
source of truth the orchestrator sweeps on restart to re-adopt running
bottles, so it lives in a durable store rather than process memory.
Attribution combines two independent signals, and needs both:
* source IP — the network-layer invariant (on Firecracker the `/31` TAP
+ nft make it unspoofable by construction; weaker on other backends).
* identity token — an application-layer per-bottle secret, defence in
depth that doesn't lean on network anti-spoof. A bottle can only ever
prove it is *itself* (it can't learn another bottle's token), so a
hostile agent gains nothing by presenting it.
"""
from __future__ import annotations
import hmac
import secrets
import sqlite3
import time
from dataclasses import dataclass
from pathlib import Path
from ..db_store import DbStore
from ..migrations import TableMigrations
from ..paths import host_db_path
# 256 bits of urandom, URL-safe — unguessable per-bottle identity token.
IDENTITY_TOKEN_BYTES = 32
def new_identity_token() -> str:
"""A fresh per-bottle identity token (PRD 0070 attribution defence)."""
return secrets.token_urlsafe(IDENTITY_TOKEN_BYTES)
def default_db_path() -> Path:
"""The shared host state DB (`bot-bottle.db`) — all bot-bottle runtime
state in one file, co-tenanted via schema_key. Host-resident so it
survives orchestrator restarts (re-adoption sweeps it)."""
return host_db_path()
@dataclass(frozen=True)
class BottleRecord:
"""One live bottle in the registry."""
bottle_id: str
source_ip: str
identity_token: str
state: str = "active"
created_at: float = 0.0
# Opaque JSON blob for forward-compat (pool slot, ...) — the registry
# doesn't interpret it, so new fields need no migration.
metadata: str = ""
# The bottle's gateway policy (opaque JSON — egress allowlist / routes /
# git config). The registry stores and serves it verbatim, keyed by
# source IP; the multi-tenant gateway interprets it.
policy: str = ""
def redacted(self) -> dict[str, object]:
"""Public view for listing — never exposes the identity token."""
return {
"bottle_id": self.bottle_id,
"source_ip": self.source_ip,
"state": self.state,
"created_at": self.created_at,
"metadata": self.metadata,
"policy": self.policy,
}
_MIGRATIONS = TableMigrations(
"orchestrator_registry",
[
# v1 — the live bottle registry.
"""
CREATE TABLE IF NOT EXISTS orchestrator_bottles (
bottle_id TEXT PRIMARY KEY,
source_ip TEXT NOT NULL,
identity_token TEXT NOT NULL,
state TEXT NOT NULL DEFAULT 'active',
created_at REAL NOT NULL,
metadata TEXT NOT NULL DEFAULT ''
)
""",
# source_ip is the data-plane attribution key — index it, and make
# the "one active bottle per source IP" check cheap.
"CREATE INDEX IF NOT EXISTS idx_orchestrator_bottles_source_ip "
"ON orchestrator_bottles (source_ip)",
# v3 — per-bottle policy (opaque JSON the gateway interprets): the
# egress allowlist / routes / git config selected by source IP. The
# multi-tenant gateway resolves it per request via `attribute`.
"ALTER TABLE orchestrator_bottles ADD COLUMN policy TEXT NOT NULL DEFAULT ''",
],
)
def _row_to_record(row: sqlite3.Row) -> BottleRecord:
"""Build a BottleRecord from a registry row."""
return BottleRecord(
bottle_id=row["bottle_id"],
source_ip=row["source_ip"],
identity_token=row["identity_token"],
state=row["state"],
created_at=row["created_at"],
metadata=row["metadata"],
policy=row["policy"],
)
class RegistryStore(DbStore):
"""SQLite-backed registry of live bottles + fail-closed attribution."""
def __init__(self, db_path: Path | None = None) -> None:
super().__init__(db_path or default_db_path(), _MIGRATIONS)
def _connect(self) -> sqlite3.Connection:
"""Open a connection with a busy timeout for the shared DB."""
conn = super()._connect()
# The registry co-tenants the shared bot-bottle.db, so a busy_timeout
# rides out brief lock contention with the other stores. WAL for the
# shared DB is a deliberate future change (it affects supervise/audit
# and is finicky over guest shares) — not flipped here.
conn.execute("PRAGMA busy_timeout=5000")
return conn
def register(
self,
source_ip: str,
*,
bottle_id: str | None = None,
identity_token: str | None = None,
metadata: str = "",
policy: str = "",
) -> BottleRecord:
"""Register (or replace) a bottle. Mints a bottle_id / identity token
when not supplied. Live — this is the control-plane reload path."""
rec = BottleRecord(
bottle_id=bottle_id or secrets.token_hex(8),
source_ip=source_ip,
identity_token=identity_token or new_identity_token(),
state="active",
created_at=time.time(),
metadata=metadata,
policy=policy,
)
with self._connect() as conn:
conn.execute(
"INSERT OR REPLACE INTO orchestrator_bottles "
"(bottle_id, source_ip, identity_token, state, created_at, metadata, policy) "
"VALUES (?, ?, ?, ?, ?, ?, ?)",
(
rec.bottle_id,
rec.source_ip,
rec.identity_token,
rec.state,
rec.created_at,
rec.metadata,
rec.policy,
),
)
self._chmod()
return rec
def set_policy(self, bottle_id: str, policy: str) -> bool:
"""Update a bottle's policy in place (live reload). Returns True if
the bottle exists."""
with self._connect() as conn:
cur = conn.execute(
"UPDATE orchestrator_bottles SET policy = ? WHERE bottle_id = ?",
(policy, bottle_id),
)
self._chmod()
return cur.rowcount > 0
def deregister(self, bottle_id: str) -> bool:
"""Remove a bottle. Returns True if a row was deleted."""
with self._connect() as conn:
cur = conn.execute(
"DELETE FROM orchestrator_bottles WHERE bottle_id = ?", (bottle_id,)
)
return cur.rowcount > 0
def get(self, bottle_id: str) -> BottleRecord | None:
"""Return the bottle by id, or None if absent."""
with self._connect() as conn:
row = conn.execute(
"SELECT * FROM orchestrator_bottles WHERE bottle_id = ?", (bottle_id,)
).fetchone()
return _row_to_record(row) if row else None
def all(self) -> list[BottleRecord]:
"""Every registered bottle, oldest first."""
with self._connect() as conn:
rows = conn.execute(
"SELECT * FROM orchestrator_bottles ORDER BY created_at"
).fetchall()
return [_row_to_record(r) for r in rows]
def by_source_ip(self, source_ip: str) -> BottleRecord | None:
"""Network-layer attribution: the single active bottle at this source
IP, or None if unknown or ambiguous (more than one — a
misconfiguration). Safe as the *sole* attributor only where the
source IP is unspoofable (Firecracker `/31` + nft) and the control
plane is reachable only by the trusted gateway; pair with the
identity token (`attribute`) elsewhere."""
with self._connect() as conn:
rows = conn.execute(
"SELECT * FROM orchestrator_bottles "
"WHERE source_ip = ? AND state = 'active'",
(source_ip,),
).fetchall()
if len(rows) != 1:
return None
return _row_to_record(rows[0])
def attribute(self, source_ip: str, identity_token: str) -> BottleRecord | None:
"""Fail-closed attribution: `by_source_ip` AND a matching identity
token (constant-time). Either signal alone is insufficient here — an
unknown/ambiguous IP, an empty token, or a token mismatch all deny."""
if not identity_token:
return None
rec = self.by_source_ip(source_ip)
if rec is None:
return None
if not hmac.compare_digest(rec.identity_token, identity_token):
return None
return rec
__all__ = [
"BottleRecord",
"RegistryStore",
"new_identity_token",
"default_db_path",
"IDENTITY_TOKEN_BYTES",
]
+121
View File
@@ -0,0 +1,121 @@
"""The per-host orchestrator core (PRD 0070).
`Orchestrator` is the single backend-neutral object the control plane talks
to: it owns the registry (runtime state) and brokers agent launches. It
never branches on backend — the `LaunchBroker` abstracts the backend-native
launch, so this same object drives docker / firecracker / apple once a real
broker is wired in.
Launch lifecycle:
* `launch_bottle` mints the bottle (registry: source IP + identity
token), sends a *signed, structured* launch request through the broker,
and returns the record. If the broker rejects/fails, the registry entry
is rolled back so a failed launch leaves no orphan.
* `teardown_bottle` sends a signed teardown request, then deregisters.
"""
from __future__ import annotations
from .broker import LaunchBroker, LaunchRequest, sign_request
from .registry import BottleRecord, RegistryStore
from .gateway import Gateway
class Orchestrator:
"""Owns the registry + brokers launches, and manages the single
consolidated per-host gateway. Backend-neutral (broker and gateway
abstract the backend-native pieces)."""
def __init__(
self,
registry: RegistryStore,
broker: LaunchBroker,
sign_secret: bytes,
gateway: Gateway | None = None,
) -> None:
self.registry = registry
self._broker = broker
self._secret = sign_secret
self._gateway = gateway
def launch_bottle(
self,
source_ip: str,
*,
image_ref: str = "",
slot: int | None = None,
metadata: str = "",
policy: str = "",
) -> BottleRecord:
"""Register a bottle (with its gateway policy) and broker its launch.
Rolls the registry entry back if the launch doesn't take, so a
failure leaves no orphan."""
rec = self.registry.register(source_ip, metadata=metadata, policy=policy)
req = LaunchRequest(
op="launch",
bottle_id=rec.bottle_id,
source_ip=source_ip,
image_ref=image_ref,
slot=slot,
)
launched = False
try:
self._broker.submit(sign_request(req, self._secret))
launched = True
finally:
if not launched:
self.registry.deregister(rec.bottle_id)
return rec
def teardown_bottle(self, bottle_id: str) -> bool:
"""Broker teardown then deregister. False if the bottle is unknown."""
rec = self.registry.get(bottle_id)
if rec is None:
return False
req = LaunchRequest(op="teardown", bottle_id=bottle_id, source_ip=rec.source_ip)
self._broker.submit(sign_request(req, self._secret))
self.registry.deregister(bottle_id)
return True
def attribute(self, source_ip: str, identity_token: str) -> BottleRecord | None:
"""Fail-closed attribution (delegates to the registry)."""
return self.registry.attribute(source_ip, identity_token)
def resolve(self, source_ip: str, identity_token: str = "") -> BottleRecord | None:
"""Resolve the bottle behind a request — the source-IP-keyed lookup
the multi-tenant gateway makes per request; the returned record
carries its `policy`. With a token, full attribution (source IP +
token); without, network-layer attribution by source IP alone
(valid where the IP is unspoofable and the control plane is
gateway-only)."""
if identity_token:
return self.registry.attribute(source_ip, identity_token)
return self.registry.by_source_ip(source_ip)
def set_policy(self, bottle_id: str, policy: str) -> bool:
"""Update a bottle's gateway policy in place (live reload). False if
the bottle is unknown."""
return self.registry.set_policy(bottle_id, policy)
# --- consolidated gateway ----------------------------------------------
def ensure_gateway(self) -> None:
"""Ensure the single per-host gateway is built and up (idempotent).
No-op when no gateway is configured."""
if self._gateway is not None:
self._gateway.ensure_built()
self._gateway.ensure_running()
def gateway_status(self) -> dict[str, object]:
"""Report the shared gateway for the control plane / console."""
if self._gateway is None:
return {"configured": False}
return {
"configured": True,
"name": self._gateway.name,
"running": self._gateway.is_running(),
}
__all__ = ["Orchestrator"]
+43
View File
@@ -0,0 +1,43 @@
"""Foundational filesystem paths for bot-bottle.
`bot_bottle_root()` is the app data root — state, queue, audit logs,
git-gate keys, and the shared DB all live under it. It defaults to
`~/.bot-bottle` and is overridable with the **`BOT_BOTTLE_ROOT`** env var.
The env override is the single knob for redirecting the root: the test
suite points it at a throwaway dir instead of monkey-patching the function
(every module and every flat/package copy reads the same env var, so one
override covers them all), and operators can relocate the root if needed.
This module has no bot-bottle imports, so it is safe to import from any
layer (and to COPY flat into the sidecar bundle).
"""
from __future__ import annotations
import os
from pathlib import Path
# The single shared host state DB. All bot-bottle SQLite stores (supervise
# queue, audit, the orchestrator registry) co-tenant this one file — the
# TableMigrations schema_key namespaces each store's tables.
HOST_DB_FILENAME = "bot-bottle.db"
def bot_bottle_root() -> Path:
"""The app data root — `$BOT_BOTTLE_ROOT` if set, else `~/.bot-bottle`."""
override = os.environ.get("BOT_BOTTLE_ROOT")
return Path(override) if override else Path.home() / ".bot-bottle"
def host_db_path() -> Path:
"""Path to the shared host state DB, `<root>/db/bot-bottle.db`.
Kept in its own `db/` subdirectory (not directly under the root) so a
backend that can only bind-mount *directories* can share this one file
with a sidecar without exposing the root's other contents (git-gate
keys, per-bottle state, ...)."""
return bot_bottle_root() / "db" / HOST_DB_FILENAME
__all__ = ["HOST_DB_FILENAME", "bot_bottle_root", "host_db_path"]
+76
View File
@@ -0,0 +1,76 @@
"""Gateway-side per-client policy resolver (PRD 0070).
The consolidated gateway serves every bottle from one process, so for each
request it must apply the *calling* bottle's policy, selected by the source
IP the attribution invariant makes unspoofable. This resolves that policy
from the orchestrator's control plane (`POST /resolve`), keyed on the
`(source_ip, identity_token)` pair.
**Always fresh — no cache.** The resolver is called rarely enough that a
round-trip doesn't matter, and correctness matters more than speed: every
resolve reflects the orchestrator's *current* view, so a revocation, a
policy change, or a bottle teardown the orchestrator knows about is honored
immediately rather than lingering for a cache TTL. (If this ever becomes a
hot path, add caching with orchestrator-driven invalidation — not a blind
TTL.)
**Fail-closed:** an unattributed client (the orchestrator answers `403`)
resolves to `None`, and the caller (the egress addon, git-gate) must then
deny — exactly as an unknown bottle should be treated. Orchestrator
*errors* (unreachable / unexpected status) raise, so the caller can fail
closed too rather than silently serving stale or empty policy.
The resolved value is the policy blob the orchestrator stores verbatim; the
consumer parses it (e.g. the egress addon's `load_config`). This module is
stdlib-only and free of bot-bottle imports so it can be COPYed flat into
the sidecar bundle.
"""
from __future__ import annotations
import json
import urllib.error
import urllib.request
DEFAULT_TIMEOUT_SECONDS = 2.0
class PolicyResolveError(RuntimeError):
"""The orchestrator was unreachable or returned an unexpected status —
distinct from a clean `403` (unattributed), which returns None."""
class PolicyResolver:
"""Resolves each client's policy from the orchestrator, fresh per call."""
def __init__(self, base_url: str, *, timeout: float = DEFAULT_TIMEOUT_SECONDS) -> None:
self._base = base_url.rstrip("/")
self._timeout = timeout
def resolve(self, source_ip: str, identity_token: str = "") -> str | None:
"""The calling bottle's policy blob, or None if unattributed. Always
fetches from the orchestrator so revocations / changes / teardowns
are honored immediately. `identity_token` is optional — omit it to
resolve by source IP alone (the network-layer attribution).
Raises `PolicyResolveError` if the orchestrator can't be reached."""
body = json.dumps(
{"source_ip": source_ip, "identity_token": identity_token}
).encode()
req = urllib.request.Request(
f"{self._base}/resolve", data=body, method="POST",
headers={"Content-Type": "application/json"},
)
try:
with urllib.request.urlopen(req, timeout=self._timeout) as resp:
payload = json.loads(resp.read())
except urllib.error.HTTPError as e:
if e.code == 403:
return None # unattributed → fail closed (caller denies)
raise PolicyResolveError(f"/resolve returned HTTP {e.code}") from e
except (urllib.error.URLError, TimeoutError, OSError, ValueError) as e:
raise PolicyResolveError(f"/resolve unreachable or malformed: {e}") from e
policy = payload.get("policy") if isinstance(payload, dict) else None
return policy if isinstance(policy, str) else ""
__all__ = ["PolicyResolver", "PolicyResolveError"]
+4 -2
View File
@@ -7,11 +7,13 @@ import sqlite3
from pathlib import Path
try:
from .supervise_types import Proposal, Response, host_db_path
from .supervise_types import Proposal, Response
from .paths import host_db_path
from .db_store import DbStore
from .migrations import TableMigrations
except ImportError:
from supervise_types import Proposal, Response, host_db_path # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from supervise_types import Proposal, Response # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from paths import host_db_path # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from db_store import DbStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from migrations import TableMigrations # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
+2 -5
View File
@@ -23,13 +23,10 @@ class StoreManager:
def __init__(self, db_path: Path | None = None) -> None:
if db_path is None:
# Lazy import to avoid a circular dependency: supervise imports
# StoreManager at module level; StoreManager must not import
# supervise at module level in return.
try:
from .supervise import host_db_path
from .paths import host_db_path
except ImportError:
from supervise import host_db_path # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from paths import host_db_path # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
db_path = host_db_path()
self.db_path = db_path
+6 -16
View File
@@ -41,7 +41,6 @@ try:
from .supervise_types import (
ACTION_OPERATOR_EDIT,
AuditEntry,
HOST_DB_FILENAME,
Proposal,
Response,
STATUSES,
@@ -54,13 +53,11 @@ try:
TOOL_EGRESS_TOKEN_ALLOW,
TOOL_GITLEAKS_ALLOW,
TOOL_LIST_EGRESS_ROUTES,
bot_bottle_root,
)
except ImportError:
from supervise_types import ( # type: ignore[import-not-found,no-redef] # pylint: disable=import-error,no-name-in-module
ACTION_OPERATOR_EDIT,
AuditEntry,
HOST_DB_FILENAME,
Proposal,
Response,
STATUSES,
@@ -73,10 +70,15 @@ except ImportError:
TOOL_EGRESS_TOKEN_ALLOW,
TOOL_GITLEAKS_ALLOW,
TOOL_LIST_EGRESS_ROUTES,
bot_bottle_root,
)
try:
from .paths import bot_bottle_root
except ImportError: # flat imports inside the sidecar bundle
from paths import bot_bottle_root # type: ignore[import-not-found,no-redef] # pylint: disable=import-error,no-name-in-module
SUPERVISE_HOSTNAME = "supervise"
SUPERVISE_PORT = 9100
@@ -106,16 +108,6 @@ def audit_dir() -> Path:
return bot_bottle_root() / "audit"
def host_db_path() -> Path:
# Calls bot_bottle_root() through this module's globals so that patches
# on supervise.bot_bottle_root propagate to callers going through
# supervise.host_db_path().
#
# Kept in its own "db" subdirectory (see supervise_types.host_db_path
# for why) — must stay in sync with that copy.
return bot_bottle_root() / "db" / HOST_DB_FILENAME
def audit_log_path(component: str, slug: str) -> Path:
return audit_dir() / f"{component}-{slug}.log"
@@ -297,8 +289,6 @@ __all__ = [
"archive_proposal",
"audit_dir",
"audit_log_path",
"bot_bottle_root",
"host_db_path",
"list_pending_proposals",
"list_all_pending_proposals",
"read_audit_entries",
+5 -32
View File
@@ -1,26 +1,19 @@
"""Shared types and path helpers for supervise stores (PRD 0013).
"""Shared types for supervise stores (PRD 0013).
Extracted from supervise.py so queue_store and audit_store can import
Proposal, Response, AuditEntry, and host_db_path without creating a
circular import (supervise imports from queue_store/audit_store and
vice-versa).
Proposal, Response, and AuditEntry without creating a circular import
(supervise imports from queue_store/audit_store and vice-versa).
Patching bot_bottle_root on this module propagates through host_db_path
because host_db_path looks up bot_bottle_root via sys.modules[__name__]
at call time rather than capturing it at import time.
The path helpers (`bot_bottle_root`, `host_db_path`, `HOST_DB_FILENAME`)
live in `paths` — import them from there.
"""
from __future__ import annotations
import dataclasses
import sys
import uuid
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
HOST_DB_FILENAME = "bot-bottle.db"
TOOL_EGRESS_BLOCK = "egress-block"
TOOL_EGRESS_ALLOW = "egress-allow"
@@ -43,23 +36,6 @@ STATUSES: tuple[str, ...] = (STATUS_APPROVED, STATUS_MODIFIED, STATUS_REJECTED)
ACTION_OPERATOR_EDIT = "operator-edit"
def bot_bottle_root() -> Path:
return Path.home() / ".bot-bottle"
def host_db_path() -> Path:
# Look up bot_bottle_root through this module's live namespace so that
# monkey-patches on supervise_types.bot_bottle_root take effect.
#
# Lives in its own "db" subdirectory, not directly under
# bot_bottle_root(), so backends that can only bind-mount
# directories (macos_container's `container run --mount`, which
# rejects file sources with "is not a directory") can share this
# one file with a sidecar without also exposing bot_bottle_root()'s
# other contents (git-gate keys, per-bottle state, etc.).
return sys.modules[__name__].bot_bottle_root() / "db" / HOST_DB_FILENAME
def _require_str(raw: dict[str, object], key: str) -> str:
value = raw.get(key)
if not isinstance(value, str):
@@ -171,7 +147,6 @@ class AuditEntry:
__all__ = [
"ACTION_OPERATOR_EDIT",
"AuditEntry",
"HOST_DB_FILENAME",
"Proposal",
"Response",
"STATUSES",
@@ -184,6 +159,4 @@ __all__ = [
"TOOL_EGRESS_TOKEN_ALLOW",
"TOOL_GITLEAKS_ALLOW",
"TOOL_LIST_EGRESS_ROUTES",
"bot_bottle_root",
"host_db_path",
]
@@ -1,10 +1,16 @@
# PRD 0069: Firecracker-native, Docker-free backend
- **Status:** Draft
- **Status:** Draft (partially superseded)
- **Author:** Claude
- **Created:** 2026-07-12
- **Issue:** #348
> **Superseded in part by [PRD 0070](0070-per-host-orchestrator.md) (#351):**
> the sidecar-consolidation framing here (Stage 1, per-host sidecar; Stage 4,
> sidecar-as-VM) is taken over by 0070's per-host orchestrator. This PRD still
> owns the docker-free **image-building** work — Stage 2 (nix-built fixed
> images, a dependency of 0070) and Stage 3 (in-VM Dockerfile builder).
## Summary
Make the Firecracker backend depend on **firecracker + KVM only**, removing
+401
View File
@@ -0,0 +1,401 @@
# PRD 0070: Per-host orchestrator service
- **Status:** Draft
- **Author:** Claude
- **Created:** 2026-07-12
- **Issue:** #351
- **Supersedes:** the Stage-1 / Stage-4 sidecar-consolidation framing of
PRD 0069 (#348). Depends on 0069's nix-built fixed images (Stage 2) for
bootstrapping; 0069 still owns the docker-free image-building work.
## Summary
Replace the **per-bottle gateway bundle** with a single **persistent,
per-host orchestrator**: one long-lived service that runs the gateway
functions (egress / git-gate / supervise), coordinates with the console,
and brokers agent launches and teardown. It is **virtualized from the
start** using each backend's native isolation primitive — a Firecracker
microVM on the Firecracker backend, an Apple container on macOS, a Docker
container on the legacy backend — and is fronted by a single
**backend-agnostic contract**. Per-backend variation lives on
`BottleBackend`, not in the orchestrator.
## Motivation
Today each bottle spins up its own gateway bundle (egress mitmproxy +
git-gate + supervise). That costs:
- **Resources.** N bottles → N heavy bundles booting and idling.
- **Operational churn.** Per-launch container/VM lifecycle for the
gateways, a control path baked at launch and torn down at exit.
- **A blurry contract.** "How a bottle talks to its gateway" is
re-implemented per backend instead of being one agreed interface.
A per-host orchestrator collapses the first two and forces the third to
be made explicit. It's also the component that will own per-host runtime
**state** (slot leases, the approval queue, the bottle registry) — today
that's ad-hoc `fcntl`-locked files.
## Security review (read this first)
Consolidation is a real change to the trust model. The goal is to **not
significantly weaken** the posture; some properties strengthen, some
weaken, and the weakened ones must be mitigated by design, not hand-waved.
### What gets stronger
- **Build/host isolation of untrusted inputs** (with 0069 Stage 3): user
Dockerfiles build in a disposable VM instead of on the host.
- **One audited privileged surface.** Today the launcher runs as the full
host user and needs the Docker socket (root-equivalent). The orchestrator
model replaces that with a **thin launch broker** (below) — a small,
structured, auditable privileged core instead of a fat socket.
- **Attribution is enforced, not assumed.** Making source-IP identity a
first-class contract invariant (below) means each backend must *prove*
it, rather than the gateway implicitly trusting network position.
### What gets weaker, and the mitigation
1. **Secret concentration.** Per-bottle gateways isolate secrets at the
process boundary — each holds only its bottle's tokens/keys. A host
orchestrator concentrates **every bottle's** egress tokens, git deploy
keys, and the console credential in one long-lived process. A single
attribution bug leaks bottle A's token into bottle B's request — a class
of bug that *cannot exist* per-bottle.
- *Mitigation:* lean on the enforced source-IP invariant for
attribution; keep the most secret-dense, least-shareable service
(**git-gate**, per-repo deploy keys, no natural source-IP scoping)
**per-bottle** unless there's a compelling reason; scope each secret
to the bottle in the state DB so a lookup can't return the wrong
bottle's secret by construction (key every secret access by the
verified source identity, never by ambient state).
2. **Shared fate.** Orchestrator down = no new launches, and running
agents lose egress / git / supervise. Compromise = the whole host's
fleet, plus launch authority, plus the console token.
- *Mitigation:* the orchestrator is itself confined (its own VM/container
with its own fail-closed egress); make it **restartable without killing
running agent VMs** (agents keep running; they briefly lose gateway
connectivity until it's back); persist state to a host volume so a
restart re-adopts live bottles rather than losing them.
3. **The launch broker is the new privileged core.** We don't eliminate
host privilege — we shrink and relocate it. If the broker accepts
arbitrary paths/commands, the orchestrator VM can escape through it.
- *Mitigation:* the broker takes **structured requests only** — "launch
bottle from *this* content-addressed, nix-built rootfs on TAP slot
*k*", never "run this argv". It validates against a fixed image set,
not caller-supplied paths. It is small enough to audit line-by-line.
4. **The egress proxy now parses every bottle's traffic in one process.**
Higher blast radius for a mitmproxy/TLS-bump bug.
- *Mitigation:* this is the argument for virtualizing the orchestrator
from the start (Stage B, not a host daemon) — the code that TLS-bumps
and parses agent traffic and holds every token runs **inside its own
confined VM**, not as a host process. If egress sharing's blast radius
feels too high, egress can stay per-bottle while supervise (near-zero
secrets) goes host-level first.
### The attribution invariant
Source-IP attribution is what makes a shared orchestrator safe: one
process serves every bottle and tells them apart by source address. The
*mechanism* is identical everywhere (read source IP → look up bottle); the
**guarantee that the address can't be forged is a per-backend
responsibility** and part of the contract:
> **Invariant:** a packet's source address, as seen by the orchestrator,
> *provably* identifies the originating bottle.
- **Firecracker** — enforced by the `/31` point-to-point TAP + the
`bot_bottle_fc` nft table (strongest; already built).
- **Docker** — the per-bottle `--internal` network + anti-spoof; weaker,
must be made explicit.
- **Apple** — the host-only network.
If a backend can't honor the invariant, source-IP consolidation is not
safe there and that backend keeps per-bottle gateways. The invariant is a
hard precondition, not an aspiration.
**Defense-in-depth — a per-bottle identity token.** On top of the
network-layer invariant, inject a per-bottle secret token into every
request the agent makes to the orchestrator (the agent already egresses
through the gateway proxy, so this is cheap to add). It gives an
**application-layer** proof of identity independent of the network layer:
- On **Firecracker** the `/31` + nft already make source IP unspoofable
*by construction*, so the token is belt-and-suspenders there — but cheap
insurance against a misconfigured invariant.
- On **weaker backends** (Docker) it is load-bearing, providing attribution
that doesn't lean on network anti-spoof.
Requirements: the token is **per-bottle, unguessable, and
non-cross-leakable** — a bottle can only ever prove it is *itself* (it
can't learn another bottle's token, given bottle isolation), so a hostile
agent gains nothing by presenting it. The orchestrator provisions the
token at launch and the gateway requires it to attribute + authorize.
Note this hardens *attribution*, not *secret exposure*: it's app-layer, so
a compromised orchestrator still sees every token (that's the
concentration problem, addressed separately under Secret handling).
### Secret handling — a FUTURE pattern (not v1)
> **Status: future / not required for the initial orchestrator.** The
> initial cut can inject secrets as today; this section records the
> direction so v1 doesn't paint itself into a corner. Tracked as its own
> work in **#355** (generic `SecretProvider`).
The residual weakness after all of the above is **long-lived credential
concentration** — the data-plane proxies must hold every bottle's upstream
tokens because the agent must never see them. You can't policy-gate the
component whose job is to *use* all the secrets, so a full RCE of a proxy
drains its authorized set regardless. Two moves bound this without
pretending to prevent it:
1. **Vault as a separate trust domain.** The long-lived *roots* live in a
distinct process (ideally its own VM) that the byte-parsing data plane
never shares memory with. The proxies request secrets from it; the
crown jewels are not in the process an agent-facing parser can pop.
2. **Derive short-lived, scoped creds where the upstream allows it.** The
vault holds the root and mints expiring, narrowly-scoped credentials
per bottle-start (or per request) — GitHub App installation tokens,
OAuth/STS token exchange, forge deploy tokens. A compromise then leaks
short-lived material, not permanent keys. For upstreams stuck on static
keys the vault passes the value through (only the at-rest / audit /
revocation benefits apply, not lifetime reduction) — this residue is
accepted and documented, not solved.
Even a plain per-request fetch (no derivation) still buys **at-rest**
reduction (process memory holds only in-flight secrets), a **detection /
rate-limit / revocation chokepoint**, and clean **cross-component
scoping** (an egress RCE can't request git-gate's creds). It does *not*
prevent abuse of currently-authorized access during the compromise window.
**Mechanism (see #355):** generalize the existing `DeployKeyProvisioner`
(PRD 0048) into a user-extensible **`SecretProvider`** droppable into the
manifest anywhere a raw token value is accepted, discovered from
`~/.bot-bottle/contrib/<name>/secret_provider.py` exactly as user
`AgentProvider`s are — because maintaining every forge/cloud/OAuth
provider in-tree is untenable. The orchestrator's vault mints via these
providers; but the abstraction is shippable independently and today's
per-bottle gateways can use it too.
## Design
### The contract (backend-agnostic)
Three surfaces; only one is per-backend.
1. **Control plane (CLI / console → orchestrator)** — an RPC:
`launch_bottle`, `teardown_bottle`, `register_policy`,
`deregister_bottle`, `supervise_queue`. Fully backend-agnostic. Both the
local `cli.py` and the remote console funnel through it, so policy is
uniform and `cli.py` becomes a thin client rather than a parallel
launcher. **Transport: HTTP** — the most universal/reliable choice on
every host (no vsock/unix-socket portability caveats); a local unix
socket is a fine optimization, but HTTP is the wire contract.
2. **Data plane (agent → orchestrator)** — the egress / git / supervise
endpoints. Already agnostic today (agents dial `http://gateway:9099`);
only the *address* and *how packets get there* are per-backend.
3. **Launch / wire (orchestrator → backend)** — the irreducibly
backend-specific part; lives on `BottleBackend`.
### One `Orchestrator`, no subclass tree
The orchestrator is a **single concrete class** holding all the
backend-neutral logic — egress addon, git-gate, supervise, source-IP
attribution, live-reload control plane, console client. It never branches
on backend; it *composes* a `BottleBackend`. That composition is what makes
the contract agnostic: there is nothing backend-specific left in the
orchestrator to leak.
Rejected alternative: an `Orchestrator` ABC with per-backend
implementations. The interesting logic (proxies, attribution, control
plane) is backend-neutral, so three subclasses would triplicate the hard
part; and a second hierarchy paralleling `BottleBackend` reintroduces the
same hand-maintained lockstep coupling we just removed from the netpool
constants (PR #350). Composition over a parallel tree.
### `BottleBackend` absorbs the per-backend variation
A small, cohesive surface — reused for launching agent bottles *and* the
orchestrator's own unit (the orchestrator is just another native unit):
```
launch_unit(spec) -> Handle # agent bottle OR the orchestrator itself
# (fc microVM / apple ctr / docker ctr)
wire(unit, endpoint) -> None # DNAT+forward (fc) | attach shared net (docker/apple)
endpoint_of(unit) -> Endpoint # address resolution
health(unit) -> Status
```
Plus the **launch broker** — the answer to "a VM/container can't spawn its
own host-network siblings." The orchestrator can't directly open host
`/dev/kvm` + a host TAP fd (Firecracker), and a container can't spawn
siblings without a root-equivalent socket (Docker). So every backend
exposes a broker the orchestrator calls to launch an agent:
- **Firecracker** — a thin, structured host shim (see security #3). This
replaces today's implicit "launcher runs as host user."
- **Docker** — the socket today (fat, root-equivalent — the thing 0069's
Stage 3 removes); a narrower broker later.
- **Apple** — the `container` CLI/daemon.
**Broker request schema:** human-readable JSON of **static flags + ids
only** (never free-form paths/argv — that's what keeps it un-coercible
into arbitrary launches), wrapped as a **signed JWT** so the broker
verifies *provenance*: the request came from the real orchestrator, not a
forged one from a compromised co-located component. The orchestrator signs;
the broker verifies with the orchestrator's public key (provisioned at
broker install). This is the concrete form of security #3's "structured
requests only." Same JSON-with-ids + JWT shape for all
orchestrator↔broker/gateway communication; cases that don't fit get
handled as they arise, with this as the default.
If `BottleBackend` bloats, the pressure valve is composition one level
down: vend a `backend.network()` / `Wiring` collaborator rather than
piling methods on — the same discipline, recursed.
### State: one SQLite DB, owned by the orchestrator
The orchestrator is the natural owner of per-host **runtime state**:
- pool **slot leases** (which bottle holds slot *i*) — replaces today's
`fcntl`-locked files with SQLite transactions;
- the **supervise approval queue** + remembered approvals;
- the **live bottle registry** (source IP → bottle → policy/secrets refs),
the lookup table the attribution invariant reads.
This is deliberately **not** a "single source of truth for all config."
Config splits into three tiers with different homes:
| Tier | Example | Home |
|---|---|---|
| Build-time constants | pool size, IP base, nft table | flat `.env` (PR #350) — must be readable by Nix eval + root bash, zero runtime |
| User-authored config | bottle manifests, egress routes, secret refs | declarative files under `~/.bot-bottle/` — trust boundary at `$HOME`, git-trackable, "unknown keys die at load" |
| Runtime state | slot leases, approvals, registry | one shared **`bot-bottle.db`**, solely owned by the orchestrator |
SQLite is right for the runtime tier (mutable, concurrent, queried) and
wrong for the other two (Nix can't read it at eval time; it fights the
declarative manifest trust model). Keep the tiers separate.
**One shared `bot-bottle.db` for all runtime state** (decided in review).
The registry co-tenants the existing host `bot-bottle.db` — the `DbStore`
framework already namespaces each store by `schema_key`, so slot
leases / approvals / registry share one file. One place to query, back up,
and integrate a console against.
- **Host-resident, for durability.** Re-adoption sweeps the registry after
an orchestrator restart, so state *must* outlive the orchestrator
instance. The file lives on the host (`bot_bottle_root()/db/bot-bottle.db`);
the orchestrator unit reaches it, it doesn't carry it.
- **Integrity by sole ownership, not mount permissions.** Agents can't
touch the DB directly wherever it lives (network-isolated in their
bottles). The risk is a *compromised agent-facing data-plane service*
(egress/git-gate, which parse hostile bytes) writing the registry and
forging attribution. Because it's now one shared file, coarse `ro`/`rw`
mount-splitting no longer isolates the registry — so the rule is stronger
and simpler: **only the orchestrator (control plane) opens `bot-bottle.db`;
the data plane and the console reach state through the control-plane RPC,
never a direct file handle.** No agent-facing component gets the file, so
none can forge attribution. (This supersedes the earlier `ro`-mount idea.)
- *Transitional caveat:* today the per-bottle **supervise gateway
rw-bind-mounts `bot-bottle.db`** to write proposals — exactly the
pattern the orchestrator removes (supervise consolidates into the
orchestrator; gateway writes become RPC calls). Until that lands, don't
put the attribution registry behind a data-plane-writable mount.
Implementation note for the VM slices: SQLite **WAL** over a guest share
(virtiofs/9p) is finicky (the `-shm`/`-wal` files need real mmap/locking),
which is a second reason the DB wants a **host-side owner** the orchestrator
reaches over the RPC rather than a shared mount into the VM. WAL on the
shared DB is therefore a deliberate, tested future change — not enabled ad
hoc. `sqlite3` itself is stdlib, so "the host needs SQLite" is a non-cost.
## Sequencing
Jump straight to the **virtualized** end state (not a host-daemon stepping
stone): a host daemon's agent→`localhost` transport is throwaway once the
orchestrator becomes a VM. Decouple the two risks instead:
- **Consolidation risk** (one process, all secrets, attribution, reload)
and **packaging/transport risk** (VM-to-VM wiring, the shim) are
independent. Develop the orchestrator **service as a plain process
dev-harness** first, so the consolidation logic (attribution, reload,
secret handling) is proven with fast iteration — *then* wrap that exact
service in the VM and solve wiring separately.
Backend order (cheapest proof → hardest → last):
1. **Docker orchestrator** — nearly free (the gateway bundle is already
containers; collapse N bundles into one persistent container). Proves
consolidation + the `BottleBackend` seam with the least moving parts.
2. **Firecracker orchestrator** — the real work: the shim + VM-to-VM
routing (host forwards `bbfcN` → orchestrator TAP; the nft table grows
forward rules where today it drops all non-DNAT egress). Built against
the dev-harness so the app logic is already proven.
3. **macOS (Apple container)** — last (container-to-container networking).
Keep the gateway **service one shared thing** throughout.
## Non-goals
- Removing OCI/Dockerfile support for agent images (0069's concern).
- A single database for *all* config (see the three-tier table).
- Changing the per-bottle isolation of agent workloads — only the gateway
is consolidated; agents stay one-VM/container-each.
## Relationship to other work
- **PRD 0069 (#348):** 0070 subsumes its Stage 1 (per-host sidecar) and
Stage 4 (sidecar-as-VM). 0069 retains Stage 2 (nix-built fixed images —
a **dependency** here: the orchestrator and agent base must be
nix-built so the broker launches from a fixed image set and bootstrapping
has no chicken-and-egg) and Stage 3 (in-VM Dockerfile builder).
- **Minimal CI runner (paused):** the Firecracker broker + no host Docker
is what lets a dedicated `gitea` runner user drop the root-equivalent
`docker` group — it only needs broker-socket access + `kvm`/pool group
membership. This work unblocks it.
- **Generic `SecretProvider` (#355):** the future secret-handling
mechanism (see "Secret handling") — generalizes PRD 0048's
`DeployKeyProvisioner` into a user-extensible provider that mints
short-lived creds. Shippable independently; the orchestrator's vault
mints through it.
- **PR #350 (netpool single-source):** the same "one source per fact,
composition over parallel hierarchies" discipline the contract follows.
## Decisions (review 2026-07-13)
- **Egress sharing:** **consolidate egress** (worth it). Treat it as a
minimal, hardened attack surface for a malicious agent rather than
keeping it per-bottle; pair it with the identity token above and the
short-lived-vault-token mitigations (Secret handling / #355).
- **Control-plane transport:** **HTTP** — most universal/reliable on every
host; unix socket is an optional local optimization (see the contract).
- **Broker request schema:** **signed-JWT JSON, static flags + ids only**
(see the launch broker) — provenance + un-coercible by construction.
- **State re-adoption:** the restart procedure is:
1. **Singleton:** a new orchestrator launch requires no pre-existing
orchestrator; any found (healthy or not) is fully shut down first.
2. Re-adoption **waits for the new orchestrator to be healthy**.
3. Once healthy, it discovers all agents needing adoption via **both the
SQLite registry and live process/VM inspection** *before serving any
other request*. The two-source sweep is what closes the *in-flight
launch* race — a launch that started (VM booting / slot claimed) but
hadn't committed to SQLite when the old orchestrator died would be
invisible to a SQLite-only sweep; process inspection catches it.
Launches should also write an **intent record ahead of committing
resources** so the sweep can reconcile intent vs. actual.
## Open questions
- **VM-to-VM routing:** per-backend, and in the design it *is*
`BottleBackend.wire()` (DNAT+forward for fc, shared-net for
docker/apple), not orchestrator logic. **Not a blocker** — resolvable at
implementation time (may require a bit of host modification, as the pool
setup already does on NixOS); an earlier "gateways in VMs" spike showed
it's feasible.
- **Live-reload protocol** for per-bottle policy over the HTTP control
plane (add/remove routes/keys/proposals without a restart).
- **Identity-token delivery:** exactly how the per-bottle token is placed
where the agent can present it but not swap in another bottle's.
+67 -93
View File
@@ -1,104 +1,70 @@
# bot-bottle Firecracker network pool — declarative NixOS module.
#
# The one-time privileged setup the Firecracker backend needs: a pool of
# user-owned point-to-point TAP devices plus a fail-closed nftables table
# that confines every microVM to its own sidecar.
# user- (or group-) owned point-to-point TAP devices plus a fail-closed
# nftables table that confines every microVM to its own sidecar.
#
# NON-INVASIVE BY DESIGN. It does NOT flip `networking.nftables.enable`
# (which would switch your whole host firewall backend) or
# `systemd.network.enable` (which would hand your interfaces to
# systemd-networkd). Instead a single systemd oneshot service brings the
# pool up on boot — the declarative equivalent of
# `sudo ./scripts/firecracker-netpool.sh up`. The `inet <tableName>` table
# is independent (its own hooks at priority -10), so it coexists with an
# iptables `networking.firewall`, Docker, ufw, firewalld, etc.
# systemd-networkd). Instead a single systemd oneshot brings the pool up
# on boot by running the SAME bring-up script as every other install
# path (`scripts/firecracker-netpool.sh`) — so the TAP/nft logic lives in
# exactly one place. The `inet <tableName>` table is independent (its own
# hooks at priority -10), so it coexists with an iptables
# `networking.firewall`, Docker, ufw, firewalld, etc.
#
# The oneshot is only restarted when this config changes, so a
# `nixos-rebuild switch` doesn't tear down TAPs out from under running
# VMs unless you actually changed the pool.
# The oneshot is only restarted when this config changes, and the shared
# script is non-destructive (it never tears down an existing TAP), so a
# `nixos-rebuild switch` won't cut TAPs out from under running VMs.
#
# The option defaults MUST match the backend constants in
# bot_bottle/backend/firecracker/netpool.py (pool size, IP base, iface
# prefix, table name). The backend reads the same values at launch from
# the BOT_BOTTLE_FC_* env vars; if you override an option here, override
# the matching env var for the CLI too (or set writeEnvFile = true below
# to have this module emit them). Keep the two sides in lockstep.
# Single source of the pool defaults: bot_bottle/backend/firecracker/
# netpool.defaults.env. This module readFile-parses it for the option
# defaults below, then passes the resolved values back to the script as
# Environment=, so the host pool and the CLI launcher can't drift.
{ config, lib, pkgs, ... }:
let
cfg = config.services.bot-bottle-firecracker;
# --- IPv4 <-> int (full 32-bit carry math) -------------------------
toOctets = s: map lib.toInt (lib.splitString "." s);
ipToInt = s:
let o = toOctets s; in
(lib.elemAt o 0) * 16777216
+ (lib.elemAt o 1) * 65536
+ (lib.elemAt o 2) * 256
+ (lib.elemAt o 3);
intToIp = n:
# --- shared single-source defaults ---------------------------------
# Parse the KEY=VALUE defaults file (the same one netpool.py and the
# shell script read). Pure eval — just readFile, no import-from-
# derivation — so it works for both flake and channel consumers.
readDefaults = file:
let
b0 = n / 16777216; r0 = n - b0 * 16777216;
b1 = r0 / 65536; r1 = r0 - b1 * 65536;
b2 = r1 / 256;
b3 = r1 - b2 * 256;
in "${toString b0}.${toString b1}.${toString b2}.${toString b3}";
lines = lib.splitString "\n" (builtins.readFile file);
keep = l: l != "" && !(lib.hasPrefix "#" l) && lib.hasInfix "=" l;
toPair = l:
let parts = lib.splitString "=" l;
in lib.nameValuePair (lib.head parts)
(lib.concatStringsSep "=" (lib.tail parts));
in lib.listToAttrs (map toPair (lib.filter keep lines));
baseInt = ipToInt cfg.ipBase;
defaults = readDefaults ../bot_bottle/backend/firecracker/netpool.defaults.env;
# Slot i: host = base + 2i (the VM's gateway), on its own /31.
slots = lib.genList (i: {
iface = "${cfg.ifacePrefix}${toString i}";
hostIp = intToIp (baseInt + 2 * i);
}) cfg.poolSize;
# The one bring-up implementation, shared with the sudo/systemd paths.
netpoolScript = ../scripts/firecracker-netpool.sh;
ip = "${pkgs.iproute2}/bin/ip";
nft = "${pkgs.nftables}/bin/nft";
# /31 alignment == an even final octet (only bit 0 matters for base+2i).
lastOctet = lib.toInt (lib.last (lib.splitString "." cfg.ipBase));
# Idempotent ruleset: (create-if-absent → delete → recreate) is atomic
# within one `nft -f`, so re-running `up` always lands a clean table.
nftFile = pkgs.writeText "bot-bottle-fc.nft" ''
table inet ${cfg.tableName}
delete table inet ${cfg.tableName}
table inet ${cfg.tableName} {
chain forward {
type filter hook forward priority -10; policy accept;
iifname != "${cfg.ifacePrefix}*" return
ct state established,related accept
ct status dnat accept
drop
}
chain input {
type filter hook input priority -10; policy accept;
iifname != "${cfg.ifacePrefix}*" return
ct state established,related accept
drop
}
}
'';
# The script needs ip/nft/sysctl + the usual coreutils. It gets every
# pool value via the unit's Environment=, so it never reads the shared
# defaults file (which isn't beside it once copied to the store).
runtimePath = with pkgs; [ iproute2 nftables procps coreutils gnused ];
# A non-owner user can't open a user-owned TAP, but any member of a
# TAP's owning GROUP can (the kernel checks owner-uid OR owning-group).
# So `group` lets an interactive user and a CI-runner user share one
# pool; `owner` is the single-user default.
tapOwn = if cfg.group != null then "group ${cfg.group}" else "user ${cfg.owner}";
ownEnv =
if cfg.group != null
then { BOT_BOTTLE_FC_GROUP = cfg.group; }
else { BOT_BOTTLE_FC_OWNER = cfg.owner; };
upScript = pkgs.writeShellScript "bot-bottle-fc-netpool-up" ''
set -eu
${lib.concatMapStringsSep "\n" (s: ''
${ip} link show ${s.iface} >/dev/null 2>&1 || ${ip} tuntap add dev ${s.iface} mode tap ${tapOwn}
${ip} addr replace ${s.hostIp}/31 dev ${s.iface}
${ip} link set ${s.iface} up
'') slots}
${nft} -f ${nftFile}
'';
downScript = pkgs.writeShellScript "bot-bottle-fc-netpool-down" ''
${nft} delete table inet ${cfg.tableName} 2>/dev/null || true
${lib.concatMapStringsSep "\n" (s: ''
${ip} link show ${s.iface} >/dev/null 2>&1 && ${ip} tuntap del dev ${s.iface} mode tap || true
'') slots}
'';
unitEnv = {
BOT_BOTTLE_FC_POOL_SIZE = toString cfg.poolSize;
BOT_BOTTLE_FC_IP_BASE = cfg.ipBase;
BOT_BOTTLE_FC_IFACE_PREFIX = cfg.ifacePrefix;
BOT_BOTTLE_FC_NFT_TABLE = cfg.tableName;
} // ownEnv;
in
{
options.services.bot-bottle-firecracker = {
@@ -106,25 +72,28 @@ in
poolSize = lib.mkOption {
type = lib.types.ints.positive;
default = 8;
default = lib.toInt defaults.BOT_BOTTLE_FC_POOL_SIZE;
defaultText = lib.literalMD "the shared `netpool.defaults.env` value";
description = "Number of pool slots (concurrent bottles). Must match BOT_BOTTLE_FC_POOL_SIZE.";
};
ipBase = lib.mkOption {
type = lib.types.str;
default = "10.243.0.0";
default = defaults.BOT_BOTTLE_FC_IP_BASE;
defaultText = lib.literalMD "the shared `netpool.defaults.env` value";
description = ''
Base IPv4 of the /31 pool; slot i uses host = base + 2i. Must
be /31-aligned (even final address) and must match
BOT_BOTTLE_FC_IP_BASE. Default is an obscure RFC-1918 /16 that
dodges docker/libvirt/k8s/LAN and, deliberately, Tailscale's
100.64.0.0/10 CGNAT range.
be /31-aligned (even final octet) and must match
BOT_BOTTLE_FC_IP_BASE. The shared default is an obscure RFC-1918
/16 that dodges docker/libvirt/k8s/LAN and, deliberately,
Tailscale's 100.64.0.0/10 CGNAT range.
'';
};
ifacePrefix = lib.mkOption {
type = lib.types.str;
default = "bbfc";
default = defaults.BOT_BOTTLE_FC_IFACE_PREFIX;
defaultText = lib.literalMD "the shared `netpool.defaults.env` value";
description = "TAP interface name prefix. Must match BOT_BOTTLE_FC_IFACE_PREFIX.";
};
@@ -153,7 +122,8 @@ in
tableName = lib.mkOption {
type = lib.types.str;
default = "bot_bottle_fc";
default = defaults.BOT_BOTTLE_FC_NFT_TABLE;
defaultText = lib.literalMD "the shared `netpool.defaults.env` value";
description = "nftables table name for the isolation boundary. Must match netpool.NFT_TABLE.";
};
@@ -171,7 +141,7 @@ in
config = lib.mkIf cfg.enable {
assertions = [
{
assertion = lib.mod baseInt 2 == 0;
assertion = lib.mod lastOctet 2 == 0;
message = "services.bot-bottle-firecracker.ipBase must be /31-aligned (even final octet); got ${cfg.ipBase}.";
}
{
@@ -183,17 +153,20 @@ in
# VM->sidecar traffic is DNAT'd and forwarded, so forwarding must be on.
boot.kernel.sysctl."net.ipv4.ip_forward" = 1;
# One oneshot brings up the whole pool (TAPs + independent nft table).
# No networking.nftables.enable / systemd.network.enable — see header.
# One oneshot brings up the whole pool (TAPs + independent nft table)
# by running the shared bring-up script. No networking.nftables.enable
# / systemd.network.enable — see header.
systemd.services."bot-bottle-firecracker-netpool" = {
description = "bot-bottle Firecracker TAP pool + nft isolation table";
wantedBy = [ "multi-user.target" ];
after = [ "network-pre.target" ];
path = runtimePath;
environment = unitEnv;
serviceConfig = {
Type = "oneshot";
RemainAfterExit = true;
ExecStart = upScript;
ExecStop = downScript;
ExecStart = "${pkgs.bash}/bin/bash ${netpoolScript} up";
ExecStop = "${pkgs.bash}/bin/bash ${netpoolScript} down";
};
};
@@ -202,6 +175,7 @@ in
BOT_BOTTLE_FC_POOL_SIZE=${toString cfg.poolSize}
BOT_BOTTLE_FC_IP_BASE=${cfg.ipBase}
BOT_BOTTLE_FC_IFACE_PREFIX=${cfg.ifacePrefix}
BOT_BOTTLE_FC_NFT_TABLE=${cfg.tableName}
'';
};
};
+40 -19
View File
@@ -33,24 +33,44 @@
# sudo ./scripts/firecracker-netpool.sh down
# ./scripts/firecracker-netpool.sh status
#
# Env overrides (must match the backend's util.py constants):
# BOT_BOTTLE_FC_POOL_SIZE number of slots (default 8)
# BOT_BOTTLE_FC_IP_BASE base IPv4 of the /31 pool (default 10.243.0.0)
# BOT_BOTTLE_FC_IFACE_PREFIX TAP name prefix (default bbfc)
# BOT_BOTTLE_FC_OWNER owning user (default $SUDO_USER or $USER)
# BOT_BOTTLE_FC_GROUP owning group; if set, TAPs are group-owned
# instead of user-owned, so any group member
# (e.g. an interactive user + a CI runner
# user) can open the pool. Overrides OWNER.
# Pool params default to the shared single-source file
# (bot_bottle/backend/firecracker/netpool.defaults.env); a matching
# env var overrides its key:
# BOT_BOTTLE_FC_POOL_SIZE number of slots
# BOT_BOTTLE_FC_IP_BASE base IPv4 of the /31 pool
# BOT_BOTTLE_FC_IFACE_PREFIX TAP name prefix
# BOT_BOTTLE_FC_NFT_TABLE isolation table name
# BOT_BOTTLE_FC_OWNER owning user (default $SUDO_USER or $USER)
# BOT_BOTTLE_FC_GROUP owning group; if set, TAPs are group-owned
# instead of user-owned, so any group member
# (e.g. an interactive user + a CI runner
# user) can open the pool. Overrides OWNER.
set -euo pipefail
POOL_SIZE="${BOT_BOTTLE_FC_POOL_SIZE:-8}"
IP_BASE="${BOT_BOTTLE_FC_IP_BASE:-10.243.0.0}"
PREFIX="${BOT_BOTTLE_FC_IFACE_PREFIX:-bbfc}"
# The single source of the pool defaults, shared with netpool.py and the
# Nix module. The Nix path passes every value as env (the script runs
# from the store, detached from this file), so this lookup only matters
# on the direct/sudo path where the script sits in the repo tree.
_SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
_DEFAULTS="$_SCRIPT_DIR/../bot_bottle/backend/firecracker/netpool.defaults.env"
_default() { # value of KEY=... from the shared file; empty if unavailable
[ -f "$_DEFAULTS" ] || return 0
sed -n "s/^$1=//p" "$_DEFAULTS" | tail -1
}
POOL_SIZE="${BOT_BOTTLE_FC_POOL_SIZE:-$(_default BOT_BOTTLE_FC_POOL_SIZE)}"
IP_BASE="${BOT_BOTTLE_FC_IP_BASE:-$(_default BOT_BOTTLE_FC_IP_BASE)}"
PREFIX="${BOT_BOTTLE_FC_IFACE_PREFIX:-$(_default BOT_BOTTLE_FC_IFACE_PREFIX)}"
TABLE="${BOT_BOTTLE_FC_NFT_TABLE:-$(_default BOT_BOTTLE_FC_NFT_TABLE)}"
OWNER="${BOT_BOTTLE_FC_OWNER:-${SUDO_USER:-$USER}}"
GROUP="${BOT_BOTTLE_FC_GROUP:-}"
TABLE="bot_bottle_fc"
# Fail loudly rather than provisioning a half/empty range if a value
# resolved to nothing (env unset AND the shared file unreadable).
for _v in POOL_SIZE IP_BASE PREFIX TABLE; do
[ -n "${!_v}" ] || { echo "error: $_v unresolved (set BOT_BOTTLE_FC_* or fix $_DEFAULTS)" >&2; exit 1; }
done
# Sidecar ports (must match the backend). egress=9099, supervise=9100,
# git-http=9420. Reached by the VM at its host-side TAP IP.
@@ -94,12 +114,13 @@ cmd_up() {
for i in $(seq 0 $((POOL_SIZE-1))); do
local dev host
dev="$(iface "$i")" ; host="$(host_ip "$i")"
if ip link show "$dev" >/dev/null 2>&1; then
ip link set "$dev" down 2>/dev/null || true
ip tuntap del dev "$dev" mode tap 2>/dev/null || true
fi
ip tuntap add dev "$dev" mode tap "${own_args[@]}"
ip addr add "$host/31" dev "$dev"
# Non-destructive + idempotent: only create a missing TAP (tearing
# an existing one down would cut a running VM), and `addr replace`
# is safe to re-run. Ownership is fixed at creation, so to change
# owner/group run `down` then `up`.
ip link show "$dev" >/dev/null 2>&1 \
|| ip tuntap add dev "$dev" mode tap "${own_args[@]}"
ip addr replace "$host/31" dev "$dev"
ip link set "$dev" up
echo " $dev host=$host guest=$(guest_ip "$i") $own_desc"
done
@@ -0,0 +1,56 @@
"""Integration: the Docker launch broker starts and removes a real container.
Gated on a reachable Docker daemon (skips cleanly otherwise). Uses a tiny
image; the container may exit immediately we only assert it exists after
launch and is gone after teardown.
"""
from __future__ import annotations
import secrets
import subprocess
import unittest
from bot_bottle.orchestrator.broker import LaunchRequest, sign_request
from bot_bottle.orchestrator.docker_broker import DockerBroker, container_name
from tests._docker import skip_unless_docker
IMAGE = "busybox"
@skip_unless_docker()
class TestDockerBrokerIntegration(unittest.TestCase):
def setUp(self) -> None:
self.secret = secrets.token_bytes(16)
self.broker = DockerBroker(self.secret)
self.bottle_id = "itest" + secrets.token_hex(4)
self.name = container_name(self.bottle_id)
self.addCleanup(
lambda: subprocess.run(
["docker", "rm", "--force", self.name],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
)
)
def _exists(self) -> bool:
proc = subprocess.run(
["docker", "ps", "-a", "--filter", f"name=^{self.name}$",
"--format", "{{.Names}}"],
stdout=subprocess.PIPE, text=True, check=False,
)
return self.name in proc.stdout.split()
def _submit(self, req: LaunchRequest) -> None:
self.broker.submit(sign_request(req, self.secret))
def test_launch_creates_then_teardown_removes(self) -> None:
self._submit(
LaunchRequest(op="launch", bottle_id=self.bottle_id, image_ref=IMAGE)
)
self.assertTrue(self._exists(), "container should exist after launch")
self._submit(LaunchRequest(op="teardown", bottle_id=self.bottle_id))
self.assertFalse(self._exists(), "container should be gone after teardown")
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,45 @@
"""Integration: the consolidated Docker gateway is an idempotent singleton.
Gated on a reachable Docker daemon. Uses a unique name so it never
collides with a real per-host gateway or a leftover from another run.
"""
from __future__ import annotations
import secrets
import subprocess
import unittest
from bot_bottle.orchestrator.gateway import DockerGateway
from tests._docker import skip_unless_docker
IMAGE = "busybox"
@skip_unless_docker()
class TestDockerGatewayIntegration(unittest.TestCase):
def setUp(self) -> None:
self.name = "bot-bottle-orch-gateway-itest-" + secrets.token_hex(4)
self.sc = DockerGateway(IMAGE, name=self.name)
self.addCleanup(self.sc.stop)
def _count(self) -> int:
proc = subprocess.run(
["docker", "ps", "-a", "--filter", f"name=^{self.name}$",
"--format", "{{.Names}}"],
stdout=subprocess.PIPE, text=True, check=False,
)
return sum(1 for n in proc.stdout.split() if n == self.name)
def test_ensure_is_idempotent_singleton(self) -> None:
self.sc.ensure_running()
self.assertEqual(1, self._count())
# A second ensure must not spawn a second container — one per host.
self.sc.ensure_running()
self.assertEqual(1, self._count())
self.sc.stop()
self.assertEqual(0, self._count())
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,36 @@
"""Integration: DockerGateway.image_exists reflects real docker state.
Gated on a reachable Docker daemon. Deliberately does NOT build the full
sidecar bundle (a heavy, slow image build) it exercises the `image_exists`
primitive that `ensure_built` gates on, against a tiny pulled image and a
name that can't exist.
"""
from __future__ import annotations
import subprocess
import unittest
from bot_bottle.orchestrator.gateway import DockerGateway
from tests._docker import skip_unless_docker
IMAGE = "busybox"
@skip_unless_docker()
class TestDockerGatewayImageExists(unittest.TestCase):
def test_image_exists_true_for_present_false_for_absent(self) -> None:
# Ensure the tiny image is present (build_if_missing is disabled here
# so this never triggers a Dockerfile.sidecars build).
subprocess.run(
["docker", "pull", IMAGE],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
)
self.assertTrue(DockerGateway(IMAGE, dockerfile=None).image_exists())
self.assertFalse(
DockerGateway("bot-bottle-nonexistent:doesnotexist", dockerfile=None).image_exists()
)
if __name__ == "__main__":
unittest.main()
+22 -3
View File
@@ -2,7 +2,7 @@
Isolates ``HOME`` to a throwaway directory for the entire unit suite so
no test ever reads or writes the real ``~/.bot-bottle`` (state, queue,
and audit dirs all derive from ``supervise.bot_bottle_root()``
and audit dirs all derive from ``paths.bot_bottle_root()``
``Path.home()``). Without this, a test that takes a ``flock`` on the
real audit log can **block indefinitely** when a live bottle's supervise
sidecar holds that lock observed as a hung ``coverage run`` at 0% CPU
@@ -10,8 +10,12 @@ and unisolated tests otherwise pollute the developer's home dir.
Individual tests that need their own ``HOME`` still override
``os.environ['HOME']`` and restore it; they now restore to this isolated
dir rather than the real one, so isolation holds either way. Tests that
patch ``supervise.bot_bottle_root`` directly are unaffected.
dir rather than the real one, so isolation holds either way.
Tests that need their own app-data root call ``use_bottle_root`` (below),
which sets ``BOT_BOTTLE_ROOT`` the supported override read by
``paths.bot_bottle_root`` instead of monkey-patching. One env setting
covers every module and every flat/package copy of the helper.
"""
from __future__ import annotations
@@ -20,6 +24,9 @@ import atexit
import os
import shutil
import tempfile
from collections.abc import Callable
from pathlib import Path
from unittest import mock
_real_home = os.environ.get("HOME")
_tmp_home = tempfile.mkdtemp(prefix="bot-bottle-unit-home.")
@@ -35,3 +42,15 @@ def _restore_home() -> None:
atexit.register(_restore_home)
def use_bottle_root(root: Path) -> Callable[[], None]:
"""Redirect ``paths.bot_bottle_root()`` to ``root`` by setting
``BOT_BOTTLE_ROOT`` the supported override. Returns a callable that
undoes it (store it as the test's restore hook, or pass to addCleanup).
Replaces monkey-patching ``paths.bot_bottle_root``; one env var covers
every module and the flat/package copies alike (they all read it)."""
patcher = mock.patch.dict(os.environ, {"BOT_BOTTLE_ROOT": str(root)})
patcher.start()
return patcher.stop
+3 -8
View File
@@ -7,7 +7,8 @@ import unittest
from pathlib import Path
from unittest.mock import patch
from bot_bottle import supervise, bottle_state
from bot_bottle import bottle_state
from tests.unit import use_bottle_root
from bot_bottle.backend import ActiveAgent
from bot_bottle.backend.freeze import get_freezer
from bot_bottle.backend.docker.freezer import DockerFreezer
@@ -18,13 +19,7 @@ from bot_bottle.backend.firecracker.freezer import FirecrackerFreezer
class _FakeHomeMixin:
def _setup_fake_home(self):
self._tmp = tempfile.TemporaryDirectory(prefix="freezer-test.")
original = supervise.bot_bottle_root
def fake_root() -> Path:
return Path(self._tmp.name) / ".bot-bottle"
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
self._restore = lambda: setattr(supervise, "bot_bottle_root", original)
self._restore = use_bottle_root(Path(self._tmp.name) / ".bot-bottle")
def _teardown_fake_home(self):
self._restore()
+3 -4
View File
@@ -13,7 +13,7 @@ from pathlib import Path
from unittest.mock import patch
from bot_bottle import bottle_state
from bot_bottle import supervise
from tests.unit import use_bottle_root
from bot_bottle.backend import BottleSpec
from bot_bottle.backend.docker import DockerBottleBackend
from bot_bottle.backend.resolve_common import mint_slug
@@ -55,11 +55,10 @@ class _FakeStateMixin:
def setUp(self) -> None:
self.tmp = tempfile.TemporaryDirectory(prefix="backend-prepare.")
self.root = Path(self.tmp.name) / ".bot-bottle"
self.original_root = supervise.bot_bottle_root
supervise.bot_bottle_root = lambda: self.root # type: ignore[assignment]
self._restore = use_bottle_root(self.root)
def tearDown(self) -> None:
supervise.bot_bottle_root = self.original_root # type: ignore[assignment]
self._restore()
self.tmp.cleanup()
+3 -4
View File
@@ -13,7 +13,7 @@ from pathlib import Path
from unittest.mock import MagicMock, call, patch
from bot_bottle import bottle_state
from bot_bottle import supervise
from tests.unit import use_bottle_root
from bot_bottle.backend import Bottle, BottleSpec, ExecResult
from bot_bottle.backend.docker import DockerBottleBackend
from bot_bottle.backend.firecracker import FirecrackerBottleBackend
@@ -55,11 +55,10 @@ class _FakeStateMixin:
self.tmp_dir = tempfile.TemporaryDirectory(prefix="backend-workspace.")
self.tmp = Path(self.tmp_dir.name)
self.root = self.tmp / ".bot-bottle"
self.original_root = supervise.bot_bottle_root
supervise.bot_bottle_root = lambda: self.root # type: ignore[assignment]
self._restore = use_bottle_root(self.root)
def tearDown(self) -> None:
supervise.bot_bottle_root = self.original_root # type: ignore[assignment]
self._restore()
self.tmp_dir.cleanup()
+2 -8
View File
@@ -6,7 +6,7 @@ import tempfile
import unittest
from pathlib import Path
from bot_bottle import supervise
from tests.unit import use_bottle_root
from bot_bottle import bottle_state
from bot_bottle.bottle_state import (
BottleMetadata,
@@ -18,13 +18,7 @@ from bot_bottle.bottle_state import (
class _FakeHomeMixin:
def _setup_fake_home(self):
self._tmp = tempfile.TemporaryDirectory(prefix="bottle-state-test.")
original = supervise.bot_bottle_root
def fake_root() -> Path:
return Path(self._tmp.name) / ".bot-bottle"
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
self._restore = lambda: setattr(supervise, "bot_bottle_root", original)
self._restore = use_bottle_root(Path(self._tmp.name) / ".bot-bottle")
def _teardown_fake_home(self):
self._restore()
+2 -8
View File
@@ -8,7 +8,7 @@ from pathlib import Path
from unittest.mock import MagicMock, patch
from bot_bottle.cli.commit import cmd_commit
from bot_bottle import supervise
from tests.unit import use_bottle_root
from bot_bottle import bottle_state
from bot_bottle.backend.freeze import CommitCancelled
@@ -16,13 +16,7 @@ from bot_bottle.backend.freeze import CommitCancelled
class _FakeHomeMixin:
def _setup_fake_home(self):
self._tmp = tempfile.TemporaryDirectory(prefix="cli-commit-test.")
original = supervise.bot_bottle_root
def fake_root() -> Path:
return Path(self._tmp.name) / ".bot-bottle"
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
self._restore = lambda: setattr(supervise, "bot_bottle_root", original)
self._restore = use_bottle_root(Path(self._tmp.name) / ".bot-bottle")
def _teardown_fake_home(self):
self._restore()
+3 -8
View File
@@ -8,7 +8,7 @@ import tempfile
import unittest
from pathlib import Path
from bot_bottle import supervise
from tests.unit import use_bottle_root
from bot_bottle import bottle_state
from bot_bottle.cli import start as start_mod
@@ -16,15 +16,10 @@ from bot_bottle.cli import start as start_mod
class _FakeHomeMixin:
def _setup_fake_home(self):
self._tmp = tempfile.TemporaryDirectory(prefix="cli-start-settle.")
self._original = supervise.bot_bottle_root
def fake_root() -> Path:
return Path(self._tmp.name) / ".bot-bottle"
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
self._restore = use_bottle_root(Path(self._tmp.name) / ".bot-bottle")
def _teardown_fake_home(self):
supervise.bot_bottle_root = self._original # type: ignore[assignment]
self._restore()
self._tmp.cleanup()
+2 -8
View File
@@ -15,7 +15,7 @@ import tempfile
import unittest
from pathlib import Path
from bot_bottle import supervise
from tests.unit import use_bottle_root
from bot_bottle import bottle_state
from bot_bottle.backend.docker.cleanup import _list_orphan_state_dirs
@@ -23,13 +23,7 @@ from bot_bottle.backend.docker.cleanup import _list_orphan_state_dirs
class _FakeHomeMixin:
def _setup_fake_home(self) -> None:
self._tmp = tempfile.TemporaryDirectory(prefix="docker-cleanup-test.")
original = supervise.bot_bottle_root
def fake_root() -> Path:
return Path(self._tmp.name) / ".bot-bottle"
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
self._restore = lambda: setattr(supervise, "bot_bottle_root", original)
self._restore = use_bottle_root(Path(self._tmp.name) / ".bot-bottle")
def _teardown_fake_home(self) -> None:
self._restore()
+2 -8
View File
@@ -23,7 +23,7 @@ import tempfile
import unittest
from pathlib import Path
from bot_bottle import supervise
from tests.unit import use_bottle_root
from bot_bottle import bottle_state
from bot_bottle.backend.docker import enumerate as _enumerate
@@ -75,13 +75,7 @@ class TestParseServicesByProject(unittest.TestCase):
class _FakeHomeMixin:
def _setup_fake_home(self) -> None:
self._tmp = tempfile.TemporaryDirectory(prefix="enum-active.")
original = supervise.bot_bottle_root
def fake_root() -> Path:
return Path(self._tmp.name) / ".bot-bottle"
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
self._restore_home = lambda: setattr(supervise, "bot_bottle_root", original)
self._restore_home = use_bottle_root(Path(self._tmp.name) / ".bot-bottle")
def _teardown_fake_home(self) -> None:
self._restore_home()
+2 -8
View File
@@ -8,7 +8,7 @@ from pathlib import Path
from types import SimpleNamespace
from unittest.mock import patch
from bot_bottle import supervise
from tests.unit import use_bottle_root
from bot_bottle.backend.egress_apply import EgressApplyError
from bot_bottle.backend.docker.egress_apply import applicator
@@ -67,14 +67,8 @@ class TestValidateRoutesContent(unittest.TestCase):
class TestApplyRoutesChange(unittest.TestCase):
def setUp(self):
self._tmp = tempfile.TemporaryDirectory(prefix="egress-apply-test.")
original = supervise.bot_bottle_root
def fake_root() -> Path:
return Path(self._tmp.name) / ".bot-bottle"
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
self.addCleanup(lambda: setattr(supervise, "bot_bottle_root", original))
self.addCleanup(self._tmp.cleanup)
self.addCleanup(use_bottle_root(Path(self._tmp.name) / ".bot-bottle"))
def test_writes_live_routes_and_signals_reload(self):
calls: list[list[str]] = []
+53
View File
@@ -0,0 +1,53 @@
"""Unit: resolve_client_config — fail-closed per-client egress config (PRD 0070)."""
from __future__ import annotations
import unittest
from bot_bottle.egress_addon_core import resolve_client_config
from bot_bottle.policy_resolver import PolicyResolveError
class _FakeResolver:
def __init__(self, result: str | None = None, raises: bool = False) -> None:
self._result = result
self._raises = raises
self.calls: list[tuple[str, str]] = []
def resolve(self, source_ip: str, identity_token: str = "") -> str | None:
self.calls.append((source_ip, identity_token))
if self._raises:
raise PolicyResolveError("orchestrator down")
return self._result
class TestResolveClientConfig(unittest.TestCase):
def test_valid_policy_is_parsed(self) -> None:
cfg = resolve_client_config(
_FakeResolver(result="routes:\n - host: example.com\n"), "10.243.0.1"
)
self.assertEqual(("example.com",), tuple(r.host for r in cfg.routes))
def test_unattributed_none_denies_all(self) -> None:
cfg = resolve_client_config(_FakeResolver(result=None), "10.243.0.9")
self.assertEqual((), cfg.routes) # no routes → default-deny
def test_empty_policy_denies_all(self) -> None:
self.assertEqual((), resolve_client_config(_FakeResolver(result=""), "10.243.0.1").routes)
def test_resolver_error_denies_all(self) -> None:
# Orchestrator unreachable/errored must never widen egress.
self.assertEqual((), resolve_client_config(_FakeResolver(raises=True), "10.243.0.1").routes)
def test_unparseable_policy_denies_all(self) -> None:
cfg = resolve_client_config(_FakeResolver(result="routes: notalist\n"), "10.243.0.1")
self.assertEqual((), cfg.routes)
def test_forwards_source_ip_and_token(self) -> None:
r = _FakeResolver(result=None)
resolve_client_config(r, "10.243.0.1", "tok")
self.assertEqual(("10.243.0.1", "tok"), r.calls[0])
if __name__ == "__main__":
unittest.main()
+50 -19
View File
@@ -67,10 +67,24 @@ class TestNetpoolRenderers(unittest.TestCase):
self.assertNotIn("networking.nftables.enable = true", mod)
self.assertNotIn("systemd.network.enable = true", mod)
self.assertIn("bot-bottle-firecracker-netpool", mod) # the oneshot
self.assertIn(netpool.NFT_TABLE, mod)
self.assertIn('iifname != "${cfg.ifacePrefix}*" return', mod)
# Supports group-owned TAPs (shared pool for a multi-user host).
self.assertIn("group ${cfg.group}", mod)
self.assertIn("BOT_BOTTLE_FC_GROUP", mod)
def test_nixos_module_delegates_and_holds_no_literals(self):
# Single-source discipline: the module must NOT re-implement the
# bring-up (it delegates to the shared script) and must NOT hard-
# code the pool defaults (it readFile-parses the shared .env), so
# nothing can drift from netpool.py / the shell script.
root = Path(__file__).resolve().parents[2]
mod = (root / "nix" / "firecracker-netpool.nix").read_text()
# Delegation to the one bring-up implementation + shared defaults.
self.assertIn("scripts/firecracker-netpool.sh", mod)
self.assertIn("netpool.defaults.env", mod)
self.assertIn("BOT_BOTTLE_FC_NFT_TABLE", mod)
# No duplicated literals or nft ruleset.
self.assertNotIn(netpool.ip_base(), mod) # 10.243.0.0
self.assertNotIn(netpool.NFT_TABLE, mod) # bot_bottle_fc
self.assertNotIn("ct status dnat", mod) # the nft ruleset
def test_shell_setup_reflects_overrides(self):
with patch.dict(os.environ, {"BOT_BOTTLE_FC_POOL_SIZE": "3"}):
@@ -262,26 +276,43 @@ class TestBottleAgentArgv(unittest.TestCase):
self.assertIn("USER=node", argv)
class TestSetupScriptConsistency(unittest.TestCase):
"""The shell setup script duplicates the pool defaults; keep them in
lockstep with the Python constants so the two setup paths agree."""
class TestNetpoolDefaultsSingleSource(unittest.TestCase):
"""The pool defaults live in one shared file (netpool.defaults.env);
Python parses it and the shell script + Nix module read the same file,
so the three setup paths can't drift."""
def _script(self) -> str:
root = Path(__file__).resolve().parent.parent.parent
return (root / "scripts" / "firecracker-netpool.sh").read_text()
def _root(self) -> Path:
return Path(__file__).resolve().parents[2]
def test_defaults_match_python(self):
script = self._script()
def _shared_defaults(self) -> dict[str, str]:
text = netpool.DEFAULTS_FILE.read_text()
out: dict[str, str] = {}
for raw in text.splitlines():
line = raw.strip()
if line and not line.startswith("#") and "=" in line:
k, _, v = line.partition("=")
out[k.strip()] = v.strip()
return out
def test_python_reads_the_shared_file(self):
d = self._shared_defaults()
with patch.dict(os.environ, {}, clear=True):
self.assertIn(f'POOL_SIZE:-{netpool.pool_size()}', script)
self.assertIn(f'BOT_BOTTLE_FC_IP_BASE:-{netpool.ip_base()}', script)
self.assertIn(f'IFACE_PREFIX:-{netpool.IFACE_PREFIX}', script)
self.assertEqual(int(d["BOT_BOTTLE_FC_POOL_SIZE"]), netpool.pool_size())
self.assertEqual(d["BOT_BOTTLE_FC_IP_BASE"], netpool.ip_base())
# Module constants resolve through the same shared file.
self.assertEqual(d["BOT_BOTTLE_FC_IFACE_PREFIX"], netpool.IFACE_PREFIX)
self.assertEqual(d["BOT_BOTTLE_FC_NFT_TABLE"], netpool.NFT_TABLE)
def test_table_name_and_ports_match(self):
script = self._script()
self.assertIn(f'TABLE="{netpool.NFT_TABLE}"', script)
for port in netpool.SIDECAR_PORTS:
self.assertIn(str(port), script)
def test_env_var_overrides_the_shared_default(self):
with patch.dict(os.environ, {"BOT_BOTTLE_FC_IP_BASE": "10.99.0.0"}):
self.assertEqual("10.99.0.0", netpool.ip_base())
def test_script_defers_to_shared_file_without_literals(self):
script = (self._root() / "scripts" / "firecracker-netpool.sh").read_text()
# Delegates its defaults to the shared file, not hardcoded values.
self.assertIn("netpool.defaults.env", script)
self.assertIn("BOT_BOTTLE_FC_POOL_SIZE:-$(_default BOT_BOTTLE_FC_POOL_SIZE)", script)
self.assertIn('TABLE="${BOT_BOTTLE_FC_NFT_TABLE:-$(_default BOT_BOTTLE_FC_NFT_TABLE)}"', script)
class TestBootArgs(unittest.TestCase):
+86
View File
@@ -0,0 +1,86 @@
"""Unit tests for the orchestrator launch broker (PRD 0070)."""
from __future__ import annotations
import secrets
import unittest
from bot_bottle.orchestrator.broker import (
BrokerAuthError,
LaunchRequest,
StubBroker,
sign_request,
verify_request,
)
class TestSignVerify(unittest.TestCase):
def setUp(self) -> None:
self.secret = secrets.token_bytes(16)
def test_launch_round_trip(self) -> None:
req = LaunchRequest(
op="launch", bottle_id="b1", source_ip="10.243.0.1",
image_ref="sha256:abc", slot=3,
)
self.assertEqual(req, verify_request(sign_request(req, self.secret), self.secret))
def test_teardown_round_trip(self) -> None:
req = LaunchRequest(op="teardown", bottle_id="b1")
got = verify_request(sign_request(req, self.secret), self.secret)
self.assertEqual(("teardown", "b1"), (got.op, got.bottle_id))
def test_wrong_secret_rejected(self) -> None:
token = sign_request(LaunchRequest(op="launch", bottle_id="b1"), self.secret)
with self.assertRaises(BrokerAuthError):
verify_request(token, secrets.token_bytes(16))
def test_tampered_payload_rejected(self) -> None:
token = sign_request(LaunchRequest(op="launch", bottle_id="b1"), self.secret)
header, payload, sig = token.split(".")
flipped = payload[:-1] + ("A" if payload[-1] != "A" else "B")
with self.assertRaises(BrokerAuthError):
verify_request(f"{header}.{flipped}.{sig}", self.secret)
def test_malformed_tokens_rejected(self) -> None:
for bad in ("", "a.b", "a.b.c.d", "not-a-token"):
with self.assertRaises(BrokerAuthError):
verify_request(bad, self.secret)
def test_unknown_op_rejected_despite_valid_signature(self) -> None:
# A correctly-signed token whose op isn't in the allow-list must
# still be refused — the schema guard is independent of provenance.
token = sign_request(LaunchRequest(op="rm-rf", bottle_id="b1"), self.secret)
with self.assertRaises(BrokerAuthError):
verify_request(token, self.secret)
class TestStubBroker(unittest.TestCase):
def setUp(self) -> None:
self.secret = secrets.token_bytes(16)
self.broker = StubBroker(self.secret)
def test_submit_launch_records_verified_request(self) -> None:
req = LaunchRequest(op="launch", bottle_id="b1", source_ip="10.243.0.1")
got = self.broker.submit(sign_request(req, self.secret))
self.assertEqual(req, got)
self.assertEqual(["b1"], [r.bottle_id for r in self.broker.launched])
self.assertEqual([], self.broker.torn_down)
def test_submit_teardown_records(self) -> None:
self.broker.submit(
sign_request(LaunchRequest(op="teardown", bottle_id="b1"), self.secret)
)
self.assertEqual(["b1"], [r.bottle_id for r in self.broker.torn_down])
def test_submit_forged_token_is_fail_closed(self) -> None:
forged = sign_request(
LaunchRequest(op="launch", bottle_id="b1"), secrets.token_bytes(16)
)
with self.assertRaises(BrokerAuthError):
self.broker.submit(forged)
self.assertEqual([], self.broker.launched) # nothing acted on
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,227 @@
"""Unit tests for the orchestrator HTTP control plane (PRD 0070).
Mostly exercises the pure `dispatch()` (socket-free, like the supervise
server tests), plus one real-socket round-trip to prove the handler wiring.
"""
from __future__ import annotations
import json
import secrets
import tempfile
import threading
import unittest
import urllib.request
from pathlib import Path
from bot_bottle.orchestrator.broker import StubBroker
from bot_bottle.orchestrator.control_plane import dispatch, make_server
from bot_bottle.orchestrator.registry import RegistryStore
from bot_bottle.orchestrator.service import Orchestrator
def _body(obj: object) -> bytes:
return json.dumps(obj).encode()
def _orchestrator(db_path: Path) -> Orchestrator:
store = RegistryStore(db_path)
store.migrate()
secret = secrets.token_bytes(16)
return Orchestrator(store, StubBroker(secret), secret)
class TestDispatch(unittest.TestCase):
def setUp(self) -> None:
self._tmp = tempfile.TemporaryDirectory()
self.orch = _orchestrator(Path(self._tmp.name) / "r.db")
def tearDown(self) -> None:
self._tmp.cleanup()
def test_health(self) -> None:
status, payload = dispatch(self.orch, "GET", "/health", b"")
self.assertEqual(200, status)
self.assertEqual("ok", payload["status"])
def test_register_returns_id_and_token(self) -> None:
status, payload = dispatch(
self.orch, "POST", "/bottles", _body({"source_ip": "10.243.0.1"})
)
self.assertEqual(201, status)
self.assertTrue(payload["bottle_id"])
self.assertTrue(payload["identity_token"])
def test_register_requires_source_ip(self) -> None:
status, _ = dispatch(self.orch, "POST", "/bottles", _body({}))
self.assertEqual(400, status)
def test_register_rejects_bad_json(self) -> None:
status, _ = dispatch(self.orch, "POST", "/bottles", b"{not json")
self.assertEqual(400, status)
def test_list_redacts_identity_token(self) -> None:
dispatch(self.orch, "POST", "/bottles", _body({"source_ip": "10.243.0.1"}))
status, payload = dispatch(self.orch, "GET", "/bottles", b"")
self.assertEqual(200, status)
bottles = payload["bottles"]
assert isinstance(bottles, list)
self.assertEqual(1, len(bottles))
first = bottles[0]
assert isinstance(first, dict)
self.assertNotIn("identity_token", first)
self.assertEqual("10.243.0.1", first["source_ip"])
def test_attribute_ok_and_forbidden(self) -> None:
_, reg = dispatch(
self.orch, "POST", "/bottles", _body({"source_ip": "10.243.0.5"})
)
token = reg["identity_token"]
ok_status, ok = dispatch(
self.orch, "POST", "/attribute",
_body({"source_ip": "10.243.0.5", "identity_token": token}),
)
self.assertEqual(200, ok_status)
self.assertEqual(reg["bottle_id"], ok["bottle_id"])
bad_status, _ = dispatch(
self.orch, "POST", "/attribute",
_body({"source_ip": "10.243.0.5", "identity_token": "nope"}),
)
self.assertEqual(403, bad_status)
def test_attribute_requires_both_fields(self) -> None:
status, _ = dispatch(
self.orch, "POST", "/attribute", _body({"source_ip": "10.243.0.5"})
)
self.assertEqual(400, status)
def test_delete(self) -> None:
_, reg = dispatch(
self.orch, "POST", "/bottles", _body({"source_ip": "10.243.0.1"})
)
bid = reg["bottle_id"]
status, payload = dispatch(self.orch, "DELETE", f"/bottles/{bid}", b"")
self.assertEqual(200, status)
self.assertEqual(True, payload["torn_down"])
self.assertEqual([], self.orch.registry.all())
def test_delete_missing_404(self) -> None:
status, _ = dispatch(self.orch, "DELETE", "/bottles/ghost", b"")
self.assertEqual(404, status)
def test_unknown_route_404(self) -> None:
status, _ = dispatch(self.orch, "GET", "/nope", b"")
self.assertEqual(404, status)
def test_trailing_slash_normalized(self) -> None:
status, _ = dispatch(self.orch, "GET", "/health/", b"")
self.assertEqual(200, status)
def test_gateway_status_unconfigured(self) -> None:
status, payload = dispatch(self.orch, "GET", "/gateway", b"")
self.assertEqual(200, status)
self.assertEqual(False, payload["configured"])
def test_launch_stores_policy_and_resolve_returns_it(self) -> None:
_, reg = dispatch(
self.orch, "POST", "/bottles",
_body({"source_ip": "10.243.0.5", "policy": '{"a":1}'}),
)
status, payload = dispatch(
self.orch, "POST", "/resolve",
_body({"source_ip": "10.243.0.5", "identity_token": reg["identity_token"]}),
)
self.assertEqual(200, status)
self.assertEqual(reg["bottle_id"], payload["bottle_id"])
self.assertEqual('{"a":1}', payload["policy"])
def test_resolve_forbidden_on_bad_token(self) -> None:
dispatch(self.orch, "POST", "/bottles", _body({"source_ip": "10.243.0.5"}))
status, _ = dispatch(
self.orch, "POST", "/resolve",
_body({"source_ip": "10.243.0.5", "identity_token": "nope"}),
)
self.assertEqual(403, status)
def test_put_policy_updates_live(self) -> None:
_, reg = dispatch(self.orch, "POST", "/bottles", _body({"source_ip": "10.243.0.1"}))
bid = reg["bottle_id"]
status, payload = dispatch(
self.orch, "PUT", f"/bottles/{bid}/policy", _body({"policy": '{"v":9}'})
)
self.assertEqual(200, status)
self.assertEqual(True, payload["updated"])
_, resolved = dispatch(
self.orch, "POST", "/resolve",
_body({"source_ip": "10.243.0.1", "identity_token": reg["identity_token"]}),
)
self.assertEqual('{"v":9}', resolved["policy"])
def test_put_policy_missing_bottle_404(self) -> None:
status, _ = dispatch(
self.orch, "PUT", "/bottles/ghost/policy", _body({"policy": "{}"})
)
self.assertEqual(404, status)
def test_put_policy_requires_string(self) -> None:
_, reg = dispatch(self.orch, "POST", "/bottles", _body({"source_ip": "10.243.0.1"}))
status, _ = dispatch(
self.orch, "PUT", f"/bottles/{reg['bottle_id']}/policy", _body({})
)
self.assertEqual(400, status)
def test_resolve_without_token_by_source_ip(self) -> None:
_, reg = dispatch(
self.orch, "POST", "/bottles",
_body({"source_ip": "10.243.0.5", "policy": "P"}),
)
status, payload = dispatch(
self.orch, "POST", "/resolve", _body({"source_ip": "10.243.0.5"})
)
self.assertEqual(200, status)
self.assertEqual(reg["bottle_id"], payload["bottle_id"])
self.assertEqual("P", payload["policy"])
def test_resolve_requires_source_ip(self) -> None:
status, _ = dispatch(self.orch, "POST", "/resolve", _body({}))
self.assertEqual(400, status)
class TestServerRoundTrip(unittest.TestCase):
def test_http_register_health_attribute(self) -> None:
tmp = tempfile.TemporaryDirectory()
self.addCleanup(tmp.cleanup)
orch = _orchestrator(Path(tmp.name) / "r.db")
server = make_server(orch, "127.0.0.1", 0)
self.addCleanup(server.server_close)
thread = threading.Thread(target=server.serve_forever, daemon=True)
thread.start()
self.addCleanup(server.shutdown)
host, port = server.server_address[0], server.server_address[1]
base = f"http://{host}:{port}"
reg = json.load(urllib.request.urlopen(
urllib.request.Request(
f"{base}/bottles", data=_body({"source_ip": "10.243.0.7"}),
method="POST", headers={"Content-Type": "application/json"},
), timeout=5,
))
self.assertTrue(reg["bottle_id"])
health = json.load(urllib.request.urlopen(f"{base}/health", timeout=5))
self.assertEqual("ok", health["status"])
attr = json.load(urllib.request.urlopen(
urllib.request.Request(
f"{base}/attribute",
data=_body({"source_ip": "10.243.0.7", "identity_token": reg["identity_token"]}),
method="POST", headers={"Content-Type": "application/json"},
), timeout=5,
))
self.assertEqual(reg["bottle_id"], attr["bottle_id"])
if __name__ == "__main__":
unittest.main()
@@ -0,0 +1,100 @@
"""Unit tests for the Docker launch broker (PRD 0070). Docker is mocked."""
from __future__ import annotations
import secrets
import unittest
from unittest.mock import Mock, patch
from bot_bottle.orchestrator.broker import (
BrokerAuthError,
LaunchRequest,
sign_request,
)
from bot_bottle.orchestrator.docker_broker import (
BOTTLE_ID_LABEL,
DockerBroker,
DockerBrokerError,
container_name,
rm_argv,
run_argv,
)
class TestArgv(unittest.TestCase):
def test_run_argv_uses_only_static_fields(self) -> None:
req = LaunchRequest(op="launch", bottle_id="b1", image_ref="busybox")
argv = run_argv(req)
self.assertEqual(["docker", "run"], argv[:2])
self.assertIn("--name", argv)
self.assertIn(container_name("b1"), argv)
self.assertIn(f"{BOTTLE_ID_LABEL}=b1", argv)
self.assertEqual("busybox", argv[-1]) # image is the terminal arg
def test_rm_argv(self) -> None:
req = LaunchRequest(op="teardown", bottle_id="b1")
self.assertEqual(["docker", "rm", "--force", container_name("b1")], rm_argv(req))
def test_container_name_is_prefixed(self) -> None:
name = container_name("b1")
self.assertTrue(name.startswith("bot-bottle-orch-"))
self.assertTrue(name.endswith("b1"))
class TestDockerBroker(unittest.TestCase):
def setUp(self) -> None:
self.secret = secrets.token_bytes(16)
self.broker = DockerBroker(self.secret)
def _submit(self, req: LaunchRequest) -> None:
self.broker.submit(sign_request(req, self.secret))
def test_launch_invokes_docker_run(self) -> None:
req = LaunchRequest(op="launch", bottle_id="b1", image_ref="busybox")
with patch.object(self.broker, "_docker", return_value=Mock(returncode=0, stderr="")) as m:
self._submit(req)
m.assert_called_once()
self.assertEqual(run_argv(req), m.call_args.args[0])
def test_launch_without_image_raises_and_skips_docker(self) -> None:
req = LaunchRequest(op="launch", bottle_id="b1", image_ref="")
with patch.object(self.broker, "_docker") as m:
with self.assertRaises(DockerBrokerError):
self._submit(req)
m.assert_not_called()
def test_launch_docker_failure_raises(self) -> None:
req = LaunchRequest(op="launch", bottle_id="b1", image_ref="busybox")
with patch.object(self.broker, "_docker", return_value=Mock(returncode=1, stderr="boom")):
with self.assertRaises(DockerBrokerError):
self._submit(req)
def test_teardown_invokes_docker_rm(self) -> None:
req = LaunchRequest(op="teardown", bottle_id="b1")
with patch.object(self.broker, "_docker", return_value=Mock(returncode=0, stderr="")) as m:
self._submit(req)
self.assertEqual(rm_argv(req), m.call_args.args[0])
def test_teardown_is_idempotent_on_missing_container(self) -> None:
req = LaunchRequest(op="teardown", bottle_id="b1")
absent = Mock(returncode=1, stderr="Error: No such container: bot-bottle-orch-b1")
with patch.object(self.broker, "_docker", return_value=absent):
self._submit(req) # must not raise
def test_teardown_other_failure_raises(self) -> None:
req = LaunchRequest(op="teardown", bottle_id="b1")
with patch.object(self.broker, "_docker", return_value=Mock(returncode=1, stderr="daemon down")):
with self.assertRaises(DockerBrokerError):
self._submit(req)
def test_forged_token_never_touches_docker(self) -> None:
req = LaunchRequest(op="launch", bottle_id="b1", image_ref="busybox")
forged = sign_request(req, secrets.token_bytes(16))
with patch.object(self.broker, "_docker") as m:
with self.assertRaises(BrokerAuthError):
self.broker.submit(forged)
m.assert_not_called()
if __name__ == "__main__":
unittest.main()
+129
View File
@@ -0,0 +1,129 @@
"""Unit tests for the consolidated Docker gateway (PRD 0070). Docker mocked."""
from __future__ import annotations
import unittest
from unittest.mock import Mock, patch
from bot_bottle.orchestrator.gateway import (
GATEWAY_NAME,
DockerGateway,
GatewayError,
)
_RUN_DOCKER = "bot_bottle.orchestrator.gateway.run_docker"
def _proc(returncode: int = 0, stdout: str = "", stderr: str = "") -> Mock:
return Mock(returncode=returncode, stdout=stdout, stderr=stderr)
class TestDockerGateway(unittest.TestCase):
def setUp(self) -> None:
self.sc = DockerGateway("bot-bottle-sidecars:latest")
def test_default_name(self) -> None:
self.assertEqual(GATEWAY_NAME, self.sc.name)
def test_is_running_reads_docker_ps(self) -> None:
with patch(_RUN_DOCKER, return_value=_proc(stdout=self.sc.name + "\n")):
self.assertTrue(self.sc.is_running())
with patch(_RUN_DOCKER, return_value=_proc(stdout="")):
self.assertFalse(self.sc.is_running())
def test_ensure_running_is_noop_when_already_up(self) -> None:
with patch(_RUN_DOCKER, return_value=_proc(stdout=self.sc.name)) as m:
self.sc.ensure_running()
# Only the is_running() ps probe — no rm / run.
self.assertEqual(1, m.call_count)
def test_ensure_running_starts_the_singleton_when_absent(self) -> None:
calls: list[list[str]] = []
def fake(argv: list[str]) -> Mock:
calls.append(argv)
return _proc(stdout="") if argv[:2] == ["docker", "ps"] else _proc()
with patch(_RUN_DOCKER, side_effect=fake):
self.sc.ensure_running()
runs = [c for c in calls if c[:2] == ["docker", "run"]]
self.assertEqual(1, len(runs))
self.assertIn(self.sc.name, runs[0])
self.assertIn("bot-bottle-sidecars:latest", runs[0])
def test_ensure_running_raises_on_docker_failure(self) -> None:
def fake(argv: list[str]) -> Mock:
if argv[:2] == ["docker", "ps"]:
return _proc(stdout="")
if argv[:2] == ["docker", "run"]:
return _proc(returncode=1, stderr="boom")
return _proc()
with patch(_RUN_DOCKER, side_effect=fake):
with self.assertRaises(GatewayError):
self.sc.ensure_running()
def test_stop_is_idempotent_on_missing(self) -> None:
absent = _proc(returncode=1, stderr="Error: No such container: x")
with patch(_RUN_DOCKER, return_value=absent):
self.sc.stop() # must not raise
def test_stop_raises_on_other_failure(self) -> None:
with patch(_RUN_DOCKER, return_value=_proc(returncode=1, stderr="daemon down")):
with self.assertRaises(GatewayError):
self.sc.stop()
class TestDockerGatewayBuild(unittest.TestCase):
def setUp(self) -> None:
self.sc = DockerGateway() # defaults to the real bundle image + dockerfile
def test_image_exists_reads_docker_inspect(self) -> None:
with patch(_RUN_DOCKER, return_value=_proc(returncode=0)):
self.assertTrue(self.sc.image_exists())
with patch(_RUN_DOCKER, return_value=_proc(returncode=1)):
self.assertFalse(self.sc.image_exists())
def test_ensure_built_is_noop_when_image_present(self) -> None:
with patch(_RUN_DOCKER, return_value=_proc(returncode=0)) as m:
self.sc.ensure_built()
self.assertEqual(1, m.call_count) # only the image-inspect probe
def test_ensure_built_builds_from_dockerfile_when_missing(self) -> None:
calls: list[list[str]] = []
def fake(argv: list[str]) -> Mock:
calls.append(argv)
if argv[:3] == ["docker", "image", "inspect"]:
return _proc(returncode=1) # missing
return _proc()
with patch(_RUN_DOCKER, side_effect=fake):
self.sc.ensure_built()
builds = [c for c in calls if c[:2] == ["docker", "build"]]
self.assertEqual(1, len(builds))
self.assertIn(self.sc.image_ref, builds[0])
self.assertIn("-f", builds[0])
self.assertTrue(any(a.endswith("Dockerfile.sidecars") for a in builds[0]))
def test_ensure_built_noop_when_no_dockerfile(self) -> None:
sc = DockerGateway("busybox", dockerfile=None)
with patch(_RUN_DOCKER) as m:
sc.ensure_built()
m.assert_not_called()
def test_ensure_built_raises_on_build_failure(self) -> None:
def fake(argv: list[str]) -> Mock:
if argv[:3] == ["docker", "image", "inspect"]:
return _proc(returncode=1)
return _proc(returncode=1, stderr="build boom")
with patch(_RUN_DOCKER, side_effect=fake):
with self.assertRaises(GatewayError):
self.sc.ensure_built()
if __name__ == "__main__":
unittest.main()
+138
View File
@@ -0,0 +1,138 @@
"""Unit tests for the orchestrator bottle registry + attribution (PRD 0070)."""
from __future__ import annotations
import tempfile
import unittest
from pathlib import Path
from bot_bottle.orchestrator.registry import (
BottleRecord,
RegistryStore,
new_identity_token,
)
class TestRegistryStore(unittest.TestCase):
def setUp(self) -> None:
self._tmp = tempfile.TemporaryDirectory()
self.db = Path(self._tmp.name) / "registry.db"
self.store = RegistryStore(self.db)
self.store.migrate()
def tearDown(self) -> None:
self._tmp.cleanup()
def test_register_mints_id_and_token(self) -> None:
rec = self.store.register("10.243.0.1")
self.assertTrue(rec.bottle_id)
self.assertTrue(rec.identity_token)
self.assertEqual("active", rec.state)
self.assertEqual(rec, self.store.get(rec.bottle_id))
def test_identity_tokens_are_unique(self) -> None:
tokens = {new_identity_token() for _ in range(200)}
self.assertEqual(200, len(tokens))
a = self.store.register("10.243.0.1")
b = self.store.register("10.243.0.3")
self.assertNotEqual(a.identity_token, b.identity_token)
def test_all_and_deregister(self) -> None:
a = self.store.register("10.243.0.1")
b = self.store.register("10.243.0.3")
self.assertEqual({a.bottle_id, b.bottle_id}, {r.bottle_id for r in self.store.all()})
self.assertTrue(self.store.deregister(a.bottle_id))
self.assertEqual([b.bottle_id], [r.bottle_id for r in self.store.all()])
def test_deregister_missing_is_false(self) -> None:
self.assertFalse(self.store.deregister("nope"))
def test_explicit_id_replaces(self) -> None:
self.store.register("10.243.0.1", bottle_id="fixed", metadata="a")
self.store.register("10.243.0.9", bottle_id="fixed", metadata="b")
rec = self.store.get("fixed")
assert rec is not None
self.assertEqual("10.243.0.9", rec.source_ip)
self.assertEqual("b", rec.metadata)
self.assertEqual(1, len(self.store.all()))
def test_attribute_success_requires_ip_and_token(self) -> None:
rec = self.store.register("10.243.0.1")
got = self.store.attribute("10.243.0.1", rec.identity_token)
assert got is not None
self.assertEqual(rec.bottle_id, got.bottle_id)
def test_attribute_wrong_token_denied(self) -> None:
self.store.register("10.243.0.1")
self.assertIsNone(self.store.attribute("10.243.0.1", "wrong-token"))
def test_attribute_unknown_ip_denied(self) -> None:
rec = self.store.register("10.243.0.1")
self.assertIsNone(self.store.attribute("10.243.9.9", rec.identity_token))
def test_attribute_empty_token_denied(self) -> None:
self.store.register("10.243.0.1")
self.assertIsNone(self.store.attribute("10.243.0.1", ""))
def test_attribute_ambiguous_ip_denied(self) -> None:
# Two active bottles on one source IP is a misconfiguration — deny
# rather than guess (fail-closed), even with a valid token.
a = self.store.register("10.243.0.1", bottle_id="a")
self.store.register("10.243.0.1", bottle_id="b")
self.assertIsNone(self.store.attribute("10.243.0.1", a.identity_token))
def test_state_persists_across_reopen(self) -> None:
rec = self.store.register("10.243.0.1")
reopened = RegistryStore(self.db)
got = reopened.get(rec.bottle_id)
assert got is not None
self.assertEqual(rec, got)
# Attribution works against the reopened (durable) store too.
self.assertIsNotNone(reopened.attribute("10.243.0.1", rec.identity_token))
def test_redacted_hides_token(self) -> None:
rec = BottleRecord(
bottle_id="x", source_ip="10.243.0.1", identity_token="secret"
)
self.assertNotIn("identity_token", rec.redacted())
self.assertEqual("10.243.0.1", rec.redacted()["source_ip"])
def test_register_with_policy_resolves_via_attribution(self) -> None:
rec = self.store.register("10.243.0.1", policy='{"allow":["x"]}')
self.assertEqual('{"allow":["x"]}', rec.policy)
got = self.store.attribute("10.243.0.1", rec.identity_token)
assert got is not None
self.assertEqual('{"allow":["x"]}', got.policy)
def test_set_policy_updates_live(self) -> None:
rec = self.store.register("10.243.0.1")
self.assertEqual("", rec.policy)
self.assertTrue(self.store.set_policy(rec.bottle_id, '{"v":2}'))
got = self.store.get(rec.bottle_id)
assert got is not None
self.assertEqual('{"v":2}', got.policy)
def test_set_policy_missing_is_false(self) -> None:
self.assertFalse(self.store.set_policy("ghost", "{}"))
def test_policy_defaults_empty_and_persists(self) -> None:
rec = self.store.register("10.243.0.1")
got = RegistryStore(self.db).get(rec.bottle_id)
assert got is not None
self.assertEqual("", got.policy)
def test_by_source_ip_returns_single_active(self) -> None:
rec = self.store.register("10.243.0.1")
got = self.store.by_source_ip("10.243.0.1")
assert got is not None
self.assertEqual(rec.bottle_id, got.bottle_id)
def test_by_source_ip_unknown_or_ambiguous_denied(self) -> None:
self.assertIsNone(self.store.by_source_ip("10.243.9.9")) # unknown
self.store.register("10.243.0.1", bottle_id="a")
self.store.register("10.243.0.1", bottle_id="b")
self.assertIsNone(self.store.by_source_ip("10.243.0.1")) # ambiguous
if __name__ == "__main__":
unittest.main()
+144
View File
@@ -0,0 +1,144 @@
"""Unit tests for the Orchestrator launch lifecycle (PRD 0070)."""
from __future__ import annotations
import secrets
import tempfile
import unittest
from pathlib import Path
from bot_bottle.orchestrator.broker import LaunchBroker, LaunchRequest, StubBroker
from bot_bottle.orchestrator.registry import RegistryStore
from bot_bottle.orchestrator.service import Orchestrator
from bot_bottle.orchestrator.gateway import Gateway
class _FailingBroker(LaunchBroker):
"""Verifies the token like any broker, then fails the launch — to
exercise the orchestrator's registry rollback."""
def _launch(self, req: LaunchRequest) -> None:
raise RuntimeError("launch failed")
def _teardown(self, req: LaunchRequest) -> None:
pass
class _FakeGateway(Gateway):
"""In-memory gateway for wiring tests."""
def __init__(self) -> None:
self.name = "fake-gateway"
self.ensured = 0
self.built = 0
self._running = False
def ensure_built(self) -> None:
self.built += 1
def ensure_running(self) -> None:
self.ensured += 1
self._running = True
def is_running(self) -> bool:
return self._running
def stop(self) -> None:
self._running = False
class TestOrchestrator(unittest.TestCase):
def setUp(self) -> None:
self._tmp = tempfile.TemporaryDirectory()
self.secret = secrets.token_bytes(16)
self.store = RegistryStore(Path(self._tmp.name) / "r.db")
self.store.migrate()
self.broker = StubBroker(self.secret)
self.orch = Orchestrator(self.store, self.broker, self.secret)
def tearDown(self) -> None:
self._tmp.cleanup()
def test_launch_registers_and_brokers_signed_request(self) -> None:
rec = self.orch.launch_bottle("10.243.0.1", image_ref="sha256:abc", slot=2)
self.assertIsNotNone(self.store.get(rec.bottle_id))
self.assertEqual(1, len(self.broker.launched))
req = self.broker.launched[0]
self.assertEqual("launch", req.op)
self.assertEqual(rec.bottle_id, req.bottle_id)
self.assertEqual("10.243.0.1", req.source_ip)
self.assertEqual("sha256:abc", req.image_ref)
self.assertEqual(2, req.slot)
def test_launch_then_attribute(self) -> None:
rec = self.orch.launch_bottle("10.243.0.3")
got = self.orch.attribute("10.243.0.3", rec.identity_token)
assert got is not None
self.assertEqual(rec.bottle_id, got.bottle_id)
self.assertIsNone(self.orch.attribute("10.243.0.3", "wrong-token"))
def test_teardown_brokers_and_deregisters(self) -> None:
rec = self.orch.launch_bottle("10.243.0.1")
self.assertTrue(self.orch.teardown_bottle(rec.bottle_id))
self.assertIsNone(self.store.get(rec.bottle_id))
self.assertEqual(["teardown"], [r.op for r in self.broker.torn_down])
def test_teardown_unknown_is_false(self) -> None:
self.assertFalse(self.orch.teardown_bottle("ghost"))
def test_launch_with_policy_is_resolvable(self) -> None:
rec = self.orch.launch_bottle("10.243.0.1", policy='{"routes":[]}')
got = self.orch.attribute("10.243.0.1", rec.identity_token)
assert got is not None
self.assertEqual('{"routes":[]}', got.policy)
def test_set_policy_live_reload(self) -> None:
rec = self.orch.launch_bottle("10.243.0.3")
self.assertTrue(self.orch.set_policy(rec.bottle_id, '{"x":1}'))
got = self.orch.attribute("10.243.0.3", rec.identity_token)
assert got is not None
self.assertEqual('{"x":1}', got.policy)
def test_set_policy_unknown_is_false(self) -> None:
self.assertFalse(self.orch.set_policy("ghost", "{}"))
def test_resolve_by_source_ip_without_token(self) -> None:
rec = self.orch.launch_bottle("10.243.0.1", policy="P")
got = self.orch.resolve("10.243.0.1") # network-layer, no token
assert got is not None
self.assertEqual(rec.bottle_id, got.bottle_id)
self.assertEqual("P", got.policy)
def test_resolve_with_token_stays_strict(self) -> None:
rec = self.orch.launch_bottle("10.243.0.3")
self.assertIsNotNone(self.orch.resolve("10.243.0.3", rec.identity_token))
self.assertIsNone(self.orch.resolve("10.243.0.3", "wrong-token"))
def test_launch_rolls_back_registry_on_broker_failure(self) -> None:
orch = Orchestrator(self.store, _FailingBroker(self.secret), self.secret)
with self.assertRaises(RuntimeError):
orch.launch_bottle("10.243.0.9")
self.assertEqual([], self.store.all()) # no orphan
def test_gateway_unconfigured_by_default(self) -> None:
self.assertEqual({"configured": False}, self.orch.gateway_status())
self.orch.ensure_gateway() # no-op, must not raise
def test_gateway_wired_and_ensured(self) -> None:
sc = _FakeGateway()
orch = Orchestrator(self.store, self.broker, self.secret, gateway=sc)
self.assertEqual(
{"configured": True, "name": "fake-gateway", "running": False},
orch.gateway_status(),
)
orch.ensure_gateway()
self.assertEqual(1, sc.built) # ensure_gateway builds first,
self.assertEqual(1, sc.ensured) # then runs
self.assertEqual(
{"configured": True, "name": "fake-gateway", "running": True},
orch.gateway_status(),
)
if __name__ == "__main__":
unittest.main()
+76
View File
@@ -0,0 +1,76 @@
"""Unit tests for the gateway-side PolicyResolver (PRD 0070). HTTP mocked."""
from __future__ import annotations
import json
import unittest
import urllib.error
from unittest.mock import MagicMock, patch
from bot_bottle.policy_resolver import PolicyResolveError, PolicyResolver
_URLOPEN = "bot_bottle.policy_resolver.urllib.request.urlopen"
def _resp(payload: object) -> MagicMock:
"""A urlopen() return value: a context manager whose read() yields JSON."""
m = MagicMock()
m.__enter__.return_value.read.return_value = json.dumps(payload).encode()
return m
def _http_error(code: int) -> urllib.error.HTTPError:
return urllib.error.HTTPError("http://x/resolve", code, "err", {}, None) # type: ignore[arg-type]
class TestPolicyResolver(unittest.TestCase):
def setUp(self) -> None:
self.r = PolicyResolver("http://orch:8080")
def test_resolve_returns_policy(self) -> None:
with patch(_URLOPEN, return_value=_resp({"bottle_id": "b1", "policy": "P"})):
self.assertEqual("P", self.r.resolve("10.243.0.1", "tok"))
def test_resolve_always_fetches_fresh(self) -> None:
# No cache — every resolve hits the orchestrator so revocations /
# policy changes are honored immediately.
with patch(_URLOPEN, return_value=_resp({"policy": "P"})) as m:
self.r.resolve("10.243.0.1", "tok")
self.r.resolve("10.243.0.1", "tok")
self.assertEqual(2, m.call_count)
def test_unattributed_403_is_none_fail_closed(self) -> None:
with patch(_URLOPEN, side_effect=_http_error(403)):
self.assertIsNone(self.r.resolve("10.243.0.9", "tok"))
def test_other_http_error_raises(self) -> None:
with patch(_URLOPEN, side_effect=_http_error(500)):
with self.assertRaises(PolicyResolveError):
self.r.resolve("10.243.0.1", "tok")
def test_unreachable_raises(self) -> None:
with patch(_URLOPEN, side_effect=urllib.error.URLError("refused")):
with self.assertRaises(PolicyResolveError):
self.r.resolve("10.243.0.1", "tok")
def test_missing_policy_field_is_empty(self) -> None:
with patch(_URLOPEN, return_value=_resp({"bottle_id": "b1"})):
self.assertEqual("", self.r.resolve("10.243.0.1", "tok"))
def test_posts_source_ip_and_token(self) -> None:
with patch(_URLOPEN, return_value=_resp({"policy": "P"})) as m:
self.r.resolve("10.243.0.7", "the-token")
req = m.call_args.args[0]
self.assertTrue(req.full_url.endswith("/resolve"))
sent = json.loads(req.data)
self.assertEqual("10.243.0.7", sent["source_ip"])
self.assertEqual("the-token", sent["identity_token"])
def test_resolve_without_token_sends_empty(self) -> None:
with patch(_URLOPEN, return_value=_resp({"policy": "P"})) as m:
self.r.resolve("10.243.0.7") # token optional
self.assertEqual("", json.loads(m.call_args.args[0].data)["identity_token"])
if __name__ == "__main__":
unittest.main()
+5 -44
View File
@@ -8,7 +8,8 @@ from datetime import datetime, timezone
from pathlib import Path
from bot_bottle import supervise
from bot_bottle import supervise_types as sv_types
from bot_bottle.paths import host_db_path
from tests.unit import use_bottle_root
from bot_bottle.audit_store import AuditStore
from bot_bottle.queue_store import QueueStore
from bot_bottle.supervise import (
@@ -21,7 +22,6 @@ from bot_bottle.supervise import (
TOOL_EGRESS_ALLOW,
TOOL_GITLEAKS_ALLOW,
archive_proposal,
host_db_path,
list_pending_proposals,
read_audit_entries,
read_proposal,
@@ -123,20 +123,7 @@ class TestQueueIO(unittest.TestCase):
self._tmp.cleanup()
def _patch_home(self, fake_home: Path):
original_sv = supervise.bot_bottle_root
original_svt = sv_types.bot_bottle_root
def fake_root() -> Path:
return fake_home / ".bot-bottle"
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
sv_types.bot_bottle_root = fake_root # type: ignore[assignment]
def restore() -> None:
supervise.bot_bottle_root = original_sv # type: ignore[assignment]
sv_types.bot_bottle_root = original_svt # type: ignore[assignment]
return restore
return use_bottle_root(fake_home / ".bot-bottle")
def test_write_and_read_proposal(self):
p = _proposal()
@@ -240,20 +227,7 @@ class TestAuditLog(unittest.TestCase):
self._tmp.cleanup()
def _patch_home(self, fake_home: Path):
original_sv = supervise.bot_bottle_root
original_svt = sv_types.bot_bottle_root
def fake_root() -> Path:
return fake_home / ".bot-bottle"
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
sv_types.bot_bottle_root = fake_root # type: ignore[assignment]
def restore() -> None:
supervise.bot_bottle_root = original_sv # type: ignore[assignment]
sv_types.bot_bottle_root = original_svt # type: ignore[assignment]
return restore
return use_bottle_root(fake_home / ".bot-bottle")
def test_write_then_read_single_entry(self):
e = AuditEntry(
@@ -400,20 +374,7 @@ class TestSupervisePrepare(unittest.TestCase):
self._tmp.cleanup()
def _patch_home(self, fake_home: Path):
original_sv = supervise.bot_bottle_root
original_svt = sv_types.bot_bottle_root
def fake_root() -> Path:
return fake_home / ".bot-bottle"
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
sv_types.bot_bottle_root = fake_root # type: ignore[assignment]
def restore() -> None:
supervise.bot_bottle_root = original_sv # type: ignore[assignment]
sv_types.bot_bottle_root = original_svt # type: ignore[assignment]
return restore
return use_bottle_root(fake_home / ".bot-bottle")
def test_prepare_creates_queue(self):
plan = _StubSupervise().prepare("dev", self.stage_dir)
+3 -16
View File
@@ -12,7 +12,7 @@ from pathlib import Path
from unittest.mock import patch
from bot_bottle import supervise
from bot_bottle import supervise_types as sv_types
from tests.unit import use_bottle_root
from bot_bottle.audit_store import AuditStore
from bot_bottle.cli import supervise as supervise_cli
from bot_bottle.queue_store import QueueStore
@@ -51,24 +51,11 @@ def _proposal(slug: str = "dev", tool: str = TOOL_EGRESS_ALLOW) -> Proposal:
class _FakeHomeMixin:
"""Patch supervise.bot_bottle_root to a temp dir for the test."""
"""Point bot_bottle_root at a temp dir (via BOT_BOTTLE_ROOT) for the test."""
def _setup_fake_home(self):
self._tmp = tempfile.TemporaryDirectory(prefix="supervise-test.")
original_sv = supervise.bot_bottle_root
original_svt = sv_types.bot_bottle_root
def fake_root() -> Path:
return Path(self._tmp.name) / ".bot-bottle"
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
sv_types.bot_bottle_root = fake_root # type: ignore[assignment]
def restore() -> None:
supervise.bot_bottle_root = original_sv # type: ignore[assignment]
sv_types.bot_bottle_root = original_svt # type: ignore[assignment]
self._restore_home = restore
self._restore_home = use_bottle_root(Path(self._tmp.name) / ".bot-bottle")
QueueStore("").migrate()
AuditStore().migrate()
+11 -12
View File
@@ -11,12 +11,13 @@ from __future__ import annotations
import contextlib
import io
import os
import tempfile
import unittest
from pathlib import Path
from unittest import mock
from bot_bottle import supervise
from tests.unit import use_bottle_root
from bot_bottle.cli import supervise as supervise_cli
from bot_bottle.log import Die, die
@@ -39,18 +40,16 @@ class TestDieCarriesMessage(unittest.TestCase):
class _FakeHomeMixin:
"""Point supervise.bot_bottle_root (what _write_crash_log resolves
through) at a temp dir so the crash log doesn't touch the real
~/.bot-bottle."""
"""Point bot_bottle_root (what _write_crash_log resolves through) at a
temp dir so the crash log doesn't touch the real ~/.bot-bottle."""
def _setup_fake_home(self):
self._tmp = tempfile.TemporaryDirectory(prefix="supervise-crash-test.")
self._orig_root = supervise.bot_bottle_root
self._root = Path(self._tmp.name) / ".bot-bottle"
supervise.bot_bottle_root = lambda: self._root # type: ignore[assignment]
self._restore_root = use_bottle_root(self._root)
def _teardown_fake_home(self):
supervise.bot_bottle_root = self._orig_root # type: ignore[assignment]
self._restore_root()
self._tmp.cleanup()
@@ -127,11 +126,11 @@ class TestWriteCrashLog(_FakeHomeMixin, unittest.TestCase):
# OSError and the helper must fall back to a tempfile.
bad = Path(self._tmp.name) / "not-a-dir"
bad.write_text("x")
supervise.bot_bottle_root = lambda: bad # type: ignore[assignment]
try:
raise RuntimeError("explode2")
except RuntimeError as e:
path = supervise_cli._write_crash_log(e)
with mock.patch.dict(os.environ, {"BOT_BOTTLE_ROOT": str(bad)}):
try:
raise RuntimeError("explode2")
except RuntimeError as e:
path = supervise_cli._write_crash_log(e)
self.assertTrue(path.exists())
self.assertIn("explode2", path.read_text())
+2 -1
View File
@@ -12,6 +12,7 @@ from unittest.mock import patch
from bot_bottle import supervise
from bot_bottle.audit_store import AuditStore
from bot_bottle.paths import bot_bottle_root
from bot_bottle.queue_store import QueueStore
from bot_bottle.supervise import (
AuditEntry,
@@ -39,7 +40,7 @@ def _proposal() -> Proposal:
class TestPathHelpers(unittest.TestCase):
def test_bot_bottle_root(self) -> None:
self.assertTrue(str(supervise.bot_bottle_root()).endswith(".bot-bottle"))
self.assertTrue(str(bot_bottle_root()).endswith(".bot-bottle"))
class TestReadMalformed(unittest.TestCase):
+3 -19
View File
@@ -10,6 +10,8 @@ import unittest
from pathlib import Path
from unittest.mock import patch
from tests.unit import use_bottle_root
# The server module loads `supervise` via same-directory import inside
# the container (Dockerfile.supervise WORKDIRs into /app). For tests
@@ -19,10 +21,8 @@ sys.path.insert(0, str(Path(__file__).resolve().parent.parent.parent / "bot_bott
import supervise as _sv # noqa: E402 # type: ignore
import queue_store as _qs # noqa: E402 # type: ignore
import audit_store as _as # noqa: E402 # type: ignore
import supervise_types as svt_flat # noqa: E402 # type: ignore
from bot_bottle import supervise_server # noqa: E402
from bot_bottle import supervise_types as svt_pkg # noqa: E402
from bot_bottle.supervise_server import (
ERR_INTERNAL,
ERR_INVALID_PARAMS,
@@ -279,23 +279,7 @@ class TestHandleToolsCall(unittest.TestCase):
self._tmp.cleanup()
def _patch_home(self, fake_home: Path):
original_sv = _sv.bot_bottle_root
original_flat = svt_flat.bot_bottle_root
original_pkg = svt_pkg.bot_bottle_root
def fake_root() -> Path:
return fake_home / ".bot-bottle"
_sv.bot_bottle_root = fake_root # type: ignore[assignment]
svt_flat.bot_bottle_root = fake_root # type: ignore[assignment]
svt_pkg.bot_bottle_root = fake_root # type: ignore[assignment]
def restore() -> None:
_sv.bot_bottle_root = original_sv # type: ignore[assignment]
svt_flat.bot_bottle_root = original_flat # type: ignore[assignment]
svt_pkg.bot_bottle_root = original_pkg # type: ignore[assignment]
return restore
return use_bottle_root(fake_home / ".bot-bottle")
def _respond_when_proposal_appears(self, status: str, notes: str = "") -> threading.Thread:
"""Background thread: poll the queue for a fresh proposal, write a