64 Commits

Author	SHA1	Message	Date
didericis	eab9d15130	refactor(docker): fold run_docker into backend/docker/util; drop the docker_cmd shim ... docker_cmd.py existed as a top-level module solely so the orchestrator's docker components could share run_docker without dragging the (then-heavy) backend layer in. Now that backend/__init__ and backend/docker/__init__ are thin, importing backend.docker.util costs 6 modules instead of 76, so the shim's whole reason to exist is gone. Move run_docker into backend/docker/util.py (where the other docker subprocess primitives live) and delete docker_cmd.py. Backend siblings import it via `from .util import run_docker`; the two docker-specific orchestrator modules (docker_broker, rotate_ca) import `from ..backend.docker.util import run_docker`. No import cycle (backend.docker.util pulls nothing from orchestrator); orchestrator.__main__ stays lean at 28 modules. run_docker patch targets are unchanged (tests patch it in the importing module). Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 15:58:23 -04:00
didericis	b96a8b44e0	refactor(docker): rename OrchestratorService -> DockerInfraService, move to backend/docker ... OrchestratorService wasn't the orchestrator — it's the host-side lifecycle of the docker infra *container* (the one that runs the Orchestrator). It read as "the orchestrator as a service" and lived in orchestrator/lifecycle.py, while its siblings (macOS MacosInfraService, Firecracker infra_vm) live under their backend package. Rename it DockerInfraService and move it to backend/docker/infra.py alongside the docker backend, with its docker-only constants (INFRA_*/ORCHESTRATOR_* image + container names, daemon list, mount paths). orchestrator/lifecycle.py keeps only the backend-neutral pieces the other infra services share — DEFAULT_PORT, DEFAULT_STARTUP_TIMEOUT_SECONDS, OrchestratorStartError, source_hash — which macOS / firecracker / client still import from there. backend/docker/infra.py imports those (backend -> orchestrator is an allowed direction). Renamed the unit test to test_docker_infra.py. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 15:26:52 -04:00
didericis	d41236c376	refactor(util): make render_diff generic (caller supplies side titles); move to root util ... Generalize render_diff so it isn't supervise-flavored: the "(current)" / "(proposed)" side titles are no longer baked in — callers pass `before_title` and `after_label` (required). Move it from orchestrator/supervisor/util.py to the base bot_bottle.util alongside sha256_hex, and delete the now-empty supervisor/util.py. The supervise callsite (Orchestrator.supervise_respond) assigns before_title="current", after_label="proposed"; the facade no longer re-exports render_diff (callers import from bot_bottle.util). Behavior at the supervise callsite is unchanged. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 15:09:27 -04:00
didericis	db6a151803	refactor(supervise): hash the proposed file inside Proposal.new; sha256_hex → root util ... Move sha256_hex out of orchestrator/supervisor/util.py to the root bot_bottle.util (it's a generic string hash, not supervise-specific), and have Proposal.new take the proposed file contents and compute current_file_hash itself instead of every caller passing sha256_hex(proposed_file). Every call site set current_file_hash to the hash of the proposed file, and nothing reads the field except the DB round-trip (schema + _row_to_proposal + from_dict, all untouched), so folding the hash into the factory is behavior-preserving and removes the boilerplate. Callers now pass just the file; the orchestrator no longer imports sha256_hex at all. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 15:03:37 -04:00
didericis	8abccf7ffe	refactor(supervise): make Supervisor a service class the orchestrator calls ... Turn the loose queue/audit functions in orchestrator/supervisor/queue.py into methods on a concrete Supervisor service. The Orchestrator now owns an injectable `self._supervisor` and calls `self._supervisor.write_proposal(...)` etc., instead of module-level free functions — the supervise dependency is explicit and mockable, and a Supervisor can be scoped to a `db_path` (defaults to the host DB) so tests can point it at a temp database. - Supervisor (in the package __init__) drops the ABC and gains write_proposal, read_proposal, list_pending_proposals, list_all_pending_proposals, write_response, read_response, archive_all_proposals, write_audit_entry, read_audit_entries, plus the launch-time prepare(). - render_diff / sha256_hex are pure and stateless, so they move to orchestrator/supervisor/util.py (re-exported from the facade) rather than becoming methods. - queue.py is deleted; service.py + the supervise tests call through a Supervisor instance. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 14:52:17 -04:00
didericis	27a122e24b	refactor(supervise): drop obsolete queue helpers (archive_proposal, wait_for_response, audit_dir/path) ... Remove the supervise queue helpers that no live path uses anymore — they're mechanisms the PRD-0070 / idempotent-poll migration superseded: - archive_proposal (+ QueueStore.archive_proposal): the poll stopped archiving on read (#469 review), so single-proposal archiving has no caller; decided proposals drop off the pending list via their response row and are reaped in bulk by archive_all_proposals on teardown/reconcile. - wait_for_response: the data plane polls the queue over the control-plane RPC now, so nothing block-waits on it. - audit_dir / audit_log_path: file-based audit paths, replaced by the SQLite AuditStore. Also drops the tests that covered those obsolete helpers. Kept the prod-unused read accessors list_pending_proposals and read_audit_entries: those aren't dead mechanisms — they're the read side of live write paths that many tests use to verify real queue/audit behavior. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 14:36:32 -04:00
didericis	3e2cbcab88	refactor(supervise): fold Supervise into the facade as Supervisor; drop dead flat-fallbacks ... Move the `Supervise` lifecycle out of its own `orchestrator/supervisor/ supervise.py` and into the package `__init__`, renaming the class to `Supervisor`. Callers now import it from `bot_bottle.orchestrator.supervisor` alongside the queue surface it belongs with. Remove the dead `try/except ImportError` flat-import fallbacks from the package-only store modules (db_store, audit_store, config_store, queue_store) and image_cache. Those fallbacks existed for when the store files were flat-copied into the gateway; post-PRD-0070 the data plane never opens the DB, so these modules are only ever imported as part of the package. The two gateway data-plane files that may still be loaded flat (egress_addon_core, git_gate_render) keep their fallbacks. Full unit suite green (2251). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 14:24:06 -04:00
didericis	44e2b5a897	refactor(supervise): split the supervise plane by tier; per-service store managers ... Give each service its own store package + manager, and cut the supervise module along the control/data-plane boundary so nothing in the shared layer reaches up into the orchestrator. Stores, by owner: - bot_bottle/store/ keeps only the shared base (DbStore, migrations) and the concrete stores that aren't service-owned (audit_store, config_store). - bot_bottle/orchestrator/store/ now houses the orchestrator-owned stores — queue_store (supervise queue), secret_store, config_store — plus a new orchestrator store_manager that migrates them (composing audit/config downward from the base). The old shared store_manager is gone. Supervise plane, by tier: - bot_bottle/supervisor/ (NEUTRAL, importable by every tier including the gateway): types.py (the Proposal/Response/AuditEntry wire types + the tool/ status/poll constants + the shared daemon constants moved out of supervise.py) and plan.py (SupervisePlan, a pure DTO). - bot_bottle/orchestrator/supervisor/ (orchestrator-only): queue.py (the queue/audit I/O wrappers + render_diff + sha256_hex) and supervise.py (the Supervise lifecycle that stages the DB via the store manager). Its __init__ re-exports the neutral vocabulary so orchestrator-side callers import from one place. The gateway now imports only bot_bottle.supervisor.types (never bot_bottle.supervise), so the data plane holds no code dependency on the orchestrator — it reaches the queue over the control-plane RPC. This removes the circular import that moving queue_store under orchestrator introduced (supervise -> orchestrator -> service -> supervise). supervise_types.py -> supervisor/types.py; supervise.py deleted (split). Full unit suite green (2251). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 14:15:04 -04:00
didericis	f77023db1d	refactor(gateway): move the data-plane daemons into a bot_bottle.gateway package ... Separate the gateway (data plane) from the orchestrator (control plane) at the module level. The gateway runtime files move out of the package root — and the backend-neutral Gateway lifecycle ABC + GATEWAY_* constants move out of orchestrator/ — into a new bot_bottle/gateway/ package: gateway/__init__.py (was orchestrator/gateway.py: Gateway ABC + consts + rotate_gateway_ca) gateway/gateway_init.py (the PID-1 daemon supervisor) gateway/egress_addon.py, egress_addon_core.py, egress_dlp_config.py, dlp_detectors.py (the egress mitmproxy daemon) gateway/git_http_backend.py (the git-http daemon) gateway/git_gate_render.py (the git-gate pre-receive rendering) gateway/supervise_server.py (the supervise MCP daemon) gateway/policy_resolver.py (the data-plane control-plane RPC client) orchestrator/ now holds only control-plane files. The shared plan/types/auth layer (egress.py=EgressPlan, git_gate.py=GitGatePlan, supervise.py, supervise_types.py, control_auth.py) and the launch-time git-gate provisioning helpers stay at root, so orchestrator/ and backend/ still own them. Because these daemons are invoked as `python3 -m bot_bottle.<name>`, loaded flat by mitmproxy, and referenced in Dockerfile.gateway, the move updates more than Python imports: the `-m` invocations (firecracker/macOS infra scripts), the Dockerfile.gateway addon shim + ENTRYPOINT, gateway_init's _DAEMONS module paths, and the git-gate CGI heredocs all now point at bot_bottle.gateway.*. No behavior change; full unit suite green (2251). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 13:25:10 -04:00
didericis	d7a58e52fd	refactor(store): consolidate the SQLite store family into bot_bottle.store ... Move the DbStore base (db_store), the shared schema-migration base (migrations / TableMigrations), the concrete stores (audit_store, config_store, queue_store), and StoreManager out of the package root into a bot_bottle/store/ package, so persistence lives in one place rather than scattered across the root. Callers use direct submodule imports (from bot_bottle.store.store_manager import StoreManager). Inside the moved modules the relative imports point back up to their non-store siblings (..paths, ..supervise_types) while co-moved siblings stay package-local; the flat-import fallbacks are untouched. The orchestrator's own stores (config_store, secret_store, registry) stay in orchestrator/ and repoint to the moved DbStore / TableMigrations. No behavior change; full unit suite green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 12:53:52 -04:00
didericis	dfce3d9505	refactor(gateway): move DockerGateway to the backend layer, drop the dead standalone-gateway path ... The `DockerGateway` container-lifecycle impl now lives in `backend/docker/gateway.py` (the shape backend gateway classes will share); `orchestrator/gateway.py` keeps only the backend-neutral pieces (the `Gateway` ABC, constants, `rotate_gateway_ca`). Delete the standalone-gateway path it was the only consumer of. `--gateway` on `python -m bot_bottle.orchestrator` was invoked nowhere — the production docker flow runs the gateway data plane inside the combined `bot-bottle-infra` container via `OrchestratorService`, never this class. Removing it takes with it `Orchestrator.ensure_gateway()` and the `gateway` ctor arg; `gateway_status()` becomes a stub reporting `configured: false` so the documented `GET /gateway` control-plane route keeps its contract. Fix a latent bug the move surfaced: `launch.py` read the shared gateway CA via `docker exec bot-bottle-orch-gateway`, a container the consolidated flow never creates — so the agent CA install was reaching a nonexistent name. Read it from `bot-bottle-infra` (INFRA_NAME) instead. Repoint the affected tests to the new module; drop the unit test + fake that covered the deleted standalone wiring. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 12:39:58 -04:00
didericis-claude	d8b61b3658	fix(supervise): make the response poll idempotent and slow it to 250ms ... The control-plane supervise poll archived the proposal on the same call that returned the decision, so a dropped connection lost the operator's decision (the retry saw `unknown`). Poll no longer archives — a re-poll returns the same decision. Decided proposals still drop off the operator's pending list (a response row exists); their rows are reaped when the bottle is torn down or reconciled. Also raise the grace-window poll interval from 50ms to 250ms now that each poll is an HTTP round trip rather than a local file read. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 06:26:06 +00:00
didericis-claude	1db2a9eb67	feat(control-plane): role-scoped signed tokens so the gateway can't drive operator routes ... Review follow-up on #469: the data plane held the same control-plane secret that authorizes every route, so a compromised egress/git-gate could queue a supervise proposal AND approve it (or rewrite policy, read injected tokens) — the (source_ip, identity_token) checks attribute the *bottle*, not the caller. Replace the single shared bearer secret with role-scoped, HMAC-signed tokens (compact HS256 JWTs, stdlib-only — no new dependency): * new `control_auth` mints/verifies `{role}` tokens; roles are `gateway` (data plane) and `cli` (host operator/launcher). * the orchestrator holds only the signing *key* and verifies; `dispatch` gates each route by role — `gateway` reaches /resolve + /supervise/ {propose,poll}, everything else is `cli`-only (401 unauthenticated, 403 wrong role). * the gateway is handed a pre-minted `gateway` token it cannot rewrite into `cli`; the host CLI mints its own `cli` token from the host key. * `gateway_init` scopes the signing key to the orchestrator process and the gateway token to the data-plane daemons, so even in the combined infra container a compromised data-plane daemon never sees the key. Launchers (docker gateway + infra, macOS infra) inject the minted token(s); Firecracker stays open behind its nft boundary. Open mode (no key) still grants full `cli` access — the fail-visible fallback for tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 05:22:56 +00:00
didericis-claude	2c496dc3d0	fix(supervise): reach the queue over RPC, get bot-bottle.db off the data plane ... PRD 0070's rule — only the orchestrator opens bot-bottle.db; the data plane reaches state through the control-plane RPC — was not in force. Three data-plane daemons held a direct read-write handle on the shared SQLite file: the supervise MCP server, the egress DLP addon (the most attack-exposed process, TLS-bumping hostile traffic), and the git-gate pre-receive hook. An RCE in any of them could read every bottle's plaintext identity_token and forge attribution fleet-wide (issue #469). Add the agent half of the supervise flow to the control plane: POST /supervise/propose -> queue a proposal, 201 {proposal_id} POST /supervise/poll -> non-blocking decision poll, 200 {status,...} Both attribute the caller by (source_ip, identity_token) exactly like /resolve — never a caller-supplied slug — so a bottle can only ever queue or read its own proposals even if the data plane is compromised. A decided poll archives server-side, preserving the archive-after-read contract. Data plane: the supervise server, egress addon, and git-gate hook now queue/poll through PolicyResolver.propose_supervise / poll_supervise instead of opening the DB. supervise_server keeps its ~30s grace window by polling the RPC; egress keeps its safelist keyed by resolved bottle; the git-gate hook gets (source_ip, identity_token) from the CGI env. Packaging: drop the DB bind-mount and SUPERVISE_DB_PATH from the data-plane containers/VMs (docker gateway + infra, macOS infra, firecracker infra). The orchestrator remains the sole opener of the one file via BOT_BOTTLE_ROOT / host_db_path(). Update PRD 0070: the rule is now in force; remove the transitional caveat. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 02:49:51 +00:00
didericis-codex	26d0f5e3b2	refactor(secrets): keep ciphertext format minimal	2026-07-23 21:39:07 +00:00
didericis-codex	854f6b5696	fix(secrets): recover encrypted tokens on all backends	2026-07-23 21:39:07 +00:00
didericis-claude	9014c07b86	feat(secrets): encrypt egress tokens at rest with per-bottle ENV_VAR_SECRET ... Implements the interim secret-provider design (PRD prd-new-secret-provider): each agent receives a random ENV_VAR_SECRET injected into its container env at launch. The host uses this key to encrypt each egress auth token value (HMAC-SHA256 CTR mode, stdlib-only) and store it in a new bottled_agent_secrets table (one row per env var, key column plaintext for auditing). The key never touches the DB. On infra container restart the in-memory token map is lost. launch_consolidated now calls _reprovision_running_bottles after ensure_running: for each registered bottle still alive on the gateway network it execs `printenv ENV_VAR_SECRET` into the agent container and posts the result to the new POST /bottles/<id>/reprovision_gateway control-plane endpoint, which decrypts the stored rows and restores _tokens — no manual intervention needed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-07-23 21:38:33 +00:00
didericis-claude	8a1b833aaa	fix(gateway): persist mitmproxy CA on the host, not a named volume (#450) ... The shared gateway self-generates a mitmproxy CA that every bottle installs to trust its TLS interception. It was persisted on a Docker named volume, which survives `docker rm` but is silently wiped by `docker volume prune` / `docker system prune --volumes` during routine host maintenance. When that happens the gateway mints a fresh CA on restart, and every already-running bottle fails the TLS handshake even after it re-resolves and reconnects to the moved gateway — a re-attachment blocker distinct from #443/#445. Move CA persistence to a host bind-mount under the app-data root (`bot_bottle_root()/gateway-ca`, via `host_gateway_ca_dir()`), mirroring how the shared DB and control-plane token already live on the host. Docker never prunes a path under the root, and it stays inspectable + rotatable from the host. mitmproxy already adopts an existing CA and generates one only on first run, so the bind-mount gives adopt-existing/generate-on-first-run for free. Add an explicit rollover path: `rotate_gateway_ca()` clears the persisted CA so the next start remints it, and `python -m bot_bottle.orchestrator.rotate_ca` wires that together with dropping the running gateway container (whose mitmproxy still holds the old CA in memory). Rotation stays an operator action — it doesn't auto-re-provision running bottles, which re-attach to pick up the new anchor. Scope: the Docker infra/gateway path (the "infra container" in the report). The macOS (`container`-only volume) and Firecracker (VM-attached ext4) backends persist the CA differently and are unaffected. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-21 15:01:38 -04:00
didericis-claude	14ff4fe186	fix(lifecycle): build gateway before infra and fix orchestrator port mapping ... - `_build_images()` now builds Dockerfile.gateway → Dockerfile.orchestrator → Dockerfile.infra in order; Dockerfile.infra starts FROM bot-bottle-gateway so the base must exist on clean hosts. - Publish mapping corrected from `self.port:self.port` to `self.port:DEFAULT_PORT` (8099) — gateway_init hardcodes the orchestrator on port 8099 inside the container, so the host-side published port must map to that fixed internal port. - BOT_BOTTLE_ORCHESTRATOR_URL inside the container now always points to 127.0.0.1:8099, not self.port, since gateway daemons reach the orchestrator over loopback at the fixed internal port. - Update test_ensure_running_builds_both_images → _all_images for the new three-step build sequence; add test_publish_maps_host_port_to_fixed_internal_port to lock in the port-mapping fix. Addresses the P1 findings from the didericis-codex review on PR #432.	2026-07-21 04:54:40 +00:00
didericis-claude	2f45f5afec	feat(docker): consolidate to single infra container under gateway_init supervise tree ... Collapses the two-container Docker model (gateway + orchestrator) into one bot-bottle-infra container, matching the macOS and Firecracker backends. - Dockerfile.infra: now a shared gateway+orchestrator base (COPY bot_bottle from orchestrator build, no CMD override) - Dockerfile.infra.fc: new Firecracker-specific layer (buildah/crun/netavark) - gateway_init: adds orchestrator daemon with _OPT_IN_DAEMONS gating so it only starts when BOT_BOTTLE_GATEWAY_DAEMONS explicitly includes it - orchestrator/lifecycle: OrchestratorService manages one infra container; builds orchestrator (intermediate) then infra; live source bind-mounted at /bot-bottle-src with PYTHONPATH so the subprocess uses the checkout - backend/consolidated_util: extracts provision_bottle + teardown_consolidated shared across all three backends; removes duplication in docker/fc/macos consolidated_launch modules - firecracker/infra_vm: builds four images (orchestrator→gateway→infra→infra.fc) - All unit tests updated and passing (1878 tests) - PRD status: Draft → Active	2026-07-21 04:54:40 +00:00
didericis-claude	1f192d785a	refactor(config_store): replace generic key/value table with explicit schema ... Switch orchestrator_config from a generic key/text store to a typed single-row table with a dedicated teardown_timeout_seconds REAL column. The CHECK (id = 1) constraint enforces at most one config row. Remove TEARDOWN_TIMEOUT_KEY constant; update OrchestratorConfigStore to expose get/set/delete_teardown_timeout_seconds() instead of the generic get/set/delete(key). Resolving the DB value in resolve_teardown_timeout() no longer needs a float() conversion since the column is already REAL.	2026-07-21 00:40:59 -04:00
didericis-claude	0b36c3eb48	fix: make orchestrator teardown timeout configurable (closes #435) ... Resolves via ENV VAR -> orchestrator DB config -> default (30 s, up from 5 s): BOT_BOTTLE_ORCHESTRATOR_TEARDOWN_TIMEOUT_SECONDS teardown_timeout_seconds key in new orchestrator_config table (bot-bottle.db) New OrchestratorConfigStore (same DbStore/TableMigrations pattern as the registry) stores the DB-level setting. resolve_teardown_timeout() implements the priority chain and is called at stack.callback registration time in all three backends (macos_container, docker, firecracker).	2026-07-21 00:40:59 -04:00
didericis	e4d53fd360	fix(orchestrator): reap registry rows whose bottle is no longer running ... A registry row only ever left the registry two ways: an explicit teardown_bottle (the launcher's cleanup callback) or the same-IP supersede sweep in register(). Neither runs when the launching CLI dies hard, so the row outlives its container. That orphan is not inert. Source IPs are recycled by the backend's DHCP and by_source_ip fail-closes on ambiguity, so a leftover row at a reused address resolves no policy at all for the next bottle that lands there — and a bottle with no policy denies every host, which surfaces to the agent as "host X is not in the allowlist" for hosts that were never the problem. Add reap_absent/reconcile and call it from the macOS launch path before registering, so each launch self-heals the registry. Restores the invariant the data plane needs: at most one active row per live address, and none for a dead one. The second half matters as much as the first — when several rows claim a *live* address the newest wins and the rest are swept, otherwise a recycled address stays ambiguous, which is exactly the bricked state. The host supplies the live set because the orchestrator runs inside the infra container and cannot see the backend. A grace window exempts rows younger than it, so reconciliation cannot race a bottle still coming up, and a reconcile failure is logged rather than blocking an otherwise-fine launch. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-21 03:27:16 +00:00
didericis-claude	32e85de16f	fix(db): close SQLite connections explicitly to suppress ResourceWarning on Python 3.13 ... `sqlite3.Connection.__exit__` only commits/rolls back a transaction — it does not close the connection. Python 3.13 (the Nix env on the KVM runner) emits `ResourceWarning: unclosed database` for every connection GC'd without an explicit close, producing noisy output in the coverage job. Add `DbStore._connection()`, a `contextmanager` that calls `self._connect()`, wraps it in the existing transaction context manager, and closes the connection in a `finally` block. Change all `with self._connect() as conn:` call sites in `db_store.py`, `audit_store.py`, `queue_store.py`, and `orchestrator/registry.py` to `with self._connection() as conn:`. `_connect()` remains as the per-subclass hook (RegistryStore overrides it to set `busy_timeout`); `_connection()` delegates to `self._connect()` so the override is respected.	2026-07-19 02:16:10 +00:00
didericis	ea1fbeeaa0	refactor(gateway): fail closed without an orchestrator URL; drop stale single-tenant comments ... Follow-ups from the #402 review of the single-tenant data-plane teardown. - egress_entrypoint.sh: refuse to launch mitmdump when BOT_BOTTLE_ORCHESTRATOR_URL is unset, so the fail-closed guarantee no longer rests solely on mitmproxy's errorcheck addon exiting on the addon's load-time raise. A misconfigured gateway can never come up as a bare TLS-bumping open proxy with no policy. - orchestrator/gateway.py: ensure_running() raises GatewayError on an empty orchestrator URL — a URL-less launch would only crash-loop the now-resolver-only daemons (egress raises, git-http exits 1, supervise exits 2). The env-injection branch is now unconditional. - Drop stale "single-tenant" / "reads routes.yaml" comments in gateway.py and egress_entrypoint.sh, and the /etc/egress/routes.yaml layout line in Dockerfile.gateway. - Tests: gateway fixtures supply an orchestrator URL; add a refuse-without-URL test and assert the URL env is injected. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UoEZHDjv84ChoZbozQERhJ	2026-07-17 22:34:28 -04:00
didericis	5c526860bc	fix(test): resolve the remaining review findings on the control-plane auth test ... Addresses the 5 lower-priority findings left as follow-up in the earlier review, now that each has a concrete answer: - Add gateway_name: str = GATEWAY_NAME to OrchestratorService.__init__ (mirrors the existing orchestrator_name param) and thread it through _gateway(). Deletes the test's _IsolatedOrchestratorService subclass, which existed only to override a private method for this one kwarg — any caller needing gateway-name isolation can now use the public constructor. Backward compatible: every existing caller constructs OrchestratorService with keyword args and a sensible default is kept. - Give the test its own fixed image tags (bot-bottle-orchestrator:itest, bot-bottle-gateway:itest) instead of the production :latest ones. _running_image_is_current() keys gateway staleness off the image tag's ID, not per-instance identity, so rebuilding the shared :latest tag from whatever's on disk during a test run could make a real host's running production gateway look stale and get force-recreated. Fixed tags (not per-run-suffixed, so they don't accumulate) fully decouple the two. - setUp -> setUpClass/tearDownClass: all 5 tests are read-only checks against the same running control plane, so one shared container lifecycle replaces 5 (each of which paid its own container-start + image-build + health-poll cycle). Cuts the file's wall-clock roughly 4x (11.5s -> 2.9-4.3s) and, combined with the network-rm cleanup from the previous commit, means one cleanup instead of five. - Reuse OrchestratorClient (bot_bottle/orchestrator/client.py) instead of a hand-rolled urllib helper — the test now exercises the same request/response code path the real host CLI uses, rather than a private copy that could silently drift from it. - Add the chown workaround test_multitenant_isolation.py already needed for this exact bind-mount: the orchestrator container has no USER directive, so it writes the registry DB as root into the throwaway host_root; chown it back before tempdir cleanup so that doesn't raise PermissionError on native Linux Docker (no UID remap, unlike Docker Desktop's macOS VM). Verified: ran the suite twice in a row (idempotency — fixed image tags don't accumulate, 5/5 pass both times, 2.99-4.33s each), the real ~/.bot-bottle/control-plane-token is untouched, zero leaked networks or containers after either run, exactly 2 :itest images (not growing), the full orchestrator unit suite (93 tests) and the sibling docker gateway/broker integration tests still pass. pyright clean, pylint 10.00/10 on both changed files. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-17 17:00:31 -04:00
didericis	ca91fc4d91	fix(security): authenticate the orchestrator control plane (issue #400 Critical) ... The control plane had no caller authentication — dispatch() routed every request straight through, and the only guards anywhere were the (source_ip, identity_token) checks inside /resolve and /attribute, which authenticate the *bottle* a request is about, not the *caller*. On the consolidated backends the agent shares a network with the control-plane port (macOS: the infra container runs 8099 on the agent's --internal network; docker: the orchestrator container sits on the shared gateway network), so a prompt-injected agent could GET /bottles (enumerate siblings + policies), PUT /bottles/<id>/policy (rewrite its own egress allowlist to allow-all), POST /resolve (read the injected upstream credentials it's never meant to see), and POST /supervise/respond (self-approve its own proposals). Every README guarantee collapsed to the agent's discretion. Fix: a per-host control-plane secret required on every route but GET /health, compared with hmac.compare_digest. It is held only by the trusted callers and never handed to an agent: - minted + persisted 0600 at <root>/control-plane-token (paths.host_control_plane_token); - injected as $BOT_BOTTLE_CONTROL_PLANE_TOKEN into the orchestrator + gateway containers via bare `--env NAME` (value inherited from the launch process, so it never lands on argv or in `container/docker inspect`); - presented by the gateway's PolicyResolver (reads the env) on /resolve, and by the host CLI's OrchestratorClient (reads the host file) on every call. The agent container is never given the env var or the host file, so from a bottle every /bottles*, /resolve, /attribute, and /supervise/* call now returns 401 — closing the enumeration, allowlist-rewrite, credential-lift, and self-approval. The existing (source_ip, identity_token) checks stay as defense-in-depth. Enforced when configured: macOS + docker inject the secret (→ enforced). With no secret set the server runs open and warns loudly at startup — a fail-visible fallback for the unit suite and for Firecracker, whose port-scoped nft already blocks agents from 8099 (wiring the secret into its infra-VM init is a clean fast-follow, left out here to avoid churning the prebuilt-artifact hash). Verified end-to-end on real Apple Container: infra comes up healthy, the host CLI (with the secret) lists bottles while an unauthenticated GET /bottles gets 401, all five issue-#400 attacks from inside the agent get 401, and egress policy still works (200 allowed / 403 denied) — proving the gateway authenticates to /resolve with the secret. 1829 unit tests pass, pyright clean, pylint 9.91. Refs #400. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-17 05:15:16 -04:00
didericis	4a607ad098	refactor(macos): one infra container (control plane + gateway), fixes shared-DB races ... Adopts the firecracker infra-VM pattern for macOS: the orchestrator control plane and the gateway data plane now run in a SINGLE Apple container instead of two. Apple Containers are lightweight VMs with separate kernels, so the prior two-container design had both guests writing one bot-bottle.db over virtiofs, where fcntl locks are not coherent across kernels — concurrent writes (the orchestrator's registry vs the gateway supervise daemon's queue) could corrupt it. One container = one kernel = coherent locking. The DB moves onto a container-only Apple volume (bot-bottle-mac-db), never bind-mounted from the host, so no host process opens the live file either. The host CLI already reaches registry + supervise state over the control-plane HTTP surface (cli/supervise.py uses OrchestratorClient), exactly as firecracker's VM-only DB requires. Two simplifications fall out of the single container: - No DNS dance: the control plane and gateway daemons reach each other over 127.0.0.1, so the orchestrator-before-gateway ordering (a workaround for Apple having no container DNS) is gone, along with the moved-IP recreate logic it needed. - Net -243 lines. Mechanics: the infra container runs from the gateway image with the control-plane source bind-mounted read-only (like the docker orchestrator, so a code change needs no rebuild) and a small sh -c init that starts both processes (mirrors firecracker's _infra_init). Also implements the macOS backend's ensure_orchestrator() and adds it to discover_orchestrator_url, so operator tools (supervise) can bring up / find the control plane on demand — previously the macOS backend died with "no orchestrator control plane". Verified end-to-end on real Apple Container 1.0.0: the single infra container comes up healthy (one address for control plane + gateway), both processes run, the DB is written on the container-only volume, host-side supervise works over HTTP, and a registered agent gets 200 for an allowed host / 403 for a denied one. 1824 unit tests pass with `container` absent (CI parity), pyright clean, pylint 9.89. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-17 04:14:14 -04:00
didericis	c69642e568	feat(macos): consolidated per-host gateway for the Apple backend (PRD 0070) ... Re-enables the macos-container backend on the shared per-host orchestrator + gateway, replacing the per-bottle companion container removed in #385. This is the last backend in PRD 0070's roadmap. Apple Container 1.0.0 forced three departures from the docker shape, each verified against the live CLI (findings recorded in the networking spike): - No `--ip`. The address is DHCP-assigned and knowable only once the container runs, so the order inverts: gateway up -> run agent -> read its address -> register. The identity token is minted by registration and therefore cannot be in the agent's run-time env; it rides the proxy URL applied at `container exec` time (bare `--env` names keep it off argv). - No container DNS. The gateway can only be handed the control plane's IP, so the orchestrator starts first and the gateway is pointed at its address. - No `network connect`. Networks are fixed at run time, so the shared host-only network is created up front; per-bottle networks would restart the gateway on every launch and defeat the consolidation. The agent runs with `--cap-drop CAP_NET_RAW`: Apple grants NET_RAW by default, which would let an agent forge a neighbour's source address on the shared segment. NET_ADMIN is already absent, so this closes the source-address half of PRD 0070's attribution invariant. Verified end-to-end on real Apple Container 1.0.0: both images build, the control plane comes up healthy, the gateway reaches it by IP, and a registered agent gets 200 for a host in its routes and 403 for one outside them. Bring-up is idempotent — a second launch does not churn the singletons. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-17 01:27:40 -04:00
didericis	b1df380ae1	feat(supervise): show the bottle's human slug, not its opaque id ... Consolidated proposals are keyed by the orchestrator-assigned bottle_id, so the supervise TUI was rendering a hex id (e.g. 3601cbe883c2786d) as the bottle name — and the resume hint printed `./cli.py resume <id>`, which resume can't take (it wants the human identity/slug). `supervise_pending` now tags each dict with `bottle_label` — the human slug resolved from registry metadata, falling back to the id when the bottle is gone. `QueuedProposal` carries the label; every display site (list rows, detail view, status lines, resume hint) shows it, while approve/reject still key on `proposal.bottle_slug` (the id). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UoEZHDjv84ChoZbozQERhJ	2026-07-16 22:05:21 -04:00
didericis	5f59df9e10	fix(supervise): resolve approval target by bottle_id, not human slug ... In consolidated mode the supervise server attributes each proposal to the orchestrator-assigned bottle_id and stores that as the proposal's `bottle_slug` (supervise_server `_attributed_config`). But `_record_for_slug` only scanned registry metadata for a matching human slug, so approving an egress proposal 409'd with "bottle <id> is no longer registered; cannot apply the route change" — the id never matched a human slug. Resolve the record by bottle_id first (the consolidated reality), keeping the metadata-slug scan as a fallback for legacy single-tenant proposals. The prior tests keyed proposals by the human slug matching the metadata, exercising only the fallback path — added a test that mirrors production (proposal keyed by bottle_id, distinct human slug in metadata). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UoEZHDjv84ChoZbozQERhJ	2026-07-16 22:01:14 -04:00
didericis	265119d601	fix(supervise): docker gateway shares the one host DB (Step 2d) ... The docker gateway container ran the supervise daemon but bind-mounted no DB and set no SUPERVISE_DB_PATH — so the daemon wrote proposals to a container-local, ephemeral SQLite file, disconnected from the host DB the orchestrator (and now the operator, over HTTP) uses. Same split-DB bug firecracker had; Dockerfile.gateway even documents the mount (`/run/supervise/bot-bottle.db bind-mounted at run time`) that ensure_running never provided. Bind-mount the host DB dir into the gateway at /run/supervise and set SUPERVISE_DB_PATH, so the daemon queues into the same file the orchestrator container opens (its BOT_BOTTLE_ROOT bind-mount) and the operator reaches over the control plane. One DB per host, shared by bind-mounts — docker now on the exact same supervise path as firecracker. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-16 19:01:45 -04:00
didericis	27fe03b612	feat(supervise): host TUI drives approvals over HTTP, not the DB (Step 2b/2c) ... The `bot-bottle supervise` operator TUI read and wrote the queue DB directly and tried a backend-specific live "apply" (which was unwired — it raised). It now talks only to the orchestrator control plane: - OrchestratorClient gains supervise_pending() + supervise_respond(). - discover_orchestrator_url() finds the one running per-host control plane by health-probing the backends' well-known :8099 addresses (docker publishes on loopback; the firecracker infra VM serves it on the orchestrator TAP) — no backend branching in the TUI. - discover_pending/approve/reject call the client; the server does the apply + response + audit atomically. The dead direct-DB apply/audit helpers and the docker/macos applicator imports are gone. - A missing control plane is now a clean one-line error up front, not a mid-curses crash. CLI tests move to mocking the client (the DB-write behaviour they used to assert is now server-side, covered by test_orchestrator_service). Docker's orchestrator-container DB wiring lands next so its /supervise endpoints hit the same shared DB. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-16 18:57:47 -04:00
didericis	9085d6f713	feat(supervise): orchestrator-side operator-approval API (Step 2a) ... The orchestrator owns the single DB *and* the live policy, so operator decisions belong there — applied server-side, reached over HTTP. Adds: GET /supervise/proposals -> pending proposals across bottles POST /supervise/respond -> apply + record an operator decision `supervise_respond` is one atomic server-side op on the one DB: approve/ modify on an egress tool rewrites the bottle's policy (so the gateway serves the new routes on its next /resolve — the live "apply" that was a documented TODO), then writes the queued Response (unblocking the agent's MCP call) and an audit entry. reject records the response + audit only. Fails closed (409) when the proposal is unknown or the bottle was torn down before the operator acted (an egress apply would have no target). This is the server half of unifying every backend onto one HTTP path for supervise; the host TUI (direct-DB today) moves onto this client next. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-16 18:51:55 -04:00
didericis	38bc555dbf	fix(supervise): single migrated DB for firecracker — orchestrator owns supervise tables ... The firecracker supervise MCP daemon 500'd (-32603) on egress-allow/block: it ran with no BOT_BOTTLE_ROOT/SUPERVISE_DB_PATH, so it targeted a stray, unmigrated SQLite file and `write_proposal` hit "no such table: supervise_proposals". Meanwhile the in-VM control plane migrated only the registry table (orchestrator_bottles) into its own DB, and the host operator reads a third, disconnected DB — three files, none shared. Consolidate to one DB per host, owned by the control plane on the persisted registry volume (/var/lib/bot-bottle/db/bot-bottle.db): - orchestrator startup now migrates the supervise queue + audit tables into the same file it migrates the registry into (StoreManager), so the control-plane DB carries every table. - the in-VM supervise daemon is pointed at that same file via SUPERVISE_DB_PATH, so daemon and control plane share one queue. `list-egress-routes` already worked (no DB); egress-allow now queues + waits on the single persisted DB instead of erroring. Validated against a live infra VM: migrate + write_proposal succeeds and the proposal is queued. The host-operator HTTP bridge (so approvals complete from the host, unifying docker onto the same path) is the follow-up. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-16 18:24:45 -04:00
didericis	d4b27ebf1f	feat(gateway): mandatory identity-token attribution on every data plane (PR #354 review) ... Codex review: the /31 TAP doesn't make source IP unspoofable, and the app-layer token was returned by launch but never delivered or enforced, so a spoofed source could select a victim bottle's policy/tokens. Make the token mandatory and deliver it on each attributed plane (anti-spoof landed separately as the network boundary). Enforcement (control plane): - `Orchestrator.resolve` now requires a matching (source_ip, identity_token) pair (constant-time) — no source-IP-only fallback. `/resolve` fail-closes (403) on a missing/empty/mismatched token. Delivery, per plane (the token is `token_urlsafe`, safe in a URL): - egress: proxy credentials (`HTTPS_PROXY=http://bottle:<token>@gw`). The addon reads `Proxy-Authorization` — from the request (HTTP) or captured at the CONNECT for HTTPS tunnels (keyed by client conn, cleared on disconnect) — validates, and strips it (+ the legacy header) before upstream. - git-http: a URL-scoped `http.<gate>/.extraHeader: x-bot-bottle-identity` in the agent's git config (only over the http transport). - supervise: `mcp add --header x-bot-bottle-identity: <token>` (claude + codex); the server reads the header and passes it to resolve. Wiring: thread `ctx.identity_token` onto the firecracker + docker plans and into the agent env/config at launch. Verified on a KVM host: egress with the correct proxy-cred token returns 200 (HTTP and HTTPS/CONNECT), and no-token / wrong-token return 403; a real `cli.py start --backend=firecracker` launch provisions git config + the supervise MCP header and reaches the agent session, all under mandatory enforcement. Fixed a `claude mcp add` arg-order bug (--header must follow the positional name/url) found by that launch. Transparent proxy for tools that ignore proxy env is deferred to a follow-up (see thread); anti-spoof + host firewall remain the fail-closed boundary. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-16 17:02:41 -04:00
didericis-claude	9369fb7de5	feat(firecracker): implement consolidated orchestrator launch (PRD 0070) ... Replace the per-bottle Docker sidecar bundle with the shared per-host orchestrator + gateway, mirroring what the Docker backend already has. - Add `bot_bottle/backend/firecracker/consolidated_launch.py`: `_FirecrackerOrchestratorService` (subclasses `OrchestratorService`, overrides `_gateway()` to return a `DockerGateway` with host port bindings so Firecracker VMs can reach it via their TAP link); `launch_consolidated()` registers the bottle by guest IP (attribution key), provisions git-gate into the shared gateway, and returns the shared CA + orchestrator URL for teardown; `teardown_consolidated()` deregisters and cleans up. - Rewrite `bot_bottle/backend/firecracker/launch.py`: removes the per-bottle sidecar bundle (`_start_sidecar_bundle`, `_stage_git_gate`, etc.) and `_mint_certs`; wires `launch_consolidated()` instead. The VM still sends to `host_tap_ip:PORT` — Docker's PREROUTING DNAT + the nft `ct status dnat accept` rule in the forward chain route the traffic to the shared gateway container. - Extend `DockerGateway` with `host_port_bindings` so the Firecracker gateway publishes its ports on the host (`0.0.0.0:PORT`). - Parameterise `OrchestratorService` with `orchestrator_name` / `orchestrator_label` so Docker and Firecracker orchestrators can coexist on the same host (`bot-bottle-orchestrator` vs `bot-bottle-fc-orchestrator`). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-07-16 14:36:54 -04:00
didericis	09393b354b	refactor(de-sidecar): purge the "sidecar" name from live code, tests, and current docs ... Completes the de-sidecar cleanup: no live code, test, current doc, script, or nix file mentions or is named "sidecar" any more. Only the dated PRD/research docs keep the term as historical record (agreed on the #385 thread). - Rename `sidecar_init.py`→`gateway_init.py` was done earlier; this pass sweeps the remaining descriptive uses: the egress / git-gate / supervise components are the gateway's *daemons*, the shared container is the *gateway*, the old per-bottle container was the *companion container*. - Rename `tests/integration/test_sidecar_bundle_image.py`→`test_gateway_image.py` and its class; update `docs/ci.md` + `tests/README.md` for the renamed/ removed integration tests. - `SIDECAR_PORTS` shell var in `scripts/firecracker-netpool.sh`→`GATEWAY_PORTS`. Full unit suite green (bar the pre-existing `/bin/sleep`-missing env errors in test_gateway_init); docker integration — gateway singleton, broker, real two-bottle multitenant isolation, and the gateway-image build — all pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-14 17:07:56 -04:00
didericis	96b84eb84d	refactor(orchestrator): split conflated sidecar image into orchestrator + gateway images ... One image — `bot-bottle-sidecars:latest`, built from `Dockerfile.sidecars` — served two unrelated roles: the egress/git-gate/supervise *data plane* and the orchestrator *control plane* (which ran the same image with the entrypoint overridden to `python3 -m bot_bottle.orchestrator`). The control plane is stdlib-only, so it needed none of the mitmproxy/git/gitleaks payload it was riding on — while being the most secret-dense process on the host (PRD 0070's "secret concentration"). Split into two purpose-built images: - `Dockerfile.gateway` -> `bot-bottle-gateway:latest` — the data plane (renamed from Dockerfile.sidecars; identical contents). - `Dockerfile.orchestrator` -> `bot-bottle-orchestrator:latest` — a lean `python:3.12-slim` runtime; the bind-mounted `bot_bottle` package supplies the code (so the #381 source-hash recreate semantics are unchanged). `OrchestratorService` now takes distinct `image` (control plane, default `ORCHESTRATOR_IMAGE`) and `gateway_image` (data plane, default `GATEWAY_IMAGE`) instead of feeding one `self.image` to both, and builds the lean image (build-if-missing) before starting the container. The per-bottle bundle constants in `backend/docker/sidecar_bundle.py` now alias the gateway constants so a bundle and the shared gateway can never drift onto different images. The `bot-bottle-sidecars` *image* name and `Dockerfile.sidecars` are gone; the per-bottle *container* name prefix (`bot-bottle-sidecars-<slug>`) is intentionally left for a separate change. Verified end-to-end: both images build; the lean image runs the control plane; `ensure_running` brings up the orchestrator on `bot-bottle-orchestrator:latest` and the gateway on `bot-bottle-gateway:latest` (distinct images) and reports healthy. Closes #384. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-14 16:24:16 -04:00
didericis	910267b8a8	fix(orchestrator): only recreate the orchestrator on a real source change ... ensure_running() unconditionally recreated the bot-bottle-orchestrator container on every bottle launch, added so bind-mounted code changes take effect (the process never reloads). But the orchestrator's per-bottle egress auth tokens live only in its process memory (Orchestrator._tokens), by design never persisted to disk. So starting a second bottle recreated the container and silently dropped the FIRST bottle's already-injected token -- its next authed egress call 403s with "env var EGRESS_TOKEN_0 is unset", and it needs a full restart to recover. Now the orchestrator container is labeled with a content hash of its bind-mounted bot_bottle source, mirroring the gateway's existing image-staleness check. ensure_running only recreates on a hash mismatch (a real code change), so a bottle launch that isn't accompanied by a code change leaves a healthy orchestrator -- and every other active bottle's in-memory tokens -- alone. Fixes #381. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-07-14 04:07:25 -04:00
didericis	ea75fab3b4	fix(orchestrator): supersede a prior active bottle at the same source IP ... register() used INSERT OR REPLACE keyed on bottle_id, so re-registering an existing source_ip with a new bottle_id left TWO active rows for that IP. by_source_ip fail-closes on ambiguity (>1 active -> None), so the reused IP was then bricked to a 403 on egress resolve — even for hosts its own policy allowed. Happy path (fresh IPs, teardown on stop) never hit it, but IP reuse, crash recovery, or a persisted registry DB across an orchestrator restart (now that ensure_running always recreates the container) all could. register() now deletes any other active row at the same source_ip before insert; the fail-closed ambiguity guard stays as defense in depth. Adds a unit test for the supersede, and a docker integration test (test_multitenant_isolation) that drives two bottles through one shared gateway and asserts each gets only its own injected token and its own allowlist — the core PRD 0070 multi-tenancy invariant, previously only verified by hand. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-14 02:38:20 -04:00
didericis	13a8bdd7ca	fix(orchestrator): recreate the orchestrator container on ensure_running ... The orchestrator runs the repo's code bind-mounted, but the Python process loads it at startup and never reloads — and ensure_running reused a healthy-but-stale container. So after a code change (e.g. the token fix), the running orchestrator kept executing OLD control-plane code that dropped the new /bottles 'tokens' field, and egress auth injection stayed broken no matter how many times the gateway image was rebuilt. ensure_running now always recreates the orchestrator container (cheap; the registry DB persists and the current launch re-registers its in-memory state). Combined with the gateway's image-staleness recreate, a fresh 'start' now runs current code end to end. Validated live: register an authed route + in-memory token -> a client at the bottle IP curling through the gateway proxy receives the injected 'Authorization: Bearer <token>' header. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-14 02:38:20 -04:00
didericis	3318bdce28	fix(orchestrator): recreate the gateway when its image is stale ... Container-level counterpart to the ensure_built cache-aware fix. ensure_built now rebuilds the gateway image on a source change, but ensure_running still reused an already-running container built from the OLD image — so a rebuild (even BOT_BOTTLE_NO_CACHE) never took effect on a warm gateway, and it kept running the pre-fix flat daemons (this is why token injection stayed broken after the rebuild). ensure_running now compares the running container's image to the current image and recreates on mismatch. Validated on docker: a stale gateway is detected and recreated with the current image + new egress addon. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-14 02:38:20 -04:00
didericis	b2f61053ad	fix(egress+orchestrator): inject per-bottle auth tokens in the shared gateway ... The cut-over dropped the per-bottle token flow, so an authed egress route on the shared gateway failed with 'env var EGRESS_TOKEN_0 is unset' — the gateway reads the token from its env, but a shared gateway has no per-bottle env. Now the bottle's egress auth tokens travel to the gateway over /resolve and the addon injects from them, mirroring what the per-bottle sidecar's env did: - launch resolves the token values from the host env and hands them to the orchestrator, which holds them IN MEMORY (keyed by bottle_id, never written to the registry DB) and serves them on /resolve; - PolicyResolver.resolve_policy_and_bottle_id + resolve_client_context now return the token map alongside policy + bottle_id (one round-trip); - the egress addon overlays the process env with the bottle's tokens per request and uses that env for auth injection AND DLP — the agent never sees the credential. Secrets stay off disk (validated: /resolve returns the token, the registry DB does not contain it). SecretProvider (#355) is the future hardening. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-14 02:38:20 -04:00
didericis	485d101430	fix(orchestrator): poll for the gateway CA (fresh-start readiness race) ... On a fresh gateway container, mitmproxy writes mitmproxy-ca-cert.pem a beat after mitmdump boots, so ca_cert_pem() read it too early and launch died with 'gateway CA cert not available'. It now polls (up to 30s) until mitmproxy has generated it. Surfaced running the docker backend locally on a cold gateway. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-14 02:38:20 -04:00
didericis	e97366dad5	fix(orchestrator): gateway image builds cache-aware, not build-if-missing ... The cut-over dropped compose (whose `build:` rebuilt the sidecar image on every up), and DockerGateway.ensure_built was build-if-missing — so a change to the gateway's flat sources (egress addon / git-http / policy_resolver / supervise) left a STALE image silently running the old single-tenant daemons (this bit e2e validation). ensure_built now always `docker build`s (cache-aware: a cache check when nothing changed, a real rebuild when sources moved), matching the old compose behavior. BOT_BOTTLE_NO_CACHE forces a full rebuild (parity with #354's `start --no-cache`). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-14 02:38:20 -04:00
didericis	96f75599a1	feat(orchestrator): slice 13d(i) — containerize the orchestrator (validated on docker) ... Real-on-host testing surfaced two issues the unit-mocked slices couldn't: 1. The host (NixOS) firewall DROPS container->host traffic, so a host-process orchestrator is unreachable from the gateway container. Fix: run the orchestrator AS a container on the shared gateway network (PRD 0070's "virtualize the orchestrator") — the gateway reaches it by container name over docker DNS (container<->container, no firewall), and the host CLI reaches it via a published loopback port. 2. The control plane crashed the connection on a dispatch error (e.g. a broker failure) instead of returning 500. Changes: - lifecycle: OrchestratorProcess (host process) -> OrchestratorService (containers). Runs the control plane in the bundle image with the repo bind-mounted (orchestrator is stdlib-only), register-only stub broker so it needs NO docker socket (the backend launches agents; the host manages both containers). Registry DB persists via a host-root mount. ensure_running is an idempotent singleton over both containers. - gateway: BOT_BOTTLE_ORCHESTRATOR_URL is now the orchestrator's *by-name* URL on the shared network (dropped the host.docker.internal hack). - control_plane: _serve wraps dispatch — a failure returns 500, never crashes the connection. - OrchestratorProcess default broker -> stub (register-only) for docker. Validated live end-to-end: both containers up, gateway->orchestrator by name OK, register -> resolve-by-source-IP returns the bottle's policy from inside the gateway. pyright 0 errors; pylint 9.83/10; unit suite green (1760 tests; the 13 test_sidecar_init /bin/sleep errors are pre-existing NixOS-local noise). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-14 02:38:20 -04:00
didericis	404369a3e0	feat(orchestrator): slice 13c(vii) — stable shared gateway CA ... The consolidated gateway TLS-intercepts every bottle, so all agents must trust ONE CA. Rather than host-generate + mount a CA, let the gateway's own mitmproxy generate its CA into a persistent named volume and extract the cert for agents — no host openssl, keeps gateway.py lean. - DockerGateway mounts GATEWAY_CA_VOLUME at /home/mitmproxy/.mitmproxy so the self-generated CA survives container recreation (it must not rotate, or already-launched agents would stop trusting the gateway). - ca_cert_pem(): read the CA cert (PEM) out of the running gateway (docker exec cat mitmproxy-ca-cert.pem) — the agent installs this into its trust store to accept the gateway's bumped certs. pyright 0 errors; pylint 9.83/10; unit suite green (the 13 test_sidecar_init /bin/sleep errors are pre-existing NixOS-local noise). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-14 02:38:20 -04:00
didericis	118d81533b	feat(orchestrator): slice 13c(iv) — gateway joins a shared network ... The consolidated gateway ran on the default bridge; the model needs it on a dedicated user-defined network that every agent bottle also joins, reaching the gateway's egress / git-http / supervise ports by its address (no host port publishing) — and the source IP the gateway attributes by is the bottle's address on this network. - GATEWAY_NETWORK = "bot-bottle-gateway"; DockerGateway gains a `network` param. - ensure_running now ensures the network exists (idempotent create, tolerating a concurrent 'already exists') then runs with `--network`. - Docker picks the subnet; the launcher reads it back (13c source-IP allocator) to hand each bottle a pinned address. pyright 0 errors; pylint 9.83/10; unit suite green (the 13 test_sidecar_init /bin/sleep errors are pre-existing NixOS-local noise). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-14 02:38:20 -04:00
didericis	353b1ad984	feat(orchestrator): slice 13c(iii) — host-side control-plane client ... The launch path's counterpart to the gateway-side PolicyResolver: a trusted host-side HTTP client for the orchestrator control plane, so the CLI can register/teardown/re-policy bottles. Stdlib-only. - OrchestratorClient.register_bottle(source_ip, *, image_ref, metadata, policy) -> RegisteredBottle(bottle_id, identity_token) (POST /bottles) - teardown_bottle(id) -> bool (DELETE; 404 -> False, idempotent for cleanup) - set_policy(id, policy) -> bool (PUT; live reload) - list_bottles() / health() - Unlike the fail-closed data-plane resolver, this is the trusted caller: a non-2xx (other than the meaningful 404s) raises OrchestratorClientError so the launch path surfaces failures rather than swallowing them. pyright 0 errors; pylint 9.82/10; unit suite green (1742 tests; the 13 test_sidecar_init /bin/sleep errors are pre-existing NixOS-local noise). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-14 02:38:20 -04:00