12 Commits

Author	SHA1	Message	Date
didericis	bef45348f5	fix(types): satisfy pyright strict on the refactored facades ... Two facade issues from the reorg: * orchestrator/__init__.py used `__all__ = list(_LAZY)`, which pyright strict can't analyze (reportUnsupportedDunderAll) — and because it isn't a static literal, the TYPE_CHECKING re-export imports read as unused. Replaced with the explicit literal list the other lazy facades already use. * manifest/index.py imported six piece types (ManifestAgentProvider, EGRESS_AUTH_SCHEMES, ManifestEgressConfig/Route, ManifestGitEntry, ManifestKeyConfig) it never referenced — the package facade re-exports those straight from their submodules, so index.py needn't import them. Dropped the dead imports. pyright: 0 errors (was 24). Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 19:11:37 -04:00
didericis	e2740842a0	refactor: make the two remaining heavy __init__ modules lazy ... Audit of package __init__ import cost turned up two eager ones: orchestrator/__init__.py (27 -> 2): was pure `from .X import Y` re-exports (registry_store, broker, docker_broker, gateway, service, server). Because importing any submodule runs the parent __init__, the CLI migration gate (orchestrator.store.store_manager, hit on every command) transitively dragged backend.docker, docker_broker, server, and http.server. Converted to the same lazy __getattr__ + _LAZY facade used by manifest/backend/egress/git_gate; the re-export API is unchanged and the docker/http drag is gone. cli/commands/__init__.py (78 -> 23): the registry imported all twelve handlers to build COMMANDS, so importing one command pulled all of them plus their deps. COMMANDS now maps each name to a thin lazy wrapper that imports its handler's module on first dispatch. Values stay callable, so the dispatcher and the patch.dict dispatch tests are unchanged; a CLI run now loads only the one command it dispatches. The remaining 23 is the dispatcher's own baseline (store_manager migration gate + help + log). Full unit suite green (2243); `bb help` + dispatch/migration-gate tests verified. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 18:54:41 -04:00
didericis	3efb014ace	refactor(orchestrator): move registry into store/registry_store ... registry.py is a SQLite-backed store (its class is already RegistryStore), so it belongs with the other orchestrator-owned stores rather than at the package root. Moved orchestrator/registry.py -> orchestrator/store/ registry_store.py, joining queue_store / secret_store / config_store. Repointed the in-package importers (.registry -> .store.registry_store) and the tests, bumped the moved file's relative-import depths, renamed the test file to match, and listed it in the store package docstring. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 18:24:20 -04:00
didericis	ca1d341d4f	refactor: unify component naming — control_plane/control_auth -> orchestrator ... The codebase used "control plane" both as an architectural role term AND as an identifier alias for the orchestrator component, producing duplicate names for one thing (control_plane_url vs orchestrator_url, CONTROL_PLANE_PORT, host_control_plane_token, …). Going forward the concrete component is always named for what it is — Gateway or Orchestrator — and the plane vocabulary is reserved for prose (module descriptions, the security argument). Renamed (identifiers + the in-repo env/wire/file string values, all setters/getters are in this repo so the change is atomic): ControlPlaneServer -> OrchestratorServer control_plane_url -> orchestrator_url probe_control_plane_url -> probe_orchestrator_url host_control_plane_token -> host_orchestrator_token CONTROL_PLANE_PORT -> ORCHESTRATOR_PORT CONTROL_PLANE_TOKEN_ENV/FILE -> ORCHESTRATOR_TOKEN_ENV/FILENAME BOT_BOTTLE_CONTROL_PLANE_TOKEN-> BOT_BOTTLE_ORCHESTRATOR_TOKEN control-plane-token (file) -> orchestrator-token control_auth (module) -> orchestrator_auth (stays top-level; the gateway imports it and must not import the orchestrator/ package) CONTROL_AUTH_HEADER -> ORCHESTRATOR_AUTH_HEADER x-bot-bottle-control-auth -> x-bot-bottle-orchestrator-auth CONTROL_AUTH_JWT_ENV -> ORCHESTRATOR_AUTH_JWT_ENV BOT_BOTTLE_CONTROL_AUTH_JWT -> BOT_BOTTLE_ORCHESTRATOR_AUTH_JWT _control_auth_headers -> _orchestrator_auth_headers Prose plane-terms ("control plane", "data plane") are preserved, including the test name test_data_plane_daemons_get_jwt_not_key (it names the security invariant). Gateway and orchestrator verified to agree on the renamed wire header; full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 18:17:48 -04:00
didericis	4166057abc	refactor(orchestrator): rename control_plane module to server ... orchestrator/control_plane.py -> orchestrator/server.py. Within the orchestrator package the "control_plane" filename stutters (the orchestrator *is* the control plane), and `orchestrator.server` reads as "the orchestrator's HTTP server", pairing with service.py (domain logic) and matching gateway/supervisor/server.py. Scope is the module name only. The class ControlPlaneServer, the CONTROL_AUTH_HEADER constant, and the "control plane" architectural term (CONTROL_PLANE_PORT, host_control_plane_token, control_plane_url, …) are deliberately unchanged — that's the load-bearing control-plane/data-plane distinction. Test file renamed to match; full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 17:57:49 -04:00
didericis	f77023db1d	refactor(gateway): move the data-plane daemons into a bot_bottle.gateway package ... Separate the gateway (data plane) from the orchestrator (control plane) at the module level. The gateway runtime files move out of the package root — and the backend-neutral Gateway lifecycle ABC + GATEWAY_* constants move out of orchestrator/ — into a new bot_bottle/gateway/ package: gateway/__init__.py (was orchestrator/gateway.py: Gateway ABC + consts + rotate_gateway_ca) gateway/gateway_init.py (the PID-1 daemon supervisor) gateway/egress_addon.py, egress_addon_core.py, egress_dlp_config.py, dlp_detectors.py (the egress mitmproxy daemon) gateway/git_http_backend.py (the git-http daemon) gateway/git_gate_render.py (the git-gate pre-receive rendering) gateway/supervise_server.py (the supervise MCP daemon) gateway/policy_resolver.py (the data-plane control-plane RPC client) orchestrator/ now holds only control-plane files. The shared plan/types/auth layer (egress.py=EgressPlan, git_gate.py=GitGatePlan, supervise.py, supervise_types.py, control_auth.py) and the launch-time git-gate provisioning helpers stay at root, so orchestrator/ and backend/ still own them. Because these daemons are invoked as `python3 -m bot_bottle.<name>`, loaded flat by mitmproxy, and referenced in Dockerfile.gateway, the move updates more than Python imports: the `-m` invocations (firecracker/macOS infra scripts), the Dockerfile.gateway addon shim + ENTRYPOINT, gateway_init's _DAEMONS module paths, and the git-gate CGI heredocs all now point at bot_bottle.gateway.*. No behavior change; full unit suite green (2251). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 13:25:10 -04:00
didericis	dfce3d9505	refactor(gateway): move DockerGateway to the backend layer, drop the dead standalone-gateway path ... The `DockerGateway` container-lifecycle impl now lives in `backend/docker/gateway.py` (the shape backend gateway classes will share); `orchestrator/gateway.py` keeps only the backend-neutral pieces (the `Gateway` ABC, constants, `rotate_gateway_ca`). Delete the standalone-gateway path it was the only consumer of. `--gateway` on `python -m bot_bottle.orchestrator` was invoked nowhere — the production docker flow runs the gateway data plane inside the combined `bot-bottle-infra` container via `OrchestratorService`, never this class. Removing it takes with it `Orchestrator.ensure_gateway()` and the `gateway` ctor arg; `gateway_status()` becomes a stub reporting `configured: false` so the documented `GET /gateway` control-plane route keeps its contract. Fix a latent bug the move surfaced: `launch.py` read the shared gateway CA via `docker exec bot-bottle-orch-gateway`, a container the consolidated flow never creates — so the agent CA install was reaching a nonexistent name. Read it from `bot-bottle-infra` (INFRA_NAME) instead. Repoint the affected tests to the new module; drop the unit test + fake that covered the deleted standalone wiring. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 12:39:58 -04:00
didericis	00418a2834	refactor(orchestrator): rename Sidecar -> Gateway for the consolidated data plane ... Retire "sidecar" for the consolidated per-host path (PRD 0070 naming decision): the orchestrator is the umbrella/control plane, and the egress/git/supervise data-plane unit it runs is the "gateway". - git mv sidecar.py -> gateway.py and the two integration + one unit test files; DockerSidecar->DockerGateway, Sidecar->Gateway, SidecarError->GatewayError, SIDECAR_*->GATEWAY_*, ensure_sidecar-> ensure_gateway, sidecar_status->gateway_status, container name bot-bottle-orch-sidecar->bot-bottle-orch-gateway. - Prose rename across broker/registry/egress/policy_resolver + PRD 0070. - Preserved: the image name bot-bottle-sidecars, the BOT_BOTTLE_SIDECAR_IMAGE env var, Dockerfile.sidecars, and PRD 0069's own stage-name cross-references (that doc still uses "sidecar"). No behavior change. Full unit suite green (1679 tests; the 13 test_sidecar_init /bin/sleep errors are pre-existing NixOS-local noise). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-14 02:38:20 -04:00
didericis	bc52c3525c	feat(orchestrator): slice 4 — consolidated per-host sidecar (#352) ... The core consolidation win: one persistent sidecar per host, shared by every bottle, instead of a sidecar bundle per bottle. Safe to share because the attribution invariant (source IP + identity token) lets the sidecar map each request to the right bottle. * orchestrator/sidecar.py — a backend-neutral `Sidecar` lifecycle contract (mirrors LaunchBroker) + a `DockerSidecar` impl. The defining behaviour is idempotent singleton: `ensure_running` starts the instance if absent and is a no-op if it's already up, so N launches never spawn N sidecars; `stop` is idempotent. * orchestrator/dockerutil.py — a shared `run_docker` helper; DockerBroker now uses it too (DRY with slice 3). * service.py — the Orchestrator holds an optional `Sidecar`, exposes `ensure_sidecar()` + `sidecar_status()`. * control_plane.py — `GET /sidecar` reports it; __main__ gains `--sidecar-image` and ensures the single sidecar on startup. Tests: unit (docker mocked) — is_running, ensure idempotent (no-op when up, starts when absent), failure raises, stop idempotent; Orchestrator sidecar wiring/status; control-plane /sidecar; integration (gated) — ensure is a real idempotent singleton (one container after two ensures), stop removes. Full suite green (only pre-existing /bin/sleep errors); integration verified locally against real docker. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-14 02:38:20 -04:00
didericis	f7b3d6aaca	feat(orchestrator): slice 3 — real Docker launch broker (#352) ... The first concrete LaunchBroker, proving the orchestrator -> backend seam on the cheapest backend (the sidecar bundle is already containers): * orchestrator/docker_broker.py — DockerBroker runs a container on a verified launch (`docker run --detach --name <bottle> --label ... <image_ref>`) and removes it on teardown (`docker rm --force`, idempotent on an already-absent container). The argv is built only from the request's static ids/flags, so nothing free-form reaches docker; provenance/schema verification is inherited from LaunchBroker.submit. * __main__.py gains `--broker {stub,docker}` so the harness can drive real containers. Slice 3 launches a single container from image_ref (the seam); the full agent + sidecar bundle is a later slice. Tests: unit (docker mocked) — argv from static fields, launch/teardown call the right commands, missing-image and docker-failure raise, teardown idempotent on missing, forged token never touches docker; integration (gated on a reachable daemon) — launch creates a real container, teardown removes it. Full suite green (only pre-existing /bin/sleep errors); integration verified locally against real docker. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-14 02:38:20 -04:00
didericis	b05f245299	feat(orchestrator): slice 2 — launch lifecycle + signed launch-broker (#352) ... Second slice of PRD 0070, still a backend-neutral dev-harness: * orchestrator/broker.py — the launch-broker contract. A LaunchRequest is structured (static ids/flags only — bottle id, pool slot, a content-addressed image_ref; never a path/argv) and signed as a compact HS256 JWT so the broker verifies PROVENANCE before acting: a compromised co-located component can't forge a launch without the shared secret. verify_request is fail-closed (bad sig / malformed / off-schema -> raise). Stdlib only (no runtime deps). Ships a StubBroker that records verified requests for the harness/tests. * orchestrator/service.py — the Orchestrator: owns the registry and brokers the lifecycle. launch_bottle mints the bottle + sends a signed launch, rolling the registry entry back if the launch fails (no orphans); teardown_bottle brokers teardown then deregisters; attribute delegates. * control_plane.py — POST /bottles now launches, DELETE tears down (both go through the Orchestrator + broker). dispatch/server take an Orchestrator. * __main__.py wires an ephemeral secret + StubBroker for the harness. Tests: broker sign/verify round-trip, tamper/wrong-secret/malformed/off-schema rejection, StubBroker fail-closed; Orchestrator launch->registry->attribute, teardown, rollback-on-broker-failure; control-plane updated for launch/teardown. Full suite green (only the pre-existing /bin/sleep errors); harness does launch -> attribute -> teardown over HTTP. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-14 02:38:20 -04:00
didericis	d9843fa41e	feat(orchestrator): slice 1 — registry + attribution + HTTP control plane (#352) ... First implementation slice of PRD 0070, the backend-neutral consolidation core as a plain-process dev-harness (no VM packaging yet): * orchestrator/registry.py — SQLite (WAL) runtime-state store on the existing DbStore/TableMigrations base. Live bottle registry keyed by source IP + per-bottle identity token, with fail-closed attribution: a request resolves to a bottle only when its source IP AND identity token both match exactly one active record (unknown/ambiguous IP, empty token, or token mismatch all deny). Tokens are 256-bit urandom. * orchestrator/control_plane.py — the HTTP control plane (the universal transport chosen in 0070): register / deregister / list / attribute / health. Routing is a pure dispatch() so it is socket-free testable; Handler/ControlPlaneServer/make_server are a thin stdlib adapter. register/deregister are the live-reload path; listing redacts tokens. * orchestrator/__main__.py — `python -m bot_bottle.orchestrator` harness. Launch/teardown, the launch broker, and the egress/git/supervise data plane come in later slices. 24 unit tests (attribution matrix, persistence, dispatch, one real-socket round-trip). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-14 02:38:20 -04:00