1386 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
didericis	7e9ad8a78d	Merge main into fix/db-off-data-plane-469 ... Brings in main's 8 commits — #470 (backend-agnostic CI guards: BackendStatus enum, quiet status(), is_backend_available/is_backend_ready, tests/_backend.py skip_unless_backend) plus firecracker status() readiness checks (kvm/kernel/ dropbear/mke2fs). Reconciled #470's additions into the reorg'd backend structure: * BackendStatus enum + the status(*, quiet=) ABC signature -> backend/base.py * is_backend_available / is_backend_ready -> backend/selection.py * exposed all three through the lazy backend facade (__init__). Integration-test conflicts were our control_plane->orchestrator renames vs main's skip_unless_docker -> skip_unless_backend swap: kept our refactored import paths + main's guard. Repointed main's new is_backend_* tests to patch selection._backends (our moved home) instead of the facade. pyright: 0 errors. Full unit suite green (2272). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 21:35:58 -04:00
didericis-claude	a68ee778f1	firecracker status(): open /dev/kvm O_RDWR; check kernel, dropbear, mke2fs ... Two reviewer findings addressed: 1. _kvm_accessible() now opens /dev/kvm with O_RDWR|O_CLOEXEC instead of read-only. VM creation requires write access; a read-only descriptor can satisfy KVM_GET_API_VERSION but fails at boot time. 2. status() now checks every hard prerequisite that require_firecracker() checks at launch: guest kernel image, static dropbear binary, and mke2fs. Previously a host with a configured TAP pool but missing artifacts could pass status() and then fail during launch. Regression coverage: TestFirecrackerKvmCheck gains test_kvm_open_rdwr_fails; TestFirecrackerArtifactCheck covers each missing artifact; existing TestFirecrackerStatus fixtures updated to stub the new checks. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-07-24 21:25:20 -04:00
didericis-claude	de02d13ccf	firecracker status(): add binary and KVM readiness checks ... Adds `_firecracker_binary_ok()` (runs `firecracker --version`) and `_kvm_accessible()` (opens /dev/kvm, issues KVM_GET_API_VERSION ioctl) and gates `status()` on both, so the launch preflight reports missing binary or inaccessible KVM before the caller ever attempts a boot. Regression tests in TestFirecrackerBinaryCheck, TestFirecrackerKvmCheck, and TestFirecrackerStatusRuntime cover all branches. Updated the pre-existing TestFirecrackerStatus fixture to also stub the two new helpers so it still exercises only the tap-pool/nft logic it was designed for. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-07-24 21:25:20 -04:00
didericis-claude	c740d1e145	fix: add missing type annotations and remove unused variables in new tests ... Pyright flagged three inner class `status` methods missing a type annotation on the `quiet` parameter, and two unused variables (`written`, `real_status`) left over from an earlier draft of test_docker_quiet_true_suppresses_stderr. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-07-24 21:25:20 -04:00
didericis-claude	177721c286	test: cover is_backend_available, is_backend_ready, and status(quiet=True) ... Add unit tests for the three new public symbols introduced by this PR: - TestIsBackendAvailable — delegates to has_backend; returns True/False correctly for available and unavailable backends - TestIsBackendReady — unknown names return False; known names return True/False based on status() return code; quiet flag is forwarded - TestBackendStatusQuiet — verifies the quiet=True branch in DockerBottleBackend, FirecrackerBottleBackend, and MacosContainerBottleBackend: return code is propagated and diagnostic stderr is suppressed Together these bring diff-coverage for the PR's changed lines to 100%. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-07-24 21:25:20 -04:00
didericis-claude	5bcf3db1f8	fix: align skip-guard tests with is_backend_ready and drop unused import ... tests/_backend.py imported is_backend_available but never called it, causing a pyright reportUnusedImport error. Remove the unused import. test_backend_skip_guards.py was patching tests._backend.has_backend but _backend.py calls is_backend_ready — the mock never intercepted the real call, producing AttributeError at test runtime. Update all patch targets to is_backend_ready and add quiet=False to the assert_called_once_with assertion to match the actual call signature. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-07-24 21:25:20 -04:00
didericis-claude	dcd658ece1	feat(backend): BackendStatus enum, quiet status(), is_backend_ready() ... Add a module-level BackendStatus(IntEnum) with READY=0 to replace magic 0 comparisons. Extend BottleBackend.status() with a quiet parameter: quiet=False (default) prints diagnostics; quiet=True returns the code silently for programmatic checks. Add is_backend_available() (cheap PATH check, alias for has_backend) and is_backend_ready(name, *, quiet=False) (full status() check) to the backend package for callers that need to distinguish availability from readiness. Update tests/_backend.py guards to use is_backend_ready(quiet=False) so diagnostic output is printed during test discovery, giving the operator a concrete reason for each skip rather than a bare "prerequisites unavailable" message. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-07-24 21:25:20 -04:00
didericis-claude	dfb56f8fe9	ci: reuse backend.has_backend / backend status for readiness ... Review feedback: the hand-rolled `/dev/kvm` + docker-daemon capability probes duplicated logic the CLI already owns. Delegate instead. - tests/_backend.py skip guards now gate on `bot_bottle.backend.has_backend` (each backend's `is_available()` classmethod — the same probe behind `./cli.py backend status`), dropping the bespoke `Capability` probes. - Remove tests/backend_preflight.py; the docker integration job runs `./cli.py backend status --backend=docker` as its preflight (clear per-check summary, non-zero exit when unready), matching the firecracker job. The firecracker preflight reverts to its original binary/KVM checks (backend status doesn't cover those). - Unit test + docs updated to match. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 21:25:20 -04:00
didericis-claude	64adf23775	ci: backend-agnostic integration guards + per-backend preflight ... Integration tests now select their backend from BOT_BOTTLE_BACKEND and skip on the capability that backend actually needs, instead of gating every backend on unrelated Docker availability. Task 1 — backend-agnostic guards (tests/_backend.py): - Capability probes: docker_capability() (reachable daemon) and firecracker_capability() (accessible /dev/kvm + firecracker on PATH, Docker-independent). backend_capability()/selected_backend() resolve the target from BOT_BOTTLE_BACKEND (default docker). - skip_unless_selected_backend_available() for backend-agnostic tests (test_sandbox_escape) — runs through whichever backend is selected and checks that backend's real capability. - skip_unless_backend("docker") for Docker-implementation tests (DockerBroker, DockerGateway, backend.docker.*) — they no-op under a non-Docker run rather than testing internals that run doesn't target. - Retires tests/_docker.py; the KVM job no longer needs SKIP_DOCKER_TESTS to steer Docker-only classes. Task 2 — explicit per-backend skip visibility: - tests/backend_preflight.py prints a clear PASS/FAIL capability line and exits non-zero when the selected backend is missing. - Both integration jobs run it as a preflight, so absent infrastructure is surfaced at the job level instead of hidden among unittest.skip lines. The docker job replaces its soft "Show environment" step; the firecracker job keeps its richer backend-status check. Docs (tests/README.md, docs/ci.md) updated; unit coverage for the probes, guards, and preflight in test_backend_skip_guards.py. Closes #414 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 21:25:20 -04:00
didericis	3d3d8fd7e8	docs(prd-0070): record remote-terminal resolution; track detail in #478 ... Remote terminal is an orchestrator operation, not a data-plane one: the agent PTY is exec_agent-sourced (ssh -t / container exec --tty / docker exec), which the orchestrator owns — the gateway never holds a PTY. A --remote flag publishes a running agent's session; local + remote clients share it via a session multiplexer (one PTY, many clients), operator-role-gated, so the split's "one authenticated remote door" holds for terminals too. Detailed design (multiplexer location, input arbitration, per-client rendering, always-on vs --remote) tracked in #478. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 21:14:20 -04:00
didericis	195e0f249d	docs(prd-0070): resolve build-placement open question — builds stay in the orchestrator ... v1 decision: agent-image builds run in the orchestrator (the control-plane side already owns launches and hosts builds in today's combined unit; the data-plane gateway must not build). Consequence recorded in the memory section: the orchestrator keeps the ~4 GB buildah ceiling and the gateway is the slim unit (~1 GB), so the split is ~memory-neutral-to-+1 GB for now — the ~1.5 GB win waits on a later move to a dedicated slim build unit (#468-adjacent). Also flags the tension of co-locating buildah with the control plane. Removed the corresponding open question. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 20:59:30 -04:00
didericis	ee26e9044f	docs(prd-0070): fold in the orchestrator/gateway plane-separation design ... Now that #469 got the DB off the data plane (removing the cross-kernel SQLite-coherence reason the planes were consolidated), separating them into distinct runtimes is possible again. Adds a Design subsection to 0070 covering: * Access — the only hop that changes is gateway->orchestrator (loopback -> network). A dedicated `bot-bottle-orchestrator` --internal control network with a dual-homed gateway gives Docker/macOS the L3 agent isolation that only Firecracker's nft currently has (both are token-only today); Firecracker gets it nearly free (agents already can't reach non-DNAT'd ports). * Memory — firecracker's mem_size_mib is a fixed ceiling (no reclaim without a balloon); the 4 GB combined VM is really buildah headroom. Right-sized: ~512 MiB orchestrator + ~1 GB gateway < today's 4 GB, if agent-image builds move off the gateway. * iroh belongs in the orchestrator (control-plane remote entry point); keep the data-plane gateway slim and P2P-free. Plus open questions: where agent-image builds run post-split, and whether agent-PTY-to-mobile relays through the orchestrator's iroh endpoint. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 20:41:34 -04:00
didericis-claude	e7fe00e2f5	fix(firecracker): keep the host key canonical instead of clobbering the host token file ... Reimplements the dropped firecracker fix on the refactored code, addressing didericis-codex's blocking review on #471. The previous firecracker path let the VM generate its own signing key on the guest volume and had `_with_signing_key()` unconditionally overwrite the single host-wide orchestrator-token with it. That breaks a co-running Docker/macOS control plane: their orchestrators still verify with the old key while new host clients start signing `cli` tokens with the guest key, so supervise/teardown/policy calls against those backends begin returning 401 (cross-backend operation is supported). Keep the host token file the single source of truth. The launcher now pushes the host-canonical key into the freshly booted infra VM over SSH (atomic write, mirroring persist_env_var_secret) via `_push_signing_key()`; the VM's init waits for it and refuses to start the control plane — rather than run OPEN — if it never arrives. Nothing writes back to the host file, so other backends are untouched, and the best-effort silent path is gone (both the push and the guest fail loudly). Ported onto the reorg'd names (host_orchestrator_token, BOT_BOTTLE_ORCHESTRATOR_ TOKEN, orchestrator-token, bot_bottle.orchestrator_auth, bot_bottle.gateway. bootstrap). pyright: 0 errors. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 19:47:59 -04:00
didericis	466f4ee13a	Merge main into fix/db-off-data-plane-469 ... Brings in the two commits main gained: #472 (CLI cleanup — remove `info`, split `list active` into a top-level `active`, simplify `list`) and the terminology glossary. #472 was written against the old flat cli/ layout, so its semantics are reconciled into the reorg'd cli/commands/ structure: * new `active` command lives at cli/commands/active.py, registered in the lazy registry and listed in `help` (not left at the flat path). * `info` removed: deleted cli/commands/info.py, dropped from the registry and `help`. * `list` simplified to list-available-only, using the reorg's os.getcwd()/constants.PROG conventions. * the `list active` -> `active` doc wording applied to the backend docstrings that our split moved into base.py / selection.py. Our reorg wins for the fully-rewritten dispatcher (cli/__init__ shim, backend facade). pyright: 0 errors. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 19:35:49 -04:00
didericis	bef45348f5	fix(types): satisfy pyright strict on the refactored facades ... Two facade issues from the reorg: * orchestrator/__init__.py used `__all__ = list(_LAZY)`, which pyright strict can't analyze (reportUnsupportedDunderAll) — and because it isn't a static literal, the TYPE_CHECKING re-export imports read as unused. Replaced with the explicit literal list the other lazy facades already use. * manifest/index.py imported six piece types (ManifestAgentProvider, EGRESS_AUTH_SCHEMES, ManifestEgressConfig/Route, ManifestGitEntry, ManifestKeyConfig) it never referenced — the package facade re-exports those straight from their submodules, so index.py needn't import them. Dropped the dead imports. pyright: 0 errors (was 24). Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 19:11:37 -04:00
didericis	e2740842a0	refactor: make the two remaining heavy __init__ modules lazy ... Audit of package __init__ import cost turned up two eager ones: orchestrator/__init__.py (27 -> 2): was pure `from .X import Y` re-exports (registry_store, broker, docker_broker, gateway, service, server). Because importing any submodule runs the parent __init__, the CLI migration gate (orchestrator.store.store_manager, hit on every command) transitively dragged backend.docker, docker_broker, server, and http.server. Converted to the same lazy __getattr__ + _LAZY facade used by manifest/backend/egress/git_gate; the re-export API is unchanged and the docker/http drag is gone. cli/commands/__init__.py (78 -> 23): the registry imported all twelve handlers to build COMMANDS, so importing one command pulled all of them plus their deps. COMMANDS now maps each name to a thin lazy wrapper that imports its handler's module on first dispatch. Values stay callable, so the dispatcher and the patch.dict dispatch tests are unchanged; a CLI run now loads only the one command it dispatches. The remaining 23 is the dispatcher's own baseline (store_manager migration gate + help + log). Full unit suite green (2243); `bb help` + dispatch/migration-gate tests verified. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 18:54:41 -04:00
didericis	3efb014ace	refactor(orchestrator): move registry into store/registry_store ... registry.py is a SQLite-backed store (its class is already RegistryStore), so it belongs with the other orchestrator-owned stores rather than at the package root. Moved orchestrator/registry.py -> orchestrator/store/ registry_store.py, joining queue_store / secret_store / config_store. Repointed the in-package importers (.registry -> .store.registry_store) and the tests, bumped the moved file's relative-import depths, renamed the test file to match, and listed it in the store package docstring. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 18:24:20 -04:00
didericis	cb3a74edb0	refactor(orchestrator): collapse supervisor package into supervisor.py ... orchestrator/supervisor/ held only __init__.py, so the package no longer earns its own directory. Moved it to orchestrator/supervisor.py and removed the dir. External imports are unchanged (orchestrator.supervisor resolves identically as a module); only the file's own relative-import depths shift up one level. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 18:22:03 -04:00
didericis	ca1d341d4f	refactor: unify component naming — control_plane/control_auth -> orchestrator ... The codebase used "control plane" both as an architectural role term AND as an identifier alias for the orchestrator component, producing duplicate names for one thing (control_plane_url vs orchestrator_url, CONTROL_PLANE_PORT, host_control_plane_token, …). Going forward the concrete component is always named for what it is — Gateway or Orchestrator — and the plane vocabulary is reserved for prose (module descriptions, the security argument). Renamed (identifiers + the in-repo env/wire/file string values, all setters/getters are in this repo so the change is atomic): ControlPlaneServer -> OrchestratorServer control_plane_url -> orchestrator_url probe_control_plane_url -> probe_orchestrator_url host_control_plane_token -> host_orchestrator_token CONTROL_PLANE_PORT -> ORCHESTRATOR_PORT CONTROL_PLANE_TOKEN_ENV/FILE -> ORCHESTRATOR_TOKEN_ENV/FILENAME BOT_BOTTLE_CONTROL_PLANE_TOKEN-> BOT_BOTTLE_ORCHESTRATOR_TOKEN control-plane-token (file) -> orchestrator-token control_auth (module) -> orchestrator_auth (stays top-level; the gateway imports it and must not import the orchestrator/ package) CONTROL_AUTH_HEADER -> ORCHESTRATOR_AUTH_HEADER x-bot-bottle-control-auth -> x-bot-bottle-orchestrator-auth CONTROL_AUTH_JWT_ENV -> ORCHESTRATOR_AUTH_JWT_ENV BOT_BOTTLE_CONTROL_AUTH_JWT -> BOT_BOTTLE_ORCHESTRATOR_AUTH_JWT _control_auth_headers -> _orchestrator_auth_headers Prose plane-terms ("control plane", "data plane") are preserved, including the test name test_data_plane_daemons_get_jwt_not_key (it names the security invariant). Gateway and orchestrator verified to agree on the renamed wire header; full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 18:17:48 -04:00
didericis	4166057abc	refactor(orchestrator): rename control_plane module to server ... orchestrator/control_plane.py -> orchestrator/server.py. Within the orchestrator package the "control_plane" filename stutters (the orchestrator *is* the control plane), and `orchestrator.server` reads as "the orchestrator's HTTP server", pairing with service.py (domain logic) and matching gateway/supervisor/server.py. Scope is the module name only. The class ControlPlaneServer, the CONTROL_AUTH_HEADER constant, and the "control plane" architectural term (CONTROL_PLANE_PORT, host_control_plane_token, control_plane_url, …) are deliberately unchanged — that's the load-bearing control-plane/data-plane distinction. Test file renamed to match; full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 17:57:49 -04:00
didericis	b276227cbb	refactor(cli): registry in commands/__init__, dispatcher main() in __main__ ... Split the cli package's two responsibilities out of __init__: * commands/__init__.py now assembles the COMMANDS registry and NO_MIGRATION_COMMANDS from the per-command modules — the command surface lives entirely under commands/. * __main__.py now owns main() (dispatch + migration gate + exit-code mapping) alongside the runnable entry guard. cli/__init__.py shrinks to a shim that re-exports main / COMMANDS / NO_MIGRATION_COMMANDS, so bot_bottle.cli.main (repo-root cli.py entry) and the tests' bot_bottle.cli.COMMANDS keep working unchanged. Verified `python -m bot_bottle.cli` (runpy) and `cli.py` both dispatch; full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 17:52:52 -04:00
didericis	3ccd308613	refactor(cli): make help a first-class command; drop usage() from dispatcher ... Move the top-level usage/command-list text out of the dispatcher into commands/help.py as cmd_help, registered in COMMANDS so `bb help` now works as a real command. The dispatcher's -h/--help, no-args, and unknown-command fallbacks call cmd_help() for the text while keeping their exit codes (0 for help/-h, 2 for bare no-args, Die for unknown). Added `help` to NO_MIGRATION_COMMANDS so it prints without a migrated DB, and listed it in the command summary. Full unit suite green (2243); dispatch + migration-gate tests pass, `bb help` verified. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 17:40:05 -04:00
didericis	a21f2358c6	refactor(cli): trim _common to a PROG constants leaf; inline os.getcwd() ... _common had rotted into a junk drawer. Trim it to what actually justifies a shared leaf module and rename for legibility: * REPO_DIR: deleted (dead — nothing imported it). * read_tty_line: dropped the pointless re-export; cleanup/start/init now import it straight from bot_bottle.util. * USER_CWD: deleted. It captured os.getcwd() at import time, but nothing chdirs and no test patched it, so it was equivalent to a live os.getcwd() — inlined at the six call sites. * PROG: kept, now the sole member. It's still a leaf (both the dispatcher and the commands it imports need it) so it can't move into __init__ without a circular import. _common.py -> constants.py. Full unit suite green (2243); CLI dispatch verified. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 17:31:14 -04:00
didericis	50a67c04bd	refactor(cli): group subcommand handlers under cli/commands/ ... Move the eleven per-command modules (backend, cleanup, commit, edit, info, init, list, login, resume, start, supervise) into a new bot_bottle/cli/commands/ package, leaving the dispatcher (__init__), entrypoint (__main__), and shared helpers (_common, tui) at the cli root. Makes the command surface obvious at a glance and separates handlers from the plumbing that registers them. Updated the dispatcher's COMMANDS imports to .commands.*, bumped the moved files' relative-import depths (.._common, .. import tui, sibling .start unchanged), and repointed test references to bot_bottle.cli.commands.*. Full unit suite green (2243); CLI dispatch verified via `python -m bot_bottle.cli --help`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 17:25:25 -04:00
didericis	82d02d2c4b	refactor(gateway): move egress entrypoint into gateway/egress ... egress_entrypoint.sh is the mitmdump launcher for the egress data plane, so it belongs with the gateway egress service, not at the package root. Moved to bot_bottle/gateway/egress/entrypoint.sh (prefix stripped now that the package namespaces it). Only the Dockerfile.gateway COPY source changes; the runtime target (/app/egress-entrypoint.sh, referenced by bootstrap's egress DaemonSpec) is unchanged, and the infra images inherit it via FROM gateway. Updated the test that resolves the script path and the infra-artifact docstring example. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 17:14:26 -04:00
didericis	74dc984cf8	refactor(gateway): rename bootstrap's _Supervisor to _DaemonManager ... The PID-1 process manager in gateway/bootstrap.py was named _Supervisor, which collided conceptually with the supervise permissions service (Supervisor / gateway.supervisor). It manages the _DaemonSpec child set, so _DaemonManager names what it does without the "supervise" echo. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 17:08:34 -04:00
didericis	ce744a85c4	refactor(gateway): split data-plane files into egress/supervisor/git_gate services ... Group the gateway's data-plane modules into three service sub-packages mirroring the host-side trio (bot_bottle.egress / .supervisor / .git_gate): gateway/egress/ addon_core, addon, dlp_config, dlp_detectors gateway/supervisor/ server (was supervise_server) gateway/git_gate/ render, http_backend Prefix-stripped filenames now that the package namespaces them; each sub-package has a thin docstring __init__ (no eager imports, cheap leaf loads). The two cross-cutting files stay at the gateway root: policy_resolver (shared per-client lookup) and gateway_init, renamed to bootstrap now that gateway/ already namespaces it. Updated all importers (bot_bottle + tests), the in-VM/container `-m` launch strings, the Dockerfile.gateway addon shim + ENTRYPOINT, and the five gateway entries in scripts/critical-modules.txt. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 17:00:51 -04:00
didericis	a446551acb	refactor(egress): make Egress a service class in an egress package ... Split the flat egress.py into an egress/ package: neutral DTOs (EgressRoute, EgressPlan) in plan.py, and the concrete Egress service class plus rendering/env helpers in service.py, behind a thin __getattr__ facade so leaf imports stay cheap. Egress drops ABC and becomes a concrete service the backends call: resolve_token_values / agent_env_entries / prepare are now methods. Backend launch paths (docker, firecracker, macos_container) call Egress().<method>(...) instead of the module-level functions. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 16:43:59 -04:00
didericis	450037b7e9	refactor(git-gate): make GitGate a service class in a git_gate package ... Turn the git_gate module into a package with GitGate as a concrete service class (dropping the ABC), mirroring the Supervisor shape. The host-side git-gate operations are now methods the backend drives: git_gate/ __init__.py — thin __getattr__ facade (keeps `from bot_bottle.git_gate import …` working; render/hook names lazily forwarded to gateway.git_gate_render) plan.py — GitGatePlan (the launch DTO the backend contract references) service.py — GitGate: prepare / provision_dynamic_keys / revoke_provisioned_keys / preflight_host_keys provision.py — deploy-key lifecycle (was git_gate_provision.py) host_key.py — host-key preflight (was git_gate_host_key.py) The backend launch + base.py preflight now call GitGate() methods instead of the free functions. The runtime rendering / in-gateway hook execution stay in gateway/git_gate_render.py (data plane) — only the host-side service moved. Because the facade forwards names lazily, the ~25 `from bot_bottle.git_gate import GitGatePlan` call-sites are unchanged, and importing git_gate.plan (the contract's dependency) no longer drags in the provisioning / forge-API code. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 16:27:56 -04:00
didericis	14c28946a7	refactor(manifest): move Manifest/ManifestIndex to manifest.index; thin the package init ... manifest/__init__.py was a facade that eagerly imported every piece (agent, bottle, egress, git, loader, schema, util) and defined the aggregate Manifest / ManifestIndex, so importing a leaf like `manifest.util` paid 19 modules. Move the aggregate model — Manifest, ManifestIndex, and the bottle-resolution helpers — into manifest/index.py (which keeps the framework imports it needs), and make __init__ a thin __getattr__ facade over the 12 __all__ names (mapping each to its submodule), with a TYPE_CHECKING block. The package docstring is preserved. `import bot_bottle.manifest.util` drops 19 -> 3; the package itself is 2. All `from bot_bottle.manifest import Manifest/ManifestIndex/ManifestError/…` call-sites keep working via the lazy facade; no importer changes needed. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 16:04:15 -04:00
didericis	eab9d15130	refactor(docker): fold run_docker into backend/docker/util; drop the docker_cmd shim ... docker_cmd.py existed as a top-level module solely so the orchestrator's docker components could share run_docker without dragging the (then-heavy) backend layer in. Now that backend/__init__ and backend/docker/__init__ are thin, importing backend.docker.util costs 6 modules instead of 76, so the shim's whole reason to exist is gone. Move run_docker into backend/docker/util.py (where the other docker subprocess primitives live) and delete docker_cmd.py. Backend siblings import it via `from .util import run_docker`; the two docker-specific orchestrator modules (docker_broker, rotate_ca) import `from ..backend.docker.util import run_docker`. No import cycle (backend.docker.util pulls nothing from orchestrator); orchestrator.__main__ stays lean at 28 modules. run_docker patch targets are unchanged (tests patch it in the importing module). Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 15:58:23 -04:00
didericis	923d44bc09	refactor(backend): thin the docker/firecracker/macos-container package inits ... The three concrete-backend package inits eagerly imported their `*BottleBackend` (which pulls the whole framework), so importing any leaf under them (e.g. `backend.docker.util`) paid ~78 modules. Give each the same thin `__getattr__` treatment as the top-level backend package: re-export the public class(es) lazily on first access, with a TYPE_CHECKING block for checkers. `import bot_bottle.backend.docker.util` drops from 78 -> 6 bot_bottle modules; the docker/firecracker/macos-container packages themselves are ~3 each. All `from bot_bottle.backend.docker import DockerBottleBackend` call-sites and the lazy selection path keep working. This is what makes backend leaves cheap — and unblocks moving docker_cmd into backend/docker/util if we want. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 15:53:48 -04:00
didericis	3e62f31d8b	refactor(backend): split the heavy backend/__init__ into base + selection, thin the package init ... backend/__init__.py eagerly imported the whole framework (manifest, egress, git-gate, env, workspace, agent-provider), so importing ANY backend.* submodule paid ~32 modules just to run the package init. Split it: - backend/base.py — the abstract contract (BottleSpec, BottlePlan, BottleCleanupPlan, ExecResult, ActiveAgent, Bottle, BottleImages, BottleBackend). Carries the framework imports; loaded only when a caller needs the contract. - backend/selection.py — backend registry / selection / enumeration (get_bottle_backend, _get_backends, _auto_select_backend, has_backend, known_backend_names, enumerate_active_agents). Concrete backends still imported lazily inside. - backend/__init__.py — thin: a __getattr__ that resolves the public names from those submodules on first access (+ a TYPE_CHECKING block for checkers). Existing `from bot_bottle.backend import X` and patch.object call-sites keep working. `import bot_bottle.backend` drops from 32 -> 2 bot_bottle modules. Repointed the backend-selection test's patch targets to the `selection` module (the functions reference each other there now; the FirecrackerBottleBackend class-method patches are unchanged — same class object). Note: backend.docker.util is still heavy because backend/docker/__init__ eagerly imports DockerBottleBackend — slimming the three sub-package inits is the follow-up that makes the leaves cheap. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 15:47:29 -04:00
didericis	b96a8b44e0	refactor(docker): rename OrchestratorService -> DockerInfraService, move to backend/docker ... OrchestratorService wasn't the orchestrator — it's the host-side lifecycle of the docker infra *container* (the one that runs the Orchestrator). It read as "the orchestrator as a service" and lived in orchestrator/lifecycle.py, while its siblings (macOS MacosInfraService, Firecracker infra_vm) live under their backend package. Rename it DockerInfraService and move it to backend/docker/infra.py alongside the docker backend, with its docker-only constants (INFRA_*/ORCHESTRATOR_* image + container names, daemon list, mount paths). orchestrator/lifecycle.py keeps only the backend-neutral pieces the other infra services share — DEFAULT_PORT, DEFAULT_STARTUP_TIMEOUT_SECONDS, OrchestratorStartError, source_hash — which macOS / firecracker / client still import from there. backend/docker/infra.py imports those (backend -> orchestrator is an allowed direction). Renamed the unit test to test_docker_infra.py. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 15:26:52 -04:00
didericis	d41236c376	refactor(util): make render_diff generic (caller supplies side titles); move to root util ... Generalize render_diff so it isn't supervise-flavored: the "(current)" / "(proposed)" side titles are no longer baked in — callers pass `before_title` and `after_label` (required). Move it from orchestrator/supervisor/util.py to the base bot_bottle.util alongside sha256_hex, and delete the now-empty supervisor/util.py. The supervise callsite (Orchestrator.supervise_respond) assigns before_title="current", after_label="proposed"; the facade no longer re-exports render_diff (callers import from bot_bottle.util). Behavior at the supervise callsite is unchanged. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 15:09:27 -04:00
didericis	db6a151803	refactor(supervise): hash the proposed file inside Proposal.new; sha256_hex → root util ... Move sha256_hex out of orchestrator/supervisor/util.py to the root bot_bottle.util (it's a generic string hash, not supervise-specific), and have Proposal.new take the proposed file contents and compute current_file_hash itself instead of every caller passing sha256_hex(proposed_file). Every call site set current_file_hash to the hash of the proposed file, and nothing reads the field except the DB round-trip (schema + _row_to_proposal + from_dict, all untouched), so folding the hash into the factory is behavior-preserving and removes the boilerplate. Callers now pass just the file; the orchestrator no longer imports sha256_hex at all. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 15:03:37 -04:00
didericis	8abccf7ffe	refactor(supervise): make Supervisor a service class the orchestrator calls ... Turn the loose queue/audit functions in orchestrator/supervisor/queue.py into methods on a concrete Supervisor service. The Orchestrator now owns an injectable `self._supervisor` and calls `self._supervisor.write_proposal(...)` etc., instead of module-level free functions — the supervise dependency is explicit and mockable, and a Supervisor can be scoped to a `db_path` (defaults to the host DB) so tests can point it at a temp database. - Supervisor (in the package __init__) drops the ABC and gains write_proposal, read_proposal, list_pending_proposals, list_all_pending_proposals, write_response, read_response, archive_all_proposals, write_audit_entry, read_audit_entries, plus the launch-time prepare(). - render_diff / sha256_hex are pure and stateless, so they move to orchestrator/supervisor/util.py (re-exported from the facade) rather than becoming methods. - queue.py is deleted; service.py + the supervise tests call through a Supervisor instance. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 14:52:17 -04:00
didericis	27a122e24b	refactor(supervise): drop obsolete queue helpers (archive_proposal, wait_for_response, audit_dir/path) ... Remove the supervise queue helpers that no live path uses anymore — they're mechanisms the PRD-0070 / idempotent-poll migration superseded: - archive_proposal (+ QueueStore.archive_proposal): the poll stopped archiving on read (#469 review), so single-proposal archiving has no caller; decided proposals drop off the pending list via their response row and are reaped in bulk by archive_all_proposals on teardown/reconcile. - wait_for_response: the data plane polls the queue over the control-plane RPC now, so nothing block-waits on it. - audit_dir / audit_log_path: file-based audit paths, replaced by the SQLite AuditStore. Also drops the tests that covered those obsolete helpers. Kept the prod-unused read accessors list_pending_proposals and read_audit_entries: those aren't dead mechanisms — they're the read side of live write paths that many tests use to verify real queue/audit behavior. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 14:36:32 -04:00
didericis	3e2cbcab88	refactor(supervise): fold Supervise into the facade as Supervisor; drop dead flat-fallbacks ... Move the `Supervise` lifecycle out of its own `orchestrator/supervisor/ supervise.py` and into the package `__init__`, renaming the class to `Supervisor`. Callers now import it from `bot_bottle.orchestrator.supervisor` alongside the queue surface it belongs with. Remove the dead `try/except ImportError` flat-import fallbacks from the package-only store modules (db_store, audit_store, config_store, queue_store) and image_cache. Those fallbacks existed for when the store files were flat-copied into the gateway; post-PRD-0070 the data plane never opens the DB, so these modules are only ever imported as part of the package. The two gateway data-plane files that may still be loaded flat (egress_addon_core, git_gate_render) keep their fallbacks. Full unit suite green (2251). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 14:24:06 -04:00
didericis	44e2b5a897	refactor(supervise): split the supervise plane by tier; per-service store managers ... Give each service its own store package + manager, and cut the supervise module along the control/data-plane boundary so nothing in the shared layer reaches up into the orchestrator. Stores, by owner: - bot_bottle/store/ keeps only the shared base (DbStore, migrations) and the concrete stores that aren't service-owned (audit_store, config_store). - bot_bottle/orchestrator/store/ now houses the orchestrator-owned stores — queue_store (supervise queue), secret_store, config_store — plus a new orchestrator store_manager that migrates them (composing audit/config downward from the base). The old shared store_manager is gone. Supervise plane, by tier: - bot_bottle/supervisor/ (NEUTRAL, importable by every tier including the gateway): types.py (the Proposal/Response/AuditEntry wire types + the tool/ status/poll constants + the shared daemon constants moved out of supervise.py) and plan.py (SupervisePlan, a pure DTO). - bot_bottle/orchestrator/supervisor/ (orchestrator-only): queue.py (the queue/audit I/O wrappers + render_diff + sha256_hex) and supervise.py (the Supervise lifecycle that stages the DB via the store manager). Its __init__ re-exports the neutral vocabulary so orchestrator-side callers import from one place. The gateway now imports only bot_bottle.supervisor.types (never bot_bottle.supervise), so the data plane holds no code dependency on the orchestrator — it reaches the queue over the control-plane RPC. This removes the circular import that moving queue_store under orchestrator introduced (supervise -> orchestrator -> service -> supervise). supervise_types.py -> supervisor/types.py; supervise.py deleted (split). Full unit suite green (2251). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 14:15:04 -04:00
didericis	f77023db1d	refactor(gateway): move the data-plane daemons into a bot_bottle.gateway package ... Separate the gateway (data plane) from the orchestrator (control plane) at the module level. The gateway runtime files move out of the package root — and the backend-neutral Gateway lifecycle ABC + GATEWAY_* constants move out of orchestrator/ — into a new bot_bottle/gateway/ package: gateway/__init__.py (was orchestrator/gateway.py: Gateway ABC + consts + rotate_gateway_ca) gateway/gateway_init.py (the PID-1 daemon supervisor) gateway/egress_addon.py, egress_addon_core.py, egress_dlp_config.py, dlp_detectors.py (the egress mitmproxy daemon) gateway/git_http_backend.py (the git-http daemon) gateway/git_gate_render.py (the git-gate pre-receive rendering) gateway/supervise_server.py (the supervise MCP daemon) gateway/policy_resolver.py (the data-plane control-plane RPC client) orchestrator/ now holds only control-plane files. The shared plan/types/auth layer (egress.py=EgressPlan, git_gate.py=GitGatePlan, supervise.py, supervise_types.py, control_auth.py) and the launch-time git-gate provisioning helpers stay at root, so orchestrator/ and backend/ still own them. Because these daemons are invoked as `python3 -m bot_bottle.<name>`, loaded flat by mitmproxy, and referenced in Dockerfile.gateway, the move updates more than Python imports: the `-m` invocations (firecracker/macOS infra scripts), the Dockerfile.gateway addon shim + ENTRYPOINT, gateway_init's _DAEMONS module paths, and the git-gate CGI heredocs all now point at bot_bottle.gateway.*. No behavior change; full unit suite green (2251). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 13:25:10 -04:00
didericis	a845cba925	refactor(manifest): consolidate the manifest modules into a bot_bottle.manifest package ... Move the nine root-level manifest modules into a bot_bottle/manifest/ package, dropping the redundant manifest_ prefix: the facade (manifest.py) becomes the package __init__ and keeps re-exporting Manifest / ManifestIndex and the piece types, while manifest_<x>.py become manifest/<x>.py (agent, bottle, egress, extends, git, loader, schema, util). Because the facade stays the package __init__, the ~13 callers that import `from bot_bottle.manifest import ...` are unchanged; only direct-piece imports move to `bot_bottle.manifest.<name>`. Inside the package, intra-manifest imports drop the prefix and the non-manifest siblings (log, yaml_subset, agent_provider) move to `..`. No behavior change; full unit suite green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 13:04:39 -04:00
didericis	d7a58e52fd	refactor(store): consolidate the SQLite store family into bot_bottle.store ... Move the DbStore base (db_store), the shared schema-migration base (migrations / TableMigrations), the concrete stores (audit_store, config_store, queue_store), and StoreManager out of the package root into a bot_bottle/store/ package, so persistence lives in one place rather than scattered across the root. Callers use direct submodule imports (from bot_bottle.store.store_manager import StoreManager). Inside the moved modules the relative imports point back up to their non-store siblings (..paths, ..supervise_types) while co-moved siblings stay package-local; the flat-import fallbacks are untouched. The orchestrator's own stores (config_store, secret_store, registry) stay in orchestrator/ and repoint to the moved DbStore / TableMigrations. No behavior change; full unit suite green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 12:53:52 -04:00
didericis	dfce3d9505	refactor(gateway): move DockerGateway to the backend layer, drop the dead standalone-gateway path ... The `DockerGateway` container-lifecycle impl now lives in `backend/docker/gateway.py` (the shape backend gateway classes will share); `orchestrator/gateway.py` keeps only the backend-neutral pieces (the `Gateway` ABC, constants, `rotate_gateway_ca`). Delete the standalone-gateway path it was the only consumer of. `--gateway` on `python -m bot_bottle.orchestrator` was invoked nowhere — the production docker flow runs the gateway data plane inside the combined `bot-bottle-infra` container via `OrchestratorService`, never this class. Removing it takes with it `Orchestrator.ensure_gateway()` and the `gateway` ctor arg; `gateway_status()` becomes a stub reporting `configured: false` so the documented `GET /gateway` control-plane route keeps its contract. Fix a latent bug the move surfaced: `launch.py` read the shared gateway CA via `docker exec bot-bottle-orch-gateway`, a container the consolidated flow never creates — so the agent CA install was reaching a nonexistent name. Read it from `bot-bottle-infra` (INFRA_NAME) instead. Repoint the affected tests to the new module; drop the unit test + fake that covered the deleted standalone wiring. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 12:39:58 -04:00
didericis-claude	aac27d8a40	fix(firecracker): provision role-scoped control-plane auth in the infra VM ... The Firecracker infra VM started the control plane with no signing key, so it ran open and granted every unauthenticated caller the `cli` role. The nft boundary only fences off the separate agent VM — the egress / supervise / git-http daemons run in this same VM and reach the orchestrator over 127.0.0.1, so a compromised data-plane daemon could still drive the operator routes (approve its own supervise proposals, rewrite policy, read injected tokens). Generate the signing key on the persistent volume, hand it only to the orchestrator process, mint a `gateway` JWT for the data-plane daemons, and mirror the key back to the host token file so the CLI signs `cli` tokens the VM verifies — the same per-process scoping the docker / macOS launchers use. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 07:03:05 +00:00
didericis-claude	d8b61b3658	fix(supervise): make the response poll idempotent and slow it to 250ms ... The control-plane supervise poll archived the proposal on the same call that returned the decision, so a dropped connection lost the operator's decision (the retry saw `unknown`). Poll no longer archives — a re-poll returns the same decision. Decided proposals still drop off the operator's pending list (a response row exists); their rows are reaped when the bottle is torn down or reconciled. Also raise the grace-window poll interval from 50ms to 250ms now that each poll is an HTTP round trip rather than a local file read. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 06:26:06 +00:00
didericis-claude	1db2a9eb67	feat(control-plane): role-scoped signed tokens so the gateway can't drive operator routes ... Review follow-up on #469: the data plane held the same control-plane secret that authorizes every route, so a compromised egress/git-gate could queue a supervise proposal AND approve it (or rewrite policy, read injected tokens) — the (source_ip, identity_token) checks attribute the *bottle*, not the caller. Replace the single shared bearer secret with role-scoped, HMAC-signed tokens (compact HS256 JWTs, stdlib-only — no new dependency): * new `control_auth` mints/verifies `{role}` tokens; roles are `gateway` (data plane) and `cli` (host operator/launcher). * the orchestrator holds only the signing *key* and verifies; `dispatch` gates each route by role — `gateway` reaches /resolve + /supervise/ {propose,poll}, everything else is `cli`-only (401 unauthenticated, 403 wrong role). * the gateway is handed a pre-minted `gateway` token it cannot rewrite into `cli`; the host CLI mints its own `cli` token from the host key. * `gateway_init` scopes the signing key to the orchestrator process and the gateway token to the data-plane daemons, so even in the combined infra container a compromised data-plane daemon never sees the key. Launchers (docker gateway + infra, macOS infra) inject the minted token(s); Firecracker stays open behind its nft boundary. Open mode (no key) still grants full `cli` access — the fail-visible fallback for tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 05:22:56 +00:00
didericis-claude	1972c8c6e9	docs: add glossary of canonical bot-bottle terminology ... Defines Agent Provider, Agent Runtime, Agent/Agent Definition, Bottle/Bottle Definition, Sealed Bottle, Bottled Agent, and Active Bottle. Links from docs/README.md and AGENTS.md for discoverability. Closes #474 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-07-24 00:12:32 -04:00
didericis-claude	5d109ea290	CLI cleanup: remove info, rename list subcommands (#472) ... - Remove `info` command - Rename `list active` → `active` (new top-level command) - Rename `list available` → `list` (no subcommand argument) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>	2026-07-24 00:02:19 -04:00
didericis-claude	72fdb1d14b	test: annotations + coverage for the supervise RPC seam ... Add pyright-strict parameter/return annotations to the fake resolvers and test helpers, and cover the new control-plane validation/403 branches (/supervise/propose + /supervise/poll) plus the supervise-server and egress poll-error fail-closed paths, so the diff-coverage gate stays above 90%. No production behavior change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 03:24:30 +00:00