2039ef635fc521a6a2c46749d3203777c2588c3e
1443 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
c4ccd74f9b |
refactor(gateway): macOS gateway under the Gateway service ABC
Extract the Apple-Container gateway data plane out of `MacosInfraService` into `MacosGateway`, the macOS implementation of the shared `Gateway` service. `connect_to_orchestrator(url, gateway_token)` runs the triple-homed gateway container (egress + agent + control networks) carrying the mitmproxy CA and the pre-minted `gateway` token; `address()` / `ca_cert_pem()` / `provisioning_transport()` round out the contract. The infra service now composes the gateway: `ensure_running` mints the role-scoped `gateway` JWT (it holds the signing key; the gateway never does — #469) and hands it to `gateway().connect_to_orchestrator(...)`, and `ca_cert_pem` delegates to the service. The inlined `_ensure_gateway_container` + CA-read are gone. `AppleGatewayTransport` + the gateway container name/label now live with the gateway module, breaking the old infra→provision coupling. Splits the gateway-run + CA tests out of test_macos_infra into a dedicated test_macos_gateway; the infra tests mock `svc.gateway`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
05df21f210 |
refactor(gateway): introduce the Gateway service ABC (docker impl)
Replace the docker backend's ad-hoc gateway wiring with the shared `Gateway` service ABC (in the gateway package), the first of the two per-host service classes that supersede the per-backend infra glue. The contract is backend-neutral: `connect_to_orchestrator(url, gateway_token)` binds the gateway to its control plane and brings it up, `address()` reports the agent-facing proxy target, `ca_cert_pem()` vends the mitmproxy CA, and `provisioning_transport()` hands back the exec/cp seam git-gate provisioning stages per-bottle repos + deploy keys through. `GatewayProvisionError` + `GatewayTransport` move to the ABC so every backend shares them. Key trust-model change: the gateway no longer mints its own token. It never holds the signing key (#469), so the orchestrator mints the role-scoped `gateway` JWT and hands it in via `connect_to_orchestrator`; the gateway only injects it. `DockerGateway.ensure_running` becomes `connect_to_orchestrator` (stashing the URL + token as instance state), and the docker launch flow reads `address()` / `provisioning_transport()` off the service rather than re-deriving the container IP + transport itself. macOS + firecracker gateways move under the ABC in following commits. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
eb9723027b |
test(macos): mock ensure_networks in TestInfraEnsureRunning
tracker-policy-pr / check-pr (pull_request) Successful in 10s
test / integration-docker (pull_request) Successful in 16s
test / unit (pull_request) Successful in 43s
test / integration-firecracker (pull_request) Failing after 2m43s
lint / lint (push) Successful in 2m46s
test / coverage (pull_request) Has been skipped
test / publish-infra (pull_request) Has been skipped
These four ensure_running() tests patched container_mod but not ensure_networks (imported into infra.py from .gateway), so they shelled out to the real Apple `container` CLI and errored on Linux (FileNotFoundError: 'container') — the dev + CI platform for this branch. Mock it so they're true unit tests, green on any platform. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
18d9b81add |
feat(firecracker): split orchestrator and gateway into separate VMs (PRD 0070)
tracker-policy-pr / check-pr (pull_request) Successful in 12s
test / integration-docker (pull_request) Successful in 19s
lint / lint (push) Successful in 53s
test / unit (pull_request) Failing after 1m46s
test / integration-firecracker (pull_request) Failing after 2m42s
test / coverage (pull_request) Has been skipped
test / publish-infra (pull_request) Has been skipped
Now that #469 got the DB off the data plane, the Firecracker infra runs as two microVMs instead of one — mirroring the docker/macos plane split: * orchestrator VM (ORCH_IFACE) — control plane + buildah image builds; sole DB opener; host-seeded signing key. No gateway daemons. * gateway VM (new GW_IFACE) — egress / git-http / supervise data plane; mitmproxy CA + a host-minted `gateway` JWT (never the key). Reaches the orchestrator only over the one nft forward rule its link allows. Both boot the SAME shared infra rootfs; a `bb_role=` kernel-cmdline arg selects which plane a VM's PID-1 init starts, so there is still one published artifact. The gateway learns the orchestrator's address via `bb_orch=` on the cmdline (no IP baked into the artifact). Isolation is nearly free: agents were already nft-dropped except the DNAT'd gateway ports, so re-pointing that single DNAT rule at the gateway VM (`dnat to gw_guest`) severs every agent's L3 route to the control plane. The only added nft is the second infra link's mirror block (masquerade egress + forward accept, which subsumes gateway->orchestrator) in the shared shell script and the NixOS module. netpool gains GW_IFACE + gw_slot() (the /31 above the orch link); firecracker_vm.boot gains extra_boot_args for the role cmdline; infra_vm ensure_running() boots + adopts the pair (orchestrator first, then the gateway that resolves policy against it) and returns an InfraEndpoint mirroring the docker/macos shape. Builds stay in the orchestrator (PRD 0070 v1); the gateway is the slim unit. Unit-tested (test_firecracker_infra_vm rewritten for two VMs; gw_slot helper test added); the KVM boot / L3-isolation checks are validated on a Firecracker host. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
a74894c6f6 |
docs(prd-0070): correct the firecracker-split difficulty framing
test / integration-docker (pull_request) Successful in 15s
tracker-policy-pr / check-pr (pull_request) Successful in 15s
test / unit (pull_request) Failing after 36s
test / integration-firecracker (pull_request) Successful in 3m23s
test / coverage (pull_request) Has been skipped
test / publish-infra (pull_request) Has been skipped
Reevaluated against the actual nft ruleset: the firecracker split's isolation is nearly free, not "the real work." Agent VMs are already dropped except the DNAT'd gateway ports, so re-pointing that single DNAT rule at the gateway VM (dnat to $(gw_guest)) isolates agents from a separate orchestrator VM with zero new agent rules; the only added nft is a mirrored gateway link + one gateway->orchestrator forward rule. The effort is the mechanical second-VM lifecycle, and it's validatable on a Firecracker host. Sharpened the Access bullet accordingly and added a note to Sequencing decoupling the original consolidation's "real work" (the broker shim) from the split, whose difficulty actually runs the other way (docker/macOS do the real work — new control network, dual-homed gateway, the token-only->L3 upgrade; firecracker is nearly free). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
b676ce3156 |
feat(macos): split orchestrator and gateway into separate containers (PRD 0070)
test / integration-docker (pull_request) Successful in 18s
tracker-policy-pr / check-pr (pull_request) Successful in 22s
test / unit (pull_request) Failing after 42s
lint / lint (push) Successful in 57s
test / integration-firecracker (pull_request) Successful in 3m22s
test / coverage (pull_request) Has been skipped
test / publish-infra (pull_request) Has been skipped
The single Apple container existed only because two guests writing one bot-bottle.db over virtiofs would race incoherent fcntl locks; #469 removed that (the data plane no longer opens the DB), so the macOS backend now runs the planes as two containers like docker: * bot-bottle-mac-orchestrator — lean control plane on the host-only `bot-bottle-mac-control` network only (image Dockerfile.orchestrator, `-m bot_bottle.orchestrator`). Sole mounter of the container-only DB volume; holds the signing key. The CLI + gateway reach it at its control-network address. * bot-bottle-mac-infra — the gateway, triple-homed on the NAT egress net, the host-only agent net, and the control net. Resolves the orchestrator by IP (Apple has no container DNS) via BOT_BOTTLE_ORCHESTRATOR_URL; holds the CA + gateway JWT. Agents sit on the agent network only, so they have no route to the control plane. ensure_networks gains the control network; MacosInfraService brings up the orchestrator then the gateway; probe_orchestrator_url + ca_cert_pem target the right containers. pyright 0 errors; unit suite green (2274). Needs a real Apple-container host to validate the networking. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
d62d19a2e7 |
feat(docker): split orchestrator and gateway into separate containers (PRD 0070)
tracker-policy-pr / check-pr (pull_request) Successful in 6s
test / integration-docker (pull_request) Successful in 13s
test / unit (pull_request) Failing after 37s
lint / lint (push) Successful in 57s
test / integration-firecracker (pull_request) Successful in 3m19s
test / coverage (pull_request) Has been skipped
test / publish-infra (pull_request) Has been skipped
Now that #469 got the DB off the data plane, the docker backend runs the control plane and data plane as two containers instead of the combined `bot-bottle-infra` container: * bot-bottle-orchestrator — the lean control-plane container (image Dockerfile.orchestrator, `-m bot_bottle.orchestrator`). Joins the new `bot-bottle-orchestrator` control network (`--internal`) only, plus a host-loopback publish for the CLI. Sole opener of bot-bottle.db; holds the signing key; no CA, no gateway daemons. * bot-bottle-orch-gateway — the data-plane container (DockerGateway), dual-homed on the agent `bot-bottle-gateway` network AND the control network, so it resolves the orchestrator by name (http://bot-bottle-orchestrator:8099) while agents — never on the control network — have no route to the control plane (the L3 block, not just the JWT). Holds the gateway JWT + the mitmproxy CA. DockerInfraService now orchestrates the pair (builds gateway + orchestrator images, brings up the orchestrator then the gateway) and exposes gateway_name; the launch flow attributes agents against the gateway container. DockerGateway gains a control_network it joins after run. rotate_ca / the CA read target the gateway container. The combined Dockerfile.infra is no longer built by docker (firecracker/macOS still use it until their splits). pyright 0 errors; unit suite green (2273). Integration tests updated for the two-container shape but need a real-docker run to validate the networking. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
7e9ad8a78d |
Merge main into fix/db-off-data-plane-469
tracker-policy-pr / check-pr (pull_request) Successful in 13s
test / integration-docker (pull_request) Successful in 35s
test / unit (pull_request) Successful in 47s
lint / lint (push) Successful in 57s
test / integration-firecracker (pull_request) Successful in 3m35s
test / coverage (pull_request) Successful in 32s
test / publish-infra (pull_request) Has been skipped
Brings in main's 8 commits — #470 (backend-agnostic CI guards: BackendStatus enum, quiet status(), is_backend_available/is_backend_ready, tests/_backend.py skip_unless_backend) plus firecracker status() readiness checks (kvm/kernel/ dropbear/mke2fs). Reconciled #470's additions into the reorg'd backend structure: * BackendStatus enum + the status(*, quiet=) ABC signature -> backend/base.py * is_backend_available / is_backend_ready -> backend/selection.py * exposed all three through the lazy backend facade (__init__). Integration-test conflicts were our control_plane->orchestrator renames vs main's skip_unless_docker -> skip_unless_backend swap: kept our refactored import paths + main's guard. Repointed main's new is_backend_* tests to patch selection._backends (our moved home) instead of the facade. pyright: 0 errors. Full unit suite green (2272). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
a68ee778f1 |
firecracker status(): open /dev/kvm O_RDWR; check kernel, dropbear, mke2fs
test / integration-docker (push) Successful in 15s
test / unit (push) Successful in 40s
Update Quality Badges / update-badges (push) Successful in 44s
lint / lint (push) Successful in 58s
test / integration-firecracker (push) Successful in 5m5s
test / coverage (push) Successful in 49s
test / publish-infra (push) Successful in 2m4s
Two reviewer findings addressed: 1. _kvm_accessible() now opens /dev/kvm with O_RDWR|O_CLOEXEC instead of read-only. VM creation requires write access; a read-only descriptor can satisfy KVM_GET_API_VERSION but fails at boot time. 2. status() now checks every hard prerequisite that require_firecracker() checks at launch: guest kernel image, static dropbear binary, and mke2fs. Previously a host with a configured TAP pool but missing artifacts could pass status() and then fail during launch. Regression coverage: TestFirecrackerKvmCheck gains test_kvm_open_rdwr_fails; TestFirecrackerArtifactCheck covers each missing artifact; existing TestFirecrackerStatus fixtures updated to stub the new checks. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
de02d13ccf |
firecracker status(): add binary and KVM readiness checks
Adds `_firecracker_binary_ok()` (runs `firecracker --version`) and `_kvm_accessible()` (opens /dev/kvm, issues KVM_GET_API_VERSION ioctl) and gates `status()` on both, so the launch preflight reports missing binary or inaccessible KVM before the caller ever attempts a boot. Regression tests in TestFirecrackerBinaryCheck, TestFirecrackerKvmCheck, and TestFirecrackerStatusRuntime cover all branches. Updated the pre-existing TestFirecrackerStatus fixture to also stub the two new helpers so it still exercises only the tap-pool/nft logic it was designed for. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
c740d1e145 |
fix: add missing type annotations and remove unused variables in new tests
Pyright flagged three inner class `status` methods missing a type annotation on the `quiet` parameter, and two unused variables (`written`, `real_status`) left over from an earlier draft of test_docker_quiet_true_suppresses_stderr. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
177721c286 |
test: cover is_backend_available, is_backend_ready, and status(quiet=True)
Add unit tests for the three new public symbols introduced by this PR: - TestIsBackendAvailable — delegates to has_backend; returns True/False correctly for available and unavailable backends - TestIsBackendReady — unknown names return False; known names return True/False based on status() return code; quiet flag is forwarded - TestBackendStatusQuiet — verifies the quiet=True branch in DockerBottleBackend, FirecrackerBottleBackend, and MacosContainerBottleBackend: return code is propagated and diagnostic stderr is suppressed Together these bring diff-coverage for the PR's changed lines to 100%. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
5bcf3db1f8 |
fix: align skip-guard tests with is_backend_ready and drop unused import
tests/_backend.py imported is_backend_available but never called it, causing a pyright reportUnusedImport error. Remove the unused import. test_backend_skip_guards.py was patching tests._backend.has_backend but _backend.py calls is_backend_ready — the mock never intercepted the real call, producing AttributeError at test runtime. Update all patch targets to is_backend_ready and add quiet=False to the assert_called_once_with assertion to match the actual call signature. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
dcd658ece1 |
feat(backend): BackendStatus enum, quiet status(), is_backend_ready()
Add a module-level BackendStatus(IntEnum) with READY=0 to replace magic 0 comparisons. Extend BottleBackend.status() with a quiet parameter: quiet=False (default) prints diagnostics; quiet=True returns the code silently for programmatic checks. Add is_backend_available() (cheap PATH check, alias for has_backend) and is_backend_ready(name, *, quiet=False) (full status() check) to the backend package for callers that need to distinguish availability from readiness. Update tests/_backend.py guards to use is_backend_ready(quiet=False) so diagnostic output is printed during test discovery, giving the operator a concrete reason for each skip rather than a bare "prerequisites unavailable" message. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
|
dfb56f8fe9 |
ci: reuse backend.has_backend / backend status for readiness
Review feedback: the hand-rolled `/dev/kvm` + docker-daemon capability probes duplicated logic the CLI already owns. Delegate instead. - tests/_backend.py skip guards now gate on `bot_bottle.backend.has_backend` (each backend's `is_available()` classmethod — the same probe behind `./cli.py backend status`), dropping the bespoke `Capability` probes. - Remove tests/backend_preflight.py; the docker integration job runs `./cli.py backend status --backend=docker` as its preflight (clear per-check summary, non-zero exit when unready), matching the firecracker job. The firecracker preflight reverts to its original binary/KVM checks (backend status doesn't cover those). - Unit test + docs updated to match. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
64adf23775 |
ci: backend-agnostic integration guards + per-backend preflight
Integration tests now select their backend from BOT_BOTTLE_BACKEND and
skip on the capability that backend actually needs, instead of gating
every backend on unrelated Docker availability.
Task 1 — backend-agnostic guards (tests/_backend.py):
- Capability probes: docker_capability() (reachable daemon) and
firecracker_capability() (accessible /dev/kvm + firecracker on PATH,
Docker-independent). backend_capability()/selected_backend() resolve
the target from BOT_BOTTLE_BACKEND (default docker).
- skip_unless_selected_backend_available() for backend-agnostic tests
(test_sandbox_escape) — runs through whichever backend is selected and
checks that backend's real capability.
- skip_unless_backend("docker") for Docker-implementation tests
(DockerBroker, DockerGateway, backend.docker.*) — they no-op under a
non-Docker run rather than testing internals that run doesn't target.
- Retires tests/_docker.py; the KVM job no longer needs SKIP_DOCKER_TESTS
to steer Docker-only classes.
Task 2 — explicit per-backend skip visibility:
- tests/backend_preflight.py prints a clear PASS/FAIL capability line and
exits non-zero when the selected backend is missing.
- Both integration jobs run it as a preflight, so absent infrastructure
is surfaced at the job level instead of hidden among unittest.skip
lines. The docker job replaces its soft "Show environment" step; the
firecracker job keeps its richer backend-status check.
Docs (tests/README.md, docs/ci.md) updated; unit coverage for the probes,
guards, and preflight in test_backend_skip_guards.py.
Closes #414
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
|
3d3d8fd7e8 |
docs(prd-0070): record remote-terminal resolution; track detail in #478
tracker-policy-pr / check-pr (pull_request) Successful in 19s
test / integration-docker (pull_request) Successful in 42s
test / unit (pull_request) Successful in 52s
test / integration-firecracker (pull_request) Successful in 3m32s
test / coverage (pull_request) Successful in 26s
test / publish-infra (pull_request) Has been skipped
Remote terminal is an orchestrator operation, not a data-plane one: the agent PTY is exec_agent-sourced (ssh -t / container exec --tty / docker exec), which the orchestrator owns — the gateway never holds a PTY. A --remote flag publishes a running agent's session; local + remote clients share it via a session multiplexer (one PTY, many clients), operator-role-gated, so the split's "one authenticated remote door" holds for terminals too. Detailed design (multiplexer location, input arbitration, per-client rendering, always-on vs --remote) tracked in #478. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
195e0f249d |
docs(prd-0070): resolve build-placement open question — builds stay in the orchestrator
tracker-policy-pr / check-pr (pull_request) Successful in 11s
test / integration-docker (pull_request) Successful in 31s
test / unit (pull_request) Successful in 46s
test / integration-firecracker (pull_request) Successful in 3m23s
test / coverage (pull_request) Successful in 42s
test / publish-infra (pull_request) Has been skipped
v1 decision: agent-image builds run in the orchestrator (the control-plane side already owns launches and hosts builds in today's combined unit; the data-plane gateway must not build). Consequence recorded in the memory section: the orchestrator keeps the ~4 GB buildah ceiling and the gateway is the slim unit (~1 GB), so the split is ~memory-neutral-to-+1 GB for now — the ~1.5 GB win waits on a later move to a dedicated slim build unit (#468-adjacent). Also flags the tension of co-locating buildah with the control plane. Removed the corresponding open question. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
ee26e9044f |
docs(prd-0070): fold in the orchestrator/gateway plane-separation design
tracker-policy-pr / check-pr (pull_request) Successful in 16s
test / integration-docker (pull_request) Successful in 41s
test / unit (pull_request) Successful in 46s
test / integration-firecracker (pull_request) Successful in 3m40s
test / coverage (pull_request) Successful in 20s
test / publish-infra (pull_request) Has been skipped
Now that #469 got the DB off the data plane (removing the cross-kernel SQLite-coherence reason the planes were consolidated), separating them into distinct runtimes is possible again. Adds a Design subsection to 0070 covering: * Access — the only hop that changes is gateway->orchestrator (loopback -> network). A dedicated `bot-bottle-orchestrator` --internal control network with a dual-homed gateway gives Docker/macOS the L3 agent isolation that only Firecracker's nft currently has (both are token-only today); Firecracker gets it nearly free (agents already can't reach non-DNAT'd ports). * Memory — firecracker's mem_size_mib is a fixed ceiling (no reclaim without a balloon); the 4 GB combined VM is really buildah headroom. Right-sized: ~512 MiB orchestrator + ~1 GB gateway < today's 4 GB, if agent-image builds move off the gateway. * iroh belongs in the orchestrator (control-plane remote entry point); keep the data-plane gateway slim and P2P-free. Plus open questions: where agent-image builds run post-split, and whether agent-PTY-to-mobile relays through the orchestrator's iroh endpoint. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
e7fe00e2f5 |
fix(firecracker): keep the host key canonical instead of clobbering the host token file
tracker-policy-pr / check-pr (pull_request) Successful in 16s
test / integration-docker (pull_request) Successful in 21s
lint / lint (push) Successful in 1m3s
test / unit (pull_request) Successful in 1m55s
test / integration-firecracker (pull_request) Successful in 3m29s
test / coverage (pull_request) Successful in 17s
test / publish-infra (pull_request) Has been skipped
Reimplements the dropped firecracker fix on the refactored code, addressing didericis-codex's blocking review on #471. The previous firecracker path let the VM generate its own signing key on the guest volume and had `_with_signing_key()` unconditionally overwrite the single host-wide orchestrator-token with it. That breaks a co-running Docker/macOS control plane: their orchestrators still verify with the old key while new host clients start signing `cli` tokens with the guest key, so supervise/teardown/policy calls against those backends begin returning 401 (cross-backend operation is supported). Keep the host token file the single source of truth. The launcher now pushes the host-canonical key into the freshly booted infra VM over SSH (atomic write, mirroring persist_env_var_secret) via `_push_signing_key()`; the VM's init waits for it and refuses to start the control plane — rather than run OPEN — if it never arrives. Nothing writes back to the host file, so other backends are untouched, and the best-effort silent path is gone (both the push and the guest fail loudly). Ported onto the reorg'd names (host_orchestrator_token, BOT_BOTTLE_ORCHESTRATOR_ TOKEN, orchestrator-token, bot_bottle.orchestrator_auth, bot_bottle.gateway. bootstrap). pyright: 0 errors. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
466f4ee13a |
Merge main into fix/db-off-data-plane-469
tracker-policy-pr / check-pr (pull_request) Successful in 13s
test / integration-docker (pull_request) Successful in 34s
test / unit (pull_request) Successful in 49s
lint / lint (push) Successful in 57s
test / integration-firecracker (pull_request) Successful in 3m29s
test / coverage (pull_request) Successful in 20s
test / publish-infra (pull_request) Has been skipped
Brings in the two commits main gained: #472 (CLI cleanup — remove `info`, split `list active` into a top-level `active`, simplify `list`) and the terminology glossary. #472 was written against the old flat cli/ layout, so its semantics are reconciled into the reorg'd cli/commands/ structure: * new `active` command lives at cli/commands/active.py, registered in the lazy registry and listed in `help` (not left at the flat path). * `info` removed: deleted cli/commands/info.py, dropped from the registry and `help`. * `list` simplified to list-available-only, using the reorg's os.getcwd()/constants.PROG conventions. * the `list active` -> `active` doc wording applied to the backend docstrings that our split moved into base.py / selection.py. Our reorg wins for the fully-rewritten dispatcher (cli/__init__ shim, backend facade). pyright: 0 errors. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
bef45348f5 |
fix(types): satisfy pyright strict on the refactored facades
tracker-policy-pr / check-pr (pull_request) Successful in 13s
test / integration-docker (pull_request) Successful in 33s
test / unit (pull_request) Successful in 51s
lint / lint (push) Successful in 57s
test / integration-firecracker (pull_request) Successful in 3m25s
test / coverage (pull_request) Successful in 43s
test / publish-infra (pull_request) Has been skipped
Two facade issues from the reorg:
* orchestrator/__init__.py used `__all__ = list(_LAZY)`, which pyright
strict can't analyze (reportUnsupportedDunderAll) — and because it
isn't a static literal, the TYPE_CHECKING re-export imports read as
unused. Replaced with the explicit literal list the other lazy
facades already use.
* manifest/index.py imported six piece types (ManifestAgentProvider,
EGRESS_AUTH_SCHEMES, ManifestEgressConfig/Route, ManifestGitEntry,
ManifestKeyConfig) it never referenced — the package facade re-exports
those straight from their submodules, so index.py needn't import them.
Dropped the dead imports.
pyright: 0 errors (was 24). Full unit suite green (2243).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
|
e2740842a0 |
refactor: make the two remaining heavy __init__ modules lazy
tracker-policy-pr / check-pr (pull_request) Successful in 13s
test / integration-docker (pull_request) Successful in 35s
test / unit (pull_request) Successful in 51s
lint / lint (push) Failing after 58s
test / integration-firecracker (pull_request) Successful in 3m32s
test / coverage (pull_request) Successful in 19s
test / publish-infra (pull_request) Has been skipped
Audit of package __init__ import cost turned up two eager ones: orchestrator/__init__.py (27 -> 2): was pure `from .X import Y` re-exports (registry_store, broker, docker_broker, gateway, service, server). Because importing any submodule runs the parent __init__, the CLI migration gate (orchestrator.store.store_manager, hit on every command) transitively dragged backend.docker, docker_broker, server, and http.server. Converted to the same lazy __getattr__ + _LAZY facade used by manifest/backend/egress/git_gate; the re-export API is unchanged and the docker/http drag is gone. cli/commands/__init__.py (78 -> 23): the registry imported all twelve handlers to build COMMANDS, so importing one command pulled all of them plus their deps. COMMANDS now maps each name to a thin lazy wrapper that imports its handler's module on first dispatch. Values stay callable, so the dispatcher and the patch.dict dispatch tests are unchanged; a CLI run now loads only the one command it dispatches. The remaining 23 is the dispatcher's own baseline (store_manager migration gate + help + log). Full unit suite green (2243); `bb help` + dispatch/migration-gate tests verified. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
3efb014ace |
refactor(orchestrator): move registry into store/registry_store
tracker-policy-pr / check-pr (pull_request) Successful in 11s
test / integration-docker (pull_request) Successful in 18s
lint / lint (push) Failing after 58s
test / unit (pull_request) Successful in 2m14s
test / integration-firecracker (pull_request) Successful in 3m25s
test / coverage (pull_request) Successful in 35s
test / publish-infra (pull_request) Has been skipped
registry.py is a SQLite-backed store (its class is already RegistryStore), so it belongs with the other orchestrator-owned stores rather than at the package root. Moved orchestrator/registry.py -> orchestrator/store/ registry_store.py, joining queue_store / secret_store / config_store. Repointed the in-package importers (.registry -> .store.registry_store) and the tests, bumped the moved file's relative-import depths, renamed the test file to match, and listed it in the store package docstring. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
cb3a74edb0 |
refactor(orchestrator): collapse supervisor package into supervisor.py
orchestrator/supervisor/ held only __init__.py, so the package no longer earns its own directory. Moved it to orchestrator/supervisor.py and removed the dir. External imports are unchanged (orchestrator.supervisor resolves identically as a module); only the file's own relative-import depths shift up one level. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
ca1d341d4f |
refactor: unify component naming — control_plane/control_auth -> orchestrator
tracker-policy-pr / check-pr (pull_request) Successful in 14s
test / integration-docker (pull_request) Successful in 18s
test / unit (pull_request) Successful in 46s
lint / lint (push) Failing after 54s
test / integration-firecracker (pull_request) Successful in 3m21s
test / coverage (pull_request) Successful in 18s
test / publish-infra (pull_request) Has been skipped
The codebase used "control plane" both as an architectural role term AND
as an identifier alias for the orchestrator component, producing
duplicate names for one thing (control_plane_url vs orchestrator_url,
CONTROL_PLANE_PORT, host_control_plane_token, …). Going forward the
concrete component is always named for what it is — Gateway or
Orchestrator — and the plane vocabulary is reserved for prose (module
descriptions, the security argument).
Renamed (identifiers + the in-repo env/wire/file string values, all
setters/getters are in this repo so the change is atomic):
ControlPlaneServer -> OrchestratorServer
control_plane_url -> orchestrator_url
probe_control_plane_url -> probe_orchestrator_url
host_control_plane_token -> host_orchestrator_token
CONTROL_PLANE_PORT -> ORCHESTRATOR_PORT
CONTROL_PLANE_TOKEN_ENV/FILE -> ORCHESTRATOR_TOKEN_ENV/FILENAME
BOT_BOTTLE_CONTROL_PLANE_TOKEN-> BOT_BOTTLE_ORCHESTRATOR_TOKEN
control-plane-token (file) -> orchestrator-token
control_auth (module) -> orchestrator_auth (stays top-level;
the gateway imports it and must not
import the orchestrator/ package)
CONTROL_AUTH_HEADER -> ORCHESTRATOR_AUTH_HEADER
x-bot-bottle-control-auth -> x-bot-bottle-orchestrator-auth
CONTROL_AUTH_JWT_ENV -> ORCHESTRATOR_AUTH_JWT_ENV
BOT_BOTTLE_CONTROL_AUTH_JWT -> BOT_BOTTLE_ORCHESTRATOR_AUTH_JWT
_control_auth_headers -> _orchestrator_auth_headers
Prose plane-terms ("control plane", "data plane") are preserved,
including the test name test_data_plane_daemons_get_jwt_not_key (it
names the security invariant). Gateway and orchestrator verified to
agree on the renamed wire header; full unit suite green (2243).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
|
4166057abc |
refactor(orchestrator): rename control_plane module to server
test / integration-docker (pull_request) Successful in 17s
test / unit (pull_request) Successful in 50s
lint / lint (push) Failing after 1m0s
test / integration-firecracker (pull_request) Successful in 3m25s
test / coverage (pull_request) Successful in 16s
test / publish-infra (pull_request) Has been skipped
tracker-policy-pr / check-pr (pull_request) Failing after 12m7s
orchestrator/control_plane.py -> orchestrator/server.py. Within the orchestrator package the "control_plane" filename stutters (the orchestrator *is* the control plane), and `orchestrator.server` reads as "the orchestrator's HTTP server", pairing with service.py (domain logic) and matching gateway/supervisor/server.py. Scope is the module name only. The class ControlPlaneServer, the CONTROL_AUTH_HEADER constant, and the "control plane" architectural term (CONTROL_PLANE_PORT, host_control_plane_token, control_plane_url, …) are deliberately unchanged — that's the load-bearing control-plane/data-plane distinction. Test file renamed to match; full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
b276227cbb |
refactor(cli): registry in commands/__init__, dispatcher main() in __main__
tracker-policy-pr / check-pr (pull_request) Successful in 8s
test / integration-docker (pull_request) Successful in 17s
test / unit (pull_request) Successful in 48s
lint / lint (push) Failing after 2m56s
test / integration-firecracker (pull_request) Successful in 3m33s
test / coverage (pull_request) Successful in 21s
test / publish-infra (pull_request) Has been skipped
Split the cli package's two responsibilities out of __init__:
* commands/__init__.py now assembles the COMMANDS registry and
NO_MIGRATION_COMMANDS from the per-command modules — the command
surface lives entirely under commands/.
* __main__.py now owns main() (dispatch + migration gate + exit-code
mapping) alongside the runnable entry guard.
cli/__init__.py shrinks to a shim that re-exports main / COMMANDS /
NO_MIGRATION_COMMANDS, so bot_bottle.cli.main (repo-root cli.py entry)
and the tests' bot_bottle.cli.COMMANDS keep working unchanged. Verified
`python -m bot_bottle.cli` (runpy) and `cli.py` both dispatch; full unit
suite green (2243).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
|
3ccd308613 |
refactor(cli): make help a first-class command; drop usage() from dispatcher
tracker-policy-pr / check-pr (pull_request) Successful in 10s
test / integration-docker (pull_request) Successful in 34s
test / unit (pull_request) Successful in 46s
lint / lint (push) Failing after 57s
test / integration-firecracker (pull_request) Successful in 3m25s
test / coverage (pull_request) Successful in 17s
test / publish-infra (pull_request) Has been skipped
Move the top-level usage/command-list text out of the dispatcher into commands/help.py as cmd_help, registered in COMMANDS so `bb help` now works as a real command. The dispatcher's -h/--help, no-args, and unknown-command fallbacks call cmd_help() for the text while keeping their exit codes (0 for help/-h, 2 for bare no-args, Die for unknown). Added `help` to NO_MIGRATION_COMMANDS so it prints without a migrated DB, and listed it in the command summary. Full unit suite green (2243); dispatch + migration-gate tests pass, `bb help` verified. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
a21f2358c6 |
refactor(cli): trim _common to a PROG constants leaf; inline os.getcwd()
tracker-policy-pr / check-pr (pull_request) Successful in 16s
test / integration-docker (pull_request) Successful in 21s
lint / lint (push) Failing after 58s
test / unit (pull_request) Successful in 2m11s
test / integration-firecracker (pull_request) Successful in 3m27s
test / coverage (pull_request) Successful in 16s
test / publish-infra (pull_request) Has been skipped
_common had rotted into a junk drawer. Trim it to what actually
justifies a shared leaf module and rename for legibility:
* REPO_DIR: deleted (dead — nothing imported it).
* read_tty_line: dropped the pointless re-export; cleanup/start/init
now import it straight from bot_bottle.util.
* USER_CWD: deleted. It captured os.getcwd() at import time, but
nothing chdirs and no test patched it, so it was equivalent to a
live os.getcwd() — inlined at the six call sites.
* PROG: kept, now the sole member. It's still a leaf (both the
dispatcher and the commands it imports need it) so it can't move
into __init__ without a circular import.
_common.py -> constants.py. Full unit suite green (2243); CLI dispatch
verified.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
|
50a67c04bd |
refactor(cli): group subcommand handlers under cli/commands/
test / integration-docker (pull_request) Successful in 18s
tracker-policy-pr / check-pr (pull_request) Successful in 17s
test / unit (pull_request) Failing after 41s
lint / lint (push) Failing after 57s
test / integration-firecracker (pull_request) Successful in 3m26s
test / coverage (pull_request) Has been skipped
test / publish-infra (pull_request) Has been skipped
Move the eleven per-command modules (backend, cleanup, commit, edit, info, init, list, login, resume, start, supervise) into a new bot_bottle/cli/commands/ package, leaving the dispatcher (__init__), entrypoint (__main__), and shared helpers (_common, tui) at the cli root. Makes the command surface obvious at a glance and separates handlers from the plumbing that registers them. Updated the dispatcher's COMMANDS imports to .commands.*, bumped the moved files' relative-import depths (.._common, .. import tui, sibling .start unchanged), and repointed test references to bot_bottle.cli.commands.*. Full unit suite green (2243); CLI dispatch verified via `python -m bot_bottle.cli --help`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
82d02d2c4b |
refactor(gateway): move egress entrypoint into gateway/egress
tracker-policy-pr / check-pr (pull_request) Successful in 15s
test / integration-docker (pull_request) Successful in 17s
test / unit (pull_request) Successful in 46s
lint / lint (push) Failing after 58s
test / integration-firecracker (pull_request) Successful in 3m35s
test / coverage (pull_request) Successful in 16s
test / publish-infra (pull_request) Has been skipped
egress_entrypoint.sh is the mitmdump launcher for the egress data plane, so it belongs with the gateway egress service, not at the package root. Moved to bot_bottle/gateway/egress/entrypoint.sh (prefix stripped now that the package namespaces it). Only the Dockerfile.gateway COPY source changes; the runtime target (/app/egress-entrypoint.sh, referenced by bootstrap's egress DaemonSpec) is unchanged, and the infra images inherit it via FROM gateway. Updated the test that resolves the script path and the infra-artifact docstring example. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
74dc984cf8 |
refactor(gateway): rename bootstrap's _Supervisor to _DaemonManager
tracker-policy-pr / check-pr (pull_request) Successful in 11s
test / integration-docker (pull_request) Successful in 17s
lint / lint (push) Failing after 53s
test / unit (pull_request) Successful in 2m4s
test / integration-firecracker (pull_request) Successful in 3m32s
test / coverage (pull_request) Successful in 18s
test / publish-infra (pull_request) Has been skipped
The PID-1 process manager in gateway/bootstrap.py was named _Supervisor, which collided conceptually with the supervise permissions service (Supervisor / gateway.supervisor). It manages the _DaemonSpec child set, so _DaemonManager names what it does without the "supervise" echo. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
ce744a85c4 |
refactor(gateway): split data-plane files into egress/supervisor/git_gate services
tracker-policy-pr / check-pr (pull_request) Successful in 11s
test / integration-docker (pull_request) Successful in 17s
test / unit (pull_request) Successful in 49s
lint / lint (push) Failing after 2m49s
test / integration-firecracker (pull_request) Successful in 3m35s
test / coverage (pull_request) Successful in 18s
test / publish-infra (pull_request) Has been skipped
Group the gateway's data-plane modules into three service sub-packages mirroring the host-side trio (bot_bottle.egress / .supervisor / .git_gate): gateway/egress/ addon_core, addon, dlp_config, dlp_detectors gateway/supervisor/ server (was supervise_server) gateway/git_gate/ render, http_backend Prefix-stripped filenames now that the package namespaces them; each sub-package has a thin docstring __init__ (no eager imports, cheap leaf loads). The two cross-cutting files stay at the gateway root: policy_resolver (shared per-client lookup) and gateway_init, renamed to bootstrap now that gateway/ already namespaces it. Updated all importers (bot_bottle + tests), the in-VM/container `-m` launch strings, the Dockerfile.gateway addon shim + ENTRYPOINT, and the five gateway entries in scripts/critical-modules.txt. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
a446551acb |
refactor(egress): make Egress a service class in an egress package
tracker-policy-pr / check-pr (pull_request) Successful in 10s
test / integration-docker (pull_request) Successful in 17s
test / unit (pull_request) Successful in 46s
lint / lint (push) Failing after 2m44s
test / integration-firecracker (pull_request) Successful in 3m35s
test / coverage (pull_request) Successful in 20s
test / publish-infra (pull_request) Has been skipped
Split the flat egress.py into an egress/ package: neutral DTOs (EgressRoute, EgressPlan) in plan.py, and the concrete Egress service class plus rendering/env helpers in service.py, behind a thin __getattr__ facade so leaf imports stay cheap. Egress drops ABC and becomes a concrete service the backends call: resolve_token_values / agent_env_entries / prepare are now methods. Backend launch paths (docker, firecracker, macos_container) call Egress().<method>(...) instead of the module-level functions. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
450037b7e9 |
refactor(git-gate): make GitGate a service class in a git_gate package
test / integration-docker (pull_request) Successful in 15s
tracker-policy-pr / check-pr (pull_request) Successful in 10s
test / unit (pull_request) Successful in 47s
lint / lint (push) Failing after 58s
test / integration-firecracker (pull_request) Successful in 3m17s
test / coverage (pull_request) Successful in 20s
test / publish-infra (pull_request) Has been skipped
Turn the git_gate module into a package with GitGate as a concrete service class
(dropping the ABC), mirroring the Supervisor shape. The host-side git-gate
operations are now methods the backend drives:
git_gate/
__init__.py — thin __getattr__ facade (keeps `from bot_bottle.git_gate
import …` working; render/hook names lazily forwarded to
gateway.git_gate_render)
plan.py — GitGatePlan (the launch DTO the backend contract references)
service.py — GitGate: prepare / provision_dynamic_keys /
revoke_provisioned_keys / preflight_host_keys
provision.py — deploy-key lifecycle (was git_gate_provision.py)
host_key.py — host-key preflight (was git_gate_host_key.py)
The backend launch + base.py preflight now call GitGate() methods instead of the
free functions. The runtime rendering / in-gateway hook execution stay in
gateway/git_gate_render.py (data plane) — only the host-side service moved.
Because the facade forwards names lazily, the ~25 `from bot_bottle.git_gate
import GitGatePlan` call-sites are unchanged, and importing git_gate.plan (the
contract's dependency) no longer drags in the provisioning / forge-API code.
Full unit suite green (2243).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
|
14c28946a7 |
refactor(manifest): move Manifest/ManifestIndex to manifest.index; thin the package init
tracker-policy-pr / check-pr (pull_request) Successful in 12s
test / integration-docker (pull_request) Successful in 19s
lint / lint (push) Failing after 1m3s
test / unit (pull_request) Successful in 2m6s
test / integration-firecracker (pull_request) Successful in 3m26s
test / coverage (pull_request) Successful in 21s
test / publish-infra (pull_request) Has been skipped
manifest/__init__.py was a facade that eagerly imported every piece (agent, bottle, egress, git, loader, schema, util) and defined the aggregate Manifest / ManifestIndex, so importing a leaf like `manifest.util` paid 19 modules. Move the aggregate model — Manifest, ManifestIndex, and the bottle-resolution helpers — into manifest/index.py (which keeps the framework imports it needs), and make __init__ a thin __getattr__ facade over the 12 __all__ names (mapping each to its submodule), with a TYPE_CHECKING block. The package docstring is preserved. `import bot_bottle.manifest.util` drops 19 -> 3; the package itself is 2. All `from bot_bottle.manifest import Manifest/ManifestIndex/ManifestError/…` call-sites keep working via the lazy facade; no importer changes needed. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
eab9d15130 |
refactor(docker): fold run_docker into backend/docker/util; drop the docker_cmd shim
tracker-policy-pr / check-pr (pull_request) Successful in 8s
test / integration-docker (pull_request) Successful in 13s
test / unit (pull_request) Successful in 45s
lint / lint (push) Successful in 57s
test / integration-firecracker (pull_request) Successful in 3m17s
test / coverage (pull_request) Successful in 35s
test / publish-infra (pull_request) Has been skipped
docker_cmd.py existed as a top-level module solely so the orchestrator's docker components could share run_docker without dragging the (then-heavy) backend layer in. Now that backend/__init__ and backend/docker/__init__ are thin, importing backend.docker.util costs 6 modules instead of 76, so the shim's whole reason to exist is gone. Move run_docker into backend/docker/util.py (where the other docker subprocess primitives live) and delete docker_cmd.py. Backend siblings import it via `from .util import run_docker`; the two docker-specific orchestrator modules (docker_broker, rotate_ca) import `from ..backend.docker.util import run_docker`. No import cycle (backend.docker.util pulls nothing from orchestrator); orchestrator.__main__ stays lean at 28 modules. run_docker patch targets are unchanged (tests patch it in the importing module). Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
923d44bc09 |
refactor(backend): thin the docker/firecracker/macos-container package inits
tracker-policy-pr / check-pr (pull_request) Successful in 15s
test / integration-docker (pull_request) Successful in 20s
lint / lint (push) Successful in 53s
test / unit (pull_request) Failing after 2m5s
test / integration-firecracker (pull_request) Successful in 3m27s
test / coverage (pull_request) Has been skipped
test / publish-infra (pull_request) Has been skipped
The three concrete-backend package inits eagerly imported their `*BottleBackend` (which pulls the whole framework), so importing any leaf under them (e.g. `backend.docker.util`) paid ~78 modules. Give each the same thin `__getattr__` treatment as the top-level backend package: re-export the public class(es) lazily on first access, with a TYPE_CHECKING block for checkers. `import bot_bottle.backend.docker.util` drops from 78 -> 6 bot_bottle modules; the docker/firecracker/macos-container packages themselves are ~3 each. All `from bot_bottle.backend.docker import DockerBottleBackend` call-sites and the lazy selection path keep working. This is what makes backend leaves cheap — and unblocks moving docker_cmd into backend/docker/util if we want. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
3e62f31d8b |
refactor(backend): split the heavy backend/__init__ into base + selection, thin the package init
tracker-policy-pr / check-pr (pull_request) Successful in 12s
test / integration-docker (pull_request) Successful in 38s
test / unit (pull_request) Successful in 52s
lint / lint (push) Successful in 1m1s
test / integration-firecracker (pull_request) Successful in 3m33s
test / coverage (pull_request) Successful in 17s
test / publish-infra (pull_request) Has been skipped
backend/__init__.py eagerly imported the whole framework (manifest, egress,
git-gate, env, workspace, agent-provider), so importing ANY backend.* submodule
paid ~32 modules just to run the package init. Split it:
- backend/base.py — the abstract contract (BottleSpec, BottlePlan,
BottleCleanupPlan, ExecResult, ActiveAgent, Bottle,
BottleImages, BottleBackend). Carries the framework
imports; loaded only when a caller needs the contract.
- backend/selection.py — backend registry / selection / enumeration
(get_bottle_backend, _get_backends, _auto_select_backend,
has_backend, known_backend_names, enumerate_active_agents).
Concrete backends still imported lazily inside.
- backend/__init__.py — thin: a __getattr__ that resolves the public names
from those submodules on first access (+ a TYPE_CHECKING
block for checkers). Existing `from bot_bottle.backend
import X` and patch.object call-sites keep working.
`import bot_bottle.backend` drops from 32 -> 2 bot_bottle modules. Repointed the
backend-selection test's patch targets to the `selection` module (the functions
reference each other there now; the FirecrackerBottleBackend class-method
patches are unchanged — same class object).
Note: backend.docker.util is still heavy because backend/docker/__init__ eagerly
imports DockerBottleBackend — slimming the three sub-package inits is the
follow-up that makes the leaves cheap.
Full unit suite green (2243).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
|
b96a8b44e0 |
refactor(docker): rename OrchestratorService -> DockerInfraService, move to backend/docker
test / integration-docker (pull_request) Successful in 22s
lint / lint (push) Successful in 1m7s
test / unit (pull_request) Successful in 2m10s
test / integration-firecracker (pull_request) Successful in 3m35s
test / coverage (pull_request) Successful in 15s
test / publish-infra (pull_request) Has been skipped
tracker-policy-pr / check-pr (pull_request) Failing after 13m2s
OrchestratorService wasn't the orchestrator — it's the host-side lifecycle of the docker infra *container* (the one that runs the Orchestrator). It read as "the orchestrator as a service" and lived in orchestrator/lifecycle.py, while its siblings (macOS MacosInfraService, Firecracker infra_vm) live under their backend package. Rename it DockerInfraService and move it to backend/docker/infra.py alongside the docker backend, with its docker-only constants (INFRA_*/ORCHESTRATOR_* image + container names, daemon list, mount paths). orchestrator/lifecycle.py keeps only the backend-neutral pieces the other infra services share — DEFAULT_PORT, DEFAULT_STARTUP_TIMEOUT_SECONDS, OrchestratorStartError, source_hash — which macOS / firecracker / client still import from there. backend/docker/infra.py imports those (backend -> orchestrator is an allowed direction). Renamed the unit test to test_docker_infra.py. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
d41236c376 |
refactor(util): make render_diff generic (caller supplies side titles); move to root util
tracker-policy-pr / check-pr (pull_request) Successful in 8s
test / integration-docker (pull_request) Successful in 36s
test / unit (pull_request) Successful in 43s
lint / lint (push) Successful in 56s
test / integration-firecracker (pull_request) Successful in 3m27s
test / coverage (pull_request) Successful in 21s
test / publish-infra (pull_request) Has been skipped
Generalize render_diff so it isn't supervise-flavored: the "(current)" / "(proposed)" side titles are no longer baked in — callers pass `before_title` and `after_label` (required). Move it from orchestrator/supervisor/util.py to the base bot_bottle.util alongside sha256_hex, and delete the now-empty supervisor/util.py. The supervise callsite (Orchestrator.supervise_respond) assigns before_title="current", after_label="proposed"; the facade no longer re-exports render_diff (callers import from bot_bottle.util). Behavior at the supervise callsite is unchanged. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
db6a151803 |
refactor(supervise): hash the proposed file inside Proposal.new; sha256_hex → root util
tracker-policy-pr / check-pr (pull_request) Successful in 16s
test / integration-docker (pull_request) Successful in 39s
test / unit (pull_request) Successful in 53s
lint / lint (push) Failing after 1m3s
test / integration-firecracker (pull_request) Successful in 3m35s
test / coverage (pull_request) Successful in 19s
test / publish-infra (pull_request) Has been skipped
Move sha256_hex out of orchestrator/supervisor/util.py to the root bot_bottle.util (it's a generic string hash, not supervise-specific), and have Proposal.new take the proposed file contents and compute current_file_hash itself instead of every caller passing sha256_hex(proposed_file). Every call site set current_file_hash to the hash of the proposed file, and nothing reads the field except the DB round-trip (schema + _row_to_proposal + from_dict, all untouched), so folding the hash into the factory is behavior-preserving and removes the boilerplate. Callers now pass just the file; the orchestrator no longer imports sha256_hex at all. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
8abccf7ffe |
refactor(supervise): make Supervisor a service class the orchestrator calls
tracker-policy-pr / check-pr (pull_request) Successful in 12s
test / integration-docker (pull_request) Successful in 19s
lint / lint (push) Failing after 55s
test / unit (pull_request) Successful in 2m8s
test / integration-firecracker (pull_request) Successful in 3m25s
test / coverage (pull_request) Successful in 39s
test / publish-infra (pull_request) Has been skipped
Turn the loose queue/audit functions in orchestrator/supervisor/queue.py into
methods on a concrete Supervisor service. The Orchestrator now owns an
injectable `self._supervisor` and calls `self._supervisor.write_proposal(...)`
etc., instead of module-level free functions — the supervise dependency is
explicit and mockable, and a Supervisor can be scoped to a `db_path` (defaults
to the host DB) so tests can point it at a temp database.
- Supervisor (in the package __init__) drops the ABC and gains write_proposal,
read_proposal, list_pending_proposals, list_all_pending_proposals,
write_response, read_response, archive_all_proposals, write_audit_entry,
read_audit_entries, plus the launch-time prepare().
- render_diff / sha256_hex are pure and stateless, so they move to
orchestrator/supervisor/util.py (re-exported from the facade) rather than
becoming methods.
- queue.py is deleted; service.py + the supervise tests call through a
Supervisor instance.
Full unit suite green (2243).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
|
27a122e24b |
refactor(supervise): drop obsolete queue helpers (archive_proposal, wait_for_response, audit_dir/path)
tracker-policy-pr / check-pr (pull_request) Successful in 10s
test / integration-docker (pull_request) Successful in 17s
lint / lint (push) Failing after 59s
test / unit (pull_request) Successful in 2m3s
test / integration-firecracker (pull_request) Successful in 3m28s
test / coverage (pull_request) Successful in 16s
test / publish-infra (pull_request) Has been skipped
Remove the supervise queue helpers that no live path uses anymore — they're
mechanisms the PRD-0070 / idempotent-poll migration superseded:
- archive_proposal (+ QueueStore.archive_proposal): the poll stopped archiving
on read (#469 review), so single-proposal archiving has no caller; decided
proposals drop off the pending list via their response row and are reaped in
bulk by archive_all_proposals on teardown/reconcile.
- wait_for_response: the data plane polls the queue over the control-plane RPC
now, so nothing block-waits on it.
- audit_dir / audit_log_path: file-based audit paths, replaced by the SQLite
AuditStore.
Also drops the tests that covered those obsolete helpers. Kept the prod-unused
read accessors list_pending_proposals and read_audit_entries: those aren't dead
mechanisms — they're the read side of live write paths that many tests use to
verify real queue/audit behavior.
Full unit suite green (2243).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
|
3e2cbcab88 |
refactor(supervise): fold Supervise into the facade as Supervisor; drop dead flat-fallbacks
tracker-policy-pr / check-pr (pull_request) Successful in 8s
test / integration-docker (pull_request) Successful in 12s
test / unit (pull_request) Successful in 43s
lint / lint (push) Failing after 59s
test / integration-firecracker (pull_request) Successful in 3m16s
test / coverage (pull_request) Successful in 16s
test / publish-infra (pull_request) Has been skipped
Move the `Supervise` lifecycle out of its own `orchestrator/supervisor/ supervise.py` and into the package `__init__`, renaming the class to `Supervisor`. Callers now import it from `bot_bottle.orchestrator.supervisor` alongside the queue surface it belongs with. Remove the dead `try/except ImportError` flat-import fallbacks from the package-only store modules (db_store, audit_store, config_store, queue_store) and image_cache. Those fallbacks existed for when the store files were flat-copied into the gateway; post-PRD-0070 the data plane never opens the DB, so these modules are only ever imported as part of the package. The two gateway data-plane files that may still be loaded flat (egress_addon_core, git_gate_render) keep their fallbacks. Full unit suite green (2251). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
44e2b5a897 |
refactor(supervise): split the supervise plane by tier; per-service store managers
test / integration-docker (pull_request) Successful in 13s
tracker-policy-pr / check-pr (pull_request) Successful in 14s
test / unit (pull_request) Successful in 42s
lint / lint (push) Failing after 54s
test / integration-firecracker (pull_request) Successful in 3m17s
test / coverage (pull_request) Successful in 17s
test / publish-infra (pull_request) Has been skipped
Give each service its own store package + manager, and cut the supervise module
along the control/data-plane boundary so nothing in the shared layer reaches up
into the orchestrator.
Stores, by owner:
- bot_bottle/store/ keeps only the shared base (DbStore, migrations) and the
concrete stores that aren't service-owned (audit_store, config_store).
- bot_bottle/orchestrator/store/ now houses the orchestrator-owned stores —
queue_store (supervise queue), secret_store, config_store — plus a new
orchestrator store_manager that migrates them (composing audit/config
downward from the base). The old shared store_manager is gone.
Supervise plane, by tier:
- bot_bottle/supervisor/ (NEUTRAL, importable by every tier including the
gateway): types.py (the Proposal/Response/AuditEntry wire types + the tool/
status/poll constants + the shared daemon constants moved out of
supervise.py) and plan.py (SupervisePlan, a pure DTO).
- bot_bottle/orchestrator/supervisor/ (orchestrator-only): queue.py (the
queue/audit I/O wrappers + render_diff + sha256_hex) and supervise.py (the
Supervise lifecycle that stages the DB via the store manager). Its __init__
re-exports the neutral vocabulary so orchestrator-side callers import from
one place.
The gateway now imports only bot_bottle.supervisor.types (never
bot_bottle.supervise), so the data plane holds no code dependency on the
orchestrator — it reaches the queue over the control-plane RPC. This removes
the circular import that moving queue_store under orchestrator introduced
(supervise -> orchestrator -> service -> supervise).
supervise_types.py -> supervisor/types.py; supervise.py deleted (split). Full
unit suite green (2251).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
|
f77023db1d |
refactor(gateway): move the data-plane daemons into a bot_bottle.gateway package
test / integration-docker (pull_request) Successful in 11s
test / unit (pull_request) Successful in 43s
lint / lint (push) Successful in 56s
test / integration-firecracker (pull_request) Successful in 3m19s
test / coverage (pull_request) Successful in 19s
test / publish-infra (pull_request) Has been skipped
tracker-policy-pr / check-pr (pull_request) Successful in 7s
Separate the gateway (data plane) from the orchestrator (control plane) at the
module level. The gateway runtime files move out of the package root — and the
backend-neutral Gateway lifecycle ABC + GATEWAY_* constants move out of
orchestrator/ — into a new bot_bottle/gateway/ package:
gateway/__init__.py (was orchestrator/gateway.py: Gateway ABC + consts
+ rotate_gateway_ca)
gateway/gateway_init.py (the PID-1 daemon supervisor)
gateway/egress_addon.py, egress_addon_core.py, egress_dlp_config.py,
dlp_detectors.py (the egress mitmproxy daemon)
gateway/git_http_backend.py (the git-http daemon)
gateway/git_gate_render.py (the git-gate pre-receive rendering)
gateway/supervise_server.py (the supervise MCP daemon)
gateway/policy_resolver.py (the data-plane control-plane RPC client)
orchestrator/ now holds only control-plane files. The shared plan/types/auth
layer (egress.py=EgressPlan, git_gate.py=GitGatePlan, supervise.py,
supervise_types.py, control_auth.py) and the launch-time git-gate provisioning
helpers stay at root, so orchestrator/ and backend/ still own them.
Because these daemons are invoked as `python3 -m bot_bottle.<name>`, loaded flat
by mitmproxy, and referenced in Dockerfile.gateway, the move updates more than
Python imports: the `-m` invocations (firecracker/macOS infra scripts), the
Dockerfile.gateway addon shim + ENTRYPOINT, gateway_init's _DAEMONS module
paths, and the git-gate CGI heredocs all now point at bot_bottle.gateway.*.
No behavior change; full unit suite green (2251).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
|
a845cba925 |
refactor(manifest): consolidate the manifest modules into a bot_bottle.manifest package
tracker-policy-pr / check-pr (pull_request) Failing after 11s
test / integration-docker (pull_request) Successful in 20s
lint / lint (push) Successful in 59s
test / unit (pull_request) Successful in 2m10s
test / integration-firecracker (pull_request) Successful in 3m29s
test / coverage (pull_request) Successful in 38s
test / publish-infra (pull_request) Has been skipped
Move the nine root-level manifest modules into a bot_bottle/manifest/ package, dropping the redundant manifest_ prefix: the facade (manifest.py) becomes the package __init__ and keeps re-exporting Manifest / ManifestIndex and the piece types, while manifest_<x>.py become manifest/<x>.py (agent, bottle, egress, extends, git, loader, schema, util). Because the facade stays the package __init__, the ~13 callers that import `from bot_bottle.manifest import ...` are unchanged; only direct-piece imports move to `bot_bottle.manifest.<name>`. Inside the package, intra-manifest imports drop the prefix and the non-manifest siblings (log, yaml_subset, agent_provider) move to `..`. No behavior change; full unit suite green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
d7a58e52fd |
refactor(store): consolidate the SQLite store family into bot_bottle.store
lint / lint (push) Successful in 54s
test / integration-docker (pull_request) Successful in 38s
test / unit (pull_request) Successful in 45s
test / integration-firecracker (pull_request) Successful in 3m31s
test / coverage (pull_request) Successful in 18s
test / publish-infra (pull_request) Has been skipped
tracker-policy-pr / check-pr (pull_request) Failing after 6s
Move the DbStore base (db_store), the shared schema-migration base (migrations / TableMigrations), the concrete stores (audit_store, config_store, queue_store), and StoreManager out of the package root into a bot_bottle/store/ package, so persistence lives in one place rather than scattered across the root. Callers use direct submodule imports (from bot_bottle.store.store_manager import StoreManager). Inside the moved modules the relative imports point back up to their non-store siblings (..paths, ..supervise_types) while co-moved siblings stay package-local; the flat-import fallbacks are untouched. The orchestrator's own stores (config_store, secret_store, registry) stay in orchestrator/ and repoint to the moved DbStore / TableMigrations. No behavior change; full unit suite green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |