7 Commits

Author	SHA1	Message	Date
didericis	7e9ad8a78d	Merge main into fix/db-off-data-plane-469 ... Brings in main's 8 commits — #470 (backend-agnostic CI guards: BackendStatus enum, quiet status(), is_backend_available/is_backend_ready, tests/_backend.py skip_unless_backend) plus firecracker status() readiness checks (kvm/kernel/ dropbear/mke2fs). Reconciled #470's additions into the reorg'd backend structure: * BackendStatus enum + the status(*, quiet=) ABC signature -> backend/base.py * is_backend_available / is_backend_ready -> backend/selection.py * exposed all three through the lazy backend facade (__init__). Integration-test conflicts were our control_plane->orchestrator renames vs main's skip_unless_docker -> skip_unless_backend swap: kept our refactored import paths + main's guard. Repointed main's new is_backend_* tests to patch selection._backends (our moved home) instead of the facade. pyright: 0 errors. Full unit suite green (2272). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 21:35:58 -04:00
didericis-claude	64adf23775	ci: backend-agnostic integration guards + per-backend preflight ... Integration tests now select their backend from BOT_BOTTLE_BACKEND and skip on the capability that backend actually needs, instead of gating every backend on unrelated Docker availability. Task 1 — backend-agnostic guards (tests/_backend.py): - Capability probes: docker_capability() (reachable daemon) and firecracker_capability() (accessible /dev/kvm + firecracker on PATH, Docker-independent). backend_capability()/selected_backend() resolve the target from BOT_BOTTLE_BACKEND (default docker). - skip_unless_selected_backend_available() for backend-agnostic tests (test_sandbox_escape) — runs through whichever backend is selected and checks that backend's real capability. - skip_unless_backend("docker") for Docker-implementation tests (DockerBroker, DockerGateway, backend.docker.*) — they no-op under a non-Docker run rather than testing internals that run doesn't target. - Retires tests/_docker.py; the KVM job no longer needs SKIP_DOCKER_TESTS to steer Docker-only classes. Task 2 — explicit per-backend skip visibility: - tests/backend_preflight.py prints a clear PASS/FAIL capability line and exits non-zero when the selected backend is missing. - Both integration jobs run it as a preflight, so absent infrastructure is surfaced at the job level instead of hidden among unittest.skip lines. The docker job replaces its soft "Show environment" step; the firecracker job keeps its richer backend-status check. Docs (tests/README.md, docs/ci.md) updated; unit coverage for the probes, guards, and preflight in test_backend_skip_guards.py. Closes #414 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 21:25:20 -04:00
didericis	ce744a85c4	refactor(gateway): split data-plane files into egress/supervisor/git_gate services ... Group the gateway's data-plane modules into three service sub-packages mirroring the host-side trio (bot_bottle.egress / .supervisor / .git_gate): gateway/egress/ addon_core, addon, dlp_config, dlp_detectors gateway/supervisor/ server (was supervise_server) gateway/git_gate/ render, http_backend Prefix-stripped filenames now that the package namespaces them; each sub-package has a thin docstring __init__ (no eager imports, cheap leaf loads). The two cross-cutting files stay at the gateway root: policy_resolver (shared per-client lookup) and gateway_init, renamed to bootstrap now that gateway/ already namespaces it. Updated all importers (bot_bottle + tests), the in-VM/container `-m` launch strings, the Dockerfile.gateway addon shim + ENTRYPOINT, and the five gateway entries in scripts/critical-modules.txt. Full unit suite green (2243). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 17:00:51 -04:00
didericis	44e2b5a897	refactor(supervise): split the supervise plane by tier; per-service store managers ... Give each service its own store package + manager, and cut the supervise module along the control/data-plane boundary so nothing in the shared layer reaches up into the orchestrator. Stores, by owner: - bot_bottle/store/ keeps only the shared base (DbStore, migrations) and the concrete stores that aren't service-owned (audit_store, config_store). - bot_bottle/orchestrator/store/ now houses the orchestrator-owned stores — queue_store (supervise queue), secret_store, config_store — plus a new orchestrator store_manager that migrates them (composing audit/config downward from the base). The old shared store_manager is gone. Supervise plane, by tier: - bot_bottle/supervisor/ (NEUTRAL, importable by every tier including the gateway): types.py (the Proposal/Response/AuditEntry wire types + the tool/ status/poll constants + the shared daemon constants moved out of supervise.py) and plan.py (SupervisePlan, a pure DTO). - bot_bottle/orchestrator/supervisor/ (orchestrator-only): queue.py (the queue/audit I/O wrappers + render_diff + sha256_hex) and supervise.py (the Supervise lifecycle that stages the DB via the store manager). Its __init__ re-exports the neutral vocabulary so orchestrator-side callers import from one place. The gateway now imports only bot_bottle.supervisor.types (never bot_bottle.supervise), so the data plane holds no code dependency on the orchestrator — it reaches the queue over the control-plane RPC. This removes the circular import that moving queue_store under orchestrator introduced (supervise -> orchestrator -> service -> supervise). supervise_types.py -> supervisor/types.py; supervise.py deleted (split). Full unit suite green (2251). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 14:15:04 -04:00
didericis	f77023db1d	refactor(gateway): move the data-plane daemons into a bot_bottle.gateway package ... Separate the gateway (data plane) from the orchestrator (control plane) at the module level. The gateway runtime files move out of the package root — and the backend-neutral Gateway lifecycle ABC + GATEWAY_* constants move out of orchestrator/ — into a new bot_bottle/gateway/ package: gateway/__init__.py (was orchestrator/gateway.py: Gateway ABC + consts + rotate_gateway_ca) gateway/gateway_init.py (the PID-1 daemon supervisor) gateway/egress_addon.py, egress_addon_core.py, egress_dlp_config.py, dlp_detectors.py (the egress mitmproxy daemon) gateway/git_http_backend.py (the git-http daemon) gateway/git_gate_render.py (the git-gate pre-receive rendering) gateway/supervise_server.py (the supervise MCP daemon) gateway/policy_resolver.py (the data-plane control-plane RPC client) orchestrator/ now holds only control-plane files. The shared plan/types/auth layer (egress.py=EgressPlan, git_gate.py=GitGatePlan, supervise.py, supervise_types.py, control_auth.py) and the launch-time git-gate provisioning helpers stay at root, so orchestrator/ and backend/ still own them. Because these daemons are invoked as `python3 -m bot_bottle.<name>`, loaded flat by mitmproxy, and referenced in Dockerfile.gateway, the move updates more than Python imports: the `-m` invocations (firecracker/macOS infra scripts), the Dockerfile.gateway addon shim + ENTRYPOINT, gateway_init's _DAEMONS module paths, and the git-gate CGI heredocs all now point at bot_bottle.gateway.*. No behavior change; full unit suite green (2251). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-24 13:25:10 -04:00
didericis	2de223a33b	test(gateway-image): probe package imports, not pre-refactor flat modules ... test_python_imports_supervise_module probed the image with `import supervise; import supervise_server`, the flat sibling modules that 5ad3449 replaced. They now live in the installed package, so the test fails with ModuleNotFoundError when actually run. It went unnoticed because the whole class carries a GITEA_ACTIONS skip — it is interactive-only, so CI never executes it. All 4 tests in the class now pass against a freshly built gateway image. Closes #429. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-07-20 13:40:14 -04:00
didericis	09393b354b	refactor(de-sidecar): purge the "sidecar" name from live code, tests, and current docs ... Completes the de-sidecar cleanup: no live code, test, current doc, script, or nix file mentions or is named "sidecar" any more. Only the dated PRD/research docs keep the term as historical record (agreed on the #385 thread). - Rename `sidecar_init.py`→`gateway_init.py` was done earlier; this pass sweeps the remaining descriptive uses: the egress / git-gate / supervise components are the gateway's *daemons*, the shared container is the *gateway*, the old per-bottle container was the *companion container*. - Rename `tests/integration/test_sidecar_bundle_image.py`→`test_gateway_image.py` and its class; update `docs/ci.md` + `tests/README.md` for the renamed/ removed integration tests. - `SIDECAR_PORTS` shell var in `scripts/firecracker-netpool.sh`→`GATEWAY_PORTS`. Full unit suite green (bar the pre-existing `/bin/sleep`-missing env errors in test_gateway_init); docker integration — gateway singleton, broker, real two-bottle multitenant isolation, and the gateway-image build — all pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck	2026-07-14 17:07:56 -04:00