Real-on-host testing surfaced two issues the unit-mocked slices couldn't:
1. The host (NixOS) firewall DROPS container->host traffic, so a host-process
orchestrator is unreachable from the gateway container. Fix: run the
orchestrator AS a container on the shared gateway network (PRD 0070's
"virtualize the orchestrator") — the gateway reaches it by container name
over docker DNS (container<->container, no firewall), and the host CLI
reaches it via a published loopback port.
2. The control plane crashed the connection on a dispatch error (e.g. a
broker failure) instead of returning 500.
Changes:
- lifecycle: OrchestratorProcess (host process) -> OrchestratorService
(containers). Runs the control plane in the bundle image with the repo
bind-mounted (orchestrator is stdlib-only), register-only stub broker so it
needs NO docker socket (the backend launches agents; the host manages both
containers). Registry DB persists via a host-root mount. ensure_running is
an idempotent singleton over both containers.
- gateway: BOT_BOTTLE_ORCHESTRATOR_URL is now the orchestrator's *by-name*
URL on the shared network (dropped the host.docker.internal hack).
- control_plane: _serve wraps dispatch — a failure returns 500, never crashes
the connection.
- OrchestratorProcess default broker -> stub (register-only) for docker.
Validated live end-to-end: both containers up, gateway->orchestrator by name
OK, register -> resolve-by-source-IP returns the bottle's policy from inside
the gateway.
pyright 0 errors; pylint 9.83/10; unit suite green (1760 tests; the 13
test_sidecar_init /bin/sleep errors are pre-existing NixOS-local noise).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
Retire "sidecar" for the consolidated per-host path (PRD 0070 naming
decision): the orchestrator is the umbrella/control plane, and the
egress/git/supervise data-plane unit it runs is the "gateway".
- git mv sidecar.py -> gateway.py and the two integration + one unit test
files; DockerSidecar->DockerGateway, Sidecar->Gateway,
SidecarError->GatewayError, SIDECAR_*->GATEWAY_*, ensure_sidecar->
ensure_gateway, sidecar_status->gateway_status, container name
bot-bottle-orch-sidecar->bot-bottle-orch-gateway.
- Prose rename across broker/registry/egress/policy_resolver + PRD 0070.
- Preserved: the image name bot-bottle-sidecars, the
BOT_BOTTLE_SIDECAR_IMAGE env var, Dockerfile.sidecars, and PRD 0069's
own stage-name cross-references (that doc still uses "sidecar").
No behavior change. Full unit suite green (1679 tests; the 13
test_sidecar_init /bin/sleep errors are pre-existing NixOS-local noise).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
Answers "where do we build the consolidated sidecar": nowhere, until now.
* sidecar.py — `Sidecar.ensure_built()` (default no-op) + `DockerSidecar`
now defaults its image to the real bundle (`bot-bottle-sidecars`) and
`ensure_built()` builds it from `Dockerfile.sidecars` when
`docker image inspect` shows it's missing (no-op when present or when no
dockerfile is configured, e.g. a pre-pulled image). `image_exists()`
added.
* service.py — `ensure_sidecar()` now builds then runs.
* __main__.py — `--sidecar` runs the consolidated bundle (build-if-missing).
Scope note: this builds + launches the bundle *container*; making the
running instance functional across bottles needs the per-bottle,
source-IP-keyed multi-tenant config + registration/reload, and routing
agent bottles to it — the next slices (added to PRD 0070's roadmap).
Tests: unit (docker mocked) — image_exists, ensure_built builds when
missing / no-op when present / no-op without a dockerfile / raises on build
failure; ensure_sidecar builds-then-runs; integration (gated, no heavy
build) — image_exists reflects real docker state. Full suite green (only
pre-existing /bin/sleep errors).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
The core consolidation win: one persistent sidecar per host, shared by
every bottle, instead of a sidecar bundle per bottle. Safe to share
because the attribution invariant (source IP + identity token) lets the
sidecar map each request to the right bottle.
* orchestrator/sidecar.py — a backend-neutral `Sidecar` lifecycle
contract (mirrors LaunchBroker) + a `DockerSidecar` impl. The defining
behaviour is idempotent singleton: `ensure_running` starts the instance
if absent and is a no-op if it's already up, so N launches never spawn
N sidecars; `stop` is idempotent.
* orchestrator/dockerutil.py — a shared `run_docker` helper; DockerBroker
now uses it too (DRY with slice 3).
* service.py — the Orchestrator holds an optional `Sidecar`, exposes
`ensure_sidecar()` + `sidecar_status()`.
* control_plane.py — `GET /sidecar` reports it; __main__ gains
`--sidecar-image` and ensures the single sidecar on startup.
Tests: unit (docker mocked) — is_running, ensure idempotent (no-op when up,
starts when absent), failure raises, stop idempotent; Orchestrator sidecar
wiring/status; control-plane /sidecar; integration (gated) — ensure is a
real idempotent singleton (one container after two ensures), stop removes.
Full suite green (only pre-existing /bin/sleep errors); integration
verified locally against real docker.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
The first concrete LaunchBroker, proving the orchestrator -> backend seam
on the cheapest backend (the sidecar bundle is already containers):
* orchestrator/docker_broker.py — DockerBroker runs a container on a
verified launch (`docker run --detach --name <bottle> --label ...
<image_ref>`) and removes it on teardown (`docker rm --force`,
idempotent on an already-absent container). The argv is built only from
the request's static ids/flags, so nothing free-form reaches docker;
provenance/schema verification is inherited from LaunchBroker.submit.
* __main__.py gains `--broker {stub,docker}` so the harness can drive real
containers.
Slice 3 launches a single container from image_ref (the seam); the full
agent + sidecar bundle is a later slice.
Tests: unit (docker mocked) — argv from static fields, launch/teardown call
the right commands, missing-image and docker-failure raise, teardown
idempotent on missing, forged token never touches docker; integration
(gated on a reachable daemon) — launch creates a real container, teardown
removes it. Full suite green (only pre-existing /bin/sleep errors);
integration verified locally against real docker.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
Second slice of PRD 0070, still a backend-neutral dev-harness:
* orchestrator/broker.py — the launch-broker contract. A LaunchRequest is
structured (static ids/flags only — bottle id, pool slot, a
content-addressed image_ref; never a path/argv) and signed as a compact
HS256 JWT so the broker verifies PROVENANCE before acting: a compromised
co-located component can't forge a launch without the shared secret.
verify_request is fail-closed (bad sig / malformed / off-schema -> raise).
Stdlib only (no runtime deps). Ships a StubBroker that records verified
requests for the harness/tests.
* orchestrator/service.py — the Orchestrator: owns the registry and brokers
the lifecycle. launch_bottle mints the bottle + sends a signed launch,
rolling the registry entry back if the launch fails (no orphans);
teardown_bottle brokers teardown then deregisters; attribute delegates.
* control_plane.py — POST /bottles now launches, DELETE tears down (both go
through the Orchestrator + broker). dispatch/server take an Orchestrator.
* __main__.py wires an ephemeral secret + StubBroker for the harness.
Tests: broker sign/verify round-trip, tamper/wrong-secret/malformed/off-schema
rejection, StubBroker fail-closed; Orchestrator launch->registry->attribute,
teardown, rollback-on-broker-failure; control-plane updated for launch/teardown.
Full suite green (only the pre-existing /bin/sleep errors); harness does
launch -> attribute -> teardown over HTTP.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
First implementation slice of PRD 0070, the backend-neutral consolidation
core as a plain-process dev-harness (no VM packaging yet):
* orchestrator/registry.py — SQLite (WAL) runtime-state store on the
existing DbStore/TableMigrations base. Live bottle registry keyed by
source IP + per-bottle identity token, with fail-closed attribution:
a request resolves to a bottle only when its source IP AND identity
token both match exactly one active record (unknown/ambiguous IP,
empty token, or token mismatch all deny). Tokens are 256-bit urandom.
* orchestrator/control_plane.py — the HTTP control plane (the universal
transport chosen in 0070): register / deregister / list / attribute /
health. Routing is a pure dispatch() so it is socket-free testable;
Handler/ControlPlaneServer/make_server are a thin stdlib adapter.
register/deregister are the live-reload path; listing redacts tokens.
* orchestrator/__main__.py — `python -m bot_bottle.orchestrator` harness.
Launch/teardown, the launch broker, and the egress/git/supervise data
plane come in later slices. 24 unit tests (attribution matrix, persistence,
dispatch, one real-socket round-trip).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
didericis