Chunk 3 of the host-control-server stack: grow the broker op vocabulary
(PRD gap 3), starting with `list_live`, and invert `reconcile` onto it.
- broker: split the closed op vocabulary into mutation (`launch`/`teardown`,
carry a bottle id + static flags) and query (`list_live`, carries nothing
but its op name) kinds. `verify_request` now enforces a **strict schema**
(open question 1, resolved yes): unknown claim keys are rejected, a mutation
must name its bottle, and a query that smuggles any id/flag is refused.
- broker verb: `LaunchBroker.list_live` / `SubmitBroker.list_live` return the
backend's live source IPs; a backend enumeration failure is converted to the
single `BrokerUnavailableError` "live set unknown" signal. `DockerBroker`
enumerates its labelled containers; `StubBroker` derives from launches (or a
test override).
- host controller: `POST /broker/live` verifies a signed `list_live` token and
returns `{source_ips}`; `BrokerClient.list_live` is its drop-in client.
- reconcile: `OrchestratorCore.reconcile()` drops the `live_source_ips`
parameter and pulls the live set from the broker itself — the tell that the
orchestrator couldn't see the backend goes away. **Fail-safe**: if the broker
can't return an authoritative set the sweep is skipped, never run against an
empty/partial set (which would reap healthy rows). The `/reconcile` HTTP
contract + `OrchestratorClient.reconcile` become a bare trigger.
The macOS launcher's Apple-container enumeration stays for now; it becomes the
host controller's `list_live` when launch itself moves behind the broker (the
pulled-forward chunk 5, next in the stack).
Tests + pyright clean; pylint 10.0 on broker.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Chunk 2 of the host-control-server stack: close the PRD's **durable
secret** gap and replace chunk 1's BOT_BOTTLE_BROKER_SECRET stopgap.
- trust_domain.py: two new domains. LAUNCH_BROKER holds the durable
HS256 key both the orchestrator (signer) and the host control server
(verifier) share for the broker's launch JWT — a host-canonical key
file minted 0600 on first use, so a restarted orchestrator re-verifies
against the same key. HOST_CONTROLLER is the separate domain for the
controller's own lifecycle endpoints, keyed by a key the orchestrator
never holds (its role is `host`, deliberately outside control-plane
ROLES). LaunchBrokerProvisioning is the fail-closed seam.
- orchestrator_auth.py: ROLE_HOST, outside ROLES.
- paths.py: key-file + env-var constants for both domains.
Key resolution is split by owner (addresses codex review on #497):
broker_secret(allow_host_file=...) — the host controller / dev-harness
(True) may mint/read the durable host key file it owns; the GUEST
orchestrator (--broker http, default False) must be *injected* the key
and fails closed if it isn't. A guest that fell back to the host file
would mint a process-local key unrelated to the host controller's, so
startup would succeed but every launch would 401 — this prevents that
silent divergence.
Tested: domain boundary + separation, provisioning fail-closed, and
broker_secret env-only (guest) vs host-file (host) resolution. pyright
clean; pylint 9.86.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The per-bottle egress-secret encryption (secret_store.py) was
unauthenticated CTR/XOR: decrypting with the WRONG ENV_VAR_SECRET
produced garbage that decrypt_value only rejected when it wasn't valid
UTF-8. For short token values that garbage is coincidentally valid UTF-8
~5% of the time, so `reprovision_from_secret` would occasionally "succeed"
with a wrong key and inject a garbage egress credential — and
test_reprovision_rejects_missing_rows_and_wrong_key failed ~5% of runs
(flaky CI, surfaced by this stack's unit job).
Switch to authenticated encrypt-then-MAC: append an HMAC-SHA256 tag over
`nonce || ciphertext`, keyed by a domain-separated MAC subkey derived from
the ENV_VAR_SECRET. decrypt_value verifies the tag (constant-time) before
returning any plaintext, so a wrong key or tampered ciphertext is rejected
deterministically. Blob format is now `nonce || ciphertext || tag`
(the stored rows are transient — re-written every launch — so no
migration is needed).
Pre-existing bug on main, unrelated to the transport work, but it blocks
this stack's CI. Tests: wrong key rejected 200/200; tampered ciphertext
rejected; round-trips unchanged. Deterministic now (was ~5% flaky).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Codex review on #496:
- **High — ambiguous delivery no longer orphans a launched bottle.** A
timeout / dropped response from the host controller is now the ambiguous
BrokerUnavailableError (distinct from the definite BrokerAuthError /
BrokerClientError). OrchestratorCore.launch_bottle keeps the registry
row on the ambiguous case instead of deregistering — deregistering would
orphan a running container with no record (reconcile reaps rows, never
containers). The row is left for reconcile to reap iff the bottle is not
actually live. Definite failures still roll back, so a real failure
leaves no orphan row.
- **Medium — the privileged endpoint bounds request bodies.** The host
server rejects an oversized Content-Length with 413 before reading it,
and sets a per-request socket timeout, so a caller that can merely reach
the socket (no signed token) can't exhaust memory or a handler thread.
Tests: ambiguous-keep vs definite-rollback in the launch path; the
BrokerUnavailableError/BrokerClientError split in BrokerClient; the 413
body cap + handler error paths (driven in-thread, since daemon request
threads lose coverage) plus a deterministic real-socket check that
declares an oversized Content-Length but sends a sliver (rejection on the
header, no unread-body reset race); and the __main__ entrypoint broker
selection. Diff-coverage 98%; pyright clean; pylint 9.8.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Chunk 1 of the host-control-server stack: close the PRD's **transport**
gap. Today LaunchBroker.submit(token) is an in-process method call from
OrchestratorCore; this makes it a real out-of-process service reached
over HTTP.
- host_server.py: the host control server. A pure dispatch() (POST
/broker verifies a signed token via the existing verify_request +
_launch/_teardown path, GET /health) wrapped by a thin http.server
adapter, mirroring orchestrator/server.py. Only the signed token
crosses the wire; provenance/schema failures are fail-closed 401s that
never touch the backend, a backend launch failure is a 502.
- broker_client.py: BrokerClient — a drop-in submit(token) that POSTs the
signed token to the host controller. A 401 re-raises as BrokerAuthError
so the launch path's rollback is identical local or remote.
- broker.py: SubmitBroker Protocol — the one method OrchestratorCore
depends on, satisfied by both LaunchBroker and BrokerClient, so the
core is unchanged (service.py annotation only).
- __main__.py: wire `--broker http` behind the shared-secret env var
(BOT_BOTTLE_BROKER_SECRET, hex) — a chunk-1 stopgap the durable
TrustDomain key (chunk 2, #476) replaces.
Tested: pure-dispatch cases, BrokerClient with HTTP mocked, and a
real-socket sign -> POST -> verify -> act round-trip (incl. fail-closed
forged token). pyright clean; pylint 9.86.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per PR review, replay protection is too heavy for the host control
server MVP. Drop it from the four-gap framing (now three gaps), remove
the enforcement design section and implementation chunk, and track the
iat-window + jti-cache work in #494 as an independent in-process change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Promote the in-process launch broker into a standalone host control
server: the single privileged host component that brokers launches, owns
orchestrator lifecycle, and is the sole writer of host-durable state.
Closes the four broker gaps (transport, durable provisioned secret,
replay protection, disciplined op vocabulary) and splits host state by
owner and lifetime (orchestrator SQLite / host JSONL audit / gateway
none). The payoff is dropping the Docker socket from the CLI.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
On a docker daemon that default-enables IPv6 (default-address-pools),
creating the gateway network with only `--subnet` lets the daemon also
attach an fdd0::/64 IPv6 subnet. Its gateway is stored as `::1/64`,
which trips docker's own netip.ParseAddr in `network inspect`/`ls`:
ParseAddr("fdd0:0:0:6::1/64"): unexpected character, want colon
That poisons every `_network_cidr`/`network ls` read and fails the
docker integration suite intermittently (whichever run the runner's
IPv6 pool index lands on a broken network). bot-bottle attribution
pins IPv4 source IPs and has no IPv6 support, so pass `--ipv6=false`
explicitly at network create to keep the gateway network IPv4-only
regardless of the daemon default.
Note: an already-poisoned runner still needs a one-time
`docker network rm bot-bottle-gateway` (and possibly a daemon
restart) to clear the malformed network.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Gitea AGit accepts pushes to refs/for/*, refs/draft/*, and
refs/for-review/* and opens pull requests backed by server-managed
refs/pull/<n>/head refs rather than ordinary refs/heads/* branches.
That breaks the git-gate branch workflow: follow-up commits can't be
pushed back through the branch, and Gitea rejects later direct updates
to the generated review ref, so recovery means recreating the PR.
Add a Phase 0 guard to the shared pre-receive hook that rejects
creation or update of those AGit review refs before any gitleaks scan
or upstream forward, with a message pointing callers at the
branch-backed PR workflow. Deletions (new == zero) stay allowed so
legacy AGit refs can still be cleaned up; normal branches and tags are
untouched.
Closes#506
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The require-numbered-prds gate previously ran on every pull request.
Scope its trigger to PRs whose base branch is main, so numbering is
only enforced at the point of merging into main.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
didericis-claude
didericis-codex
Gitea Actions