PRD: Host control server - transport (chunk 1/5) #496
Open
didericis-claude
wants to merge 3 commits from
host-control-server-1-transport into prd-new-host-control-server
pull from: host-control-server-1-transport
merge into: didericis:prd-new-host-control-server
didericis:main
didericis:fix/quality-cleanup-consolidated
didericis:feat/macos-install-test-harness
didericis:prd-0081-reconcile-1-ca
didericis:fix/shared-storage-permissions
didericis:fix/authoritative-cleanup-identities
didericis:fix/bounded-heavy-operations
didericis:fix/cleanup-execution-integrity
didericis:fix/gateway-output-pump-shutdown
didericis:fix/gateway-request-boundaries
didericis:fix/authoritative-cleanup-revalidation
didericis:refactor/egress-policy-pipeline
didericis:refactor/orchestrator-fastapi
didericis:refactor/supervisor-mcp-dispatch
didericis:chore/misc-quality-cleanup
didericis:prd-forge-subroles
didericis:prd-gateway-reprovision-on-bringup
didericis:host-control-server-2-secret
didericis:host-control-server-3-list-live
didericis:prd-new-host-control-server
didericis:fix/gitgate-persist-512
didericis:fix/firecracker-gateway-ca-persist-510
didericis:didericis/disable-agit-refs
didericis:didericis/prd-audit-event-schema
didericis:fix/pr-495-audit-contract
didericis:prd-egress-control-plane
didericis:prd-macos-container-ci-runner
didericis:refactor/control-plane-auth-provisioning-476
didericis:fix/db-off-data-plane-469
didericis:feat/encrypted-egress-secrets
didericis:spike/rootless-docker-macos
didericis:fix/ci-coverage-artifact-paths
didericis:claude-forward-host-credentials-rebased
didericis:fix-gateway-gitleaks-arch
didericis:fix/websocket-response-dlp-multitenant
didericis:orchestrator-agent-compose
didericis:orchestrator-gateway-ca
didericis:orchestrator-consolidated-launch
didericis:orchestrator-gateway-provision
didericis:orchestrator-gateway-network
didericis:orchestrator-client
didericis:orchestrator-gateway-net
didericis:orchestrator-gitgate-provision
didericis:orchestrator-registration
didericis:orchestrator-lifecycle
didericis:orchestrator-supervise-writers
didericis:orchestrator-supervise-multitenant
didericis:orchestrator-gitgate-multitenant
didericis:orchestrator-rename-gateway
didericis:orchestrator-slice8
didericis:orchestrator-slice7
didericis:orchestrator-slice6
didericis:prd-orchestrator
didericis:orchestrator-slice5
didericis:orchestrator-slice4
didericis:orchestrator-slice3
didericis:orchestrator-slice2
didericis:firecracker-backend
didericis:forge-native-integration
didericis:prd-smolmachines-linux
didericis:manifest-break-import-cycle
didericis:dlp-supervise-quality-fixes
didericis:table-drive-dlp-tests
didericis:fix-integration-test-failures
didericis:fix/macos-container-relative-dockerfile
didericis:prd-0054-install-script
didericis:commit-bottle-state
didericis:pr-211
didericis:move-codex-auth-to-contrib
didericis:feat/pipelock-skip-scan-extensions
didericis:prd-0049-named-labelled-agents
didericis:harden-git-gate-shell-rendering
3 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
7a48ea2b0c |
fix(secret): authenticate bottled-secret encryption (#468)
refresh-image-locks / refresh (push) Successful in 42s
lint / lint (push) Successful in 1m7s
test / image-input-builds (pull_request) Successful in 1m12s
test / unit (pull_request) Successful in 44s
test / integration-docker (pull_request) Successful in 55s
test / coverage (pull_request) Successful in 13s
tracker-policy-pr / check-pr (pull_request) Successful in 7s
The per-bottle egress-secret encryption (secret_store.py) was unauthenticated CTR/XOR: decrypting with the WRONG ENV_VAR_SECRET produced garbage that decrypt_value only rejected when it wasn't valid UTF-8. For short token values that garbage is coincidentally valid UTF-8 ~5% of the time, so `reprovision_from_secret` would occasionally "succeed" with a wrong key and inject a garbage egress credential — and test_reprovision_rejects_missing_rows_and_wrong_key failed ~5% of runs (flaky CI, surfaced by this stack's unit job). Switch to authenticated encrypt-then-MAC: append an HMAC-SHA256 tag over `nonce || ciphertext`, keyed by a domain-separated MAC subkey derived from the ENV_VAR_SECRET. decrypt_value verifies the tag (constant-time) before returning any plaintext, so a wrong key or tampered ciphertext is rejected deterministically. Blob format is now `nonce || ciphertext || tag` (the stored rows are transient — re-written every launch — so no migration is needed). Pre-existing bug on main, unrelated to the transport work, but it blocks this stack's CI. Tests: wrong key rejected 200/200; tampered ciphertext rejected; round-trips unchanged. Deterministic now (was ~5% flaky). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
ec953ceda7 |
fix(orchestrator): address review on host control server transport (#468)
Codex review on #496: - **High — ambiguous delivery no longer orphans a launched bottle.** A timeout / dropped response from the host controller is now the ambiguous BrokerUnavailableError (distinct from the definite BrokerAuthError / BrokerClientError). OrchestratorCore.launch_bottle keeps the registry row on the ambiguous case instead of deregistering — deregistering would orphan a running container with no record (reconcile reaps rows, never containers). The row is left for reconcile to reap iff the bottle is not actually live. Definite failures still roll back, so a real failure leaves no orphan row. - **Medium — the privileged endpoint bounds request bodies.** The host server rejects an oversized Content-Length with 413 before reading it, and sets a per-request socket timeout, so a caller that can merely reach the socket (no signed token) can't exhaust memory or a handler thread. Tests: ambiguous-keep vs definite-rollback in the launch path; the BrokerUnavailableError/BrokerClientError split in BrokerClient; the 413 body cap + handler error paths (driven in-thread, since daemon request threads lose coverage) plus a deterministic real-socket check that declares an oversized Content-Length but sends a sliver (rejection on the header, no unread-body reset race); and the __main__ entrypoint broker selection. Diff-coverage 98%; pyright clean; pylint 9.8. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
ed0f95f445 |
feat(orchestrator): host control server transport (#468)
Chunk 1 of the host-control-server stack: close the PRD's **transport** gap. Today LaunchBroker.submit(token) is an in-process method call from OrchestratorCore; this makes it a real out-of-process service reached over HTTP. - host_server.py: the host control server. A pure dispatch() (POST /broker verifies a signed token via the existing verify_request + _launch/_teardown path, GET /health) wrapped by a thin http.server adapter, mirroring orchestrator/server.py. Only the signed token crosses the wire; provenance/schema failures are fail-closed 401s that never touch the backend, a backend launch failure is a 502. - broker_client.py: BrokerClient — a drop-in submit(token) that POSTs the signed token to the host controller. A 401 re-raises as BrokerAuthError so the launch path's rollback is identical local or remote. - broker.py: SubmitBroker Protocol — the one method OrchestratorCore depends on, satisfied by both LaunchBroker and BrokerClient, so the core is unchanged (service.py annotation only). - __main__.py: wire `--broker http` behind the shared-secret env var (BOT_BOTTLE_BROKER_SECRET, hex) — a chunk-1 stopgap the durable TrustDomain key (chunk 2, #476) replaces. Tested: pure-dispatch cases, BrokerClient with HTTP mocked, and a real-socket sign -> POST -> verify -> act round-trip (incl. fail-closed forged token). pyright clean; pylint 9.86. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |