fix(secret): authenticate bottled-secret encryption (#468)

The per-bottle egress-secret encryption (secret_store.py) was
unauthenticated CTR/XOR: decrypting with the WRONG ENV_VAR_SECRET
produced garbage that decrypt_value only rejected when it wasn't valid
UTF-8. For short token values that garbage is coincidentally valid UTF-8
~5% of the time, so `reprovision_from_secret` would occasionally "succeed"
with a wrong key and inject a garbage egress credential — and
test_reprovision_rejects_missing_rows_and_wrong_key failed ~5% of runs
(flaky CI, surfaced by this stack's unit job).

Switch to authenticated encrypt-then-MAC: append an HMAC-SHA256 tag over
`nonce || ciphertext`, keyed by a domain-separated MAC subkey derived from
the ENV_VAR_SECRET. decrypt_value verifies the tag (constant-time) before
returning any plaintext, so a wrong key or tampered ciphertext is rejected
deterministically. Blob format is now `nonce || ciphertext || tag`
(the stored rows are transient — re-written every launch — so no
migration is needed).

Pre-existing bug on main, unrelated to the transport work, but it blocks
this stack's CI. Tests: wrong key rejected 200/200; tampered ciphertext
rejected; round-trips unchanged. Deterministic now (was ~5% flaky).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

.gitea/workflows/test.yml

@@ -102,20 +102,6 @@ jobs:
           python3 --version
           python3 cli.py backend status --backend=docker
 
-      - name: Preflight — clear any leftover poisoned gateway network
-        run: |
-          # The gateway network has a fixed name and persists across jobs on
-          # this shared runner. A pre-fix or concurrent launch can leave it with
-          # a malformed IPv6 subnet that trips docker's own ParseAddr in
-          # `network inspect` (see PR #515); the code now self-heals it, but the
-          # heal can't run if `network inspect` is what's broken on some daemon
-          # versions. Drop the network here so this run recreates it IPv4-only.
-          # Remove the attached gateway container first (else `network rm` fails
-          # on active endpoints); both are recreated by ensure_running. Harmless
-          # when absent.
-          docker rm --force bot-bottle-orch-gateway 2>/dev/null || true
-          docker network rm bot-bottle-gateway 2>/dev/null || true
-
       - name: Run integration tests (docker) with coverage
         env:
           BOT_BOTTLE_BACKEND: docker
@@ -298,7 +284,7 @@ jobs:
       - name: Combined coverage (unit + docker integration)
         run: PYTHON=python3 bash scripts/coverage.sh aggregate critical
 
-      - name: Diff-coverage gate (changed lines >= 80%)
+      - name: Diff-coverage gate (changed lines >= 90%)
         run: |
           git fetch --no-tags origin main:refs/remotes/origin/main
-          python3 scripts/diff_coverage.py --base origin/main --min 80
+          python3 scripts/diff_coverage.py --base origin/main --min 90

README.md

@@ -71,21 +71,7 @@ When the agent exits, `cli.py` tears down every gateway and both networks; nothi
 
 ## Quickstart
 
-```sh
-curl -fsSL https://gitea.dideric.is/didericis/bot-bottle/raw/branch/main/install.sh | sh
-```
-
-The installer is a bootstrapper: it finds a suitable Python, installs bot-bottle with `pipx` (falling back to `pip --user`), creates `~/.bot-bottle`, and runs `bot-bottle doctor`. It is idempotent and never uses `sudo`. Python-native users can skip it entirely with `pipx install bot-bottle` or `uv tool install bot-bottle`.
-
-### Requirements
-
-**Python ≥ 3.11**, and this is the one that trips people up on macOS: the `python3` Apple ships at `/usr/bin/python3` is **3.9.6**, which is too old. Bare `python3` resolves to that stub far more often than people expect. `path_helper` builds a login shell's `PATH` from `/etc/paths` and then appends `/etc/paths.d/*`, and `/usr/bin` sits in the former — so even when `/opt/homebrew/bin` *is* on the `PATH` (via `/etc/paths.d/homebrew`), it comes after `/usr/bin` and loses. Prepending a newer Python is something your shell profile does, and a fresh account, a launchd job, or a CI runner has no such profile. So the installer looks past bare `python3` before giving up: it tries `python3`, then the versioned `python3.11`–`python3.14` names, then `/opt/homebrew/bin`, `/usr/local/bin`, `~/.local/bin`, and python.org framework builds — and tells you which one it picked when it isn't the obvious one. Point it somewhere specific with `BOT_BOTTLE_PYTHON=/path/to/python3`.
-
-**No `pipx` required.** If `pipx` is present the installer uses it and stays out of the way. If it isn't, bot-bottle installs into a private venv at `~/.bot-bottle/venv` (override with `BOT_BOTTLE_VENV`) and symlinks the entry point into `~/.local/bin`. There is deliberately no `pip install --user` path: Homebrew, python.org and Debian/Ubuntu interpreters are all externally managed (PEP 668), which blocks `--user` outright — so on a Mac it is never the fallback it appears to be. A venv is exempt from PEP 668, and `venv` is stdlib, so unlike `pipx` there is nothing to bootstrap first.
-
-**`git`**, because the default install spec is a `git+` URL. Set `BOT_BOTTLE_INSTALL_SPEC` to a wheel path or index name to avoid it.
-
-**A backend**, which the installer deliberately does *not* install for you — `doctor` reports what's missing afterwards. On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The Firecracker backend (Linux) requires Docker on the host for the gateway plus the `firecracker` binary and KVM. The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
+On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The Firecracker backend (Linux) requires Docker on the host for the gateway plus the `firecracker` binary and KVM. The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
 
 Use `BOT_BOTTLE_BACKEND=docker ./cli.py start <agent>` on hosts where neither Apple Container nor KVM is available and Docker is the desired backend.
 

bot_bottle/backend/docker/gateway.py

@@ -140,43 +140,12 @@ class DockerGateway(Gateway):
             marker = inspected.stdout.strip()
             if marker in {"", self._subnet}:
                 return
-            # Inspectable but mislabelled: the stale auto-IPAM network created
-            # by older releases. Replace it below.
-            stale = True
-        else:
-            # inspect failed. Classify by stderr — do NOT assume "not absent"
-            # implies "poisoned": a transient daemon/API error, permission
-            # failure, timeout, or bad context also fails here, and destroying
-            # the shared gateway on that guess would tear the network out from
-            # under every live bottle.
-            err = inspected.stderr.lower()
-            if "no such network" in err or "not found" in err:
-                # Absent: nothing to replace — create it below.
-                stale = False
-            elif "parseaddr" in err:
-                # Present but poisoned. A daemon that default-enables IPv6
-                # attaches an fdd0::/64 subnet whose `::1/64` gateway trips
-                # docker's own netip.ParseAddr in `network inspect`/`ls`, so the
-                # command exits non-zero with that signature. A fixed release
-                # never *creates* such a network, but one can survive on a
-                # shared host from an older or concurrent launch — and
-                # `--ipv6=false` alone can't heal it, since the create below only
-                # no-ops on "already exists". Force-replace it so later reads
-                # (e.g. `_network_cidr` pinning a source IP) stop failing.
-                stale = True
-            else:
-                # Unrecognized failure: no evidence the network is malformed.
-                # Surface it rather than mutate shared state on a guess.
-                raise GatewayError(
-                    f"gateway network {self.network} could not be inspected: "
-                    f"{inspected.stderr.strip()}"
-                )
-        if stale:
-            # Migrate the stale/poisoned network. Removing the fixed gateway is
-            # safe here: this launch recreates it.
+        if inspected.returncode == 0:
+            # Migrate the stale auto-IPAM network created by older releases.
+            # Removing the fixed gateway is safe here: this launch recreates it.
             run_docker(["docker", "rm", "--force", self.name])
             removed = run_docker(["docker", "network", "rm", self.network])
-            if removed.returncode != 0 and "no such network" not in removed.stderr.lower():
+            if removed.returncode != 0:
                 raise GatewayError(
                     f"gateway network {self.network} needs explicit subnet "
                     f"{self._subnet} but could not be replaced: "

bot_bottle/orchestrator/__main__.py

@@ -17,7 +17,9 @@ from pathlib import Path
 
 from .. import log
 from .store.store_manager import StoreManager
-from .broker import LaunchBroker, StubBroker
+from .broker import StubBroker, SubmitBroker
+from .broker_client import BrokerClient
+from .host_server import BROKER_SECRET_ENV, DEFAULT_PORT, broker_secret_from_env
 from .server import make_server
 from .docker_broker import DockerBroker
 from .store.registry_store import RegistryStore, default_db_path
@@ -34,8 +36,13 @@ def main(argv: list[str] | None = None) -> int:
         help=f"registry DB path (default: {default_db_path()})",
     )
     parser.add_argument(
-        "--broker", choices=("stub", "docker"), default="stub",
-        help="launch broker: 'stub' records requests; 'docker' runs containers",
+        "--broker", choices=("stub", "docker", "http"), default="stub",
+        help="launch broker: 'stub' records requests; 'docker' runs containers "
+             "in-process; 'http' relays signed requests to a host control server",
+    )
+    parser.add_argument(
+        "--host-controller-url", default=f"http://127.0.0.1:{DEFAULT_PORT}",
+        help="host control server URL (used only with --broker http)",
     )
     args = parser.parse_args(argv)
 
@@ -47,11 +54,25 @@ def main(argv: list[str] | None = None) -> int:
     # operator reaches it over HTTP (never a second, disconnected DB).
     StoreManager(registry.db_path).migrate()
 
-    # An ephemeral signing secret ties the orchestrator (signer) to its
-    # broker (verifier). 'stub' records launches instead of starting
-    # anything; 'docker' runs real containers (firecracker drops in later).
-    secret = secrets.token_bytes(32)
-    broker: LaunchBroker = DockerBroker(secret) if args.broker == "docker" else StubBroker(secret)
+    # A signing secret ties the orchestrator (signer) to its broker (verifier).
+    # 'stub' records launches instead of starting anything; 'docker' runs real
+    # containers in-process; 'http' relays signed requests to a separate host
+    # control server, which verifies and launches. For 'stub'/'docker' the
+    # secret is ephemeral (signer and verifier share this process); for 'http'
+    # it must be the SAME secret the host controller holds, so it is read from
+    # the shared env var (the chunk-1 stand-in for out-of-band provisioning).
+    broker: SubmitBroker
+    if args.broker == "http":
+        secret = broker_secret_from_env()
+        if secret is None:
+            parser.error(
+                f"--broker http requires a shared signing secret in "
+                f"${BROKER_SECRET_ENV} (hex), matching the host control server"
+            )
+        broker = BrokerClient(args.host_controller_url)
+    else:
+        secret = secrets.token_bytes(32)
+        broker = DockerBroker(secret) if args.broker == "docker" else StubBroker(secret)
     orchestrator = OrchestratorCore(registry, broker, secret)
 
     server = make_server(orchestrator, host=args.host, port=args.port)

bot_bottle/orchestrator/broker.py

@@ -29,6 +29,7 @@ import json
 import secrets
 import time
 from dataclasses import dataclass
+from typing import Protocol
 
 _JWT_HEADER = {"alg": "HS256", "typ": "JWT"}
 _ALLOWED_OPS = ("launch", "teardown")
@@ -37,7 +38,21 @@ _ALLOWED_OPS = ("launch", "teardown")
 class BrokerAuthError(Exception):
     """A broker request failed provenance or schema verification —
     bad/absent signature, malformed token, or a payload that doesn't match
-    the fixed launch-request shape. Fail-closed: the broker must not act."""
+    the fixed launch-request shape. Fail-closed: the broker must not act.
+
+    A **definite** negative: nothing was launched, so a caller may safely roll
+    back as if the op never happened."""
+
+
+class BrokerUnavailableError(Exception):
+    """A brokered request could not be carried to a verdict: the broker (or the
+    wire to it) was unreachable, timed out, or dropped the response.
+
+    Crucially **ambiguous** — unlike `BrokerAuthError`, the op MAY already have
+    taken effect on the backend before the response was lost, so a caller must
+    NOT assume it did nothing (e.g. must not roll a registry row back as if no
+    launch happened, which would orphan a running container). Only the in-process
+    brokers never raise this; the out-of-process `BrokerClient` does."""
 
 
 @dataclass(frozen=True)
@@ -123,6 +138,16 @@ def verify_request(token: str, secret: bytes) -> LaunchRequest:
 
 # --- the broker itself ------------------------------------------------------
 
+class SubmitBroker(Protocol):
+    """The single method `OrchestratorCore` depends on: verify a signed token and
+    perform its op, returning the verified request. Both the in-process
+    `LaunchBroker` and the out-of-process `BrokerClient` (which relays the token
+    to the host control server) satisfy it structurally, so the core is unchanged
+    whether the backend is local or a real host service."""
+
+    def submit(self, token: str) -> LaunchRequest: ...
+
+
 class LaunchBroker(abc.ABC):
     """Verifies a signed request came from the orchestrator, then performs
     the backend-native launch/teardown. Subclasses implement `_launch` /
@@ -168,7 +193,9 @@ class StubBroker(LaunchBroker):
 
 __all__ = [
     "BrokerAuthError",
+    "BrokerUnavailableError",
     "LaunchRequest",
+    "SubmitBroker",
     "LaunchBroker",
     "StubBroker",
     "sign_request",

bot_bottle/orchestrator/broker_client.py

@@ -0,0 +1,126 @@
+"""Orchestrator-side broker transport (issue #468, chunk 1).
+
+The signer's half of the launch-broker transport gap. `BrokerClient` satisfies
+the exact `submit(token)` contract `OrchestratorCore` already depends on (see
+`broker.SubmitBroker`), but instead of verifying and launching in-process it POSTs
+the signed token to the host control server over HTTP (stdlib `urllib`, like
+`orchestrator/client.py`). Because it is drop-in for that interface, wiring a real
+out-of-process backend does not change the core: it still signs a request and
+calls `submit()`; only the wire is new.
+
+A provenance/schema rejection from the host controller (HTTP 401) is re-raised as
+the same `BrokerAuthError` the in-process broker raises, so the launch path's
+rollback-on-failure (`OrchestratorCore.launch_bottle`) behaves identically whether
+the broker is local or remote.
+"""
+
+from __future__ import annotations
+
+import json
+import urllib.error
+import urllib.request
+
+from .broker import BrokerAuthError, BrokerUnavailableError, LaunchRequest
+
+DEFAULT_TIMEOUT_SECONDS = 5.0
+
+
+class BrokerClientError(RuntimeError):
+    """The host control server *responded*, but with an unexpected status other
+    than the fail-closed 401 (which surfaces as `BrokerAuthError`) — e.g. a 502
+    backend failure or a malformed body. A definite negative: the host processed
+    the request and it did not launch. (A *no-response* failure — unreachable /
+    timeout / dropped — is the ambiguous `BrokerUnavailableError` instead.)"""
+
+
+class BrokerClient:
+    """Drop-in `submit(token)` that relays a signed request to the host control
+    server. Holds no secret — provenance rides entirely in the signed token, so a
+    caller that can reach this client still cannot forge a launch."""
+
+    def __init__(self, base_url: str, *, timeout: float = DEFAULT_TIMEOUT_SECONDS) -> None:
+        self._base = base_url.rstrip("/")
+        self._timeout = timeout
+
+    def submit(self, token: str) -> LaunchRequest:
+        """POST the signed token to the host controller and return the request it
+        verified and acted on.
+
+        Raises `BrokerAuthError` on a fail-closed 401 (bad provenance/schema —
+        the same exception the in-process broker raises); `BrokerClientError` if
+        the host *responds* with any other non-success status or a malformed
+        body (a definite negative); or `BrokerUnavailableError` if no response is
+        obtained (unreachable / timeout / dropped) — the **ambiguous** case, where
+        the host may already have acted, so the caller must not roll back."""
+        data = json.dumps({"token": token}).encode()
+        req = urllib.request.Request(
+            f"{self._base}/broker", data=data, method="POST",
+            headers={"Content-Type": "application/json"},
+        )
+        try:
+            with urllib.request.urlopen(req, timeout=self._timeout) as resp:
+                return _request_from(_json_object(resp.read()))
+        except urllib.error.HTTPError as e:
+            detail = _error_detail(e)
+            if e.code == 401:
+                raise BrokerAuthError(
+                    detail or "host controller rejected the request"
+                ) from e
+            raise BrokerClientError(
+                f"POST /broker: HTTP {e.code} {detail}".rstrip()
+            ) from e
+        except (urllib.error.URLError, TimeoutError, OSError) as e:
+            # No usable response — unreachable, timed out, or the connection
+            # dropped mid-exchange. Ambiguous: the request may already have
+            # launched the bottle, so this is NOT a definite failure.
+            raise BrokerUnavailableError(f"POST /broker: {e}") from e
+
+
+def _json_object(raw: bytes) -> dict[str, object]:
+    """Parse a JSON object, tolerating an empty or malformed body (→ {}), like
+    the orchestrator client — a bad body becomes a clean 'missing field' error
+    downstream rather than an opaque JSON crash."""
+    if not raw:
+        return {}
+    try:
+        obj = json.loads(raw)
+    except ValueError:
+        return {}
+    return obj if isinstance(obj, dict) else {}
+
+
+def _error_detail(e: urllib.error.HTTPError) -> str:
+    """The `error` string from a structured error response, best-effort — an
+    error body may be absent or unreadable, in which case there is no detail."""
+    try:
+        detail = _json_object(e.read()).get("error", "")
+    except Exception:  # noqa: BLE001 — the error body is advisory only
+        return ""
+    return detail if isinstance(detail, str) else ""
+
+
+def _request_from(payload: dict[str, object]) -> LaunchRequest:
+    """Reconstruct the verified `LaunchRequest` the controller echoed, so the
+    returned value matches the in-process broker's (which returns the request it
+    acted on). A missing op/bottle_id means a malformed response."""
+    op = payload.get("op")
+    bottle_id = payload.get("bottle_id")
+    if not isinstance(op, str) or not isinstance(bottle_id, str) or not bottle_id:
+        raise BrokerClientError("host controller response missing op/bottle_id")
+    source_ip = payload.get("source_ip")
+    image_ref = payload.get("image_ref")
+    slot = payload.get("slot")
+    return LaunchRequest(
+        op=op,
+        bottle_id=bottle_id,
+        source_ip=source_ip if isinstance(source_ip, str) else "",
+        image_ref=image_ref if isinstance(image_ref, str) else "",
+        slot=slot if isinstance(slot, int) and not isinstance(slot, bool) else None,
+    )
+
+
+__all__ = [
+    "BrokerClient",
+    "BrokerClientError",
+    "DEFAULT_TIMEOUT_SECONDS",
+]

bot_bottle/orchestrator/host_server.py

@@ -0,0 +1,270 @@
+"""Host control server (issue #468) — the launch broker as a real host service.
+
+Chunk 1 of the host-control-server stack closes the **transport** gap the PRD
+opens with: today `LaunchBroker.submit(token)` is an in-process method call from
+`OrchestratorCore`, and a real host service needs it reachable over the wire.
+This module is that service — the single privileged host component — reached over
+**HTTP** (the universal transport 0070 chose), mirroring the orchestrator control
+plane's shape (`orchestrator/server.py`): a pure `dispatch()` for socket-free
+testing, wrapped by a thin stdlib `http.server` adapter.
+
+    GET  /health  -> 200 {"status": "ok"}
+    POST /broker  -> 200 {"op", "bottle_id", "source_ip", "image_ref", "slot"}
+                     400 (bad body) | 401 (bad provenance/schema) | 502 (backend)
+                     body: {"token": "<signed launch/teardown JWT>"}
+
+Only the **signed token** crosses the wire; the server holds the shared HS256
+secret and a real `LaunchBroker` (e.g. `DockerBroker`) and runs the existing
+`verify_request` + `_launch`/`_teardown` path behind the endpoint, so nothing
+free-form ever reaches it. Provenance/schema failures are fail-closed 401s that
+never touch the backend (`LaunchBroker.submit` verifies before acting), and a
+backend launch failure is a 502 the caller must surface — neither takes the
+controller down.
+
+The signed launch token *is* the endpoint's authentication (its provenance is the
+whole point of the JWS), so `/broker` needs no separate caller credential; the
+host controller's own lifecycle endpoints, which do, arrive with the durable
+`TrustDomain` key in a later chunk.
+
+The shared signing secret is read from `$BOT_BOTTLE_BROKER_SECRET` (hex). That is
+a **chunk-1 stopgap**: it must be provisioned to signer and verifier out of band,
+which is exactly what the durable `TrustDomain` key in chunk 2 (#476) replaces.
+"""
+
+from __future__ import annotations
+
+import argparse
+import http.server
+import json
+import os
+import socketserver
+import sys
+import typing
+from urllib.parse import urlsplit
+
+from .. import log
+from .broker import BrokerAuthError, LaunchBroker
+from .docker_broker import DockerBroker
+
+# JSON body payload type (parsed request / rendered response).
+Json = dict[str, object]
+
+# The hex-encoded HS256 secret shared with the request signer (the orchestrator).
+# Chunk-1 stopgap for the durable, out-of-band `TrustDomain` key of chunk 2.
+BROKER_SECRET_ENV = "BOT_BOTTLE_BROKER_SECRET"
+
+# Default host-controller port. Distinct from the orchestrator control plane
+# (8099) — a separate privileged component listening on its own socket.
+DEFAULT_PORT = 8091
+
+# Cap on the request body. A signed broker request is tiny, so rejecting anything
+# larger *before reading it* keeps a caller that can merely reach the socket (no
+# signed token needed) from exhausting memory or a handler thread with a huge
+# Content-Length — the signed token, not mere reachability, is the authority.
+MAX_BODY_BYTES = 64 * 1024
+
+# Per-request socket timeout, bounding how long a stalled / slow-loris caller can
+# hold a handler thread on this privileged listener.
+REQUEST_TIMEOUT_SECONDS = 15
+
+
+def _parse_json_object(body: bytes) -> Json:
+    """Parse a JSON object body. Raises ValueError for non-objects / bad JSON."""
+    if not body:
+        return {}
+    obj = json.loads(body)  # raises json.JSONDecodeError (a ValueError)
+    if not isinstance(obj, dict):
+        raise ValueError("request body must be a JSON object")
+    return obj
+
+
+def broker_secret_from_env(environ: typing.Mapping[str, str] | None = None) -> bytes | None:
+    """The shared HS256 secret from `$BOT_BOTTLE_BROKER_SECRET` (hex), or None
+    when unset or not valid hex. The signer (orchestrator, `--broker http`) and
+    the verifier (this server) read the same env var so both hold the same key —
+    the chunk-1 stand-in for out-of-band provisioning."""
+    env = os.environ if environ is None else environ
+    raw = env.get(BROKER_SECRET_ENV, "").strip()
+    if not raw:
+        return None
+    try:
+        return bytes.fromhex(raw)
+    except ValueError:
+        return None
+
+
+def dispatch(  # pylint: disable=too-many-return-statements
+    broker: LaunchBroker, method: str, path: str, body: bytes,
+) -> tuple[int, Json]:
+    """Route one host-control request to a (status, payload) pair. Pure — the
+    only side effect is the broker's own backend launch — so routing is testable
+    without a socket.
+
+    Total by design: a provenance/schema failure becomes 401 and a backend launch
+    failure becomes 502 rather than raising, so one bad request can neither act
+    on the backend nor take the controller down for the next caller."""
+    route = urlsplit(path).path.rstrip("/") or "/"
+
+    if method == "GET" and route == "/health":
+        return 200, {"status": "ok"}
+
+    if method == "POST" and route == "/broker":
+        try:
+            data = _parse_json_object(body)
+        except ValueError as e:
+            return 400, {"error": f"invalid JSON: {e}"}
+        token = data.get("token")
+        if not isinstance(token, str) or not token:
+            return 400, {"error": "token (string) is required"}
+        try:
+            req = broker.submit(token)
+        except BrokerAuthError as e:
+            # Fail-closed: bad signature, malformed token, or off-schema payload.
+            # `submit` verifies before acting, so nothing was launched.
+            return 401, {"error": f"broker auth failed: {e}"}
+        except Exception as e:  # noqa: BLE001 — a backend launch failure (docker
+            # down, image gone) is operational, not a control-plane bug; the
+            # caller must see it as a distinct 502, and the server must stay up.
+            return 502, {"error": f"backend launch failed: {e}"}
+        return 200, {
+            "op": req.op,
+            "bottle_id": req.bottle_id,
+            "source_ip": req.source_ip,
+            "image_ref": req.image_ref,
+            "slot": req.slot,
+        }
+
+    return 404, {"error": "not found"}
+
+
+class Handler(http.server.BaseHTTPRequestHandler):
+    """Thin stdlib adapter: read the body, call `dispatch`, write JSON."""
+
+    # Socket timeout per request (applied by StreamRequestHandler.setup) so a
+    # stalled caller can't pin a handler thread on this privileged listener.
+    timeout = REQUEST_TIMEOUT_SECONDS
+
+    # Quiet by default; opt back into stdlib access logging with
+    # BOT_BOTTLE_HOST_CONTROLLER_DEBUG (the controller has its own logging).
+    def log_message(self, format: str, *args: typing.Any) -> None:  # noqa: A002
+        if os.environ.get("BOT_BOTTLE_HOST_CONTROLLER_DEBUG"):
+            super().log_message(format, *args)
+
+    def _serve(self, method: str) -> None:
+        """Read the request body (bounded), dispatch it, and write the JSON
+        reply. A dispatch that raises (it shouldn't — dispatch is total) still
+        returns a 500 rather than dropping the connection."""
+        server = self.server
+        assert isinstance(server, HostControlServer)
+        try:
+            length = int(self.headers.get("Content-Length") or 0)
+        except ValueError:
+            self._reply(400, {"error": "invalid Content-Length"})
+            return
+        if length < 0 or length > MAX_BODY_BYTES:
+            # Reject before reading: nothing legitimate is this big, so an
+            # oversized declared length is a bug or a resource-exhaustion attempt.
+            self._reply(413, {"error": "request body too large"})
+            return
+        body = self.rfile.read(length) if length > 0 else b""
+        try:
+            status, payload = dispatch(server.broker, method, self.path, body)
+        except Exception as e:  # noqa: BLE001 — the controller must stay up
+            sys.stderr.write(f"host controller: {method} {self.path} failed: {e!r}\n")
+            sys.stderr.flush()
+            status, payload = 500, {"error": f"internal error: {e}"}
+        self._reply(status, payload)
+
+    def _reply(self, status: int, payload: typing.Mapping[str, object]) -> None:
+        """Write one JSON response with an explicit Content-Length."""
+        data = json.dumps(payload).encode()
+        self.send_response(status)
+        self.send_header("Content-Type", "application/json")
+        self.send_header("Content-Length", str(len(data)))
+        self.end_headers()
+        self.wfile.write(data)
+
+    def do_GET(self) -> None:
+        self._serve("GET")
+
+    def do_POST(self) -> None:
+        self._serve("POST")
+
+
+class HostControlServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
+    """Threading HTTP server that carries the launch broker for its handlers.
+
+    The broker holds the shared signing secret and performs the backend-native
+    launch/teardown; the server itself keeps no secret of its own — provenance
+    rides entirely in each request's signed token."""
+
+    daemon_threads = True
+    allow_reuse_address = True
+
+    def __init__(self, address: tuple[str, int], broker: LaunchBroker) -> None:
+        self.broker = broker
+        super().__init__(address, Handler)
+
+
+def make_host_server(
+    broker: LaunchBroker, host: str = "127.0.0.1", port: int = DEFAULT_PORT
+) -> HostControlServer:
+    """Build (but do not start) a host control server. `port=0` binds an
+    ephemeral port — read `server.server_address` for the actual one."""
+    return HostControlServer((host, port), broker)
+
+
+def main(argv: list[str] | None = None) -> int:
+    """Run the host control server as a plain process (dev-harness).
+
+        python -m bot_bottle.orchestrator.host_server [--host H] [--port P]
+
+    Fail-closed: without a shared `$BOT_BOTTLE_BROKER_SECRET` the server can
+    verify no request's provenance, so it refuses to start rather than run a
+    launcher that accepts unsigned input."""
+    parser = argparse.ArgumentParser(prog="bot_bottle.orchestrator.host_server")
+    parser.add_argument("--host", default="127.0.0.1", help="bind address")
+    parser.add_argument("--port", type=int, default=DEFAULT_PORT, help="bind port (0 = ephemeral)")
+    args = parser.parse_args(argv)
+
+    secret = broker_secret_from_env()
+    if secret is None:
+        sys.stderr.write(
+            f"host controller: refusing to start without a shared signing secret "
+            f"(${BROKER_SECRET_ENV}, hex) — it could verify no request's "
+            "provenance and would relay unsigned launches to the backend\n"
+        )
+        sys.stderr.flush()
+        return 2
+
+    broker = DockerBroker(secret)
+    server = make_host_server(broker, host=args.host, port=args.port)
+    bound_host, bound_port = server.server_address[0], server.server_address[1]
+    log.info(
+        "host control server listening",
+        context={"host": bound_host, "port": bound_port},
+    )
+    try:
+        server.serve_forever()
+    except KeyboardInterrupt:
+        log.info("host controller shutting down")
+    finally:
+        server.server_close()
+    return 0
+
+
+__all__ = [
+    "dispatch",
+    "Handler",
+    "HostControlServer",
+    "make_host_server",
+    "broker_secret_from_env",
+    "main",
+    "Json",
+    "BROKER_SECRET_ENV",
+    "DEFAULT_PORT",
+]
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

bot_bottle/orchestrator/service.py

@@ -25,7 +25,7 @@ import json
 from collections.abc import Iterable
 from datetime import datetime, timezone
 
-from .broker import LaunchBroker, LaunchRequest, sign_request
+from .broker import BrokerUnavailableError, LaunchRequest, SubmitBroker, sign_request
 from .store.registry_store import DEFAULT_REAP_GRACE_SECONDS, BottleRecord, RegistryStore
 from .supervisor import (
     AuditEntry,
@@ -62,7 +62,7 @@ class OrchestratorCore:
     def __init__(
         self,
         registry: RegistryStore,
-        broker: LaunchBroker,
+        broker: SubmitBroker,
         sign_secret: bytes,
         supervisor: Supervisor | None = None,
     ) -> None:
@@ -111,14 +111,23 @@ class OrchestratorCore:
             image_ref=image_ref,
             slot=slot,
         )
-        launched = False
         try:
             self._broker.submit(sign_request(req, self._secret))
-            launched = True
-        finally:
-            if not launched:
-                self.registry.deregister(rec.bottle_id)
-                self._tokens.pop(rec.bottle_id, None)
+        except BrokerUnavailableError:
+            # Ambiguous delivery failure (timeout / dropped response): the broker
+            # may already have launched the bottle before the response was lost.
+            # Do NOT deregister — that would orphan a running container with no
+            # registry row (reconcile reaps rows, never containers). Keep the row
+            # so reconcile reaps it iff the bottle is not actually live; surface
+            # the error so the caller knows the launch is unconfirmed.
+            raise
+        except Exception:
+            # A definite failure — a fail-closed rejection, a backend launch
+            # error, or the host reporting it did not launch: nothing is running,
+            # so roll the registry entry back to leave no orphan.
+            self.registry.deregister(rec.bottle_id)
+            self._tokens.pop(rec.bottle_id, None)
+            raise
         return rec
 
     def teardown_bottle(self, bottle_id: str) -> bool:

bot_bottle/orchestrator/store/secret_store.py

@@ -12,12 +12,22 @@ reattachment path reads ENV_VAR_SECRET from the running agent container via
 ``POST /bottles/<id>/reprovision_gateway``; the orchestrator decrypts the
 stored rows and re-populates ``_tokens``.
 
-Encryption scheme: HMAC-SHA256 used as a PRF in CTR mode (stdlib-only,
-no external deps). Each value is encrypted independently. The output blob is
-``nonce (16 bytes) || ciphertext`` encoded as URL-safe base64 (no padding).
+Encryption scheme: HMAC-SHA256 used as a PRF in CTR mode, **authenticated**
+encrypt-then-MAC (stdlib-only, no external deps). Each value is encrypted
+independently. The output blob is ``nonce (16 bytes) || ciphertext || tag
+(32 bytes)`` encoded as URL-safe base64 (no padding).
 
     keystream_block_i = HMAC-SHA256(key, nonce || i.to_bytes(4, "big"))
     ciphertext_i      = plaintext_i XOR keystream_block_i[:len(plaintext_i)]
+    mac_key           = HMAC-SHA256(key, "bottled-secret-mac-v1")
+    tag               = HMAC-SHA256(mac_key, nonce || ciphertext)
+
+The tag is what makes a **wrong key deterministically detectable**: without it,
+CTR decryption with the wrong key yields garbage that only fails when it isn't
+valid UTF-8 (so ``reprovision`` would sometimes "succeed" with a wrong
+ENV_VAR_SECRET and inject garbage egress tokens). The MAC key is derived from
+the ENV_VAR_SECRET by a domain-separated HMAC so the same key never both
+generates the keystream and signs the tag with the same message shape.
 """
 
 from __future__ import annotations
@@ -29,6 +39,7 @@ import secrets
 
 _KEY_BYTES = 32    # 256-bit key from ENV_VAR_SECRET
 _NONCE_BYTES = 16  # 128-bit random nonce per encrypt call
+_TAG_BYTES = 32    # HMAC-SHA256 authentication tag
 _BLOCK = 32        # HMAC-SHA256 output width == one keystream block
 
 # Env-var name the agent container receives at startup.
@@ -50,45 +61,58 @@ def _keystream(key: bytes, nonce: bytes, block_index: int) -> bytes:
     ).digest()
 
 
+def _tag(key: bytes, nonce: bytes, ciphertext: bytes) -> bytes:
+    """The authentication tag over ``nonce || ciphertext``, keyed by a MAC
+    subkey domain-separated from the keystream key."""
+    mac_key = hmac.new(key, b"bottled-secret-mac-v1", hashlib.sha256).digest()
+    return hmac.new(mac_key, nonce + ciphertext, hashlib.sha256).digest()
+
+
+def _ctr(key: bytes, nonce: bytes, data: bytes) -> bytes:
+    """CTR keystream XOR — its own inverse, so it both encrypts and decrypts."""
+    out = bytearray()
+    for i in range(0, len(data), _BLOCK):
+        chunk = data[i : i + _BLOCK]
+        ks = _keystream(key, nonce, i)[: len(chunk)]
+        out.extend(b ^ k for b, k in zip(chunk, ks))
+    return bytes(out)
+
+
 def encrypt_value(secret_b64: str, plaintext: str) -> str:
     """Encrypt a single string value with *secret_b64* (the ENV_VAR_SECRET).
 
-    Returns a URL-safe base64 blob ``nonce || ciphertext`` suitable for
+    Returns a URL-safe base64 blob ``nonce || ciphertext || tag`` suitable for
     the ``bottled_agent_secrets.value`` column."""
     key = _b64dec(secret_b64)
-    pt = plaintext.encode()
     nonce = secrets.token_bytes(_NONCE_BYTES)
-    ct = bytearray()
-    for i in range(0, len(pt), _BLOCK):
-        chunk = pt[i : i + _BLOCK]
-        ks = _keystream(key, nonce, i)[: len(chunk)]
-        ct.extend(p ^ k for p, k in zip(chunk, ks))
-    return base64.urlsafe_b64encode(nonce + bytes(ct)).rstrip(b"=").decode()
+    ct = _ctr(key, nonce, plaintext.encode())
+    tag = _tag(key, nonce, ct)
+    return base64.urlsafe_b64encode(nonce + ct + tag).rstrip(b"=").decode()
 
 
 def decrypt_value(secret_b64: str, blob_b64: str) -> str:
     """Decrypt a blob produced by :func:`encrypt_value`.
 
     Returns the original plaintext string. Raises ``ValueError`` for malformed
-    input or a key mismatch (wrong key produces garbage, not an error, unless
-    the plaintext is non-UTF-8 — treat all such failures as wrong key)."""
+    input, a **wrong key**, or a tampered ciphertext — all caught by the
+    authentication tag before any plaintext is returned, so a wrong
+    ENV_VAR_SECRET is rejected deterministically (never a garbage token)."""
     key = _b64dec(secret_b64)
     try:
         blob = _b64dec(blob_b64)
     except Exception as exc:
         raise ValueError(f"invalid ciphertext blob: {exc}") from exc
-    if len(blob) < _NONCE_BYTES:
+    if len(blob) < _NONCE_BYTES + _TAG_BYTES:
         raise ValueError("ciphertext blob too short")
-    nonce, ciphertext = blob[:_NONCE_BYTES], blob[_NONCE_BYTES:]
-    pt = bytearray()
-    for i in range(0, len(ciphertext), _BLOCK):
-        chunk = ciphertext[i : i + _BLOCK]
-        ks = _keystream(key, nonce, i)[: len(chunk)]
-        pt.extend(c ^ k for c, k in zip(chunk, ks))
+    nonce = blob[:_NONCE_BYTES]
+    tag = blob[-_TAG_BYTES:]
+    ciphertext = blob[_NONCE_BYTES:-_TAG_BYTES]
+    if not hmac.compare_digest(tag, _tag(key, nonce, ciphertext)):
+        raise ValueError("ciphertext failed authentication (wrong key or tampered)")
     try:
-        return bytes(pt).decode()
-    except UnicodeDecodeError as exc:
-        raise ValueError(f"decryption produced non-UTF-8 output (wrong key?): {exc}") from exc
+        return _ctr(key, nonce, ciphertext).decode()
+    except UnicodeDecodeError as exc:  # pragma: no cover - authenticated, so unreachable
+        raise ValueError(f"decryption produced non-UTF-8 output: {exc}") from exc
 
 
 __all__ = ["ENV_VAR_SECRET_NAME", "new_env_var_secret", "encrypt_value", "decrypt_value"]

docs/decisions/0004-coverage-policy.md

@@ -3,10 +3,6 @@
 - **Status:** Accepted
 - **Date:** 2026-06-25
 - **Deciders:** didericis
-- **Revised:** 2026-07-27 — thresholds relaxed (critical minimum 90→85%,
-  diff-coverage gate 90→80%) to cut low-value test churn on changed lines.
-  The risk-weighting structure and the "global is informational" rule are
-  unchanged.
 
 ## Context
 
@@ -38,7 +34,7 @@ a regression (Goodhart's law).
 Coverage is **risk-weighted**, measured over the **combined unit +
 integration** suites, with three rules:
 
-1. **Critical modules must remain ≥ 85%.** The curated security/logic core
+1. **Critical modules must remain ≥ 90%.** The curated security/logic core
    covers the host and gateway egress policy, manifest trust boundary,
    git-gate enforcement, supervise protocol/server, YAML parser, and bottle
    state. The concrete module list lives in `scripts/critical-modules.txt`;
@@ -59,7 +55,7 @@ integration** suites, with three rules:
 
 The forward-looking guard is a **diff-coverage gate**
 (`scripts/diff_coverage.py`): new/changed executable lines on a branch
-must be ≥ 80% covered. This catches regressions where they are
+must be ≥ 90% covered. This catches regressions where they are
 introduced without forcing a back-fill crusade through legacy glue. The
 gate skips lines in omitted files (there is no coverage data for them),
 so the omit list cannot launder *new* logic into the dark: anything that

docs/prds/prd-new-host-control-server.md

@@ -0,0 +1,273 @@
+# PRD prd-new: Host control server
+
+- **Status:** Draft
+- **Author:** Claude
+- **Created:** 2026-07-26
+- **Issue:** #468
+
+## Summary
+
+Promote the in-process launch broker into a standalone **host control
+server**: the single privileged component on the host. Both the CLI and the
+orchestrator drive it over HTTP; it brokers agent launches, owns the
+orchestrator's own lifecycle, and is the sole writer of host-durable state (the
+tamper-evident audit record). This closes the three gaps between today's
+well-formed broker *contract* ([`orchestrator/broker.py`](../../bot_bottle/orchestrator/broker.py))
+and a real out-of-process service — transport, durable provisioned secret,
+and a disciplined op vocabulary — and splits host state by
+owner and lifetime. The prize: **the CLI no longer needs the Docker socket**,
+which is what finally lets a dedicated Gitea runner user drop the
+root-equivalent `docker` group (PRD 0070, "Relationship to other work").
+
+## Problem
+
+Container launches run directly from a short-lived CLI process against the
+Docker socket. That socket is root-equivalent, so every host that launches
+bottles hands root to whoever invokes the CLI — including a CI runner user we
+want to keep unprivileged. PRD 0070 already argues for replacing the fat socket
+with a **thin, structured, auditable** launch broker, and the contract for that
+broker exists and is tested in-process. But it is *only* in-process:
+`LaunchBroker.submit(token)` is a method call from
+`OrchestratorCore.launch_bottle` ([`service.py:116`](../../bot_bottle/orchestrator/service.py)),
+and `DockerBroker` is on no production path — every backend starts the
+orchestrator with `--broker stub` ([`__main__.py:54`](../../bot_bottle/orchestrator/__main__.py)).
+
+Three gaps stand between that scaffold and a host service:
+
+1. **No transport.** `submit` is an in-process call. A real service needs a
+   `BrokerClient` that POSTs the signed token and a host-side HTTP server that
+   verifies and acts.
+2. **The signing secret is ephemeral and self-generated.**
+   [`__main__.py:53`](../../bot_bottle/orchestrator/__main__.py) does
+   `secrets.token_bytes(32)` and hands the *same value* to signer and verifier —
+   viable only because they share a process. A separate daemon needs the secret
+   provisioned out of band and durable across orchestrator restarts.
+3. **The op vocabulary is `launch` / `teardown` only.** Everything else
+   host-privileged still lives in the CLI, so the schema has to grow — carefully,
+   since PRD 0070's security argument rests on "structured requests only, static
+   flags + ids."
+
+Separately, host state has no clear owner. `OrchestratorCore.reconcile` takes
+`live_source_ips` as a parameter *only because the orchestrator cannot see the
+backend* ([`service.py:137`](../../bot_bottle/orchestrator/service.py)); the
+egress traffic log is written to the container's stderr; and there is no durable,
+tamper-evident home for the audit record that survives orchestrator destruction.
+
+## Goals / Success Criteria
+
+- A standalone host control server that the CLI and orchestrator reach over
+  **HTTP**, with three entry paths working end to end:
+  - `web console -(iroh)-> orchestrator -(http)-> host controller -> launch`
+  - `cli -(http)-> orchestrator -(http)-> host controller -> launch`
+  - `cli -(http)-> host controller` — start / restart / status of the
+    orchestrator **itself** (the bootstrap/recovery path #391 targets).
+- The launch op is expressed as a **signed JWT of static flags + ids only**,
+  verified against a closed schema.
+- The signing secret is **provisioned out of band and durable** across
+  orchestrator restarts (a `TrustDomain` per #476, with a key the orchestrator
+  never holds for the host controller's *own* endpoints).
+- Host-privileged operations move off the CLI to the control server; **the CLI
+  no longer opens the Docker socket** for bottle operations.
+- `Orchestrator.reconcile` no longer takes `live_source_ips` — live-bottle
+  enumeration becomes an internal control-server call.
+- Host-durable state lands as an **append-only, hash-chained JSONL** audit log
+  owned solely by the host controller; operational state stays SQLite owned
+  solely by the orchestrator.
+
+## Non-goals
+
+- **Removing standing privilege.** This converts on-demand privilege (a CLI the
+  user invokes) into standing privilege (a daemon under launchd/systemd). The
+  win is that the privilege is *narrower* (structured requests vs. a raw socket),
+  not that it disappears. "Always running" is an accepted new property.
+- **Asymmetric signing.** We stay HS256 — see Design / "Signing stays
+  symmetric."
+- **Integrity against a live compromised orchestrator.** Host-location of the
+  audit log does not buy this: the orchestrator makes the decisions being audited
+  and can forge or omit entries wherever the file lives. An off-box copy is the
+  answer, tracked separately.
+- **A single unified DB for all state.** Impossible over a guest-kernel share
+  (SQLite locking is not coherent); state is split by owner and lifetime instead.
+- **The generic `SecretProvider` (#355)** and **remote terminal design (#478)** —
+  both ride the same door but are their own work.
+
+## Design
+
+### Topology
+
+The host controller is the sole privileged component. The orchestrator becomes a
+client of it for launches, and the CLI becomes a client of it for *both* bottle
+operations (indirectly, through the orchestrator) and orchestrator lifecycle
+(directly, for bootstrap/recovery — startup can't route through the thing being
+started).
+
+```
+web console ─(iroh)─▶ orchestrator ─┐
+                                    ├─(http, signed JWT)─▶ host controller ─▶ launch
+cli ────────(http)──▶ orchestrator ─┘
+cli ────────(http, bearer)──────────────────────────────▶ host controller   (orchestrator lifecycle)
+```
+
+### Transport: `BrokerClient` + host server
+
+`LaunchBroker.submit(token)` keeps its exact signature and semantics; only the
+*wire* changes. A new `BrokerClient` implements the same submit contract by
+POSTing the signed token to the host controller (stdlib `urllib`, like the
+existing [`orchestrator/client.py`](../../bot_bottle/orchestrator/client.py)),
+and the host controller's launch handler is the existing `verify_request` +
+`_launch`/`_teardown` path, now reached over HTTP instead of a method call. The
+in-process `StubBroker` stays for the dev-harness and tests; `DockerBroker`'s
+`_launch`/`_teardown` bodies move behind the server unchanged. Because the client
+satisfies the same interface `OrchestratorCore` already depends on, the core does
+not change to gain a real backend.
+
+### Signing stays symmetric (HS256)
+
+PRD 0070 nominally specifies asymmetric; the code is HS256 and we keep it.
+Asymmetric matters when the verifier is *less* privileged than the signer — here
+it is the reverse: the host controller (verifier) is strictly more privileged
+than the orchestrator (signer), and a controller that could forge orchestrator
+requests gains nothing, since it is already the component that launches. Staying
+symmetric also honors the no-runtime-deps policy (stdlib has no Ed25519). This
+matches the reasoning already inlined in `broker.py`'s module docstring.
+
+### Replay protection is out of scope (tracked in #494)
+
+Once the launch token travels over a wire, a captured token could be replayed —
+`sign_request` already emits `jti`/`iat` but `verify_request` reads neither, so
+there is no expiry window or `jti` cache today. Enforcing that (an `iat` window +
+a self-trimming `jti` cache) is a pure in-process change that lands independently
+of this work, and it is deferred to **#494** rather than gating the MVP of the
+host control server. Nothing here depends on it; it can merge before or after.
+
+### Op vocabulary and the "ids + static flags" rule (gap 3)
+
+Each op moved off the CLI widens the privileged surface, so growth is governed by
+one explicit rule, enforced in `verify_request`'s schema check:
+
+> A broker op carries **only ids and enumerated static flags** — a bottle id, a
+> pool slot, a **content-addressed** image ref chosen from a fixed set, an op
+> name from a closed vocabulary. Never a free-form path, argv, command, or
+> caller-supplied filesystem location. If an operation cannot be expressed that
+> way, it does not become a broker op.
+
+Operations that fit and move off the CLI (all today in
+`backend/*/consolidated_launch.py`, driven by a short-lived CLI process):
+
+| Op | What it does | Fits the rule because |
+|---|---|---|
+| `launch` / `teardown` | existing | ids + slot + image ref |
+| `orchestrator.ensure_running` | start the infra container | no arguments |
+| `orchestrator.{start,restart,status}` | lifecycle (the #391 path) | no arguments |
+| `list_live` | enumerate running bottles for reconcile | no arguments; returns ids/IPs |
+| `allocate_ip` | `next_free_ip` over `_network_container_ips` | no arguments; returns an IP |
+| `provision_git_gate` | `cp`/`exec` a per-bottle deploy key into the gateway | bottle id + key handle, no path |
+| `reprovision` | `docker exec printenv <ENV_VAR_SECRET>` on a live agent | bottle id + secret *name* |
+
+Image **builds** stay with the orchestrator for v1 (PRD 0070 §Memory: builds run
+control-plane-side; a dedicated slim build unit is later, #468-adjacent), so no
+`build` broker op is added here.
+
+With `list_live` as an internal control-server call, `Orchestrator.reconcile`'s
+`live_source_ips` parameter goes away — the tell PRD 0070 called out that the
+orchestrator couldn't see the backend disappears with it.
+
+### Secret provisioning (gap 2)
+
+The shared HS256 secret becomes a durable, out-of-band artifact via the
+**`TrustDomain`** seam (#476,
+[`trust_domain.py`](../../bot_bottle/trust_domain.py)):
+
+- The **launch-broker secret** is a `TrustDomain` whose key
+  (`host_signing_key(<file>)`, minted 0600 on first use, durable under
+  `bot_bottle_root()`) is provisioned to the orchestrator (signer) and the host
+  controller (verifier). Durability across orchestrator restarts is what makes
+  re-adoption work — a restart re-verifies against the same key.
+- The **host controller's own lifecycle endpoints** (the direct `cli -> host
+  controller` path) get a **separate** `TrustDomain` key the orchestrator never
+  holds — exactly the second domain #476's PRD reserves. The orchestrator must
+  not be able to mint the credentials used to start and stop it.
+
+This reuses the seam #476 landed rather than re-deriving provisioning per
+backend (the PR #471 bug class).
+
+### One daemon, structurally separate handlers (open decision 1)
+
+The audit writer and the broker live in **one daemon** for install simplicity,
+but with **no shared parsing** and **different credentials per handler**:
+
+- the **launch** handler requires the signed launch **JWT** (provenance +
+  un-coercible schema);
+- the **audit-append** handler takes a plain **bearer token** and writes to the
+  JSONL log.
+
+This does not defend against orchestrator compromise (it holds both creds) — it
+stops a bug in the boring audit path from reaching the privileged launch path.
+The launcher stays small enough to audit line-by-line, per PRD 0070.
+
+### State ownership: split by owner and lifetime
+
+A single mounted DB is impossible — SQLite locking is not coherent across guest
+kernels over a share, which is why the macOS backend already uses a container-only
+volume (`INFRA_DB_VOLUME`). So state splits three ways (depends on #469, which
+gets `bot-bottle.db` off the data plane first):
+
+| Owner | State | Home | Shape |
+|---|---|---|---|
+| **Orchestrator** | `orchestrator_bottles` registry; `bottled_agent_secrets` (encrypted egress tokens); `supervise_proposals` / `supervise_responses` | volume nothing else mounts (generalizing the macOS design) | **SQLite** — mutable, transactional, queried |
+| **Host controller** | supervise audit entries; egress traffic log (today → container stderr); host-side config | host filesystem, survives orchestrator/volume destruction | **JSONL** — append-only |
+| **Gateway** | none | — | after #469 the data plane holds no DB state |
+
+The historical record is **JSONL, not SQLite**, because it is append-only, never
+updated, never transactionally queried: `O_APPEND` writes are atomic, there is no
+locking protocol to get wrong, hash-chaining for tamper-evidence is cheap, and it
+survives container-runtime volume pruning (the #450 lesson) and stays readable
+without the orchestrator running. Both halves of "the audit record" — supervise
+decisions and the egress traffic log — land in the one place.
+
+The orchestrator is **sole mounter and sole writer** of its SQLite volume; the
+host controller is **sole writer** of the JSONL log, over the authenticated
+audit-append channel.
+
+## Implementation chunks
+
+Ordered, each independently mergeable:
+
+1. **`BrokerClient` + host launch server** over HTTP, reusing `verify_request`
+   and the existing `DockerBroker` bodies. Wire `OrchestratorCore` to a
+   `BrokerClient` behind a flag; keep `StubBroker` for the dev-harness. Closes
+   gap 1.
+2. **Durable secret via `TrustDomain`** — provision the launch-broker key to
+   signer + verifier; add the host controller's own lifecycle `TrustDomain`.
+   Closes gap 2.
+3. **Grow the op vocabulary** one op at a time (`list_live` first — it also
+   removes `reconcile`'s `live_source_ips`), each behind the ids + static-flags
+   rule. Closes gap 3.
+4. **JSONL audit log** — the host-controller-owned, hash-chained historical
+   record with the plain-bearer audit-append handler; redirect the egress traffic
+   log into it.
+5. **Drop the Docker socket from the CLI** once every host-privileged op it used
+   is a broker op — the payoff that unblocks the unprivileged Gitea runner user.
+
+## Open questions
+
+1. **Schema-width rule enforcement.** The "ids + static flags" rule is stated;
+   should `verify_request` reject unknown claim keys outright (strict schema) to
+   keep the surface from drifting? Leaning yes.
+2. **Audit-append back-pressure.** What the audit handler does if the JSONL sink
+   is unavailable (fail-closed vs. buffer) — resolve before shipping chunk 5.
+
+## References
+
+- **PRD 0070** — the contract, the launch broker, and the state tiers this
+  implements.
+- **#469** — get `bot-bottle.db` off the data plane (lands underneath this).
+- **#476** ([`prd-new-control-plane-auth-provisioning`](prd-new-control-plane-auth-provisioning.md))
+  — the `TrustDomain` seam this plugs the host controller's key into.
+- **#391** — backend-agnostic orchestrator restart (the bootstrap path).
+- **#494** — enforce broker replay protection (`iat` window + `jti` cache); split
+  out of this PRD as an independent in-process change.
+- **#386** — prebuilt images from the Gitea OCI registry (the fixed image set the
+  broker validates against).
+- **#355** — generic `SecretProvider`.
+- **#478** — remote terminal design.

docs/research/agent-sandbox-landscape.md

@@ -32,17 +32,9 @@ not a principled scope exclusion: both are major hosted sandbox platforms and
 belong in this landscape even though they target platform builders rather than
 bot-bottle's local single-operator workflow.
 
-Updated 2026-07-27 after a scan of recent Show HN launches: **Black LLAB,
-Eve, CloudRouter, Nucleus, yolo-cage, and Sandbox Agent SDK** added as a
-dated entrant cohort. They sharpen the comparison on three axes the original
-table underweighted: the browser/preview loop, parallel-agent operator UX, and
-a provider-neutral automation/session API.
-
 ## Summary
 
-The main table compares bot-bottle against fifteen canonical
-isolation/sandbox tools; a later section evaluates six recent HN entrants
-without widening an already unwieldy table.
+The main table compares bot-bottle against fifteen isolation/sandbox tools.
 Governance/pre-action authorization and credential-only layers are covered
 separately because they don't provide VM or container isolation. None
 duplicate bot-bottle's combination of local
@@ -550,199 +542,6 @@ them.
   framework runtime is not compromised.
 - **Maturity**: Specification + reference implementation, 2026.
 
-## Recent HN entrants (added 2026-07-27)
-
-These are grouped by launch date rather than promoted into the main table.
-Several are young or sparsely documented, and putting them beside mature
-runtime platforms with false precision would obscure the useful comparison.
-The HN launch posts are the evidence snapshot; feature claims should be
-rechecked against their repositories before relying on them for a security
-decision.
-
-### Black LLAB
-
-- **Source**: https://github.com/isaacdear/black-llab ;
-  HN launch https://news.ycombinator.com/item?id=47402394
-- **Isolation/locality**: Local Docker environment, with an isolated container
-  created for each agent task. Shared host kernel; no stronger boundary is
-  claimed.
-- **Agent integration**: General local/cloud model workspace. Its headline is
-  dynamic routing of simple prompts to local models and complex prompts to
-  hosted models, with code execution and web scraping inside the task
-  container.
-- **Network/credentials**: No default-deny egress, payload inspection, or
-  host-side credential injection documented in the launch.
-- **Competitive read**: Superficial overlap ("a container per agent task"),
-  but not a direct security-policy competitor. Its useful challenge is the
-  integrated model-selection UX, which bot-bottle intentionally leaves to the
-  selected agent provider.
-- **Maturity**: Early solo project; HN launch received 1 point.
-
-### Eve
-
-- **Source**: https://eve.new/ ;
-  HN launch https://news.ycombinator.com/item?id=47721255
-- **Isolation/locality**: Managed, hosted Linux sandbox per user/session
-  (claimed 2 vCPU, 4 GB RAM, 10 GB disk), with filesystem, code execution,
-  headless Chromium, and service connectors.
-- **Agent integration**: End-user OpenClaw-style agent product. An orchestrator
-  routes subtasks to specialist models and can run parallel subagents that
-  coordinate through a shared filesystem. Web UI and iMessage are primary
-  interaction surfaces.
-- **Network/credentials**: Broad connectors are a product feature; the launch
-  does not document bot-bottle-style default-deny route policy, content DLP,
-  or credentials held outside the sandbox.
-- **Competitive read**: Adjacent, not direct. Eve sells a managed colleague;
-  bot-bottle lets an operator run existing coding-agent CLIs under local
-  containment. Eve nevertheless demonstrates the appeal of background work,
-  live progress, browser capability, and mobile notification.
-- **Maturity**: Commercial hosted product; HN launch received 71 points and
-  39 comments.
-
-### CloudRouter
-
-- **Source**: https://github.com/manaflow-ai/manaflow/tree/main/packages/cloudrouter ;
-  HN launch https://news.ycombinator.com/item?id=47006393
-- **Isolation/locality**: Claude Code or Codex runs locally and provisions
-  remote cloud VMs/GPUs for execution. Project files are uploaded to the VM;
-  each machine exposes auth-protected VNC, VS Code, and Jupyter surfaces.
-- **Agent integration**: A skill plus CLI lets the coding agent itself start,
-  command, inspect, and tear down machines. Browser automation is integrated,
-  including snapshots and screenshots. Parallel disposable compute is the
-  central workflow.
-- **Network/credentials**: The launch emphasizes remote resource isolation and
-  authenticated UI endpoints, not default-deny guest egress, payload DLP, or
-  proxy-held application credentials.
-- **Competitive read**: The closest recent workflow competitor. It directly
-  addresses parallel coding agents, environmental conflict, and closing the
-  browser/test loop, but trades local custody for elastic cloud compute.
-  Cloud VMs and GPUs could be a future bot-bottle backend; they do not replace
-  its manifest/policy layer.
-- **Maturity**: Active open-source monorepo project; HN launch received
-  138 points and 36 comments.
-
-### Nucleus
-
-- **Source**: https://github.com/coproduct-opensource/nucleus ;
-  HN launch https://news.ycombinator.com/item?id=46855770
-- **Isolation/locality**: Firecracker microVM with an enforcing MCP tool proxy.
-- **Agent integration/config**: Compositional permission envelope for
-  read/write/run actions. The envelope is non-escalating and can tighten or
-  terminate, with scoped approval tokens for gated operations.
-- **Network/credentials**: Default-deny egress, DNS allowlist, iptables drift
-  detection, time/budget caps, and hash-chained audit logging are claimed.
-  Remote append-only audit storage and attestation were roadmap items at
-  launch.
-- **Competitive read**: Direct on security architecture, especially
-  non-escalating policy and tamper-evident audit. It is an early execution/tool
-  proxy rather than a provider-neutral, one-command coding-agent product. Its
-  tool-level action envelope is semantically finer than bot-bottle's network
-  boundary; bot-bottle is stronger on turnkey agent/provider integration,
-  credential custody, Git mediation, and long-running operator workflow.
-- **Maturity**: Early OSS experiment; HN launch received 3 points.
-
-### yolo-cage
-
-- **Source**: https://github.com/borenstein/yolo-cage ;
-  HN launch https://news.ycombinator.com/item?id=46706796
-- **Isolation/locality**: Local sandbox for running multiple coding agents in
-  YOLO mode. The launch discussion describes a VM boundary.
-- **Agent integration**: Built around the native Claude Code experience and
-  motivated by running many agents in parallel without permission-prompt
-  fatigue.
-- **Network/Git/credentials**: Strict egress filtering, configurable HTTP
-  middleware, and mediated `git`/`gh` dispatch are the main value. The launch
-  discussion explicitly identifies provider credential handling as unfinished
-  and difficult because Claude state spans multiple host paths.
-- **Competitive read**: The closest new threat-model competitor. It shares
-  bot-bottle's premise that filesystem isolation alone is insufficient and
-  that Git plus authorized HTTP channels need mediation. bot-bottle currently
-  leads on cross-provider support, proxy-held Claude/Codex/forge credentials,
-  typed per-role manifests, content DLP, and supervision. yolo-cage's simpler
-  pitch and narrower Claude-first setup may be easier to explain.
-- **Maturity**: Early local tool; HN launch received 60 points and 76 comments.
-
-### Sandbox Agent SDK
-
-- **Source**: https://github.com/rivet-dev/sandbox-agent ;
-  HN launch https://news.ycombinator.com/item?id=46795584
-- **Isolation/locality**: Does not provide the isolation primitive. It runs
-  inside E2B, Daytona, Modal, Cloudflare Containers, Agent Computer, BoxLite,
-  Docker, or another sandbox provider. Embedded mode can also run locally
-  without a sandbox.
-- **Agent integration**: Provider-neutral Rust server/SDK exposing a common
-  HTTP/SSE/OpenAPI interface across Claude Code, Codex, OpenCode, Cursor, Amp,
-  and Pi, plus a universal event/session schema for external storage and
-  replay. It also exposes filesystem, managed-process, terminal, MCP, skills,
-  custom-tool, and computer-use APIs. TypeScript is the primary SDK surface.
-- **Network/credentials**: Delegated to the chosen sandbox provider.
-- **Credential posture**: Its documented convenience command extracts real
-  OpenAI/Anthropic credentials from local agent configuration and passes them
-  as environment variables into the sandbox. That is materially weaker than
-  bot-bottle's host-side credential custody, but it is an integration choice,
-  not a structural limitation: a sandbox provider could put a credential
-  proxy underneath the same SDK.
-- **Competitive read**: A serious architectural threat despite not supplying
-  isolation. Sandbox Agent is trying to standardize the boundary *above* the
-  sandbox: one client protocol, session model, and UI/control surface across
-  every coding agent and runtime. If that boundary becomes the ecosystem
-  standard, users and application builders may choose a sandbox provider plus
-  Sandbox Agent rather than a vertically integrated launcher. bot-bottle's
-  manifests would then be valuable chiefly as a local policy/backend
-  implementation unless they expose an equally usable control contract.
-- **Maturity**: Apache 2.0, ~1.5k stars and 426 commits at the 2026-07-27
-  check; HN launch received 41 points.
-
-#### Why the Sandbox Agent architecture is strategically different
-
-The manifest and the universal control protocol solve different layers:
-
-- A bot-bottle manifest is a **trusted launch-time policy composition**. It
-  selects the agent role, isolation backend, image, skills, egress routes,
-  credentials, Git mediation, and supervision policy. Crucially, identity and
-  secret references live on the host side of the trust boundary.
-- Sandbox Agent is a **runtime control and observation protocol**. A remote
-  client creates sessions, sends messages, handles permissions, configures
-  skills/MCP, manipulates files/processes/desktops, and streams normalized
-  events. It deliberately delegates sandbox lifecycle, Git management,
-  storage, network policy, and credential security to other products.
-
-That makes it complementary in a component diagram but competitive in product
-architecture. The layer that becomes the stable integration point tends to own
-the ecosystem. Three plausible threat paths matter:
-
-1. **Standard control plane, interchangeable runtimes.** Applications integrate
-   once with Sandbox Agent and treat E2B, Daytona, BoxLite, Docker, or a future
-   local microVM as replaceable compute. A provider that bundles adequate
-   egress and credential custody makes bot-bottle's end-to-end launcher less
-   necessary.
-2. **Policy grows upward.** Sandbox Agent already configures permissions,
-   skills, MCP, custom tools, filesystem/process access, and computer use. If
-   it adds a declarative, host-verifiable policy document, the overlap with
-   agent/bottle manifests becomes substantial even if enforcement remains
-   delegated.
-3. **UI and session ownership.** Its universal transcript schema, Inspector,
-   React components, event replay, and remote terminal/computer APIs can become
-   the natural basis for desktop, web, and mobile agent managers. bot-bottle's
-   security layer could remain stronger while losing the operator surface and
-   distribution channel.
-
-The counter-position is not to claim that manifests and an API are mutually
-exclusive. The defensible split is:
-
-- bot-bottle owns the trusted policy and enforcement plane outside the agent;
-- a provider-neutral protocol owns agent process control and normalized
-  events; and
-- the operator UI consumes both.
-
-This suggests an explicit compatibility decision rather than parallel,
-accidental protocol design: evaluate running Sandbox Agent inside a bottle and
-exposing it only through the authenticated bot-bottle control plane. If its
-schema is suitable, adopting it could turn a threat into an integration while
-keeping manifests as the higher-trust policy source. If it is unsuitable,
-bot-bottle should still publish a stable provider-neutral session/event API so
-frontends do not depend on Claude/Codex/Pi-specific process behavior.
-
 ## Comparison table
 
 *Isolation/sandbox tools only. AGT and OAP are governance layers — see their per-project notes above.*
@@ -817,70 +616,6 @@ would be a *backend* bot-bottle could call, not a competitor to its
 manifest layer. endo-familiar is in a different paradigm entirely:
 capability passing rather than kernel boundaries.
 
-**Recent entrants change two parts of this read.** yolo-cage is closer to the
-actual threat model than agent-safehouse or litterbox: it combines a VM-style
-boundary with mediated Git and filtered HTTP specifically for parallel coding
-agents. Sandbox Agent SDK is the more important strategic entrant even though
-it supplies no isolation. It can become the standard agent-control layer above
-all of these runtimes, including a future bot-bottle backend. CloudRouter is
-the clearest workflow challenge because its browser/desktop/GPU loop makes
-parallel agents visibly more capable, not merely safer.
-
-## Gap evaluation after the 2026-07-27 entrant scan
-
-### Material gaps
-
-1. **A stable provider-neutral control and event protocol.** This is the
-   largest newly visible gap. bot-bottle normalizes launch/provisioning across
-   providers, but an external UI or orchestrator still lacks one documented
-   contract for creating a Claude/Codex/Pi session, sending input, handling
-   permission/supervision events, streaming normalized output, reconnecting,
-   and replaying history. Sandbox Agent SDK addresses exactly this layer and
-   is already portable across many sandbox providers.
-2. **Browser/preview closure.** CloudRouter and Eve make a browser or desktop
-   part of the standard agent environment and expose screenshots/live viewing
-   to the operator. bot-bottle can run dev servers and supports nested
-   containers, but it does not present a first-class browser/computer-use
-   primitive or an auth-protected preview surface. For coding agents expected
-   to verify UI work, this is a real product gap.
-3. **Unified parallel-session operator UX.** Named persistent bottles and
-   supervision provide the substrate, but the recent products make task
-   switching, live progress, notifications, terminal attach, diffs, and
-   session history the product. Security depth will not compensate for a
-   visibly rougher daily loop.
-4. **Normalized transcript persistence and replay.** bot-bottle preserves
-   provider-specific state for resume; it does not expose a provider-neutral
-   event record suitable for audit, replay, analytics, or a web/mobile client.
-   This is both a UX gap and an audit gap.
-
-### Important, but not necessarily bot-bottle features
-
-- **Cloud VM/GPU provisioning.** Valuable for elastic workloads and could be a
-  backend, but it conflicts with the local-custody default and should not
-  displace core policy work.
-- **Automatic model routing.** Black LLAB and Eve sell task-to-model routing.
-  bot-bottle's provider-template boundary can host that choice without making
-  it part of the trusted sandbox policy.
-- **A thousand SaaS connectors.** This broadens capability and blast radius.
-  The bot-bottle-native answer should remain explicit, scoped forge/egress
-  associations rather than connector count as a goal.
-- **SDK-driven sandbox lifecycle as the primary configuration model.** Useful
-  for platform builders, but not a replacement for reviewable, host-owned
-  manifests. A control API and a declarative policy source are compatible;
-  neither should silently become the other.
-
-### Areas where bot-bottle remains ahead
-
-- real provider and forge credentials remain outside the agent process rather
-  than being extracted into its environment;
-- authorized HTTP payloads are scanned, not merely destination-filtered;
-- Git writes traverse a distinct gate with secret scanning and host-held
-  upstream credentials;
-- role policy is host-owned, composable, and separate from untrusted repo
-  content; and
-- local Firecracker/Apple Container execution preserves operator custody
-  without requiring a hosted sandbox platform.
-
 ## Borrowable ideas
 
 ### Already shipped or otherwise addressed
@@ -907,19 +642,6 @@ parallel agents visibly more capable, not merely safer.
 
 ### Still worth considering
 
-- **Sandbox Agent compatibility or an equivalent stable protocol (highest
-  priority):** spike running its server inside a bottle behind bot-bottle's
-  authenticated control plane. Compare its session/event schema, permission
-  model, restore semantics, and provider coverage with current provider
-  adapters. Adopt compatibility if it preserves the host-owned trust boundary;
-  otherwise specify bot-bottle's own stable API before building another UI.
-- **First-class browser/preview loop** (from CloudRouter and Eve): give a
-  bottle an optional browser/computer-use capability plus an operator-visible,
-  authenticated preview/screenshot surface. Treat its network access as part
-  of the bottle policy, not an implicit bypass.
-- **Provider-neutral transcript/event persistence** (from Sandbox Agent SDK):
-  retain enough normalized structure for replay and audit while preserving the
-  provider-native state needed for exact resume.
 - **Live network activity in the supervisor TUI** (from Docker sbx): show
   allowed and blocked connections and let the operator propose policy changes
   from the existing supervision surface.
@@ -930,11 +652,10 @@ parallel agents visibly more capable, not merely safer.
   closer review. This needs a carefully specified trust model before it can be
   more than a heuristic.
 
-Not worth borrowing: SDK-first *policy configuration* as used by boxlite /
-microsandbox (cuts against the reviewable declarative-manifest stance), and
-the hosted-SaaS custody model of tilde.run (cuts against the "infrastructure I
-control" goal). A provider-neutral runtime-control API is a separate concern
-and is worth borrowing.
+Not worth borrowing: the SDK-first programmatic API style of boxlite /
+microsandbox (cuts against the declarative-manifest stance), and the
+hosted-SaaS dashboard model of tilde.run (cuts against the
+"infrastructure I control" goal).
 
 ## Publishing and positioning verdict
 
@@ -958,15 +679,9 @@ bot-bottle remains unusual in combining:
 The practical wedge is “as easy as native yolo, with declarative role policy
 and self-hosted custody,” including scoped access to private LAN/Tailnet
 services that cloud-first runtimes cannot provide without additional network
-plumbing. The main competitive risks are now:
-
-- a local wrapper such as yolo-cage, claudebox, or Docker sbx growing a
-  role-manifest and credential-custody layer;
-- Sandbox Agent SDK becoming the standard control/session boundary and making
-  the runtime beneath it interchangeable; and
-- GUI products such as SuperHQ or CloudRouter adding equivalent policy and
-  audit depth before bot-bottle closes the browser/preview and
-  parallel-session UX gaps.
+plumbing. The main competitive risks are a local wrapper such as claudebox or
+Docker sbx growing a role-manifest layer, and GUI products such as SuperHQ
+adding equivalent policy and audit depth.
 
 ## Caveats
 

docs/research/testing-clean-install-on-macos.md

@@ -1,258 +0,0 @@
-# Testing a clean bot-bottle install on macOS
-
-How do you exercise `install.sh` (and, ideally, a first `bot-bottle start`)
-the way a brand-new user would — on a pristine macOS environment you can
-throw away afterward — *without* permanently polluting your daily-driver
-Mac? The user's framing: is there a VM or boundary that avoids creating a
-separate account, or is spinning up and tearing down a throwaway macOS
-user on the CLI easy enough to just do that?
-
-## Summary
-
-There is no lightweight, in-place macOS sandbox that hands you a clean home
-directory and wipeable system state without *either* a VM or a separate
-user account. `sandbox-exec` (Seatbelt) is deprecated and confines a
-process, not an environment; App Sandbox is for shipping apps, not for
-provisioning a fresh dev host. So the real choice is exactly the two the
-user named: **a disposable macOS VM** or **a throwaway user account** —
-and which one is right turns on a detail specific to *this* project.
-
-bot-bottle's default macOS backend is Apple's `container`, which runs each
-container in its own lightweight VM via `Virtualization.framework`
-([`README.md:27`](../README.md), [`apple-container-backend.md`](apple-container-backend.md)).
-That means a full end-to-end test — install *and* `bot-bottle start` —
-needs virtualization to work wherever bot-bottle runs. Inside a macOS guest
-VM that requires **nested virtualization, which Apple gates to M3 or newer
-chips on macOS 15+**. On M1/M2 you cannot run the Apple Container backend
-(or Docker Desktop, same reason) inside a macOS VM at all.
-
-The recommendation splits on what you're testing and what silicon you have:
-
-- **Install-script correctness only** (does `curl | sh` → pipx → config dir
-  → `doctor`'s Python/config checks pass?): a **disposable Tart VM** is the
-  cleanest boundary and works on any Apple Silicon Mac. `doctor` will report
-  the backend as not-ready inside the VM on M1/M2, which is fine — you're
-  testing the installer, not the runtime.
-- **Full runtime** (actually launch a bottle) on **M3/M4**: a **disposable
-  Tart VM from a golden base image, cloned per run** is the gold standard —
-  a genuine kernel/state boundary that wipes to nothing.
-- **Full runtime** on **M1/M2**, or when you'd rather not fight nested virt:
-  a **throwaway admin user via `sysadminctl`** is the pragmatic pick. It
-  tests the real backend because the backend runs on the host hypervisor —
-  but it is a *hygiene* boundary, not a security one, and it does **not**
-  clean the system-level footprint (see below).
-
-Prefer the VM. Reach for the throwaway user only when nested virt is off the
-table and you accept an imperfect wipe.
-
-## Why "a boundary without a separate user" doesn't really exist on macOS
-
-macOS has no namespace/overlay story like Linux `unshare` + tmpfs. The
-options that sound like in-place sandboxes don't fit:
-
-| Mechanism | Why it doesn't give you a clean, wipeable env |
-|---|---|
-| `sandbox-exec` / Seatbelt | Officially deprecated; confines *one process's* syscalls against a profile. It cannot present a fresh `$HOME` or a pristine `/usr/local`, and it won't let the Apple Container system service work. |
-| App Sandbox | Entitlement-based confinement for signed `.app` bundles, not a provisioning tool for a CLI dev environment. |
-| A second `$HOME` via `HOME=/tmp/foo` | Redirects only what honors `$HOME`. `install.sh` mostly does (it writes `~/.bot-bottle` and pipx/pip `--user` paths), but the Apple `container` install lands in `/usr/local` + a **system service**, and Homebrew lands in `/opt/homebrew` — all outside any `$HOME` you set. You'd get a false sense of "clean." |
-| APFS snapshot rollback (`tmutil localsnapshot`) | You can't roll the live boot volume back to a local snapshot without booting to Recovery; it's not a per-run userspace undo. |
-
-So the honest answer to "is there some boundary that avoids a separate
-user?": yes — a **VM** — and it's the *stronger* boundary anyway. The only
-lighter-weight option is the separate user, with the caveats below.
-
-## What a clean install actually touches (the footprint that decides "wipeable")
-
-Grounding the teardown story in what `install.sh` and the backend create:
-
-| Artifact | Location | In `$HOME`? | Survives user deletion? |
-|---|---|---|---|
-| Config / state / db | `~/.bot-bottle/{agents,bottles,contrib,state,db}` ([`install.sh:80-83`](../install.sh), [`bot_bottle/paths.py:59`](../bot_bottle/paths.py)) | ✅ | ❌ removed with home |
-| pipx venv + shim | `~/.local/pipx/venvs/bot-bottle`, shim in `~/.local/bin` ([`install.sh:87-89`](../install.sh)) | ✅ | ❌ removed with home |
-| private venv fallback (no pipx) | `~/.bot-bottle/venv` + symlink in `~/.local/bin` ([`install.sh`](../install.sh)) | ✅ | ❌ removed with home |
-| PATH / token exports | shell profile (`~/.zprofile`, etc.); `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN` ([`README.md:74`](../README.md)) | ✅ | ❌ removed with home |
-| **Apple `container` install** | `/usr/local/...` + notarized `.pkg` receipts | ❌ | ✅ **stays** |
-| **Apple `container` system service** | launchd system service (`container system start`) | ❌ | ✅ **stays** |
-| **Homebrew** (if used for `container`/python) | `/opt/homebrew` | ❌ | ✅ **stays** |
-| Rosetta 2 (needed for image builds) | system | ❌ | ✅ **stays** |
-
-The three bold rows are the crux: **deleting the throwaway user does not
-uninstall the Apple Container runtime, its system service, Homebrew, or
-Rosetta.** A VM, by contrast, wipes 100% of the above by definition —
-that's its entire advantage for this task.
-
-## Option A — Disposable Tart VM (recommended)
-
-[Tart](https://tart.run) is a CLI-first macOS/Linux VM manager built on
-`Virtualization.framework`, purpose-built for exactly this "does it work on
-a clean macOS, without my settings/permissions/data" workflow. Keep one
-pristine *golden* image, clone a throwaway per run, delete it after.
-
-```sh
-brew install cirruslabs/cli/tart
-
-# One-time: build a golden base (either a prebuilt image or a vanilla IPSW).
-tart clone ghcr.io/cirruslabs/macos-tahoe-base:latest golden   # ~25 GB pull
-# — or a truly vanilla install you click through once —
-# tart create golden --from-ipsw latest --disk-size 60
-
-# Per test run: clone → boot → test → destroy.
-tart clone golden test-run
-tart run test-run &
-ssh admin@"$(tart ip test-run)"
-#   inside the guest:
-#   curl -fsSL https://gitea.dideric.is/didericis/bot-bottle/raw/branch/main/install.sh | sh
-#   bot-bottle doctor
-tart stop test-run
-tart delete test-run          # back to pristine; golden is untouched
-```
-
-Cloning is cheap (sparse files), so the golden image is your reset button —
-every `tart clone` is a fresh macOS. This is the closest thing to a Linux
-`docker run --rm` for a whole Mac.
-
-**The nested-virt caveat (read before relying on it for runtime tests).**
-The Apple Container backend inside the guest needs
-`Virtualization.framework` to work *inside* the VM. Apple enables nested
-virtualization only on **M3 or newer**, on **macOS 15 (Sequoia) or later**;
-M2 and earlier are excluded by Apple, confirmed by Apple DTS. Consequences:
-
-- **M3/M4 host:** full runtime works in the guest. `bot-bottle doctor`
-  reports the backend ready and `start` can launch a bottle. Gold standard.
-- **M1/M2 host:** the guest can install bot-bottle and pass the Python /
-  config-dir checks, but `doctor`'s backend check will fail and you cannot
-  launch a bottle in the VM. Still perfectly good for testing *the
-  installer*; not for the runtime.
-- **M4-specific:** a known bug blocks pre-Ventura guests on M4; use a
-  current macOS guest (which you want anyway, since Apple `container`
-  targets macOS 26 Tahoe).
-
-UTM is the GUI equivalent on the same framework (and was first to expose
-nested virt) if you'd rather click; Tart wins for a scriptable
-spin-up/tear-down loop.
-
-## Option B — Throwaway user via `sysadminctl` (pragmatic fallback)
-
-Creating and deleting a user from the CLI is genuinely a two-liner, and it
-tests the **real** backend on any Apple Silicon Mac because the backend runs
-on the host hypervisor — no nested virt needed.
-
-```sh
-# Create a self-contained admin user (admin needed for the container service).
-sudo sysadminctl -addUser bbtest -fullName "bot-bottle test" \
-    -password 'throwaway' -admin
-
-# Log into that account (fast-user-switch or the login window), then run the
-# installer as bbtest exactly as a new user would. When done:
-
-sudo sysadminctl -deleteUser bbtest -secure   # -secure erases the home dir
-```
-
-Honest accounting of what this does and doesn't buy you:
-
-- **Boundary strength:** it's a *hygiene / fresh-`$HOME`* boundary, **not a
-  security boundary.** Same kernel, same admin group; an admin test user can
-  touch system state. If the point is "clean environment," fine. If the point
-  is "contain something untrusted," this is the wrong tool — use a VM.
-- **Wipe completeness:** `-secure` erases the home dir (so `~/.bot-bottle`,
-  the pipx venv, and profile exports go away), but as the footprint table
-  shows, the **Apple Container runtime, its launchd system service,
-  Homebrew, and Rosetta persist.** For a truly repeatable "did a *system with
-  nothing installed* work?" test, that residue defeats the purpose — the
-  second run isn't clean.
-- **Operational gotchas:** don't pass real passwords on the command line (they
-  land in `ps` and history — this is a throwaway credential, so it's
-  tolerable here). Deletion must run as root from a normally-booted, admin-
-  logged-in session; the Terminal needs **Full Disk Access** or you'll hit
-  error `-14120` and a half-deleted account. Prefer letting the system place
-  the home dir (don't pass `-home`), or deletion can orphan it.
-
-Use this when you're on M1/M2, you specifically want to exercise the live
-backend, and you can tolerate the system-level runtime staying installed
-between runs (or you uninstall Apple `container` / brew by hand to reset).
-
-## Honorable mentions
-
-- **External bootable macOS volume.** A fresh macOS on an external SSD (or a
-  separate APFS volume) is bare-metal disposable: no nested-virt limit, real
-  backend works, and you `diskutil` the volume away to reset. Cost is reboot
-  friction per run — good for an occasional thorough pass, poor for a tight
-  loop.
-- **Rented / cloud Mac.** AWS EC2 Mac (dedicated Mac minis), Scaleway Apple
-  silicon, or MacStadium give a genuinely throwaway host you release when
-  done. Overkill for local iteration, but this is essentially what the
-  project's own advisory `integration-macos` CI job needs — a self-hosted
-  Apple Silicon runner with the `container` CLI, Python ≥ 3.11, and coverage
-  on the launchd service's PATH ([`README.md:78`](../README.md)). If you end
-  up standing up a cloud Mac for install testing, it doubles as that runner.
-
-## Recommendation
-
-Default to a **disposable Tart VM** — it's the only option that wipes the
-*entire* footprint (including the Apple Container system service that a user
-deletion leaves behind), it's a real boundary, and the spin-up/tear-down
-loop is a two-command `tart clone` / `tart delete`. Confirm your chip first:
-on **M3/M4** it tests install *and* runtime end-to-end; on **M1/M2** it still
-cleanly tests `install.sh` + `doctor`'s Python/config path, and you fall back
-to a **throwaway `sysadminctl` admin user** for live-backend testing —
-accepting that it's a hygiene boundary and that you'll manually uninstall the
-Apple Container runtime / Homebrew between runs to get back to truly clean.
-
-There is no third, lighter-weight "in-place boundary without a user" that
-actually delivers a clean, wipeable macOS — the VM *is* that answer, and it's
-the better one.
-
-## Harness
-
-The throwaway-user loop is scripted in
-[`scripts/macos-install-test.sh`](../../scripts/macos-install-test.sh):
-`up` creates the account, `run` pipes *this checkout's* `install.sh` into it
-headlessly (so a PR is verifiable before it lands) and lets the installer run
-`doctor`, `down` deletes the account and its home (the full reset), and
-`deep-reset` additionally uninstalls the host `container` runtime. It leans on
-the footprint analysis above — the reset is just user deletion because
-everything `install.sh` writes is user-home-local.
-
-`test` chains `up → run → status → down` into the one-shot cycle you normally
-want:
-
-```sh
-sudo ./scripts/macos-install-test.sh test
-```
-
-It refuses to start against an existing account (a reused home is not a clean
-install), and it tears the account down from an `EXIT`/`INT` trap armed the
-moment the account exists, so a failed or Ctrl-C'd run still leaves the machine
-clean. Its verdict is deliberately stricter than the installer's own: note that
-`install.sh` exits **0** when it finishes but `doctor` reports unmet
-prerequisites, so "the installer succeeded" is not the assertion — `test` fails
-if the install fails, if `bot-bottle` never reached the new user's `PATH`, or if
-`doctor` is unhappy. `BB_TEST_KEEP=1` skips the teardown to poke at a failure.
-
-### What a fresh account actually inherits
-
-Expect the first honest run on a developer Mac to fail at the *Python* gate,
-and expect that to be correct. A new account's `PATH` is just `/etc/paths`
-(`/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin`),
-which notably does **not** include `/opt/homebrew/bin`. Homebrew's `shellenv`
-line lives in the *installing* user's `~/.zprofile` and is not inherited, so a
-throwaway user resolves `python3` to `/usr/bin/python3` — the Command Line
-Tools stub, still **3.9.6** on macOS 26 — and `install.sh` correctly dies on its
-`3.11+` requirement. Your own shell resolving `python3` to a 3.14 Homebrew
-build says nothing about what a new user sees; that gap is exactly what this
-harness exists to expose.
-
-## Sources
-
-- [Apple Containers on macOS: technical comparison with Docker — The New Stack](https://thenewstack.io/apple-containers-on-macos-a-technical-comparison-with-docker/)
-- [How to Set Up Apple Containerization on macOS 26 — Stéphane Paquet](https://spaquet.medium.com/how-to-set-up-apple-containerization-on-macos-26-f870cc8c26cd)
-- [Install Apple Container CLI (macOS 15/26) — 4sysops](https://4sysops.com/archives/install-apple-container-cli-running-containers-natively-on-macos-15-sequoia-and-macos-26-tahoe/)
-- [Nested virtualization on Apple Silicon (M3+, macOS 15) — UTM issue #6700](https://github.com/utmapp/UTM/issues/6700)
-- [macOS 15 Sequoia nested virtualization for M3+ — Parallels Forums](https://forum.parallels.com/threads/macos-15-sequoia-nested-virtualization-for-m3-macs.364397/)
-- [M2 nested virtualization restriction (Apple DTS) — Apple Developer Forums](https://developer.apple.com/forums/thread/756723)
-- [M4 can't virtualize older macOS — Yahoo/Tech](https://tech.yahoo.com/computing/articles/m4-mac-computers-cant-virtualize-175122301.html)
-- [Tart — macOS/Linux VMs on Apple Silicon (Cirrus Labs)](https://tart.run/quick-start/)
-- [Tart GitHub](https://github.com/cirruslabs/tart)
-- [macOS VMs in a single command — frr.dev](https://www.frr.dev/posts/tart-macos-vms-from-terminal/)
-- [sysadminctl reference — SS64](https://ss64.com/mac/sysadminctl.html)
-- [User management from the macOS command line — macnotes](https://macnotes.wordpress.com/2019/03/28/user-management-create-remove-change-password-secure-token-from-macos-command-line/)

install.sh

@@ -8,20 +8,14 @@
 #   pipx install bot-bottle        # from a checkout or a published index
 #   uv tool install bot-bottle
 #
-# This script is a thin bootstrapper: it finds a Python 3.11+ interpreter,
-# installs the package with pipx (falling back to a private venv), creates the
-# config dir, and runs `bot-bottle doctor`. It is idempotent (safe to re-run)
-# and never uses sudo. It does NOT install Docker or a VM backend for you —
-# `doctor` reports what's missing after install.
-#
-# Env:
-#   BOT_BOTTLE_PYTHON        interpreter to install with (skips the search)
-#   BOT_BOTTLE_INSTALL_SPEC  pip/git spec to install instead of the default
-#   BOT_BOTTLE_VENV          where the non-pipx install lives (~/.bot-bottle/venv)
+# This script is a thin bootstrapper: it checks prerequisites, installs the
+# package with pipx (falling back to pip --user), creates the config dir, and
+# runs `bot-bottle doctor`. It is idempotent (safe to re-run) and never uses
+# sudo. It does NOT install Docker or a VM backend for you — `doctor` reports
+# what's missing after install.
 set -eu
 
 PACKAGE_SPEC="${BOT_BOTTLE_INSTALL_SPEC:-git+https://gitea.dideric.is/didericis/bot-bottle.git}"
-VENV="${BOT_BOTTLE_VENV:-${HOME}/.bot-bottle/venv}"
 MIN_PYTHON_MAJOR=3
 MIN_PYTHON_MINOR=11
 
@@ -34,97 +28,17 @@ die() {
     exit 1
 }
 
-# --- prerequisites: find an interpreter new enough ----------------------------
+# --- prerequisites -----------------------------------------------------------
 
-# Is $1 an interpreter that exists and meets the floor?
-python_ok() {
-    [ -n "${1:-}" ] || return 1
-    command -v "$1" >/dev/null 2>&1 || return 1
-    "$1" - "$MIN_PYTHON_MAJOR" "$MIN_PYTHON_MINOR" <<'PY' >/dev/null 2>&1
+command -v python3 >/dev/null 2>&1 \
+    || die "python3 ${MIN_PYTHON_MAJOR}.${MIN_PYTHON_MINOR}+ is required but was not found"
+
+python3 - "$MIN_PYTHON_MAJOR" "$MIN_PYTHON_MINOR" <<'PY' || die "python3 ${MIN_PYTHON_MAJOR}.${MIN_PYTHON_MINOR} or newer is required"
 import sys
 
 want = (int(sys.argv[1]), int(sys.argv[2]))
 raise SystemExit(0 if sys.version_info[:2] >= want else 1)
 PY
-}
-
-python_version() {
-    "$1" -c 'import sys; print("%d.%d.%d" % sys.version_info[:3])' 2>/dev/null
-}
-
-# `python3` on PATH is often NOT the newest interpreter installed, and on macOS
-# it is usually the oldest: a fresh login shell's PATH is just /etc/paths, so
-# python3 resolves to the Command Line Tools stub (3.9.x) while the usable
-# 3.11+ build sits in /opt/homebrew/bin or a python.org framework directory,
-# reachable only via a line in the *installing* user's shell profile. A new
-# account inherits none of that. Look past PATH before giving up, so the common
-# case installs instead of dead-ending on a version error.
-find_python() {
-    for candidate in \
-        "${BOT_BOTTLE_PYTHON:-}" \
-        python3 \
-        python3.14 python3.13 python3.12 python3.11 \
-        /opt/homebrew/bin/python3 \
-        /usr/local/bin/python3 \
-        "${HOME}/.local/bin/python3" \
-        /Library/Frameworks/Python.framework/Versions/*/bin/python3
-    do
-        # An unmatched glob arrives here literally; python_ok rejects it.
-        if python_ok "$candidate"; then
-            command -v "$candidate"
-            return 0
-        fi
-    done
-    return 1
-}
-
-# An explicit choice that doesn't work is an error, not a reason to quietly
-# search elsewhere and install somewhere the caller didn't ask for.
-if [ -n "${BOT_BOTTLE_PYTHON:-}" ] && ! python_ok "${BOT_BOTTLE_PYTHON}"; then
-    if command -v "${BOT_BOTTLE_PYTHON}" >/dev/null 2>&1; then
-        die "BOT_BOTTLE_PYTHON=${BOT_BOTTLE_PYTHON} is $(python_version "${BOT_BOTTLE_PYTHON}"), "\
-"below the ${MIN_PYTHON_MAJOR}.${MIN_PYTHON_MINOR} floor. Unset it to search for a newer one."
-    fi
-    die "BOT_BOTTLE_PYTHON=${BOT_BOTTLE_PYTHON} is not an executable interpreter."
-fi
-
-PYTHON="$(find_python || true)"
-
-if [ -z "${PYTHON}" ]; then
-    path_python="$(command -v python3 2>/dev/null || true)"
-    if [ -n "${path_python}" ]; then
-        found="the python3 on your PATH is ${path_python} ($(python_version "${path_python}")), which is too old"
-    else
-        found="no python3 was found on your PATH"
-    fi
-    case "$(uname -s)" in
-        Darwin) fix="  brew install python@3.12
-  # or install from https://www.python.org/downloads/macos/
-  # macOS itself ships only /usr/bin/python3, which is too old" ;;
-        *) fix="  sudo apt install python3.12    # Debian/Ubuntu
-  sudo dnf install python3.12    # Fedora/RHEL" ;;
-    esac
-    die "bot-bottle needs python3 ${MIN_PYTHON_MAJOR}.${MIN_PYTHON_MINOR} or newer, and none was found.
-  ${found}.
-  Also checked: python3.11-3.14, /opt/homebrew/bin, /usr/local/bin,
-  ~/.local/bin, and python.org framework builds.
-
-Install a newer Python, then re-run this installer:
-${fix}
-
-Already have one somewhere? Point at it directly:
-  BOT_BOTTLE_PYTHON=/path/to/python3 sh install.sh"
-fi
-
-# Be explicit when the interpreter isn't the obvious one, so nobody is left
-# wondering which Python their install ended up on.
-path_python="$(command -v python3 2>/dev/null || true)"
-if [ "${PYTHON}" != "${path_python}" ]; then
-    say "using ${PYTHON} ($(python_version "${PYTHON}"))"
-    if [ -n "${path_python}" ]; then
-        say "note: 'python3' on your PATH is ${path_python} ($(python_version "${path_python}")), which is below the ${MIN_PYTHON_MAJOR}.${MIN_PYTHON_MINOR} floor"
-    fi
-fi
 
 # Installing a `git+` spec (the default) shells out to git under the hood,
 # whether via pipx or pip. Fail early with a clear message rather than deep
@@ -137,6 +51,30 @@ case "${PACKAGE_SPEC}" in
         ;;
 esac
 
+# The pip fallback needs a usable pip. Externally-managed interpreters
+# (PEP 668, common on Debian/Ubuntu/Homebrew) reject `pip install --user`;
+# pipx sidesteps that, so recommend it when pip can't be used.
+if ! command -v pipx >/dev/null 2>&1; then
+    python3 -m pip --version >/dev/null 2>&1 || die \
+        "neither pipx nor a usable 'python3 -m pip' was found. Install pipx "\
+"(recommended): 'python3 -m pip install --user pipx' or your OS package manager."
+    if python3 - <<'PY'
+import os
+import sys
+import sysconfig
+
+# PEP 668: an EXTERNALLY-MANAGED marker in the stdlib dir means pip refuses
+# to install into this interpreter without --break-system-packages.
+marker = os.path.join(sysconfig.get_path("stdlib"), "EXTERNALLY-MANAGED")
+raise SystemExit(0 if os.path.exists(marker) else 1)
+PY
+    then
+        die "this Python is externally managed (PEP 668), so 'pip install --user' is "\
+"blocked. Install pipx and re-run: 'python3 -m pip install --user --break-system-packages pipx', "\
+"then 'pipx ensurepath'."
+    fi
+fi
+
 # --- config directories ------------------------------------------------------
 
 mkdir -p \
@@ -146,50 +84,32 @@ mkdir -p \
 
 # --- install -----------------------------------------------------------------
 
-BIN_DIR="${HOME}/.local/bin"
-
 if command -v pipx >/dev/null 2>&1; then
-    # --python pins the venv to the interpreter we vetted. Without it pipx uses
-    # whichever Python it was itself installed with, which is not necessarily
-    # the one that passed the version check above.
-    say "installing with pipx (python: ${PYTHON})"
-    pipx install --python "${PYTHON}" --force "${PACKAGE_SPEC}"
-    # Ask pipx where it puts entry points rather than assuming ~/.local/bin.
-    pipx_bin="$(pipx environment --value PIPX_BIN_DIR 2>/dev/null || true)"
-    [ -n "${pipx_bin}" ] && BIN_DIR="${pipx_bin}"
+    say "installing with pipx"
+    pipx install --force "${PACKAGE_SPEC}"
 else
-    # No `pip install --user` fallback: PEP 668 makes it unusable on nearly
-    # every interpreter a Mac offers (Homebrew and python.org are both
-    # externally managed), and on Debian/Ubuntu too. A private venv sidesteps
-    # that entirely — PEP 668 does not apply inside a venv — and `venv` is
-    # stdlib, so unlike pipx there is nothing to bootstrap first.
-    say "pipx not found; installing into a managed venv at ${VENV}"
-    "${PYTHON}" -m venv --clear "${VENV}" || die \
-        "could not create a virtualenv at ${VENV} using ${PYTHON}. On Debian/Ubuntu "\
-"the venv module ships separately: 'sudo apt install python3-venv'."
-    "${VENV}/bin/python" -m pip install --upgrade "${PACKAGE_SPEC}"
-    # Expose the entry point outside the venv, the way pipx would.
-    mkdir -p "${BIN_DIR}"
-    ln -sf "${VENV}/bin/bot-bottle" "${BIN_DIR}/bot-bottle"
+    say "pipx not found; installing with 'python3 -m pip install --user'"
+    python3 -m pip install --user --upgrade "${PACKAGE_SPEC}"
 fi
 
 # --- locate the entry point --------------------------------------------------
 
+# The pip --user scripts directory is platform-specific: ~/.local/bin on Linux,
+# but ~/Library/Python/<X.Y>/bin on a python.org macOS interpreter. Ask the
+# interpreter for its own user-scheme scripts dir instead of hardcoding.
+USER_SCRIPTS="$(python3 - <<'PY'
+import sysconfig
+print(sysconfig.get_path("scripts", sysconfig.get_preferred_scheme("user")))
+PY
+)"
+
 if command -v bot-bottle >/dev/null 2>&1; then
     BOT_BOTTLE_BIN="bot-bottle"
-elif [ -x "${BIN_DIR}/bot-bottle" ]; then
-    BOT_BOTTLE_BIN="${BIN_DIR}/bot-bottle"
-    # Name the file the user's own login shell actually reads. ~/.profile is
-    # the safe default for non-zsh: bash falls back to it, and suggesting
-    # ~/.bash_profile could shadow an existing ~/.profile.
-    case "${SHELL:-}" in
-        */zsh) profile="~/.zprofile" ;;
-        *) profile="~/.profile" ;;
-    esac
-    say "note: add ${BIN_DIR} to your PATH to run 'bot-bottle' directly:"
-    say "    echo 'export PATH=\"${BIN_DIR}:\$PATH\"' >> ${profile}"
+elif [ -n "${USER_SCRIPTS}" ] && [ -x "${USER_SCRIPTS}/bot-bottle" ]; then
+    BOT_BOTTLE_BIN="${USER_SCRIPTS}/bot-bottle"
+    say "note: add ${USER_SCRIPTS} to your PATH to run 'bot-bottle' directly"
 else
-    die "bot-bottle was installed but no entry point turned up in ${BIN_DIR}"
+    die "bot-bottle was installed but is not on PATH; add ${USER_SCRIPTS:-your user scripts dir} to PATH and re-run"
 fi
 
 # --- verify ------------------------------------------------------------------

scripts/coverage.sh

@@ -13,7 +13,7 @@
 #     are re-executed; no KVM or Docker dependency.
 #
 # Pass "critical" as the last argument in either mode to also report just the
-# critical modules (ADR 0004 target: 85%).
+# critical modules (ADR 0004 target: 90%).
 set -euo pipefail
 
 cd "$(dirname "$0")/.."
@@ -34,8 +34,8 @@ if [ "${1:-}" = "aggregate" ]; then
     "$PY" -m coverage report -m
 
     if [ "${2:-}" = "critical" ]; then
-        echo "== critical modules (ADR 0004 minimum: 85%) ==" >&2
-        "$PY" -m coverage report --include="$CRITICAL" --fail-under=85
+        echo "== critical modules (ADR 0004 minimum: 90%) ==" >&2
+        "$PY" -m coverage report --include="$CRITICAL" --fail-under=90
     fi
     exit 0
 fi
@@ -55,6 +55,6 @@ echo "== combined report ==" >&2
 "$PY" -m coverage report -m
 
 if [ "${1:-}" = "critical" ]; then
-    echo "== critical modules (ADR 0004 minimum: 85%) ==" >&2
-    "$PY" -m coverage report --include="$CRITICAL" --fail-under=85
+    echo "== critical modules (ADR 0004 minimum: 90%) ==" >&2
+    "$PY" -m coverage report --include="$CRITICAL" --fail-under=90
 fi

scripts/critical-modules.txt

@@ -1,4 +1,4 @@
-# Critical security/logic core held to the >=85% coverage bar by
+# Critical security/logic core held to the >=90% coverage bar by
 # docs/decisions/0004-coverage-policy.md.
 #
 # SINGLE SOURCE OF TRUTH: scripts/coverage.sh (the `critical` report) and

scripts/diff_coverage.py

@@ -13,8 +13,8 @@ policy.
 
 Usage:
     scripts/coverage.sh                 # produce .coverage first
-    python3 scripts/diff_coverage.py    # gate against origin/main, min 80%
-    python3 scripts/diff_coverage.py --base main --min 75
+    python3 scripts/diff_coverage.py    # gate against origin/main, min 90%
+    python3 scripts/diff_coverage.py --base main --min 85
 """
 
 from __future__ import annotations
@@ -74,7 +74,7 @@ def main() -> int:
     ap = argparse.ArgumentParser()
     ap.add_argument("--base", default="origin/main",
                     help="git ref to diff against (default: origin/main)")
-    ap.add_argument("--min", type=float, default=80.0,
+    ap.add_argument("--min", type=float, default=90.0,
                     help="minimum %% of changed executable lines covered")
     args = ap.parse_args()
 

scripts/macos-install-test.sh

@@ -1,280 +0,0 @@
-#!/usr/bin/env bash
-# Clean-install test harness for the macOS (Apple `container`) path.
-#
-# Exercises install.sh the way a brand-new user would, inside a throwaway
-# macOS account you create and delete from the CLI. install.sh's entire
-# footprint is user-home-local — the pipx venv under ~/.local, or the private
-# venv at ~/.bot-bottle/venv plus a ~/.local/bin symlink, and the ~/.bot-bottle
-# config dir. It writes no shell-profile PATH line, and never installs the
-# backend (see
-# the header of install.sh), so deleting the user is a complete,
-# deterministic reset of everything the installer touched. The Apple
-# `container` runtime is a HOST prerequisite installed once and kept;
-# `deep-reset` is the rare escape hatch that also removes it.
-#
-# Why a throwaway user and not a disposable VM: bot-bottle's default macOS
-# backend is Apple `container`, which runs each container in its own
-# Virtualization.framework microVM. Running that backend inside a macOS
-# guest VM needs nested virtualization, which Apple gates to M3+ silicon.
-# On M1/M2 a separate user account is the only way to get a clean $HOME
-# while still reaching the real host backend. Full rationale in
-# docs/research/testing-clean-install-on-macos.md.
-#
-# Usage:
-#   sudo ./scripts/macos-install-test.sh test        # up -> run -> status -> down
-#   sudo ./scripts/macos-install-test.sh up          # create the throwaway user
-#   sudo ./scripts/macos-install-test.sh run         # run install.sh (+doctor) as it
-#        ./scripts/macos-install-test.sh status      # user present? backend ready?
-#   sudo ./scripts/macos-install-test.sh down        # delete user + home (the reset)
-#   sudo ./scripts/macos-install-test.sh deep-reset  # ALSO uninstall host `container`
-#
-# `test` is the one-shot clean cycle and the command you normally want: it
-# refuses to start if the account already exists (a reused home is not a clean
-# install), and it tears the account down on the way out however it exits, so
-# a failed run never leaves an orphan behind. It exits non-zero if the install
-# fails, if `bot-bottle` is missing from the new user's PATH, or if `doctor`
-# reports unmet prerequisites — note install.sh itself exits 0 in that last
-# case, so `test` is a stricter gate than running the installer by hand.
-#
-# Config via env:
-#   BB_TEST_USER         account short name             (default: bbtest)
-#   BB_TEST_FULLNAME     account full name              (default: "bot-bottle install test")
-#   BB_TEST_ADMIN        1=admin (reach container svc), 0=standard  (default: 1)
-#   BB_TEST_INSTALL_URL  curl this install.sh instead of piping the local checkout
-#   BB_TEST_KEEP         1=`test` skips its teardown, to poke at a failure
-#   BOT_BOTTLE_INSTALL_SPEC   passed through to install.sh (pip / git spec)
-#
-# Notes:
-#   * Run from a normally-booted admin session. Grant Terminal *Full Disk
-#     Access* (System Settings -> Privacy & Security) or `down` half-fails
-#     with error -14120 and leaves an orphaned account.
-#   * `sysadminctl` always exits 0 even on failure, so `up`/`down` verify
-#     the result with `dscl` and fail loudly on a mismatch.
-#   * The account is created without a password: `run` drives it headlessly
-#     via `sudo -u`, which never needs the target's password. The account
-#     cannot GUI-login, which this harness does not require.
-#   * `run` covers the installer + `bot-bottle doctor`. Actually launching a
-#     bottle from the throwaway user may need a full launchd user session
-#     (`launchctl asuser`); on M1/M2 the backend can't run under nested virt
-#     anyway, so this harness stops at install + doctor.
-
-set -euo pipefail
-
-USER_NAME="${BB_TEST_USER:-bbtest}"
-FULL_NAME="${BB_TEST_FULLNAME:-bot-bottle install test}"
-ADMIN="${BB_TEST_ADMIN:-1}"
-
-_SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-_REPO_ROOT="$(cd "$_SCRIPT_DIR/.." && pwd)"
-
-# Set by `test`, which chains the steps itself and so suppresses the
-# "here's the next command to run" hints the individual steps print.
-IN_TEST=0
-
-# --- guards ----------------------------------------------------------
-require_macos() {
-    [ "$(uname -s)" = "Darwin" ] \
-        || { echo "error: this harness is macOS-only (uname is $(uname -s))" >&2; exit 1; }
-}
-
-require_root() {
-    if [ "$(id -u)" -ne 0 ]; then
-        echo "error: '$1' needs root; re-run under sudo" >&2
-        exit 1
-    fi
-}
-
-user_exists() { dscl . -read "/Users/$USER_NAME" >/dev/null 2>&1; }
-
-# Run a shell snippet as the throwaway user in a fresh login shell.
-run_as_user() { sudo -u "$USER_NAME" -i sh -c "$1"; }
-
-# `bot-bottle doctor` as the throwaway user. Non-zero when the entry point
-# never made it onto that user's PATH, or when doctor itself is unhappy.
-doctor_as_user() {
-    if ! run_as_user 'command -v bot-bottle >/dev/null 2>&1'; then
-        echo "  bot-bottle is not on PATH for $USER_NAME"
-        return 1
-    fi
-    run_as_user 'bot-bottle doctor'
-}
-
-# --- commands --------------------------------------------------------
-cmd_up() {
-    require_macos
-    require_root up
-    if user_exists; then
-        echo "$USER_NAME already exists; nothing to do (run 'down' first to reset)"
-        return 0
-    fi
-    local admin_flag=()
-    [ "$ADMIN" = "1" ] && admin_flag=(-admin)
-    # No -password: the account is only ever driven headlessly via `sudo -u`,
-    # which doesn't need one. sysadminctl warns about FileVault here; that's
-    # irrelevant to a headless test account.
-    sysadminctl -addUser "$USER_NAME" -fullName "$FULL_NAME" "${admin_flag[@]}" || true
-    # sysadminctl exits 0 regardless of outcome, so confirm the account landed.
-    user_exists || { echo "error: failed to create $USER_NAME" >&2; return 1; }
-    if [ "$IN_TEST" = 1 ]; then
-        echo "created $USER_NAME (admin=$ADMIN)"
-    else
-        echo "created $USER_NAME (admin=$ADMIN). Install into it with: sudo $0 run"
-    fi
-}
-
-cmd_run() {
-    require_macos
-    require_root run
-    user_exists || { echo "error: $USER_NAME does not exist; run 'sudo $0 up' first" >&2; return 1; }
-
-    local spec_env=""
-    [ -n "${BOT_BOTTLE_INSTALL_SPEC:-}" ] \
-        && spec_env="BOT_BOTTLE_INSTALL_SPEC='$BOT_BOTTLE_INSTALL_SPEC' "
-
-    echo "== installing bot-bottle as $USER_NAME =="
-    if [ -n "${BB_TEST_INSTALL_URL:-}" ]; then
-        run_as_user "curl -fsSL '$BB_TEST_INSTALL_URL' | ${spec_env}sh"
-    else
-        # Test THIS checkout's install.sh, not the published one, so a PR is
-        # verifiable before it lands. Feed it in on stdin rather than staging a
-        # copy somewhere the throwaway user can read: the redirect is opened by
-        # root before sudo drops privileges, so the tester's mode-700 home is a
-        # non-issue, there's no temp file to leak if the run is interrupted, and
-        # `sh -s` is the same shape as the documented `curl … | sh` install.
-        run_as_user "${spec_env}sh -s" < "$_REPO_ROOT/install.sh"
-    fi
-    [ "$IN_TEST" = 1 ] \
-        || echo "== install.sh runs 'doctor' itself; re-check anytime with: $0 status =="
-}
-
-# Informational, with one teeth-bearing case: when it can actually reach
-# doctor (root, account present) its exit status is doctor's, so `test` and
-# any other caller can use it as the post-install assertion.
-cmd_status() {
-    require_macos
-    local rc=0
-    if user_exists; then
-        echo "user:    $USER_NAME present"
-        if [ "$(id -u)" -eq 0 ]; then
-            echo "doctor (as $USER_NAME):"
-            doctor_as_user || rc=1
-        else
-            echo "  (re-run under sudo to run 'bot-bottle doctor' as $USER_NAME)"
-        fi
-    else
-        echo "user:    $USER_NAME absent"
-    fi
-    if command -v container >/dev/null 2>&1; then
-        echo "backend: apple 'container' present ($(container --version 2>/dev/null | head -1))"
-    else
-        echo "backend: apple 'container' NOT on PATH (host prerequisite; install once)"
-    fi
-    return "$rc"
-}
-
-cmd_down() {
-    require_macos
-    require_root down
-    if ! user_exists; then
-        echo "$USER_NAME not present; nothing to remove"
-        return 0
-    fi
-    # A plain -deleteUser removes the home dir, which is the whole reset.
-    # -secure is a no-op on modern macOS (secure erase of the home folder
-    # was removed in Sierra), so it buys nothing here.
-    sysadminctl -deleteUser "$USER_NAME" || true
-    if user_exists; then
-        echo "error: $USER_NAME still present after delete." >&2
-        echo "  - grant Terminal Full Disk Access (System Settings > Privacy & Security), or" >&2
-        echo "  - it may hold the last Secure Token (won't happen while another admin exists)" >&2
-        return 1
-    fi
-    echo "removed $USER_NAME and its home — install surface is clean."
-}
-
-# Teardown half of `test`, installed as an EXIT trap the moment the account
-# exists so that a failure — or a Ctrl-C — still leaves the machine clean.
-_test_teardown() {
-    local rc=$?
-    trap - EXIT INT TERM
-    if [ "${BB_TEST_KEEP:-0}" = "1" ]; then
-        echo
-        echo "== [4/4] down: SKIPPED (BB_TEST_KEEP=1) =="
-        echo "   $USER_NAME is still around; remove it with: sudo $0 down"
-        exit "$rc"
-    fi
-    echo
-    echo "== [4/4] down =="
-    cmd_down || rc=1
-    if [ "$rc" -eq 0 ]; then
-        echo
-        echo "PASS: a brand-new user can install bot-bottle and pass doctor."
-    else
-        echo
-        echo "FAIL: see above (the throwaway account was torn down regardless)." >&2
-    fi
-    exit "$rc"
-}
-
-cmd_test() {
-    require_macos
-    require_root test
-    # A pre-existing account means a pre-existing home, which is the one thing
-    # this harness exists to rule out. Don't silently test a dirty install.
-    if user_exists; then
-        echo "error: $USER_NAME already exists, so this would not be a clean install." >&2
-        echo "  reset first:  sudo $0 down" >&2
-        return 1
-    fi
-    IN_TEST=1
-
-    echo "== [1/4] up =="
-    cmd_up
-    trap _test_teardown EXIT INT TERM
-
-    echo
-    echo "== [2/4] run =="
-    cmd_run
-
-    echo
-    echo "== [3/4] status =="
-    # install.sh exits 0 even when doctor reports unmet prerequisites, so the
-    # install succeeding is not the verdict — this is.
-    cmd_status || {
-        echo "error: doctor is unhappy for a freshly installed user (see above)." >&2
-        echo "  re-run with BB_TEST_KEEP=1 to keep $USER_NAME around and dig in." >&2
-        return 1
-    }
-}
-
-cmd_deep_reset() {
-    require_macos
-    require_root deep-reset
-    # Remove the user first (idempotent), then the HOST-level container
-    # runtime that a user deletion leaves behind under /usr/local + launchd.
-    cmd_down || true
-    if command -v container >/dev/null 2>&1; then
-        # The service can run in more than one launchd context (the invoking
-        # user's and root's), so stop both, best-effort.
-        [ -n "${SUDO_USER:-}" ] && sudo -u "$SUDO_USER" container system stop 2>/dev/null || true
-        container system stop 2>/dev/null || true
-        if [ -x /usr/local/bin/uninstall-container.sh ]; then
-            /usr/local/bin/uninstall-container.sh -d || true
-            echo "uninstalled the host Apple 'container' runtime"
-        else
-            echo "note: /usr/local/bin/uninstall-container.sh not found; runtime left as-is" >&2
-        fi
-    else
-        echo "no 'container' runtime on PATH; nothing further to remove"
-    fi
-}
-
-case "${1:-}" in
-    test) cmd_test ;;
-    up) cmd_up ;;
-    run) cmd_run ;;
-    status) cmd_status ;;
-    down) cmd_down ;;
-    deep-reset) cmd_deep_reset ;;
-    *) echo "usage: $0 {test|up|run|status|down|deep-reset}" >&2 ; exit 2 ;;
-esac

tests/unit/test_install_script.py

@@ -9,7 +9,7 @@ create the config tree, install the package, and verify with `doctor`.
 from __future__ import annotations
 
 import os
-import re
+import sysconfig
 import unittest
 from pathlib import Path
 
@@ -17,22 +17,6 @@ REPO_ROOT = Path(__file__).resolve().parents[2]
 INSTALL_SH = REPO_ROOT / "install.sh"
 
 
-def code_only(text: str) -> str:
-    """Script text with string literals and comments removed.
-
-    Both are places the script *talks about* commands rather than running
-    them — remediation advice quite reasonably says "sudo apt install …" —
-    so assertions about what the script actually executes must not see them.
-    Strings are stripped before comments because a '#' inside a quoted string
-    is not a comment, and several literals here span multiple lines.
-    """
-    without_strings = re.sub(r"\"(?:[^\"\\]|\\.)*\"|'[^']*'", "", text)
-    return "\n".join(
-        ln for ln in without_strings.splitlines()
-        if ln.strip() and not ln.lstrip().startswith("#")
-    )
-
-
 class TestInstallScript(unittest.TestCase):
     @classmethod
     def setUpClass(cls):
@@ -48,45 +32,20 @@ class TestInstallScript(unittest.TestCase):
         self.assertIn("set -eu", self.text)
 
     def test_never_uses_sudo(self):
-        # The installer must never *invoke* sudo. It may print it: the "no
-        # usable python" error suggests 'sudo apt install python3.12'.
-        self.assertNotIn("sudo", code_only(self.text))
+        # Only executable lines matter; the header comment may mention sudo.
+        code = [
+            ln for ln in self.text.splitlines()
+            if ln.strip() and not ln.lstrip().startswith("#")
+        ]
+        self.assertNotIn("sudo", "\n".join(code))
 
     def test_creates_config_tree(self):
         self.assertIn(".bot-bottle/agents", self.text)
         self.assertIn(".bot-bottle/bottles", self.text)
 
-    def test_installs_via_pipx_with_venv_fallback(self):
+    def test_installs_via_pipx_with_pip_fallback(self):
         self.assertIn("pipx install", self.text)
-        self.assertIn("-m venv", self.text)
-
-    def test_no_pip_user_fallback(self):
-        # `pip install --user` is not a fallback, it's a dead end: PEP 668
-        # blocks it on Homebrew, python.org and Debian/Ubuntu interpreters,
-        # which is every Python a Mac realistically offers. A private venv is
-        # exempt from PEP 668 and needs no bootstrap, since venv is stdlib.
-        # code_only, because the comment explaining the absence says the words.
-        code = code_only(self.text)
-        self.assertNotIn("pip install --user", code)
-        self.assertNotIn("--break-system-packages", code)
-
-    def test_venv_lives_under_the_config_dir(self):
-        # Keeps the whole install footprint inside ~/.bot-bottle (plus the
-        # entry-point symlink), which is what makes deleting a throwaway
-        # account a complete reset in scripts/macos-install-test.sh.
-        self.assertIn(".bot-bottle/venv", self.text)
-        self.assertIn("BOT_BOTTLE_VENV", self.text)
-
-    def test_venv_failure_is_actionable(self):
-        # Debian/Ubuntu ship venv separately; failing there must say so rather
-        # than dumping ensurepip's error.
-        self.assertIn("python3-venv", self.text)
-
-    def test_entry_point_is_exposed_outside_the_venv(self):
-        # A venv's bin dir is never on PATH, so the console script has to be
-        # linked somewhere conventional or `bot-bottle` is unreachable.
-        self.assertIn(".local/bin", self.text)
-        self.assertIn("ln -sf", self.text)
+        self.assertIn("pip install --user", self.text)
 
     def test_runs_doctor_after_install(self):
         self.assertIn("doctor", self.text)
@@ -101,43 +60,42 @@ class TestInstallScript(unittest.TestCase):
         self.assertIn("command -v git", self.text)
         self.assertIn("git+*|*.git", self.text)
 
-    def test_installs_into_the_venv_with_its_own_pip(self):
-        # The venv's pip, not the base interpreter's — the base one may not
-        # exist, and using it would install outside the venv.
-        self.assertIn("${VENV}/bin/python\" -m pip install", self.text)
+    def test_checks_pip_usable_before_fallback(self):
+        self.assertIn("python3 -m pip --version", self.text)
 
-    def test_pipx_is_preferred_when_present(self):
-        # The venv is a fallback, not a takeover: someone who already manages
-        # their Python apps with pipx keeps doing so.
-        self.assertIn("command -v pipx", self.text)
+    def test_detects_externally_managed_python(self):
+        # PEP 668: 'pip install --user' is blocked on externally-managed
+        # interpreters; the script must detect this and point at pipx.
+        self.assertIn("EXTERNALLY-MANAGED", self.text)
+        self.assertIn("pipx", self.text)
 
-    def test_asks_pipx_where_its_bin_dir_is(self):
-        # PIPX_BIN_DIR is configurable, so the post-install "is it on PATH?"
-        # check must ask rather than assume ~/.local/bin.
-        self.assertIn("PIPX_BIN_DIR", self.text)
+    def test_resolves_user_scripts_dir_not_hardcoded(self):
+        # The pip --user scripts dir differs by platform; the script must ask
+        # the interpreter (sysconfig + the preferred *user* scheme) rather than
+        # hardcoding Linux's ~/.local/bin (which is wrong on macOS python.org).
+        self.assertIn("get_preferred_scheme", self.text)
+        self.assertIn("sysconfig", self.text)
+        # No hardcoded Linux path in executable lines (a comment may mention it).
+        code = "\n".join(
+            ln for ln in self.text.splitlines()
+            if ln.strip() and not ln.lstrip().startswith("#")
+        )
+        self.assertNotIn(".local/bin", code)
 
-    def test_searches_beyond_path_for_an_interpreter(self):
-        # `python3` on PATH is the *oldest* interpreter on a stock Mac: a fresh
-        # account's PATH is /etc/paths, so python3 is the 3.9.6 CLT stub while
-        # the usable build sits somewhere only a shell profile puts on PATH.
-        # Giving up at that point dead-ends every new macOS user.
-        for candidate in ("python3.11", "/opt/homebrew/bin", "Python.framework"):
-            self.assertIn(candidate, self.text)
+    def test_macos_user_scheme_is_not_dot_local_bin(self):
+        # The case the fix exists for: a python.org macOS interpreter uses the
+        # osx_framework_user scheme, whose scripts land under
+        # ~/Library/Python/<X.Y>/bin — NOT ~/.local/bin. Drive the same
+        # sysconfig lookup install.sh uses, with a mac-like userbase, to prove
+        # it resolves a non-~/.local/bin directory.
+        self.assertIn("osx_framework_user", sysconfig.get_scheme_names())
+        scripts = sysconfig.get_path(
+            "scripts", "osx_framework_user",
+            vars={"userbase": "/Users/dev/Library/Python/3.11"},
+        )
+        self.assertEqual("/Users/dev/Library/Python/3.11/bin", scripts)
+        self.assertNotIn("/.local/bin", scripts)
 
-    def test_interpreter_is_overridable(self):
-        self.assertIn("BOT_BOTTLE_PYTHON", self.text)
-
-    def test_pipx_is_pinned_to_the_vetted_interpreter(self):
-        # Without --python, pipx builds the venv with whichever interpreter
-        # pipx itself was installed with, which need not be the one that
-        # passed the version check.
-        self.assertIn("pipx install --python", self.text)
-
-    def test_version_failure_is_actionable(self):
-        # The failure a new macOS user actually hits must say what to do about
-        # it, not just state the requirement.
-        self.assertIn("brew install python@", self.text)
-        self.assertIn("BOT_BOTTLE_PYTHON=/path/to/python3", self.text)
 
 if __name__ == "__main__":
     unittest.main()

tests/unit/test_orchestrator_broker_client.py

@@ -0,0 +1,117 @@
+"""Unit: orchestrator-side broker client (issue #468, chunk 1). HTTP mocked."""
+
+from __future__ import annotations
+
+import io
+import json
+import unittest
+import urllib.error
+from unittest.mock import MagicMock, patch
+
+from bot_bottle.orchestrator.broker import (
+    BrokerAuthError,
+    BrokerUnavailableError,
+    LaunchRequest,
+)
+from bot_bottle.orchestrator.broker_client import BrokerClient, BrokerClientError
+
+_URLOPEN = "bot_bottle.orchestrator.broker_client.urllib.request.urlopen"
+
+
+def _resp(payload: object) -> MagicMock:
+    m = MagicMock()
+    m.__enter__.return_value.read.return_value = json.dumps(payload).encode()
+    return m
+
+
+def _http_error(code: int, payload: object = None) -> urllib.error.HTTPError:
+    body = json.dumps(payload).encode() if payload is not None else b""
+    return urllib.error.HTTPError(
+        "http://host/broker", code, "err", {}, io.BytesIO(body))  # type: ignore[arg-type]
+
+
+class TestSubmit(unittest.TestCase):
+    def setUp(self) -> None:
+        self.c = BrokerClient("http://host:8091")
+
+    def test_returns_the_verified_request(self) -> None:
+        echo = {
+            "op": "launch", "bottle_id": "b1", "source_ip": "10.0.0.1",
+            "image_ref": "img", "slot": 3,
+        }
+        with patch(_URLOPEN, return_value=_resp(echo)):
+            got = self.c.submit("tok")
+        self.assertEqual(
+            LaunchRequest(op="launch", bottle_id="b1", source_ip="10.0.0.1",
+                          image_ref="img", slot=3),
+            got,
+        )
+
+    def test_posts_token_to_broker_endpoint(self) -> None:
+        with patch(_URLOPEN, return_value=_resp({"op": "teardown", "bottle_id": "b1"})) as m:
+            self.c.submit("signed-token")
+        request = m.call_args.args[0]
+        self.assertEqual("POST", request.get_method())
+        self.assertTrue(request.full_url.endswith("/broker"))
+        self.assertEqual({"token": "signed-token"}, json.loads(request.data))
+
+    def test_401_raises_broker_auth_error(self) -> None:
+        # A fail-closed provenance/schema rejection surfaces as the SAME exception
+        # the in-process broker raises, so the launch path's rollback is identical.
+        with patch(_URLOPEN, side_effect=_http_error(401, {"error": "bad signature"})):
+            with self.assertRaises(BrokerAuthError):
+                self.c.submit("forged")
+
+    def test_502_is_a_definite_client_error(self) -> None:
+        # The host responded — it processed the request and did not launch, so a
+        # definite BrokerClientError (the caller may safely roll back).
+        with patch(_URLOPEN, side_effect=_http_error(502, {"error": "docker down"})):
+            with self.assertRaises(BrokerClientError):
+                self.c.submit("tok")
+
+    def test_unreachable_is_ambiguous_unavailable(self) -> None:
+        # No response at all — the request may already have launched, so the
+        # AMBIGUOUS BrokerUnavailableError (the caller must NOT roll back).
+        with patch(_URLOPEN, side_effect=urllib.error.URLError("refused")):
+            with self.assertRaises(BrokerUnavailableError):
+                self.c.submit("tok")
+
+    def test_timeout_is_ambiguous_unavailable(self) -> None:
+        # A dropped/late response after the request was sent is the exact orphan
+        # risk: the host may have launched. Must be ambiguous, not a definite fail.
+        with patch(_URLOPEN, side_effect=TimeoutError("read timed out")):
+            with self.assertRaises(BrokerUnavailableError):
+                self.c.submit("tok")
+
+    def test_malformed_success_body_raises(self) -> None:
+        with patch(_URLOPEN, return_value=_resp({"op": "launch"})):  # missing bottle_id
+            with self.assertRaises(BrokerClientError):
+                self.c.submit("tok")
+
+    def test_empty_error_body_is_tolerated(self) -> None:
+        # An error with no readable JSON body still classifies by status code.
+        with patch(_URLOPEN, side_effect=_http_error(401)):
+            with self.assertRaises(BrokerAuthError):
+                self.c.submit("forged")
+
+    def test_non_json_success_body_raises(self) -> None:
+        # A 200 whose body isn't JSON is tolerated into {} then fails the
+        # missing-field check — a definite client error, not a crash.
+        m = MagicMock()
+        m.__enter__.return_value.read.return_value = b"not json at all"
+        with patch(_URLOPEN, return_value=m):
+            with self.assertRaises(BrokerClientError):
+                self.c.submit("tok")
+
+    def test_unreadable_error_body_is_tolerated(self) -> None:
+        # An HTTPError whose body can't be read (fp=None) still classifies by
+        # status — the error detail is best-effort.
+        err = urllib.error.HTTPError(
+            "http://host/broker", 502, "err", {}, None)  # type: ignore[arg-type]
+        with patch(_URLOPEN, side_effect=err):
+            with self.assertRaises(BrokerClientError):
+                self.c.submit("tok")
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/test_orchestrator_gateway.py

@@ -248,88 +248,6 @@ class TestDockerGateway(unittest.TestCase):
             calls,
         )
 
-    def test_ensure_running_replaces_poisoned_ipv6_network(self) -> None:
-        # A daemon that default-enables IPv6 leaves the gateway network with a
-        # malformed fdd0::/64 gateway, so `docker network inspect` exits
-        # non-zero with a ParseAddr error (not "No such network"). `--ipv6=false`
-        # can't heal an already-poisoned network — the create just no-ops on
-        # "already exists" — so _ensure_network must force-remove and recreate
-        # it, else every later subnet read keeps failing.
-        calls: list[list[str]] = []
-
-        def fake(argv: list[str], **_kw: object) -> Mock:
-            calls.append(argv)
-            if argv[:2] == ["docker", "ps"]:
-                return _proc(stdout="")
-            if argv[:3] == ["docker", "network", "inspect"]:
-                return _proc(
-                    returncode=1,
-                    stderr='ParseAddr("fdd0:0:0:4::1/64"): unexpected character, '
-                    'want colon (at "/64")',
-                )
-            return _proc()
-
-        with patch(_RUN_DOCKER, side_effect=fake):
-            self.sc.connect_to_orchestrator(_ORCH_URL, _TOKEN)
-        self.assertIn(["docker", "rm", "--force", self.sc.name], calls)
-        self.assertIn(["docker", "network", "rm", self.sc.network], calls)
-        creates = [c for c in calls if c[:3] == ["docker", "network", "create"]]
-        self.assertEqual(
-            [[
-                "docker", "network", "create",
-                "--ipv6=false",
-                "--subnet", DEFAULT_GATEWAY_SUBNET,
-                "--label",
-                f"bot-bottle.gateway-subnet={DEFAULT_GATEWAY_SUBNET}",
-                self.sc.network,
-            ]],
-            creates,
-        )
-
-    def test_ensure_running_creates_network_when_inspect_reports_absent(self) -> None:
-        # The absent case (inspect fails with "No such network") must NOT try to
-        # remove anything — it just creates. Guards the poisoned-vs-absent split.
-        calls: list[list[str]] = []
-
-        def fake(argv: list[str], **_kw: object) -> Mock:
-            calls.append(argv)
-            if argv[:3] == ["docker", "network", "inspect"]:
-                return _proc(returncode=1, stderr="Error: No such network: x")
-            return _proc(stdout="") if argv[:2] == ["docker", "ps"] else _proc()
-
-        with patch(_RUN_DOCKER, side_effect=fake):
-            self.sc.connect_to_orchestrator(_ORCH_URL, _TOKEN)
-        self.assertNotIn(["docker", "network", "rm", self.sc.network], calls)
-        creates = [c for c in calls if c[:3] == ["docker", "network", "create"]]
-        self.assertEqual(1, len(creates))
-
-    def test_ensure_running_does_not_destroy_on_generic_inspect_error(self) -> None:
-        # A generic inspect failure (daemon hiccup, permission, timeout) is NOT
-        # evidence of a poisoned network. Only the ParseAddr poison signature may
-        # take the destructive heal path; anything else must surface as an error
-        # without tearing down a possibly-healthy shared gateway.
-        calls: list[list[str]] = []
-
-        def fake(argv: list[str], **_kw: object) -> Mock:
-            calls.append(argv)
-            if argv[:2] == ["docker", "ps"]:
-                return _proc(stdout="")
-            if argv[:3] == ["docker", "network", "inspect"]:
-                return _proc(
-                    returncode=1,
-                    stderr="Cannot connect to the Docker daemon at unix:///var/run/docker.sock",
-                )
-            return _proc()
-
-        with patch(_RUN_DOCKER, side_effect=fake):
-            with self.assertRaises(GatewayError):
-                self.sc.connect_to_orchestrator(_ORCH_URL, _TOKEN)
-        # No mutation of the shared gateway: neither the container nor the
-        # network is removed, and nothing is recreated.
-        self.assertNotIn(["docker", "network", "rm", self.sc.network], calls)
-        self.assertFalse(any(c[:3] == ["docker", "rm", "--force"] for c in calls))
-        self.assertEqual([], [c for c in calls if c[:3] == ["docker", "network", "create"]])
-
     def test_ca_cert_pem_reads_from_container(self) -> None:
         with patch(_RUN_DOCKER, return_value=_proc(stdout=_CA_PEM)) as m:
             self.assertEqual(_CA_PEM, self.sc.ca_cert_pem())

tests/unit/test_orchestrator_host_server.py

@@ -0,0 +1,285 @@
+"""Unit tests for the host control server (issue #468, chunk 1).
+
+Mostly exercises the pure `dispatch()` (socket-free, like the orchestrator
+server tests), plus a real-socket round-trip through `BrokerClient` that proves
+the full sign -> POST -> verify -> act seam over HTTP.
+"""
+
+from __future__ import annotations
+
+import http.client
+import io
+import json
+import secrets
+import threading
+import typing
+import unittest
+from unittest.mock import MagicMock, patch
+
+from bot_bottle.orchestrator.broker import (
+    BrokerAuthError,
+    LaunchBroker,
+    LaunchRequest,
+    StubBroker,
+    sign_request,
+)
+from bot_bottle.orchestrator.broker_client import BrokerClient
+from bot_bottle.orchestrator.host_server import (
+    MAX_BODY_BYTES,
+    Handler,
+    HostControlServer,
+    broker_secret_from_env,
+    dispatch,
+    main,
+    make_host_server,
+)
+
+
+def _body(obj: object) -> bytes:
+    return json.dumps(obj).encode()
+
+
+class _RaisingBroker(LaunchBroker):
+    """A broker whose backend launch always fails — exercises the 502 path (an
+    operational backend failure, distinct from a fail-closed provenance 401)."""
+
+    def _launch(self, req: LaunchRequest) -> None:
+        raise RuntimeError("docker down")
+
+    def _teardown(self, req: LaunchRequest) -> None:
+        raise RuntimeError("docker down")
+
+
+class TestDispatch(unittest.TestCase):
+    def setUp(self) -> None:
+        self.secret = secrets.token_bytes(16)
+        self.broker = StubBroker(self.secret)
+
+    def _token(self, **kwargs: object) -> str:
+        return sign_request(LaunchRequest(**kwargs), self.secret)  # type: ignore[arg-type]
+
+    def test_health(self) -> None:
+        status, payload = dispatch(self.broker, "GET", "/health", b"")
+        self.assertEqual(200, status)
+        self.assertEqual("ok", payload["status"])
+
+    def test_broker_launch_verifies_and_acts(self) -> None:
+        token = self._token(
+            op="launch", bottle_id="b1", source_ip="10.243.0.1",
+            image_ref="img", slot=2,
+        )
+        status, payload = dispatch(self.broker, "POST", "/broker", _body({"token": token}))
+        self.assertEqual(200, status)
+        self.assertEqual("launch", payload["op"])
+        self.assertEqual("b1", payload["bottle_id"])
+        self.assertEqual("img", payload["image_ref"])
+        self.assertEqual(2, payload["slot"])
+        self.assertEqual(["b1"], [r.bottle_id for r in self.broker.launched])
+
+    def test_broker_teardown_acts(self) -> None:
+        token = self._token(op="teardown", bottle_id="b1")
+        status, _ = dispatch(self.broker, "POST", "/broker", _body({"token": token}))
+        self.assertEqual(200, status)
+        self.assertEqual(["b1"], [r.bottle_id for r in self.broker.torn_down])
+
+    def test_forged_token_is_401_and_nothing_acted(self) -> None:
+        forged = sign_request(
+            LaunchRequest(op="launch", bottle_id="b1"), secrets.token_bytes(16))
+        status, payload = dispatch(self.broker, "POST", "/broker", _body({"token": forged}))
+        self.assertEqual(401, status)
+        self.assertIn("broker auth failed", str(payload["error"]))
+        self.assertEqual([], self.broker.launched)  # fail-closed: never launched
+
+    def test_backend_failure_is_502(self) -> None:
+        broker = _RaisingBroker(self.secret)
+        token = self._token(op="launch", bottle_id="b1", image_ref="img")
+        status, payload = dispatch(broker, "POST", "/broker", _body({"token": token}))
+        self.assertEqual(502, status)
+        self.assertIn("backend launch failed", str(payload["error"]))
+
+    def test_missing_token_is_400(self) -> None:
+        status, _ = dispatch(self.broker, "POST", "/broker", _body({}))
+        self.assertEqual(400, status)
+
+    def test_bad_json_is_400(self) -> None:
+        status, _ = dispatch(self.broker, "POST", "/broker", b"{not json")
+        self.assertEqual(400, status)
+
+    def test_empty_body_is_missing_token_400(self) -> None:
+        # Empty body parses to {} (no token) → 400, never reaching the broker.
+        status, _ = dispatch(self.broker, "POST", "/broker", b"")
+        self.assertEqual(400, status)
+        self.assertEqual([], self.broker.launched)
+
+    def test_non_object_body_is_400(self) -> None:
+        status, _ = dispatch(self.broker, "POST", "/broker", b"[1, 2]")
+        self.assertEqual(400, status)
+
+    def test_unknown_route_404(self) -> None:
+        status, _ = dispatch(self.broker, "GET", "/nope", b"")
+        self.assertEqual(404, status)
+
+    def test_trailing_slash_normalized(self) -> None:
+        status, _ = dispatch(self.broker, "GET", "/health/", b"")
+        self.assertEqual(200, status)
+
+
+class TestBrokerSecretFromEnv(unittest.TestCase):
+    def test_reads_hex_secret(self) -> None:
+        s = secrets.token_bytes(16)
+        self.assertEqual(s, broker_secret_from_env({"BOT_BOTTLE_BROKER_SECRET": s.hex()}))
+
+    def test_unset_is_none(self) -> None:
+        self.assertIsNone(broker_secret_from_env({}))
+
+    def test_invalid_hex_is_none(self) -> None:
+        self.assertIsNone(broker_secret_from_env({"BOT_BOTTLE_BROKER_SECRET": "not-hex"}))
+
+
+class TestSeamRoundTrip(unittest.TestCase):
+    """The whole point of chunk 1: a request signed by the orchestrator side is
+    POSTed to a real host control server, verified there, and acted on — over
+    HTTP, not an in-process call."""
+
+    def _serve(self, broker: LaunchBroker) -> BrokerClient:
+        server = make_host_server(broker, "127.0.0.1", 0)
+        self.addCleanup(server.server_close)
+        threading.Thread(target=server.serve_forever, daemon=True).start()
+        self.addCleanup(server.shutdown)
+        host, port = server.server_address[0], server.server_address[1]
+        return BrokerClient(f"http://{host}:{port}")
+
+    def test_sign_post_verify_act_over_http(self) -> None:
+        secret = secrets.token_bytes(16)
+        broker = StubBroker(secret)
+        client = self._serve(broker)
+        req = LaunchRequest(
+            op="launch", bottle_id="b1", source_ip="10.0.0.1", image_ref="img", slot=1)
+        got = client.submit(sign_request(req, secret))
+        self.assertEqual(req, got)  # the controller echoes the verified request
+        self.assertEqual(["b1"], [r.bottle_id for r in broker.launched])
+
+    def test_forged_token_raises_broker_auth_error_over_http(self) -> None:
+        secret = secrets.token_bytes(16)
+        broker = StubBroker(secret)
+        client = self._serve(broker)
+        forged = sign_request(
+            LaunchRequest(op="launch", bottle_id="b1"), secrets.token_bytes(16))
+        with self.assertRaises(BrokerAuthError):
+            client.submit(forged)
+        self.assertEqual([], broker.launched)  # fail-closed across the wire
+
+
+class TestRequestLimits(unittest.TestCase):
+    """The privileged listener must not let a caller that can merely reach the
+    socket (no signed token) exhaust it via an oversized declared body — and it
+    rejects on the Content-Length *header*, before reading the body."""
+
+    def _addr(self) -> tuple[str, int]:
+        self.broker = StubBroker(secrets.token_bytes(16))
+        server = make_host_server(self.broker, "127.0.0.1", 0)
+        self.addCleanup(server.server_close)
+        threading.Thread(target=server.serve_forever, daemon=True).start()
+        self.addCleanup(server.shutdown)
+        host, port = server.server_address[:2]
+        return typing.cast(str, host), port
+
+    def test_oversized_content_length_is_rejected_before_reading(self) -> None:
+        host, port = self._addr()
+        conn = http.client.HTTPConnection(host, port, timeout=5)
+        self.addCleanup(conn.close)
+        # Declare an oversized body but send only a sliver: the server must reject
+        # on the header before reading, so the caller gets a clean, deterministic
+        # 413 (no large unread body to race a connection reset).
+        conn.putrequest("POST", "/broker", skip_accept_encoding=True)
+        conn.putheader("Content-Type", "application/json")
+        conn.putheader("Content-Length", str(MAX_BODY_BYTES + 1))
+        conn.endheaders()
+        conn.send(b"{}")  # far short of the declared length; never read
+        resp = conn.getresponse()
+        self.assertEqual(413, resp.status)
+        self.assertEqual([], self.broker.launched)  # never reached the broker
+
+
+class TestServeUnit(unittest.TestCase):
+    """Drive `Handler._serve` directly (no socket). The real per-request handler
+    runs in a daemon thread whose coverage/trace data is lost, so the
+    bounded-body and error paths are exercised here in the main thread instead."""
+
+    def _handler(self, broker: LaunchBroker, headers: dict[str, str],
+                 body: bytes = b"") -> tuple[Handler, MagicMock]:
+        server = HostControlServer.__new__(HostControlServer)
+        server.broker = broker
+        h = Handler.__new__(Handler)
+        h.server = server
+        h.headers = headers  # type: ignore[assignment] — dict is a valid .get() stand-in
+        h.path = "/broker"
+        h.rfile = io.BytesIO(body)
+        h.wfile = io.BytesIO()
+        send_response = MagicMock()
+        h.send_response = send_response  # type: ignore[method-assign]
+        h.send_header = MagicMock()  # type: ignore[method-assign]
+        h.end_headers = MagicMock()  # type: ignore[method-assign]
+        return h, send_response
+
+    def test_oversized_content_length_is_413(self) -> None:
+        broker = StubBroker(secrets.token_bytes(16))
+        h, send_response = self._handler(broker, {"Content-Length": str(MAX_BODY_BYTES + 1)})
+        h.do_POST()  # exercises do_POST -> _serve
+        send_response.assert_called_once_with(413)
+        self.assertEqual([], broker.launched)  # rejected before the broker
+
+    def test_invalid_content_length_is_400(self) -> None:
+        h, send_response = self._handler(StubBroker(secrets.token_bytes(16)),
+                                         {"Content-Length": "not-a-number"})
+        h._serve("POST")
+        send_response.assert_called_once_with(400)
+
+    def test_valid_request_dispatches_200(self) -> None:
+        secret = secrets.token_bytes(16)
+        broker = StubBroker(secret)
+        body = _body({"token": sign_request(
+            LaunchRequest(op="teardown", bottle_id="b1"), secret)})
+        h, send_response = self._handler(broker, {"Content-Length": str(len(body))}, body)
+        h._serve("POST")
+        send_response.assert_called_once_with(200)
+        self.assertEqual(["b1"], [r.bottle_id for r in broker.torn_down])
+
+    def test_dispatch_exception_becomes_500(self) -> None:
+        # dispatch is total, but the handler still guards it: a raised dispatch
+        # returns 500 rather than dropping the connection.
+        h, send_response = self._handler(
+            StubBroker(secrets.token_bytes(16)), {"Content-Length": "0"})
+        with patch("bot_bottle.orchestrator.host_server.dispatch",
+                   side_effect=RuntimeError("boom")):
+            h._serve("POST")
+        send_response.assert_called_once_with(500)
+
+    def test_health_over_do_get(self) -> None:
+        h, send_response = self._handler(StubBroker(secrets.token_bytes(16)), {})
+        h.path = "/health"
+        h.do_GET()
+        send_response.assert_called_once_with(200)
+
+
+class TestMain(unittest.TestCase):
+    def test_fail_closed_without_secret(self) -> None:
+        with patch("bot_bottle.orchestrator.host_server.broker_secret_from_env",
+                   return_value=None):
+            self.assertEqual(2, main(["--port", "0"]))
+
+    def test_serves_then_shuts_down_cleanly(self) -> None:
+        fake = MagicMock()
+        fake.server_address = ("127.0.0.1", 0)
+        fake.serve_forever.side_effect = KeyboardInterrupt
+        with patch("bot_bottle.orchestrator.host_server.broker_secret_from_env",
+                   return_value=b"k"), \
+             patch("bot_bottle.orchestrator.host_server.make_host_server",
+                   return_value=fake):
+            self.assertEqual(0, main(["--port", "0"]))
+        fake.serve_forever.assert_called_once()
+        fake.server_close.assert_called_once()
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/test_orchestrator_main.py

@@ -0,0 +1,64 @@
+"""Unit: the orchestrator dev-harness entrypoint (`python -m bot_bottle.orchestrator`).
+
+Exercises broker selection (stub / docker / http) and the fail-closed http path,
+patching `make_server` so the serve loop returns instead of blocking.
+"""
+
+from __future__ import annotations
+
+import os
+import secrets
+import tempfile
+import unittest
+from pathlib import Path
+from unittest.mock import MagicMock, patch
+
+from bot_bottle.orchestrator.__main__ import main
+
+
+def _fake_server() -> MagicMock:
+    fake = MagicMock()
+    fake.server_address = ("127.0.0.1", 0)
+    # Break out of serve_forever immediately, exercising the try/finally.
+    fake.serve_forever.side_effect = KeyboardInterrupt
+    return fake
+
+
+class TestMain(unittest.TestCase):
+    def _run(self, broker: str, env: dict[str, str] | None = None) -> tuple[int, MagicMock]:
+        fake = _fake_server()
+        with tempfile.TemporaryDirectory() as d:
+            argv = ["--db", str(Path(d) / "r.db"), "--port", "0", "--broker", broker]
+            with patch("bot_bottle.orchestrator.__main__.make_server", return_value=fake), \
+                    patch.dict("os.environ", env or {}, clear=False):
+                if env is None:
+                    os.environ.pop("BOT_BOTTLE_BROKER_SECRET", None)
+                rc = main(argv)
+        return rc, fake
+
+    def test_stub_broker_serves_and_closes(self) -> None:
+        rc, fake = self._run("stub")
+        self.assertEqual(0, rc)
+        fake.serve_forever.assert_called_once()
+        fake.server_close.assert_called_once()
+
+    def test_docker_broker_serves(self) -> None:
+        rc, _ = self._run("docker")
+        self.assertEqual(0, rc)
+
+    def test_http_broker_with_secret_serves(self) -> None:
+        rc, _ = self._run(
+            "http", env={"BOT_BOTTLE_BROKER_SECRET": secrets.token_bytes(16).hex()})
+        self.assertEqual(0, rc)
+
+    def test_http_broker_without_secret_exits(self) -> None:
+        # Fail-closed: --broker http with no shared secret is a usage error.
+        with tempfile.TemporaryDirectory() as d:
+            with patch.dict("os.environ", {}, clear=False):
+                os.environ.pop("BOT_BOTTLE_BROKER_SECRET", None)
+                with self.assertRaises(SystemExit):
+                    main(["--db", str(Path(d) / "r.db"), "--broker", "http"])
+
+
+if __name__ == "__main__":
+    unittest.main()

tests/unit/test_orchestrator_secret_store.py

@@ -2,10 +2,12 @@
 
 from __future__ import annotations
 
+import base64
 import unittest
 
 from bot_bottle.orchestrator.store.secret_store import (
     ENV_VAR_SECRET_NAME,
+    _NONCE_BYTES,
     decrypt_value,
     encrypt_value,
     new_env_var_secret,
@@ -65,22 +67,27 @@ class TestDecryptErrors(unittest.TestCase):
     def setUp(self) -> None:
         self.secret = new_env_var_secret()
 
-    def test_wrong_key_raises_value_error(self) -> None:
+    def test_wrong_key_always_raises_value_error(self) -> None:
+        # Deterministic: the authentication tag rejects a wrong key every time,
+        # so reprovision can never inject a garbage token. Repeat across many
+        # random keys (the old unauthenticated scheme let ~5% through when the
+        # garbage happened to decode as valid UTF-8).
+        for _ in range(200):
+            ct = encrypt_value(self.secret, "secret-token")
+            with self.assertRaises(ValueError):
+                decrypt_value(new_env_var_secret(), ct)
+
+    def test_tampered_ciphertext_raises_value_error(self) -> None:
         ct = encrypt_value(self.secret, "secret-token")
-        other_key = new_env_var_secret()
-        # Wrong key produces garbage bytes; decrypt_value raises ValueError
-        # when the result is non-UTF-8 (which is very likely for 12-char data).
-        # We allow it to succeed only if garbage happens to be valid UTF-8, but
-        # the plaintext must not match.
-        try:
-            result = decrypt_value(other_key, ct)
-            self.assertNotEqual("secret-token", result)
-        except ValueError:
-            pass
+        raw = bytearray(base64.urlsafe_b64decode(ct + "=" * (-len(ct) % 4)))
+        raw[_NONCE_BYTES] ^= 0x01  # flip a bit in the ciphertext body → tag mismatch
+        tampered = base64.urlsafe_b64encode(bytes(raw)).rstrip(b"=").decode()
+        with self.assertRaises(ValueError):
+            decrypt_value(self.secret, tampered)
 
     def test_truncated_blob_raises_value_error(self) -> None:
         with self.assertRaises(ValueError):
-            decrypt_value(self.secret, "dG9vc2hvcnQ")  # "tooshort" — under 16 nonce bytes
+            decrypt_value(self.secret, "dG9vc2hvcnQ")  # "tooshort" — under nonce+tag
 
     def test_invalid_base64_raises_value_error(self) -> None:
         with self.assertRaises(ValueError):

tests/unit/test_orchestrator_service.py

@@ -11,7 +11,12 @@ from contextlib import closing
 from pathlib import Path
 from unittest.mock import patch
 
-from bot_bottle.orchestrator.broker import LaunchBroker, LaunchRequest, StubBroker
+from bot_bottle.orchestrator.broker import (
+    BrokerUnavailableError,
+    LaunchBroker,
+    LaunchRequest,
+    StubBroker,
+)
 from bot_bottle.orchestrator.store.registry_store import RegistryStore
 from bot_bottle.orchestrator.service import OrchestratorCore
 from bot_bottle.orchestrator.store.secret_store import new_env_var_secret
@@ -25,8 +30,8 @@ from bot_bottle.orchestrator.supervisor import (
 
 
 class _FailingBroker(LaunchBroker):
-    """Verifies the token like any broker, then fails the launch — to
-    exercise the orchestrator's registry rollback."""
+    """Verifies the token like any broker, then fails the launch *definitely* —
+    to exercise the orchestrator's registry rollback."""
 
     def _launch(self, req: LaunchRequest) -> None:
         raise RuntimeError("launch failed")
@@ -35,6 +40,18 @@ class _FailingBroker(LaunchBroker):
         pass
 
 
+class _UnavailableBroker(LaunchBroker):
+    """Verifies the token, then raises the *ambiguous* BrokerUnavailableError —
+    the host may already have launched — so the orchestrator must KEEP the
+    registry row rather than orphan a running container."""
+
+    def _launch(self, req: LaunchRequest) -> None:
+        raise BrokerUnavailableError("delivery dropped after send")
+
+    def _teardown(self, req: LaunchRequest) -> None:
+        pass
+
+
 class TestOrchestrator(unittest.TestCase):
     def setUp(self) -> None:
         self._tmp = tempfile.TemporaryDirectory()
@@ -144,11 +161,20 @@ class TestOrchestrator(unittest.TestCase):
         self.assertIsNotNone(self.orch.resolve("10.243.0.3", rec.identity_token))
         self.assertIsNone(self.orch.resolve("10.243.0.3", "wrong-token"))
 
-    def test_launch_rolls_back_registry_on_broker_failure(self) -> None:
+    def test_launch_rolls_back_registry_on_definite_broker_failure(self) -> None:
         orch = OrchestratorCore(self.store, _FailingBroker(self.secret), self.secret)
         with self.assertRaises(RuntimeError):
             orch.launch_bottle("10.243.0.9")
-        self.assertEqual([], self.store.all())  # no orphan
+        self.assertEqual([], self.store.all())  # no orphan row
+
+    def test_launch_keeps_registry_on_ambiguous_broker_failure(self) -> None:
+        # The host may already have launched the bottle before the response was
+        # lost, so deregistering would orphan a running container with no row.
+        # The row is kept for reconcile to reap iff the bottle is not live.
+        orch = OrchestratorCore(self.store, _UnavailableBroker(self.secret), self.secret)
+        with self.assertRaises(BrokerUnavailableError):
+            orch.launch_bottle("10.243.0.9")
+        self.assertEqual(1, len(self.store.all()))  # row survives — no orphan container
 
     def test_gateway_status_reports_unconfigured(self) -> None:
         # The orchestrator no longer owns a standalone gateway lifecycle; the