Compare commits

..

10 Commits

Author SHA1 Message Date
didericis-claude 605146d287 ci(integration-macos): dump infra logs on failure before teardown
test / integration-macos (pull_request) Has been skipped
test / integration-docker (pull_request) Successful in 20s
tracker-policy-pr / check-pr (pull_request) Successful in 23s
test / unit (pull_request) Successful in 40s
test / integration-firecracker (pull_request) Successful in 2m8s
test / coverage (pull_request) Successful in 21s
test / publish-infra (pull_request) Has been skipped
A control-plane failure (e.g. the orchestrator becoming unreachable at
bottle registration) is undiagnosable from CI as-is: the `if: always()`
teardown runs `MacosInfraService().stop()`, which deletes the
orchestrator and gateway containers — and their logs — on every run.

Add an `if: failure()` step that dumps `container ls`/`network ls`, plus
`inspect` and `logs` for `bot-bottle-mac-orchestrator` and
`bot-bottle-mac-infra`, ordered before the teardown so the evidence is
captured while the containers still exist. Best-effort (never fails the
job; tolerates an already-removed container).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-26 02:22:52 +00:00
didericis-claude 27dea58ae1 integration-macos: dispatch-only (drop push-to-main trigger)
test / integration-macos (pull_request) Has been skipped
tracker-policy-pr / check-pr (pull_request) Successful in 11s
test / integration-docker (pull_request) Successful in 34s
test / unit (pull_request) Successful in 44s
lint / lint (push) Successful in 1m2s
test / integration-firecracker (pull_request) Successful in 2m6s
test / coverage (pull_request) Successful in 17s
test / publish-infra (pull_request) Has been skipped
Make the advisory macOS Apple Container job run on workflow_dispatch
only, removing the push-to-`main` trigger so a single non-redundant
laptop never runs unattended on every push. Updates the job comment,
docs/ci.md, README CI note, and the PRD accordingly.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 22:05:26 -04:00
didericis 5401f036a9 docs: correct git-gate status on macOS backend (not deferred)
git-gate is fully implemented on the macos-container backend via the
gateway's consolidated git-http daemon with dynamic key
provisioning/revocation; only the legacy per-bottle git:// daemon is
unused. Fixes a stale README claim that git-gate is "deferred" and
removes the PRD open-question built on that false premise (sandbox-escape
attack 5 runs the same on macOS as on the other backends).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 22:05:26 -04:00
didericis 238f5f7614 ci: add advisory macOS Apple Container integration runner
Adds an `integration-macos` job that runs the integration suite against
BOT_BOTTLE_BACKEND=macos-container on a self-hosted host-mode macOS runner
(label `macos`), plus the PRD and provisioning docs.

The job is advisory (push-to-main + workflow_dispatch only, never PRs, not
in coverage.needs) since it targets a single non-redundant laptop. It
preflights `container`/`backend status` so a misprovisioned runner fails
loudly, serializes on a concurrency group, and tears down the
`bot-bottle-mac-infra` singleton on exit (#425).

Also relaxes the TestSandboxEscape CI skip guard: it skipped every backend
but firecracker under GITEA_ACTIONS, which would also skip on a host-mode
macOS runner. The guard's real target is the containerized act_runner, so
it now allows both host-mode backends (firecracker, macos-container)
through — otherwise the macOS job would go green while skipping the one
end-to-end test that proves the backend launches.

Closes #426

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-25 22:05:26 -04:00
Quality Badge Bot 2644759b0d chore: update quality badges
- Coverage: 84%
- Core coverage: 94%

[skip ci]
2026-07-26 01:49:46 +00:00
didericis-claude e53104d5c1 refactor(orchestrator): fail closed unconditionally, drop topology opt-out
tracker-policy-pr / check-pr (pull_request) Successful in 7s
test / integration-docker (pull_request) Successful in 15s
test / unit (pull_request) Successful in 39s
test / integration-firecracker (pull_request) Successful in 3m55s
test / coverage (pull_request) Successful in 22s
test / publish-infra (pull_request) Has been skipped
prd-number / assign-numbers (push) Failing after 21s
test / integration-docker (push) Successful in 22s
lint / lint (push) Successful in 56s
Update Quality Badges / update-badges (push) Successful in 53s
test / unit (push) Successful in 1m52s
test / integration-firecracker (push) Successful in 5m2s
test / coverage (push) Successful in 16s
test / publish-infra (push) Successful in 1m56s
Remove the empty-key opt-out codex flagged: ControlPlaneProvisioning
no longer lets the orchestrator start without a signing key when a
backend declares an "isolated" topology. Host separation is not the
safety condition — the gateway (and any other caller) must reach the
control-plane listener for /resolve, so an open orchestrator would
still grant them full `cli`. The signing key is now mandatory for
every backend.

Drops the now-purposeless Topology/COLOCATED machinery (no backend
declared a non-default topology) and the contractual
test_orchestrator_key_allows_empty_when_isolated. Updates the PRD's
invariant 4 and lifecycle docstring accordingly. Docs/behaviour only
otherwise.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-26 01:40:19 +00:00
didericis-claude 8f6148d571 docs(orchestrator): tighten auth-provisioning wording to concrete services
test / integration-docker (pull_request) Successful in 18s
tracker-policy-pr / check-pr (pull_request) Successful in 21s
test / integration-firecracker (pull_request) Successful in 3m51s
test / unit (pull_request) Failing after 11m53s
test / coverage (pull_request) Has been skipped
test / publish-infra (pull_request) Has been skipped
Address review on #482: drop the generic "credential boundary" framing in the
PRD and docstrings and talk about the specific services — the orchestrator holds
the control-plane key; the host controller (#468) gets a separate key the
orchestrator never holds, so the orchestrator can't mint the credentials it uses
to talk to the host controller that owns its lifecycle. Lead with that concrete
win. No code behaviour change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-26 01:32:53 +00:00
didericis-claude 45f3cefbc5 refactor(orchestrator): uniform control-plane auth provisioning per trust domain
Hoist control-plane auth provisioning out of the per-backend launchers into
one shared contract, parameterized per trust domain (#476). Every blocking
finding in PR #471 was the same integration-bug class: each launcher
re-derived, by hand, how to generate the signing key, scope it to the
orchestrator, mint the gateway JWT, and keep the host key canonical.

Introduces `trust_domain.py`:

  * `TrustDomain` — one credential boundary (host-canonical key file + role
    set + env vars). `mint`/`verify` are scoped to the domain's roles, so a
    future host-controller domain (#468) uses its own key/verifier/roles
    rather than a `host` role on the control plane's frozenset (which the
    orchestrator key could then forge).
  * `ControlPlaneProvisioning` — the single seam answering the four
    invariants: host-canonical key, split key-vs-token credential, CLI token
    valid across co-running backends, and fail-closed (no open mode) for any
    co-located topology.
  * `Topology` — the backend declares what it is; the default is co-located +
    fail-closed, so a backend need not redeclare it.

The `Orchestrator` ABC gets `control_plane_key()` (fail-closed) and routes
`mint_gateway_token()` through the contract; docker/macOS/firecracker
orchestrators, the server (verify), and the host CLI client (mint cli) all go
through the domain instead of reading the host key directly. `orchestrator_auth`
gains an optional `roles=` arg (default unchanged) so a domain scopes its own
role set; `paths.host_signing_key(filename)` generalizes host_orchestrator_token.

Adds unit coverage for the domain boundary + provisioning invariants and a PRD
capturing the durable rationale. No change to the auth primitive's HMAC, the
plane split, or the server's documented open-mode fallback.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Closes #476
2026-07-26 01:32:53 +00:00
Quality Badge Bot 0e70d26af4 chore: update quality badges
- Coverage: 83%
- Core coverage: 94%

[skip ci]
2026-07-25 23:45:18 +00:00
didericis-codex f2d8158742 Merge pull request 'fix(supervise): get bot-bottle.db off the data plane (supervise + egress proposals over RPC)' (#471) from fix/db-off-data-plane-469 into main
test / integration-docker (push) Successful in 35s
test / unit (push) Successful in 49s
Update Quality Badges / update-badges (push) Successful in 50s
lint / lint (push) Successful in 57s
test / integration-firecracker (push) Successful in 5m10s
test / coverage (push) Successful in 16s
test / publish-infra (push) Successful in 2m18s
2026-07-25 19:44:20 -04:00
21 changed files with 694 additions and 53 deletions
+83
View File
@@ -212,6 +212,89 @@ jobs:
name: firecracker-inputs name: firecracker-inputs
path: /var/cache/bot-bottle-fc/dropbear path: /var/cache/bot-bottle-fc/dropbear
# Integration tests against the macOS Apple Container backend. Runs on a
# self-hosted macOS runner (label `macos`) registered in HOST mode — Apple
# Container needs the host `container` CLI + virtualization framework and
# cannot run inside a Linux container, so this cannot reuse the KVM runner.
#
# Advisory only: workflow_dispatch (manual) exclusively — never push or
# pull_request. A single non-redundant laptop that sleeps/roams must not run
# unattended on every push to main, let alone block a PR merge, so this job is
# deliberately NOT in the `coverage` job's `needs` and its coverage never
# feeds the diff-coverage gate. Dispatch-only also means no fork PR (or any
# push) ever executes on the host-mode runner.
#
# The infra container is a singleton (`bot-bottle-mac-infra`); the
# `concurrency` group serializes runs so two never collide on it (#425), and
# the always-run teardown removes it so a crashed run can't wedge the next.
#
# Runner prerequisites (provision once; see README "macOS Apple Container"):
# the `container` CLI on PATH with `container system status` running, and a
# Python >=3.11 with `coverage` importable on the launchd service PATH.
integration-macos:
runs-on: [self-hosted, macos]
if: github.event_name == 'workflow_dispatch'
concurrency:
group: integration-macos-infra
cancel-in-progress: false
steps:
- name: Checkout
uses: actions/checkout@v4
# Fail loudly if the backend this job promises isn't actually usable,
# rather than letting every test silently `unittest.skip` and the job go
# green on zero coverage. `backend status` exits non-zero (and prints the
# per-check summary) when the `container` CLI or its system service is
# missing — the same readiness check the skip guards gate on.
- name: Preflight — Apple Container backend is ready
run: |
command -v container >/dev/null || {
echo "container CLI not on PATH — provision the runner (README: macOS Apple Container)"; exit 1; }
container system status || {
echo "container system service not running — run 'container system start'"; exit 1; }
python3 cli.py backend status --backend=macos-container
# `coverage` comes from the runner's provisioned Python (no pip install
# into the host interpreter). Advisory job: report coverage in-line for
# visibility but don't upload — it never feeds the combined gate.
- name: Run integration tests (macos-container) with coverage
env:
BOT_BOTTLE_BACKEND: macos-container
COVERAGE_FILE: ${{ github.workspace }}/.coverage.macos
run: python3 -m coverage run -m unittest discover -t . -s tests/integration -v
- name: Report macos coverage
env:
COVERAGE_FILE: ${{ github.workspace }}/.coverage.macos
run: python3 -m coverage report -m
# On failure, capture the infra containers' state and logs BEFORE the
# teardown below removes them — otherwise a control-plane crash is
# undiagnosable from CI, since `stop()` deletes the orchestrator (and its
# logs) on every run. Best-effort: never let the diagnostics themselves
# fail the job, and keep going if a container is already gone.
- name: Dump infra diagnostics (on failure)
if: failure()
run: |
set +e
echo "=== containers ==="
container ls -a | grep bot-bottle-mac || echo "(no bot-bottle-mac containers)"
echo "=== networks ==="
container network ls | grep bot-bottle-mac || echo "(no bot-bottle-mac networks)"
for c in bot-bottle-mac-orchestrator bot-bottle-mac-infra; do
echo "=== inspect $c ==="
container inspect "$c" || echo "($c not found)"
echo "=== logs $c ==="
container logs "$c" || echo "($c logs unavailable)"
done
exit 0
# Remove the singleton infra container so a crashed or cancelled run
# cannot leave `bot-bottle-mac-infra` wedged for the next job.
- name: Teardown infra singleton
if: always()
run: python3 -c 'from bot_bottle.backend.macos_container.infra import MacosInfraService; MacosInfraService().stop()'
# Combined coverage gate: aggregates .coverage.* artifacts uploaded by each # Combined coverage gate: aggregates .coverage.* artifacts uploaded by each
# test job, then runs the diff-coverage gate (new/changed lines >= 90%). # test job, then runs the diff-coverage gate (new/changed lines >= 90%).
# #
+3 -1
View File
@@ -30,7 +30,7 @@
## Architecture ## Architecture
On the default macOS Apple Container backend, a bottle is an agent container on a host-only internal network plus a gateway attached to both that internal network and a NAT egress network. The agent gets HTTP(S)_PROXY and CA bundle env vars pointing at the gateway's internal-network IP, so HTTP/HTTPS traffic flows through the gateway instead of direct egress. `bottle.git` / git-gate is intentionally deferred on this backend until a safe Apple Container key-delivery path exists. On the default macOS Apple Container backend, a bottle is an agent container on a host-only internal network plus a gateway attached to both that internal network and a NAT egress network. The agent gets HTTP(S)_PROXY and CA bundle env vars pointing at the gateway's internal-network IP, so HTTP/HTTPS traffic flows through the gateway instead of direct egress. git-gate runs over the gateway's consolidated `git-http` daemon (the legacy per-bottle `git://` daemon is not used on this backend); keys are provisioned dynamically at launch and revoked on teardown.
On the Firecracker backend, a bottle is an agent microVM plus a Docker gateway for egress, git-gate, and supervise. The VM reaches the gateway over a per-bottle point-to-point TAP link; a dedicated fail-closed `nftables` table (`inet bot_bottle_fc`) confines the guest to that link, so nothing leaves the box except through the gateway. The TAP pool and nft table are provisioned once (root); per-launch needs no privilege. On the Firecracker backend, a bottle is an agent microVM plus a Docker gateway for egress, git-gate, and supervise. The VM reaches the gateway over a per-bottle point-to-point TAP link; a dedicated fail-closed `nftables` table (`inet bot_bottle_fc`) confines the guest to that link, so nothing leaves the box except through the gateway. The TAP pool and nft table are provisioned once (root); per-launch needs no privilege.
@@ -75,6 +75,8 @@ On compatible macOS hosts, the default backend requires Apple's `container` CLI
Use `BOT_BOTTLE_BACKEND=docker ./cli.py start <agent>` on hosts where neither Apple Container nor KVM is available and Docker is the desired backend. Use `BOT_BOTTLE_BACKEND=docker ./cli.py start <agent>` on hosts where neither Apple Container nor KVM is available and Docker is the desired backend.
> **CI (macOS Apple Container):** the `integration-macos` job (`.gitea/workflows/test.yml`) runs the integration suite against `BOT_BOTTLE_BACKEND=macos-container` on a self-hosted macOS runner labelled `macos`, because Apple Container needs the host virtualization framework and cannot run in a Linux container (so it can't reuse the `kvm` runner). Provision an Apple Silicon host with the `container` CLI on `PATH` and `container system status` running, then register the runner in **host mode** (not docker mode) with the `macos` label — `brew install gitea-runner` (the `act_runner` rename). Give it a Python ≥ 3.11 with `coverage` importable on the launchd service's `PATH` (a launchd service doesn't inherit your shell profile, so pin `node` and the Python env explicitly). The job is **advisory** — `workflow_dispatch` (manual) only, never triggered by push or PR — since a single laptop that sleeps/roams must not block merges or churn on every push to main; its coverage doesn't feed the gate. The infra container is a singleton (`bot-bottle-mac-infra`), so keep runner concurrency at 1.
### Containers inside a bottle ### Containers inside a bottle
A bottle may set `nested_containers: true`. On the macOS backend this starts a A bottle may set `nested_containers: true`. On the macOS backend this starts a
+3 -2
View File
@@ -19,7 +19,6 @@ from .util import run_docker
from ...paths import ( from ...paths import (
ORCHESTRATOR_TOKEN_ENV, ORCHESTRATOR_TOKEN_ENV,
bot_bottle_root, bot_bottle_root,
host_orchestrator_token,
) )
from ...gateway import GatewayError from ...gateway import GatewayError
from ...orchestrator.lifecycle import ( from ...orchestrator.lifecycle import (
@@ -171,7 +170,9 @@ class DockerOrchestrator(Orchestrator):
fixed-name container first).""" fixed-name container first)."""
self._ensure_control_network() self._ensure_control_network()
run_docker(["docker", "rm", "--force", self.name]) run_docker(["docker", "rm", "--force", self.name])
_signing_key = host_orchestrator_token() # The signing key comes through the shared provisioning contract (#476),
# which fail-closes rather than yield an empty key that would run OPEN.
_signing_key = self.control_plane_key()
proc = run_docker([ proc = run_docker([
"docker", "run", "--detach", "docker", "run", "--detach",
"--name", self.name, "--name", self.name,
@@ -21,7 +21,6 @@ import time
from pathlib import Path from pathlib import Path
from ...log import die, info from ...log import die, info
from ...paths import host_orchestrator_token
from ...orchestrator.lifecycle import ( from ...orchestrator.lifecycle import (
DEFAULT_STARTUP_TIMEOUT_SECONDS, DEFAULT_STARTUP_TIMEOUT_SECONDS,
Orchestrator, Orchestrator,
@@ -108,11 +107,13 @@ class FirecrackerOrchestrator(Orchestrator):
data_drive=self._ensure_registry_volume(), data_drive=self._ensure_registry_volume(),
) )
# Push the host-canonical signing key (the init waits for it before # Push the host-canonical signing key (the init waits for it before
# starting the control plane). The host token file stays the single # starting the control plane). It comes through the shared provisioning
# source of truth, so a co-running docker/macOS control plane keeps # contract (#476) — the same host token file every backend uses, so a
# working; the guest verifies tokens with the same key the CLI signs from. # co-running docker/macOS control plane keeps working and the guest
# verifies tokens with the same key the CLI signs from; fail-closed, so
# the guest is never handed an empty key that would run it OPEN.
infra_vm.push_secret( infra_vm.push_secret(
vm, host_orchestrator_token(), infra_vm._GUEST_SIGNING_KEY_PATH, vm, self.control_plane_key(), infra_vm._GUEST_SIGNING_KEY_PATH,
"the control-plane signing key to the orchestrator VM " "the control-plane signing key to the orchestrator VM "
"(its control plane will not start)", "(its control plane will not start)",
) )
@@ -18,10 +18,7 @@ import urllib.request
from pathlib import Path from pathlib import Path
from ... import log from ... import log
from ...paths import ( from ...paths import ORCHESTRATOR_TOKEN_ENV
ORCHESTRATOR_TOKEN_ENV,
host_orchestrator_token,
)
from ...orchestrator.lifecycle import ( from ...orchestrator.lifecycle import (
DEFAULT_HEALTH_TIMEOUT_SECONDS, DEFAULT_HEALTH_TIMEOUT_SECONDS,
DEFAULT_PORT, DEFAULT_PORT,
@@ -134,7 +131,9 @@ class MacosOrchestrator(Orchestrator):
def _run_container(self, current_hash: str) -> None: def _run_container(self, current_hash: str) -> None:
container_mod.force_remove_container(self.name) container_mod.force_remove_container(self.name)
_signing_key = host_orchestrator_token() # The signing key comes through the shared provisioning contract (#476),
# which fail-closes rather than yield an empty key that would run OPEN.
_signing_key = self.control_plane_key()
argv = [ argv = [
"container", "run", "--detach", "container", "run", "--detach",
"--name", self.name, "--name", self.name,
+3 -3
View File
@@ -18,8 +18,8 @@ import urllib.request
from collections.abc import Iterable from collections.abc import Iterable
from dataclasses import dataclass from dataclasses import dataclass
from ..orchestrator_auth import ROLE_CLI, mint from ..orchestrator_auth import ROLE_CLI
from ..paths import host_orchestrator_token from ..trust_domain import CONTROL_PLANE
from .server import ORCHESTRATOR_AUTH_HEADER from .server import ORCHESTRATOR_AUTH_HEADER
DEFAULT_TIMEOUT_SECONDS = 5.0 DEFAULT_TIMEOUT_SECONDS = 5.0
@@ -32,7 +32,7 @@ def _host_auth_token() -> str:
"" means 'send no auth header' — correct against an open (unconfigured) "" means 'send no auth header' — correct against an open (unconfigured)
control plane, and harmlessly rejected by a secured one.""" control plane, and harmlessly rejected by a secured one."""
try: try:
return mint(ROLE_CLI, host_orchestrator_token()) return CONTROL_PLANE.mint(ROLE_CLI)
except (OSError, ValueError): except (OSError, ValueError):
return "" return ""
+19 -5
View File
@@ -21,8 +21,7 @@ import urllib.error
import urllib.request import urllib.request
from pathlib import Path from pathlib import Path
from ..orchestrator_auth import ROLE_GATEWAY, mint from ..trust_domain import ControlPlaneProvisioning
from ..paths import host_orchestrator_token
DEFAULT_PORT = 8099 DEFAULT_PORT = 8099
DEFAULT_STARTUP_TIMEOUT_SECONDS = 45.0 DEFAULT_STARTUP_TIMEOUT_SECONDS = 45.0
@@ -58,6 +57,12 @@ class Orchestrator(abc.ABC):
so it — not the gateway — mints the gateway's role-scoped token. so it — not the gateway — mints the gateway's role-scoped token.
Backend-neutral.""" Backend-neutral."""
# The shared control-plane auth provisioning contract (#476). Every backend
# gets its signing key + gateway token through this one seam rather than
# re-deriving the wiring; it is fail-closed for every backend — the
# orchestrator never starts without its signing key.
provisioning: ControlPlaneProvisioning = ControlPlaneProvisioning()
def ensure_built(self) -> None: def ensure_built(self) -> None:
"""Ensure the orchestrator's image / rootfs exists, building it if """Ensure the orchestrator's image / rootfs exists, building it if
needed. Default: nothing to build (e.g. a pre-pulled image). Call before needed. Default: nothing to build (e.g. a pre-pulled image). Call before
@@ -105,9 +110,17 @@ class Orchestrator(abc.ABC):
def mint_gateway_token(self) -> str: def mint_gateway_token(self) -> str:
"""Mint a role-scoped `gateway` JWT from the host signing key for the """Mint a role-scoped `gateway` JWT from the host signing key for the
gateway to present. The orchestrator holds the key; the gateway never gateway to present. The orchestrator holds the key; the gateway never
does (#469). Backend-neutral — the same host token file is the single does (#469). Routed through the shared provisioning contract (#476), so
source of truth across backends.""" the same host token file is the single source of truth across backends."""
return mint(ROLE_GATEWAY, host_orchestrator_token()) return self.provisioning.gateway_token()
def control_plane_key(self) -> str:
"""The raw signing key the control-plane *process* must receive — the ONE
place a backend obtains it (docker/macOS inject it as `key_env`;
firecracker pushes it to the guest). Fail-closed via the provisioning
contract: it raises rather than yield an empty key that would run the
server OPEN (#476)."""
return self.provisioning.orchestrator_key()
__all__ = [ __all__ = [
@@ -117,4 +130,5 @@ __all__ = [
"OrchestratorStartError", "OrchestratorStartError",
"source_hash", "source_hash",
"Orchestrator", "Orchestrator",
"ControlPlaneProvisioning",
] ]
+7 -5
View File
@@ -63,8 +63,8 @@ import sys
import typing import typing
from urllib.parse import urlsplit from urllib.parse import urlsplit
from ..orchestrator_auth import ROLE_CLI, ROLES, verify from ..orchestrator_auth import ROLE_CLI, ROLES
from ..paths import ORCHESTRATOR_TOKEN_ENV from ..trust_domain import CONTROL_PLANE
from ..supervisor.types import TOOLS from ..supervisor.types import TOOLS
from .service import OrchestratorCore from .service import OrchestratorCore
@@ -413,11 +413,13 @@ class OrchestratorServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
def __init__(self, address: tuple[str, int], orchestrator: OrchestratorCore) -> None: def __init__(self, address: tuple[str, int], orchestrator: OrchestratorCore) -> None:
self.orchestrator = orchestrator self.orchestrator = orchestrator
self._signing_key = os.environ.get(ORCHESTRATOR_TOKEN_ENV, "").strip() # The control-plane trust domain's signing key, as injected into THIS
# (the owning) process by the launcher (#476). Unset → open mode below.
self._signing_key = CONTROL_PLANE.key_from_env()
if not self._signing_key: if not self._signing_key:
sys.stderr.write( sys.stderr.write(
"orchestrator: WARNING — no control-plane signing key " "orchestrator: WARNING — no control-plane signing key "
f"(${ORCHESTRATOR_TOKEN_ENV}); running WITHOUT caller " f"(${CONTROL_PLANE.key_env}); running WITHOUT caller "
"authentication. Any client that can reach this port can drive " "authentication. Any client that can reach this port can drive "
"it. Backends that put the control plane on an agent-reachable " "it. Backends that put the control plane on an agent-reachable "
"network MUST set this.\n" "network MUST set this.\n"
@@ -433,7 +435,7 @@ class OrchestratorServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
role (→ per-route 401/403 in `dispatch`).""" role (→ per-route 401/403 in `dispatch`)."""
if not self._signing_key: if not self._signing_key:
return ROLE_CLI return ROLE_CLI
return verify(presented, self._signing_key) return CONTROL_PLANE.verify(presented, self._signing_key)
def make_server( def make_server(
+14 -8
View File
@@ -59,12 +59,17 @@ _HEADER_SEGMENT = _b64url_encode(
) )
def mint(role: str, secret: str) -> str: def mint(role: str, secret: str, *, roles: frozenset[str] = ROLES) -> str:
"""A compact HS256 token asserting `role`, signed with `secret`. """A compact HS256 token asserting `role`, signed with `secret`.
Raises ValueError for an unknown role (mint only what the control plane will `roles` is the set the signing key is allowed to sign (default: the
accept) or an empty signing key (an unsigned credential is never valid).""" orchestrator's `{gateway, cli}`). A separate service (e.g. the host
if role not in ROLES: controller) passes its own key + role set so its tokens can't be forged with
the orchestrator's key — see `trust_domain.py`, issues #476/#468.
Raises ValueError for a role outside `roles`, or an empty signing key (an
unsigned credential is never valid)."""
if role not in roles:
raise ValueError(f"unknown control-plane role {role!r}") raise ValueError(f"unknown control-plane role {role!r}")
if not secret: if not secret:
raise ValueError("cannot mint a control-plane token without a signing key") raise ValueError("cannot mint a control-plane token without a signing key")
@@ -73,10 +78,11 @@ def mint(role: str, secret: str) -> str:
return f"{signing_input}.{_sign(secret, signing_input)}" return f"{signing_input}.{_sign(secret, signing_input)}"
def verify(token: str, secret: str) -> str | None: def verify(token: str, secret: str, *, roles: frozenset[str] = ROLES) -> str | None:
"""The role a valid `token` carries, or None if it is malformed, wrongly """The role a valid `token` carries, or None if it is malformed, wrongly
signed, or names an unknown role. Constant-time signature check; rejects any signed, or names a role outside `roles` (the verifying trust domain's set —
header whose alg isn't HS256 (no alg-confusion / `none`).""" default `{gateway, cli}`). Constant-time signature check; rejects any header
whose alg isn't HS256 (no alg-confusion / `none`)."""
if not token or not secret: if not token or not secret:
return None return None
parts = token.split(".") parts = token.split(".")
@@ -94,7 +100,7 @@ def verify(token: str, secret: str) -> str | None:
if not isinstance(header, dict) or header.get("alg") != _ALG: if not isinstance(header, dict) or header.get("alg") != _ALG:
return None return None
role = payload.get("role") if isinstance(payload, dict) else None role = payload.get("role") if isinstance(payload, dict) else None
return role if isinstance(role, str) and role in ROLES else None return role if isinstance(role, str) and role in roles else None
__all__ = ["ROLE_GATEWAY", "ROLE_CLI", "ROLES", "mint", "verify"] __all__ = ["ROLE_GATEWAY", "ROLE_CLI", "ROLES", "mint", "verify"]
+19 -9
View File
@@ -97,16 +97,17 @@ def host_gateway_ca_dir() -> Path:
return ca_dir return ca_dir
def host_orchestrator_token() -> str: def host_signing_key(filename: str) -> str:
"""The per-host control-plane secret, minted (256-bit, url-safe) and """A per-host signing key at `<root>/<filename>`, minted (256-bit, url-safe)
persisted 0600 on first use, then reused. and persisted 0600 on first use, then reused.
This is the shared secret the launchers inject into the control-plane and The generic form of `host_orchestrator_token()`: each service names its own
gateway containers and that the host CLI presents on every call. It is a key file (`trust_domain.py`), so the orchestrator and a separate service like
*host* artifact the file lives under the root the agent never mounts, and the host controller (#468) get distinct keys neither can read. It is a *host*
the env var is set only on the trusted containers so reading it here is artifact the file lives under the root the agent never mounts, and its value
safe on the host launch path but the value never reaches a bottle.""" is injected only into the trusted control-plane process so reading it here
path = bot_bottle_root() / ORCHESTRATOR_TOKEN_FILENAME is safe on the launch path but the value never reaches a bottle."""
path = bot_bottle_root() / filename
try: try:
existing = path.read_text().strip() existing = path.read_text().strip()
if existing: if existing:
@@ -128,6 +129,14 @@ def host_orchestrator_token() -> str:
return token return token
def host_orchestrator_token() -> str:
"""The per-host control-plane signing key — the host-canonical key the
launchers inject into the control-plane process and the host CLI mints its
own `cli` token from. The `control-plane` trust domain's specialization of
`host_signing_key()`."""
return host_signing_key(ORCHESTRATOR_TOKEN_FILENAME)
__all__ = [ __all__ = [
"HOST_DB_FILENAME", "HOST_DB_FILENAME",
"ORCHESTRATOR_TOKEN_FILENAME", "ORCHESTRATOR_TOKEN_FILENAME",
@@ -138,5 +147,6 @@ __all__ = [
"host_db_path", "host_db_path",
"host_db_dir", "host_db_dir",
"host_gateway_ca_dir", "host_gateway_ca_dir",
"host_signing_key",
"host_orchestrator_token", "host_orchestrator_token",
] ]
+140
View File
@@ -0,0 +1,140 @@
"""Per-service control-plane signing keys (issue #476).
A `TrustDomain` is one service's signing material: its host-canonical key file,
the roles that key may sign, and the env vars its key and a pre-minted token ride
in. Scoping `mint`/`verify` to a domain's roles keeps one service's key from
signing (or accepting) another service's tokens.
Today there is one domain, `CONTROL_PLANE` the orchestrator's key (roles
`{gateway, cli}`): the orchestrator holds it and mints the gateway's and CLI's
tokens. The host controller (#468) will add a **second** domain with its own key
the orchestrator never holds. That is the point: the host controller starts and
stops the orchestrator, so the orchestrator must not be able to mint the
credentials it uses to talk to it. Adding a `host` role to `CONTROL_PLANE`
instead would defeat that the orchestrator holds that key, so it could forge
`host` tokens.
`ControlPlaneProvisioning` is the one seam every backend launcher uses to get the
orchestrator its key and the gateway its token, instead of re-deriving that
wiring per backend (the bug class behind PR #471 — see
`docs/prds/prd-new-control-plane-auth-provisioning.md`).
Stdlib-only: the HMAC lives in `orchestrator_auth`, the key file in `paths`.
"""
from __future__ import annotations
import os
from collections.abc import Mapping
from dataclasses import dataclass
from . import orchestrator_auth
from .orchestrator_auth import ROLE_GATEWAY
from .paths import (
ORCHESTRATOR_AUTH_JWT_ENV,
ORCHESTRATOR_TOKEN_ENV,
ORCHESTRATOR_TOKEN_FILENAME,
host_signing_key,
)
class ProvisioningError(RuntimeError):
"""A control-plane auth invariant would be violated (e.g. starting the
orchestrator without its signing key which would run OPEN)."""
@dataclass(frozen=True)
class TrustDomain:
"""One service's signing material: a host-canonical key file, the roles that
key may sign, and the env vars its key and a minted token ride in.
The service that *owns* the domain (e.g. the orchestrator) receives the raw
key via `key_env`; a delegate (e.g. the gateway) receives only a pre-minted,
role-scoped token via `token_env` it cannot rewrite. `mint`/`verify` are
scoped to `roles`, so this service's key can neither sign nor accept another
service's role."""
name: str
key_filename: str
roles: frozenset[str]
key_env: str
token_env: str
def signing_key(self) -> str:
"""This service's host-canonical signing key (minted 0600 on first use).
Host-side only the value is injected into the owning process."""
return host_signing_key(self.key_filename)
def key_from_env(self, environ: Mapping[str, str] | None = None) -> str:
"""The signing key as the owning process sees it — read from `key_env`
(default `os.environ`). "" when unset; the caller decides whether that is
fatal (`ControlPlaneProvisioning`) or the open-mode fallback
(`OrchestratorServer`)."""
env = os.environ if environ is None else environ
return env.get(self.key_env, "").strip()
def mint(self, role: str) -> str:
"""A role-scoped token for a delegate, signed with this service's key.
Raises ValueError for a role this service doesn't sign."""
if role not in self.roles:
raise ValueError(f"role {role!r} is not in trust domain {self.name!r}")
return orchestrator_auth.mint(role, self.signing_key(), roles=self.roles)
def verify(self, token: str, key: str) -> str | None:
"""The role `token` carries under `key`, or None. `key` is passed in
(not read from disk) because the verifier the control-plane process
holds it in `key_env`, not on disk in its guest."""
return orchestrator_auth.verify(token, key, roles=self.roles)
# The orchestrator's domain: the key the orchestrator (and host CLI) holds, the
# `gateway` token it mints for the data plane, and the `cli` token the CLI mints
# for itself. #468's host controller will add a second, separate domain.
CONTROL_PLANE = TrustDomain(
name="control-plane",
key_filename=ORCHESTRATOR_TOKEN_FILENAME,
roles=orchestrator_auth.ROLES,
key_env=ORCHESTRATOR_TOKEN_ENV,
token_env=ORCHESTRATOR_AUTH_JWT_ENV,
)
@dataclass(frozen=True)
class ControlPlaneProvisioning:
"""The one seam every backend launcher uses to provision control-plane auth,
instead of re-deriving the four invariants that each cost a PR #471 review
round: the orchestrator gets the raw key (`orchestrator_key`), the gateway
gets a minted `gateway` token (`gateway_token`), the host CLI mints its own
`cli` token from the same host-canonical key, and the orchestrator never
starts open."""
domain: TrustDomain = CONTROL_PLANE
def orchestrator_key(self) -> str:
"""The raw signing key the orchestrator process must receive (carry it in
`domain.key_env`). Fail-closed: raises rather than return "", since an
empty key runs the server open and being on a separate host does not
stop the gateway from reaching the control plane (it must, for
`/resolve`), so an open orchestrator would treat that gateway as `cli`."""
key = self.domain.signing_key()
if not key:
raise ProvisioningError(
f"refusing to start the {self.domain.name} orchestrator without "
"a signing key: an open orchestrator authenticates no one and "
"grants every caller that reaches it full `cli` (#476)"
)
return key
def gateway_token(self) -> str:
"""The `gateway`-role token the gateway receives (carry it in
`domain.token_env`) minted from the key, never the key itself, so a
compromised gateway cannot forge a `cli` token."""
return self.domain.mint(ROLE_GATEWAY)
__all__ = [
"ProvisioningError",
"TrustDomain",
"CONTROL_PLANE",
"ControlPlaneProvisioning",
]
+12 -1
View File
@@ -2,11 +2,22 @@
The test workflow lives at [`.gitea/workflows/test.yml`](../.gitea/workflows/test.yml). The test workflow lives at [`.gitea/workflows/test.yml`](../.gitea/workflows/test.yml).
It runs the unit suite plus one integration job per backend It runs the unit suite plus one integration job per backend
(`integration-docker`, `integration-firecracker`) on: (`integration-docker`, `integration-firecracker`, `integration-macos`) on:
- every push to a branch with an open pull request, and - every push to a branch with an open pull request, and
- every push to `main`. - every push to `main`.
`integration-macos` is the exception: it is **advisory**, running only on
`workflow_dispatch` (manual dispatch), never on push or pull requests. It targets the
Apple Container backend on a self-hosted macOS runner (label `macos`,
registered in host mode — Apple Container can't run in a Linux container, so it
can't reuse the `kvm` runner). A single non-redundant laptop must not be able
to block a PR merge, so the job stays out of the `coverage` job's `needs` and
its coverage never feeds the diff-coverage gate. Because the infra container is
a singleton (`bot-bottle-mac-infra`), the job declares a `concurrency` group
and tears the container down on exit; keep runner concurrency at 1. See the
README "macOS Apple Container" CI note for runner provisioning.
Each integration job selects its backend via `BOT_BOTTLE_BACKEND` and Each integration job selects its backend via `BOT_BOTTLE_BACKEND` and
runs a **preflight** (`./cli.py backend status --backend=<name>`) that runs a **preflight** (`./cli.py backend status --backend=<name>`) that
prints a clear per-check readiness summary and fails the job when the prints a clear per-check readiness summary and fails the job when the
@@ -0,0 +1,89 @@
# PRD prd-new: Per-service signing keys for control-plane auth
- **Status:** Draft
- **Author:** claude
- **Created:** 2026-07-26
- **Issue:** #476
## Summary
Provision control-plane signing keys **per service**, through one shared seam, so
no service can mint another's credentials. Concretely: the orchestrator holds the
control-plane key and mints the gateway's and CLI's tokens; the host controller
(#468, next) gets a **separate** key the orchestrator never holds — so the
orchestrator cannot forge the credentials it uses to talk to the host controller
that starts and stops it. Landing this seam also retires the per-backend
provisioning duplication that made PR #471 take three review rounds.
## Problem
**1. The orchestrator could forge host-controller credentials.** The
orchestrator's key signs roles `{gateway, cli}`. The tempting way to add the host
controller (#468) is a third role, `host`, on that same key. But then the
orchestrator — which holds the key — can mint `host` tokens, and the host
controller, which owns the orchestrator's lifecycle, must not trust anything the
orchestrator can mint. The two services need separate keys.
**2. Every backend provisioned auth by hand.** Each launcher (docker
gateway/infra, macOS infra, firecracker infra) re-derived how to generate the
signing key, scope it to the orchestrator, mint the gateway JWT, and keep the
host key file canonical. All three PR #471 High-severity findings were this one
integration bug in different launchers: the data plane got the full `cli` token;
the firecracker control plane ran open; the firecracker guest clobbered the host
key.
## Goals / Success Criteria
- The orchestrator and the host controller sign with **different** keys; neither
can mint the other's tokens. (This PR provisions the orchestrator's key and
leaves a drop-in seam for the host controller's.)
- One shared provisioning seam every backend uses — a new backend or daemon
implements it instead of rediscovering these four invariants:
1. the signing key is host-canonical: a guest is handed it, never generates or
overwrites it;
2. only the orchestrator process gets the raw key; the gateway gets a
pre-minted `gateway` token it can't rewrite into `cli`;
3. the host CLI's `cli` token is minted from the same key, so it stays valid
across co-running backends;
4. the orchestrator never runs open — the signing key is mandatory, with no
topology opt-out (a separate host does not stop a caller from reaching the
control-plane listener, so it cannot make open mode safe).
## Non-goals
- The host controller itself (#468) — this only provisions the orchestrator's
key and the seam #468 plugs into.
- Rewriting the HMAC primitive: `orchestrator_auth.mint/verify` gain an optional
`roles=` arg (default unchanged) so a key can carry a different role set;
nothing else changes.
- Network topology, the plane split (#469), or the server's open-mode fallback
for tests.
## Design
A **`TrustDomain`** is one service's signing material: its host-canonical key
file, the roles that key may sign, and the env vars its key and a pre-minted
token ride in. `mint`/`verify` are scoped to that domain's roles, so a token
signed by one service's key neither carries nor verifies another service's role.
- `CONTROL_PLANE` — the orchestrator's domain: key `orchestrator-token`, roles
`{gateway, cli}`. The orchestrator process holds the key; the gateway holds
only a minted `gateway` token; the host CLI mints its own `cli` token.
- The host controller (#468) will add a second `TrustDomain` — its own key file
and role(s) — that the orchestrator never holds.
**`ControlPlaneProvisioning`** is the seam the backends call.
`orchestrator_key()` returns the raw key for the control-plane process
(fail-closed for every backend: it raises rather than hand back an empty key that
would run the server open). `gateway_token()` mints the gateway's token. Each backend applies these through its own transport —
docker/macOS inject env vars, firecracker pushes over SSH — but none re-derives
*which* key or role.
`paths.host_signing_key(filename)` generalizes `host_orchestrator_token()` so each
domain names its own key file.
## Open questions
None blocking. #468 adds its `TrustDomain` and a second
`ControlPlaneProvisioning`-shaped consumer; renaming that class to something
service-neutral is a cosmetic call to make then.
@@ -0,0 +1,142 @@
# PRD prd-new: macOS (Apple Container) CI runner
- **Status:** Draft
- **Author:** Claude
- **Created:** 2026-07-25
- **Issue:** #426
## Summary
CI has no runner for the `macos-container` (Apple Container) backend.
`.gitea/workflows/test.yml` exercises Docker (`ubuntu-latest`) and
Firecracker (self-hosted `kvm`) but never the macOS backend. This PRD adds a
self-hosted macOS runner (label `macos`) and an advisory `integration-macos`
job that runs the integration suite against `BOT_BOTTLE_BACKEND=macos-container`,
so the backend that is the default on macOS stops shipping unexercised.
## Problem
The gap is not theoretical. `5ad3449` moved `bot_bottle` from flat files under
`/app` into a pip-installed package but left init scripts spawning the
supervisor as `python3 /app/gateway_init.py`, which no longer exists. Both the
Firecracker and macOS backends carried the identical bug:
- **firecracker** — caught and fixed in `127ba49` because the KVM runner
(added in `c193b04`, PR #349) runs that backend's integration suite.
- **macos-container** — survived on `main` and only surfaced when a human ran
`bot-bottle start` by hand.
The failure mode is expensive to debug: the supervisor never starts, so
mitmdump never generates its CA, and launch dies downstream with
`GatewayError: gateway CA not available`, which points at TLS rather than at
the supervisor. Unit tests did not help — `test_macos_infra` asserted the
substring `"gateway_init.py"`, which the *broken* path satisfies. (That
specific assertion has since been tightened to the module form
`bot_bottle.gateway_init`, matching its Firecracker twin, so the exact
regression is now covered on `ubuntu-latest`. What remains missing is the
end-to-end runner that would catch the *next* macOS-only launch regression.)
PR #470 (#414) already made the integration suite backend-agnostic:
`skip_unless_selected_backend_available()` gates on the *selected* backend's
own `is_backend_ready()` rather than `docker_available()`, and each
integration job runs `./cli.py backend status --backend=<name>` as a preflight
that fails loudly when the backend is missing. That is the machinery this job
plugs into; this PRD supplies the runner and the job.
## Goals / Success criteria
- A macOS runner is registered and picks up jobs by the `macos` label.
- An `integration-macos` job runs the integration suite against
`BOT_BOTTLE_BACKEND=macos-container`.
- The job **fails, not skips**, when the backend is unavailable on the runner
(via the `backend status` preflight).
- Reverting the `macos_container/infra.py` supervisor fix makes the job fail:
the broken supervisor path throws `GatewayError` at bottle launch, which is
`TestSandboxEscape.setUpClass`, failing the whole class before any individual
attack runs.
## Non-goals
- Making `integration-macos` a **required** PR check. It runs on
`workflow_dispatch` (manual dispatch) only — never on push or PRs. A single
non-redundant laptop that sleeps and roams must never be able to block a PR
merge or churn unattended on every push to main, and it is deliberately kept
out of the `coverage` job's `needs` so the diff-coverage gate never depends on
it.
- Multi-machine or hosted macOS runners. Apple Container needs the host
virtualization framework, so the runner must be a physical/VM macOS host on
Apple Silicon — it cannot reuse the KVM runner or run in a Linux container.
- Coverage aggregation from the macOS job into the combined gate (would couple
the gate to the laptop).
## Design
### Runner (operational, provisioned once)
- Apple Silicon macOS host with Apple's `container` CLI installed and
`container system status` reporting `running`.
- Install the runner: `brew install gitea-runner` (the `act_runner` rename),
registered in **host mode** with label `macos` — not docker mode, because
Apple Container needs the host `container` CLI and virtualization framework,
not a nested container.
- A Python ≥ 3.11 with `coverage` importable on the runner's `PATH`. Because a
launchd service does not inherit an interactive shell's `PATH`, pin `node`
(for the JS `actions/*`) and the Python env explicitly in the service
environment rather than relying on `nvm`/shell profile.
- Concurrency 1. The infra container is a singleton (`bot-bottle-mac-infra`),
so two simultaneous runs on one host collide (#425). The job also declares a
`concurrency` group as belt-and-suspenders and tears the singleton down after
each run.
### `integration-macos` job
Modeled on `integration-firecracker`:
- `runs-on: [self-hosted, macos]`.
- `if:` `workflow_dispatch` only (advisory, manual dispatch; never push or PRs,
so no fork-PR exposure, no merge-blocking, and no unattended runs on push).
- `concurrency: { group: integration-macos-infra, cancel-in-progress: false }`
to serialize runs against the singleton.
- **Preflight**`command -v container`, `container system status`, then
`./cli.py backend status --backend=macos-container`; any failure exits
non-zero so a misprovisioned runner fails loudly instead of silently
skipping.
- Run the integration suite under coverage with
`BOT_BOTTLE_BACKEND=macos-container` and print a `coverage report -m` for
visibility (no upload, not in the gate).
- **Teardown** (`if: always()`) — `MacosInfraService().stop()` removes the
singleton so a crashed run cannot wedge the next one.
### The `test_sandbox_escape` CI guard (the trap #470 left)
`TestSandboxEscape` is the only backend-agnostic integration test that boots a
real bottle, so it is the one that would catch a macOS launch regression. It
still carries a second guard that skips under `GITEA_ACTIONS` for every backend
except `firecracker`:
```python
@unittest.skipIf(
os.environ.get("GITEA_ACTIONS") == "true"
and os.environ.get("BOT_BOTTLE_BACKEND") != "firecracker",
...,
)
```
The skip exists because the *containerized* `act_runner` (docker on
`ubuntu-latest`) can't see a host bind mount and hides sibling-gateway network
topology. Those constraints do not apply to a **host-mode** runner — neither
the KVM host runner nor a macOS host runner is containerized. This PRD relaxes
the guard to allow both host-mode backends (`firecracker`, `macos-container`)
through while still skipping on the containerized Docker job. Without this
change the macOS job would run green while skipping the exact test that proves
the backend launches — the very false-green this issue is about.
## Open questions
- None known that block the job. git-gate is fully implemented on the macOS
backend (the gateway's consolidated `git-http` daemon plus dynamic key
provisioning/revocation), so `TestSandboxEscape` attack 5 — secret exfil
pushed through git-gate, rejected by the gitleaks hook before the upstream
push — runs the same as on the other backends. Any genuinely
macOS-specific test adjustment would surface at first runner bring-up, but
none is anticipated from the current backend implementation.
+14 -3
View File
@@ -67,14 +67,25 @@ _DUMMY_HOST_KEY = (
) )
# Backends whose CI runner is HOST-mode (self-hosted), so the test process
# and the backend share a host. The containerized act_runner (docker on
# ubuntu-latest) is the one that can't see the host bind mount egress_tls_init
# uses and hides sibling-gateway network topology; host-mode runners
# (firecracker/KVM, macos-container) don't have those constraints, so the test
# runs there. Keep this in sync with the `runs-on` labels in
# .gitea/workflows/test.yml.
_HOST_MODE_CI_BACKENDS = frozenset({"firecracker", "macos-container"})
@skip_unless_selected_backend_available() @skip_unless_selected_backend_available()
@unittest.skipIf( @unittest.skipIf(
os.environ.get("GITEA_ACTIONS") == "true" os.environ.get("GITEA_ACTIONS") == "true"
and os.environ.get("BOT_BOTTLE_BACKEND") != "firecracker", and os.environ.get("BOT_BOTTLE_BACKEND") not in _HOST_MODE_CI_BACKENDS,
"skipped under act_runner unless BOT_BOTTLE_BACKEND=firecracker: " "skipped under the containerized act_runner (docker on ubuntu-latest): "
"egress_tls_init uses a host bind mount the runner container can't " "egress_tls_init uses a host bind mount the runner container can't "
"see, and the network topology hides sibling-gateway visibility — " "see, and the network topology hides sibling-gateway visibility — "
"these constraints don't apply on the self-hosted KVM runner", "these constraints don't apply on the self-hosted host-mode runners "
"(firecracker/KVM, macos-container)",
) )
class TestSandboxEscape(unittest.TestCase): class TestSandboxEscape(unittest.TestCase):
"""End-to-end attacks against a real bottle. The bottle stays """End-to-end attacks against a real bottle. The bottle stays
+3 -1
View File
@@ -19,7 +19,9 @@ _ORCH = "bot_bottle.backend.docker.orchestrator"
_RUN = f"{_ORCH}.run_docker" _RUN = f"{_ORCH}.run_docker"
_SLEEP = f"{_ORCH}.time.sleep" _SLEEP = f"{_ORCH}.time.sleep"
_MONOTONIC = f"{_ORCH}.time.monotonic" _MONOTONIC = f"{_ORCH}.time.monotonic"
_TOKEN = f"{_ORCH}.host_orchestrator_token" # The signing key is read through the shared provisioning contract (#476); patch
# its host-canonical key file read to keep it off the real host file.
_TOKEN = "bot_bottle.trust_domain.host_signing_key"
# The ABC's is_healthy probes /health via urllib in the lifecycle module. # The ABC's is_healthy probes /health via urllib in the lifecycle module.
_URLOPEN = "bot_bottle.orchestrator.lifecycle.urllib.request.urlopen" _URLOPEN = "bot_bottle.orchestrator.lifecycle.urllib.request.urlopen"
+1 -1
View File
@@ -44,7 +44,7 @@ class TestEnsureRunning(unittest.TestCase):
vm = infra_vm.InfraVm(guest_ip="10.243.255.1", private_key=Path("/k"), vm=MagicMock()) vm = infra_vm.InfraVm(guest_ip="10.243.255.1", private_key=Path("/k"), vm=MagicMock())
with patch.object(infra_vm, "boot_vm", return_value=vm) as boot, \ with patch.object(infra_vm, "boot_vm", return_value=vm) as boot, \
patch.object(infra_vm, "push_secret") as push, \ patch.object(infra_vm, "push_secret") as push, \
patch(f"{_ORCH}.host_orchestrator_token", return_value="host-key"), \ patch("bot_bottle.trust_domain.host_signing_key", return_value="host-key"), \
patch.object(orch, "is_running", return_value=False), \ patch.object(orch, "is_running", return_value=False), \
patch.object(orch, "_ensure_registry_volume", return_value=Path("/reg")), \ patch.object(orch, "_ensure_registry_volume", return_value=Path("/reg")), \
patch.object(orch, "_wait_for_health"): patch.object(orch, "_wait_for_health"):
+2 -2
View File
@@ -33,7 +33,7 @@ class TestMacosOrchestratorRun(unittest.TestCase):
def _run(self) -> list[str]: def _run(self) -> list[str]:
run = Mock(return_value=_ok()) run = Mock(return_value=_ok())
with patch(f"{_ORCH}.container_mod") as mod, \ with patch(f"{_ORCH}.container_mod") as mod, \
patch(f"{_ORCH}.host_orchestrator_token", return_value="k"): patch("bot_bottle.trust_domain.host_signing_key", return_value="k"):
mod.dns_server.return_value = "1.1.1.1" mod.dns_server.return_value = "1.1.1.1"
mod.bind_mount_spec.side_effect = _spec mod.bind_mount_spec.side_effect = _spec
mod.run_container_argv = run mod.run_container_argv = run
@@ -65,7 +65,7 @@ class TestMacosOrchestratorRun(unittest.TestCase):
def test_start_failure_raises(self) -> None: def test_start_failure_raises(self) -> None:
with patch(f"{_ORCH}.container_mod") as mod, \ with patch(f"{_ORCH}.container_mod") as mod, \
patch(f"{_ORCH}.host_orchestrator_token", return_value="k"): patch("bot_bottle.trust_domain.host_signing_key", return_value="k"):
mod.dns_server.return_value = "1.1.1.1" mod.dns_server.return_value = "1.1.1.1"
mod.run_container_argv = Mock(return_value=_fail()) mod.run_container_argv = Mock(return_value=_fail())
with self.assertRaises(OrchestratorStartError): with self.assertRaises(OrchestratorStartError):
+21
View File
@@ -82,5 +82,26 @@ class TestMintVerify(unittest.TestCase):
mint(ROLE_GATEWAY, "") mint(ROLE_GATEWAY, "")
class TestCustomRoleSet(unittest.TestCase):
"""The `roles=` seam a trust domain other than the control plane uses (#476):
mint/verify are scoped to the passed role set, not the module default."""
_ROLES = frozenset({"host"})
def test_round_trips_a_role_in_the_custom_set(self) -> None:
tok = mint("host", _KEY, roles=self._ROLES)
self.assertEqual("host", verify(tok, _KEY, roles=self._ROLES))
def test_mint_rejects_a_role_outside_the_custom_set(self) -> None:
with self.assertRaises(ValueError):
mint(ROLE_CLI, _KEY, roles=self._ROLES)
def test_verify_rejects_a_role_outside_the_verifiers_set(self) -> None:
# A validly signed token whose role isn't in the verifier's set fails —
# this is what keeps one domain's key from asserting another's role.
tok = mint(ROLE_CLI, _KEY) # a control-plane `cli` token
self.assertIsNone(verify(tok, _KEY, roles=self._ROLES))
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()
+4 -2
View File
@@ -20,13 +20,15 @@ _URLOPEN = "bot_bottle.orchestrator.client.urllib.request.urlopen"
class TestHostAuthToken(unittest.TestCase): class TestHostAuthToken(unittest.TestCase):
def test_mints_a_cli_token_from_the_host_key(self) -> None: def test_mints_a_cli_token_from_the_host_key(self) -> None:
with patch("bot_bottle.orchestrator.client.host_orchestrator_token", # The CLI mints its `cli` token from the control-plane trust domain's
# host-canonical key (#476) — patch the key file read underneath it.
with patch("bot_bottle.trust_domain.host_signing_key",
return_value="signing-key"): return_value="signing-key"):
tok = _host_auth_token() tok = _host_auth_token()
self.assertEqual(ROLE_CLI, verify(tok, "signing-key")) self.assertEqual(ROLE_CLI, verify(tok, "signing-key"))
def test_returns_empty_when_key_unreadable(self) -> None: def test_returns_empty_when_key_unreadable(self) -> None:
with patch("bot_bottle.orchestrator.client.host_orchestrator_token", with patch("bot_bottle.trust_domain.host_signing_key",
side_effect=OSError("no host root")): side_effect=OSError("no host root")):
self.assertEqual("", _host_auth_token()) self.assertEqual("", _host_auth_token())
+105
View File
@@ -0,0 +1,105 @@
"""Unit: trust domains + the shared control-plane provisioning contract (#476)."""
from __future__ import annotations
import unittest
from unittest.mock import patch
from bot_bottle import orchestrator_auth
from bot_bottle.orchestrator_auth import ROLE_CLI, ROLE_GATEWAY
from bot_bottle.trust_domain import (
CONTROL_PLANE,
ControlPlaneProvisioning,
ProvisioningError,
TrustDomain,
)
# A second, unrelated domain — the shape #468's host controller would take: its
# own key file, its own role, its own env vars. Distinct from CONTROL_PLANE.
_HOST_CTRL = TrustDomain(
name="host-controller",
key_filename="host-controller-token",
roles=frozenset({"host"}),
key_env="BOT_BOTTLE_HOST_CONTROLLER_TOKEN",
token_env="BOT_BOTTLE_HOST_CONTROLLER_JWT",
)
class TestTrustDomainMintVerify(unittest.TestCase):
def test_mint_verify_round_trips_within_a_domain(self) -> None:
with patch("bot_bottle.trust_domain.host_signing_key", return_value="k"):
tok = CONTROL_PLANE.mint(ROLE_GATEWAY)
self.assertEqual(ROLE_GATEWAY, CONTROL_PLANE.verify(tok, "k"))
def test_mint_rejects_a_role_outside_the_domain(self) -> None:
# `host` is a valid role in _HOST_CTRL but not in the control plane —
# the control-plane key must refuse to mint it.
with patch("bot_bottle.trust_domain.host_signing_key", return_value="k"):
with self.assertRaises(ValueError):
CONTROL_PLANE.mint("host")
def test_a_custom_role_set_verifies_its_own_role(self) -> None:
with patch("bot_bottle.trust_domain.host_signing_key", return_value="k"):
tok = _HOST_CTRL.mint("host")
self.assertEqual("host", _HOST_CTRL.verify(tok, "k"))
def test_key_from_env_reads_the_domains_env_var(self) -> None:
self.assertEqual(
"secret", CONTROL_PLANE.key_from_env({CONTROL_PLANE.key_env: " secret "}))
self.assertEqual("", CONTROL_PLANE.key_from_env({}))
class TestDomainBoundary(unittest.TestCase):
"""The #476/#468 invariant: two domains, two keys, two role sets — one
domain's key can neither mint nor verify the other's tokens."""
def test_a_control_plane_token_does_not_verify_under_another_domains_key(
self,
) -> None:
# Even with an IDENTICAL underlying key, a `cli` token minted under the
# control-plane domain must not verify as a role in the host-controller
# domain — the role isn't in that domain's set.
cli_tok = orchestrator_auth.mint(ROLE_CLI, "shared-bytes")
self.assertIsNone(_HOST_CTRL.verify(cli_tok, "shared-bytes"))
def test_distinct_keys_do_not_cross_verify(self) -> None:
# The realistic case: distinct host-canonical keys per domain. A token
# signed by one key never verifies under the other.
host_tok = orchestrator_auth.mint(
"host", "host-ctrl-key", roles=_HOST_CTRL.roles)
self.assertIsNone(_HOST_CTRL.verify(host_tok, "control-plane-key"))
self.assertEqual("host", _HOST_CTRL.verify(host_tok, "host-ctrl-key"))
class TestControlPlaneProvisioning(unittest.TestCase):
def test_orchestrator_key_returns_the_canonical_key(self) -> None:
prov = ControlPlaneProvisioning()
with patch("bot_bottle.trust_domain.host_signing_key", return_value="key"):
self.assertEqual("key", prov.orchestrator_key())
def test_orchestrator_key_fail_closes_when_empty(self) -> None:
# Invariant 4: the orchestrator must never start without a key — it would
# run OPEN and grant every caller that reaches it full `cli`. There is no
# topology opt-out: a separate host does not stop the gateway (or any
# other caller) from reaching the control-plane listener.
prov = ControlPlaneProvisioning()
with patch("bot_bottle.trust_domain.host_signing_key", return_value=""):
with self.assertRaises(ProvisioningError):
prov.orchestrator_key()
def test_gateway_token_is_a_verifiable_gateway_role_token(self) -> None:
prov = ControlPlaneProvisioning()
with patch("bot_bottle.trust_domain.host_signing_key", return_value="k"):
tok = prov.gateway_token()
self.assertEqual(ROLE_GATEWAY, CONTROL_PLANE.verify(tok, "k"))
def test_gateway_token_cannot_be_reused_as_cli(self) -> None:
# The data plane's token is `gateway`-scoped: it never carries `cli`.
prov = ControlPlaneProvisioning()
with patch("bot_bottle.trust_domain.host_signing_key", return_value="k"):
tok = prov.gateway_token()
self.assertNotEqual(ROLE_CLI, CONTROL_PLANE.verify(tok, "k"))
if __name__ == "__main__":
unittest.main()