Compare commits

..

5 Commits

Author SHA1 Message Date
didericis-claude 10c46fc584 fix: make installed wheel self-contained + harden install.sh prereqs
tracker-policy-pr / check-pr (pull_request) Successful in 14s
test / integration-docker (pull_request) Successful in 16s
lint / lint (push) Successful in 58s
test / unit (pull_request) Successful in 2m19s
test / integration-firecracker (pull_request) Successful in 3m49s
test / coverage (pull_request) Successful in 16s
test / publish-infra (pull_request) Has been skipped
Addresses the review on PR #481.

Self-contained wheel (review point 1): the gateway/infra/orchestrator
images build from a context that must hold bot_bottle/, pyproject.toml,
and the root-level Dockerfiles. Modules previously located these by
walking __file__ to the repo root, so an installed wheel (package in
site-packages, no repo root) passed `doctor` but failed `start`.

- Add bot_bottle/resources.py: build_root() returns the repo root in a
  checkout (unchanged) or a staged copy from the wheel's bundled
  _resources/ otherwise; dockerfile()/nix_netpool_module()/
  netpool_script() derive from it.
- setup.py bundles the root Dockerfiles, nix module, netpool script, and
  pyproject.toml into bot_bottle/_resources/ at build; MANIFEST.in ships
  them in the sdist.
- Route every _REPO_ROOT/_REPO_DIR call site (docker/macos launch, macos
  infra, firecracker infra_vm/infra_artifact/setup, orchestrator
  lifecycle/gateway) through resources. Checkout behavior is unchanged.

install.sh prerequisites (review point 2): check for git when installing
a git+ spec, and — before the pip fallback — that pip is usable and the
interpreter isn't externally managed (PEP 668), pointing at pipx.

Tests: test_resources covers checkout + staged-wheel layouts;
test_wheel_install builds the wheel, installs it into an isolated venv,
and asserts `doctor` runs and build_root() yields a valid context.
Running `start` end-to-end still needs a Docker/KVM host (CI).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-26 01:50:10 +00:00
didericis-claude 18dbdb828c feat: add quick install script and packaging (#197)
Give bot-bottle a real distribution path so new users can install
without cloning the repo:

- pyproject.toml: full project metadata, a `bot-bottle` console-script
  entry point (bot_bottle.cli:main), and package-data for the runtime
  assets (Dockerfiles, egress entrypoint, netpool defaults, macos init).
  Still zero runtime pip dependencies.
- install.sh: POSIX, sudo-free, idempotent bootstrapper — checks Python
  >= 3.11, creates ~/.bot-bottle/{agents,bottles,contrib}, installs via
  pipx (pip --user fallback), then runs `bot-bottle doctor`.
- `bot-bottle doctor`: new store-free subcommand reporting Python
  version, backend availability (reuses is_backend_available rather than
  hardcoding Docker), and config-dir presence. Exits non-zero when a hard
  prerequisite is unmet.
- PRD prd-new-install-script and unit tests for doctor, the packaging
  contract, and the install script.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-26 01:49:31 +00:00
didericis-claude e53104d5c1 refactor(orchestrator): fail closed unconditionally, drop topology opt-out
tracker-policy-pr / check-pr (pull_request) Successful in 7s
test / integration-docker (pull_request) Successful in 15s
test / unit (pull_request) Successful in 39s
test / integration-firecracker (pull_request) Successful in 3m55s
test / coverage (pull_request) Successful in 22s
test / publish-infra (pull_request) Has been skipped
prd-number / assign-numbers (push) Failing after 21s
test / integration-docker (push) Successful in 22s
lint / lint (push) Successful in 56s
Update Quality Badges / update-badges (push) Successful in 53s
test / unit (push) Successful in 1m52s
test / integration-firecracker (push) Successful in 5m2s
test / coverage (push) Successful in 16s
test / publish-infra (push) Successful in 1m56s
Remove the empty-key opt-out codex flagged: ControlPlaneProvisioning
no longer lets the orchestrator start without a signing key when a
backend declares an "isolated" topology. Host separation is not the
safety condition — the gateway (and any other caller) must reach the
control-plane listener for /resolve, so an open orchestrator would
still grant them full `cli`. The signing key is now mandatory for
every backend.

Drops the now-purposeless Topology/COLOCATED machinery (no backend
declared a non-default topology) and the contractual
test_orchestrator_key_allows_empty_when_isolated. Updates the PRD's
invariant 4 and lifecycle docstring accordingly. Docs/behaviour only
otherwise.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-26 01:40:19 +00:00
didericis-claude 8f6148d571 docs(orchestrator): tighten auth-provisioning wording to concrete services
test / integration-docker (pull_request) Successful in 18s
tracker-policy-pr / check-pr (pull_request) Successful in 21s
test / integration-firecracker (pull_request) Successful in 3m51s
test / unit (pull_request) Failing after 11m53s
test / coverage (pull_request) Has been skipped
test / publish-infra (pull_request) Has been skipped
Address review on #482: drop the generic "credential boundary" framing in the
PRD and docstrings and talk about the specific services — the orchestrator holds
the control-plane key; the host controller (#468) gets a separate key the
orchestrator never holds, so the orchestrator can't mint the credentials it uses
to talk to the host controller that owns its lifecycle. Lead with that concrete
win. No code behaviour change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-26 01:32:53 +00:00
didericis-claude 45f3cefbc5 refactor(orchestrator): uniform control-plane auth provisioning per trust domain
Hoist control-plane auth provisioning out of the per-backend launchers into
one shared contract, parameterized per trust domain (#476). Every blocking
finding in PR #471 was the same integration-bug class: each launcher
re-derived, by hand, how to generate the signing key, scope it to the
orchestrator, mint the gateway JWT, and keep the host key canonical.

Introduces `trust_domain.py`:

  * `TrustDomain` — one credential boundary (host-canonical key file + role
    set + env vars). `mint`/`verify` are scoped to the domain's roles, so a
    future host-controller domain (#468) uses its own key/verifier/roles
    rather than a `host` role on the control plane's frozenset (which the
    orchestrator key could then forge).
  * `ControlPlaneProvisioning` — the single seam answering the four
    invariants: host-canonical key, split key-vs-token credential, CLI token
    valid across co-running backends, and fail-closed (no open mode) for any
    co-located topology.
  * `Topology` — the backend declares what it is; the default is co-located +
    fail-closed, so a backend need not redeclare it.

The `Orchestrator` ABC gets `control_plane_key()` (fail-closed) and routes
`mint_gateway_token()` through the contract; docker/macOS/firecracker
orchestrators, the server (verify), and the host CLI client (mint cli) all go
through the domain instead of reading the host key directly. `orchestrator_auth`
gains an optional `roles=` arg (default unchanged) so a domain scopes its own
role set; `paths.host_signing_key(filename)` generalizes host_orchestrator_token.

Adds unit coverage for the domain boundary + provisioning invariants and a PRD
capturing the durable rationale. No change to the auth primitive's HMAC, the
plane split, or the server's documented open-mode fallback.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Closes #476
2026-07-26 01:32:53 +00:00
16 changed files with 440 additions and 48 deletions
+3 -2
View File
@@ -20,7 +20,6 @@ from .util import run_docker
from ...paths import (
ORCHESTRATOR_TOKEN_ENV,
bot_bottle_root,
host_orchestrator_token,
)
from ...gateway import GatewayError
from ...orchestrator.lifecycle import (
@@ -172,7 +171,9 @@ class DockerOrchestrator(Orchestrator):
fixed-name container first)."""
self._ensure_control_network()
run_docker(["docker", "rm", "--force", self.name])
_signing_key = host_orchestrator_token()
# The signing key comes through the shared provisioning contract (#476),
# which fail-closes rather than yield an empty key that would run OPEN.
_signing_key = self.control_plane_key()
proc = run_docker([
"docker", "run", "--detach",
"--name", self.name,
@@ -21,7 +21,6 @@ import time
from pathlib import Path
from ...log import die, info
from ...paths import host_orchestrator_token
from ...orchestrator.lifecycle import (
DEFAULT_STARTUP_TIMEOUT_SECONDS,
Orchestrator,
@@ -108,11 +107,13 @@ class FirecrackerOrchestrator(Orchestrator):
data_drive=self._ensure_registry_volume(),
)
# Push the host-canonical signing key (the init waits for it before
# starting the control plane). The host token file stays the single
# source of truth, so a co-running docker/macOS control plane keeps
# working; the guest verifies tokens with the same key the CLI signs from.
# starting the control plane). It comes through the shared provisioning
# contract (#476) — the same host token file every backend uses, so a
# co-running docker/macOS control plane keeps working and the guest
# verifies tokens with the same key the CLI signs from; fail-closed, so
# the guest is never handed an empty key that would run it OPEN.
infra_vm.push_secret(
vm, host_orchestrator_token(), infra_vm._GUEST_SIGNING_KEY_PATH,
vm, self.control_plane_key(), infra_vm._GUEST_SIGNING_KEY_PATH,
"the control-plane signing key to the orchestrator VM "
"(its control plane will not start)",
)
@@ -19,10 +19,7 @@ from pathlib import Path
from ... import log
from ... import resources
from ...paths import (
ORCHESTRATOR_TOKEN_ENV,
host_orchestrator_token,
)
from ...paths import ORCHESTRATOR_TOKEN_ENV
from ...orchestrator.lifecycle import (
DEFAULT_HEALTH_TIMEOUT_SECONDS,
DEFAULT_PORT,
@@ -136,7 +133,9 @@ class MacosOrchestrator(Orchestrator):
def _run_container(self, current_hash: str) -> None:
container_mod.force_remove_container(self.name)
_signing_key = host_orchestrator_token()
# The signing key comes through the shared provisioning contract (#476),
# which fail-closes rather than yield an empty key that would run OPEN.
_signing_key = self.control_plane_key()
argv = [
"container", "run", "--detach",
"--name", self.name,
+3 -3
View File
@@ -18,8 +18,8 @@ import urllib.request
from collections.abc import Iterable
from dataclasses import dataclass
from ..orchestrator_auth import ROLE_CLI, mint
from ..paths import host_orchestrator_token
from ..orchestrator_auth import ROLE_CLI
from ..trust_domain import CONTROL_PLANE
from .server import ORCHESTRATOR_AUTH_HEADER
DEFAULT_TIMEOUT_SECONDS = 5.0
@@ -32,7 +32,7 @@ def _host_auth_token() -> str:
"" means 'send no auth header' — correct against an open (unconfigured)
control plane, and harmlessly rejected by a secured one."""
try:
return mint(ROLE_CLI, host_orchestrator_token())
return CONTROL_PLANE.mint(ROLE_CLI)
except (OSError, ValueError):
return ""
+19 -5
View File
@@ -21,8 +21,7 @@ import urllib.error
import urllib.request
from pathlib import Path
from ..orchestrator_auth import ROLE_GATEWAY, mint
from ..paths import host_orchestrator_token
from ..trust_domain import ControlPlaneProvisioning
DEFAULT_PORT = 8099
DEFAULT_STARTUP_TIMEOUT_SECONDS = 45.0
@@ -58,6 +57,12 @@ class Orchestrator(abc.ABC):
so it — not the gateway — mints the gateway's role-scoped token.
Backend-neutral."""
# The shared control-plane auth provisioning contract (#476). Every backend
# gets its signing key + gateway token through this one seam rather than
# re-deriving the wiring; it is fail-closed for every backend — the
# orchestrator never starts without its signing key.
provisioning: ControlPlaneProvisioning = ControlPlaneProvisioning()
def ensure_built(self) -> None:
"""Ensure the orchestrator's image / rootfs exists, building it if
needed. Default: nothing to build (e.g. a pre-pulled image). Call before
@@ -105,9 +110,17 @@ class Orchestrator(abc.ABC):
def mint_gateway_token(self) -> str:
"""Mint a role-scoped `gateway` JWT from the host signing key for the
gateway to present. The orchestrator holds the key; the gateway never
does (#469). Backend-neutral — the same host token file is the single
source of truth across backends."""
return mint(ROLE_GATEWAY, host_orchestrator_token())
does (#469). Routed through the shared provisioning contract (#476), so
the same host token file is the single source of truth across backends."""
return self.provisioning.gateway_token()
def control_plane_key(self) -> str:
"""The raw signing key the control-plane *process* must receive — the ONE
place a backend obtains it (docker/macOS inject it as `key_env`;
firecracker pushes it to the guest). Fail-closed via the provisioning
contract: it raises rather than yield an empty key that would run the
server OPEN (#476)."""
return self.provisioning.orchestrator_key()
__all__ = [
@@ -117,4 +130,5 @@ __all__ = [
"OrchestratorStartError",
"source_hash",
"Orchestrator",
"ControlPlaneProvisioning",
]
+7 -5
View File
@@ -63,8 +63,8 @@ import sys
import typing
from urllib.parse import urlsplit
from ..orchestrator_auth import ROLE_CLI, ROLES, verify
from ..paths import ORCHESTRATOR_TOKEN_ENV
from ..orchestrator_auth import ROLE_CLI, ROLES
from ..trust_domain import CONTROL_PLANE
from ..supervisor.types import TOOLS
from .service import OrchestratorCore
@@ -413,11 +413,13 @@ class OrchestratorServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
def __init__(self, address: tuple[str, int], orchestrator: OrchestratorCore) -> None:
self.orchestrator = orchestrator
self._signing_key = os.environ.get(ORCHESTRATOR_TOKEN_ENV, "").strip()
# The control-plane trust domain's signing key, as injected into THIS
# (the owning) process by the launcher (#476). Unset → open mode below.
self._signing_key = CONTROL_PLANE.key_from_env()
if not self._signing_key:
sys.stderr.write(
"orchestrator: WARNING — no control-plane signing key "
f"(${ORCHESTRATOR_TOKEN_ENV}); running WITHOUT caller "
f"(${CONTROL_PLANE.key_env}); running WITHOUT caller "
"authentication. Any client that can reach this port can drive "
"it. Backends that put the control plane on an agent-reachable "
"network MUST set this.\n"
@@ -433,7 +435,7 @@ class OrchestratorServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
role (→ per-route 401/403 in `dispatch`)."""
if not self._signing_key:
return ROLE_CLI
return verify(presented, self._signing_key)
return CONTROL_PLANE.verify(presented, self._signing_key)
def make_server(
+14 -8
View File
@@ -59,12 +59,17 @@ _HEADER_SEGMENT = _b64url_encode(
)
def mint(role: str, secret: str) -> str:
def mint(role: str, secret: str, *, roles: frozenset[str] = ROLES) -> str:
"""A compact HS256 token asserting `role`, signed with `secret`.
Raises ValueError for an unknown role (mint only what the control plane will
accept) or an empty signing key (an unsigned credential is never valid)."""
if role not in ROLES:
`roles` is the set the signing key is allowed to sign (default: the
orchestrator's `{gateway, cli}`). A separate service (e.g. the host
controller) passes its own key + role set so its tokens can't be forged with
the orchestrator's key — see `trust_domain.py`, issues #476/#468.
Raises ValueError for a role outside `roles`, or an empty signing key (an
unsigned credential is never valid)."""
if role not in roles:
raise ValueError(f"unknown control-plane role {role!r}")
if not secret:
raise ValueError("cannot mint a control-plane token without a signing key")
@@ -73,10 +78,11 @@ def mint(role: str, secret: str) -> str:
return f"{signing_input}.{_sign(secret, signing_input)}"
def verify(token: str, secret: str) -> str | None:
def verify(token: str, secret: str, *, roles: frozenset[str] = ROLES) -> str | None:
"""The role a valid `token` carries, or None if it is malformed, wrongly
signed, or names an unknown role. Constant-time signature check; rejects any
header whose alg isn't HS256 (no alg-confusion / `none`)."""
signed, or names a role outside `roles` (the verifying trust domain's set —
default `{gateway, cli}`). Constant-time signature check; rejects any header
whose alg isn't HS256 (no alg-confusion / `none`)."""
if not token or not secret:
return None
parts = token.split(".")
@@ -94,7 +100,7 @@ def verify(token: str, secret: str) -> str | None:
if not isinstance(header, dict) or header.get("alg") != _ALG:
return None
role = payload.get("role") if isinstance(payload, dict) else None
return role if isinstance(role, str) and role in ROLES else None
return role if isinstance(role, str) and role in roles else None
__all__ = ["ROLE_GATEWAY", "ROLE_CLI", "ROLES", "mint", "verify"]
+19 -9
View File
@@ -97,16 +97,17 @@ def host_gateway_ca_dir() -> Path:
return ca_dir
def host_orchestrator_token() -> str:
"""The per-host control-plane secret, minted (256-bit, url-safe) and
persisted 0600 on first use, then reused.
def host_signing_key(filename: str) -> str:
"""A per-host signing key at `<root>/<filename>`, minted (256-bit, url-safe)
and persisted 0600 on first use, then reused.
This is the shared secret the launchers inject into the control-plane and
gateway containers and that the host CLI presents on every call. It is a
*host* artifact — the file lives under the root the agent never mounts, and
the env var is set only on the trusted containers — so reading it here is
safe on the host launch path but the value never reaches a bottle."""
path = bot_bottle_root() / ORCHESTRATOR_TOKEN_FILENAME
The generic form of `host_orchestrator_token()`: each service names its own
key file (`trust_domain.py`), so the orchestrator and a separate service like
the host controller (#468) get distinct keys neither can read. It is a *host*
artifact — the file lives under the root the agent never mounts, and its value
is injected only into the trusted control-plane process — so reading it here
is safe on the launch path but the value never reaches a bottle."""
path = bot_bottle_root() / filename
try:
existing = path.read_text().strip()
if existing:
@@ -128,6 +129,14 @@ def host_orchestrator_token() -> str:
return token
def host_orchestrator_token() -> str:
"""The per-host control-plane signing key — the host-canonical key the
launchers inject into the control-plane process and the host CLI mints its
own `cli` token from. The `control-plane` trust domain's specialization of
`host_signing_key()`."""
return host_signing_key(ORCHESTRATOR_TOKEN_FILENAME)
__all__ = [
"HOST_DB_FILENAME",
"ORCHESTRATOR_TOKEN_FILENAME",
@@ -138,5 +147,6 @@ __all__ = [
"host_db_path",
"host_db_dir",
"host_gateway_ca_dir",
"host_signing_key",
"host_orchestrator_token",
]
+140
View File
@@ -0,0 +1,140 @@
"""Per-service control-plane signing keys (issue #476).
A `TrustDomain` is one service's signing material: its host-canonical key file,
the roles that key may sign, and the env vars its key and a pre-minted token ride
in. Scoping `mint`/`verify` to a domain's roles keeps one service's key from
signing (or accepting) another service's tokens.
Today there is one domain, `CONTROL_PLANE` — the orchestrator's key (roles
`{gateway, cli}`): the orchestrator holds it and mints the gateway's and CLI's
tokens. The host controller (#468) will add a **second** domain with its own key
the orchestrator never holds. That is the point: the host controller starts and
stops the orchestrator, so the orchestrator must not be able to mint the
credentials it uses to talk to it. Adding a `host` role to `CONTROL_PLANE`
instead would defeat that — the orchestrator holds that key, so it could forge
`host` tokens.
`ControlPlaneProvisioning` is the one seam every backend launcher uses to get the
orchestrator its key and the gateway its token, instead of re-deriving that
wiring per backend (the bug class behind PR #471 — see
`docs/prds/prd-new-control-plane-auth-provisioning.md`).
Stdlib-only: the HMAC lives in `orchestrator_auth`, the key file in `paths`.
"""
from __future__ import annotations
import os
from collections.abc import Mapping
from dataclasses import dataclass
from . import orchestrator_auth
from .orchestrator_auth import ROLE_GATEWAY
from .paths import (
ORCHESTRATOR_AUTH_JWT_ENV,
ORCHESTRATOR_TOKEN_ENV,
ORCHESTRATOR_TOKEN_FILENAME,
host_signing_key,
)
class ProvisioningError(RuntimeError):
"""A control-plane auth invariant would be violated (e.g. starting the
orchestrator without its signing key — which would run OPEN)."""
@dataclass(frozen=True)
class TrustDomain:
"""One service's signing material: a host-canonical key file, the roles that
key may sign, and the env vars its key and a minted token ride in.
The service that *owns* the domain (e.g. the orchestrator) receives the raw
key via `key_env`; a delegate (e.g. the gateway) receives only a pre-minted,
role-scoped token via `token_env` it cannot rewrite. `mint`/`verify` are
scoped to `roles`, so this service's key can neither sign nor accept another
service's role."""
name: str
key_filename: str
roles: frozenset[str]
key_env: str
token_env: str
def signing_key(self) -> str:
"""This service's host-canonical signing key (minted 0600 on first use).
Host-side only — the value is injected into the owning process."""
return host_signing_key(self.key_filename)
def key_from_env(self, environ: Mapping[str, str] | None = None) -> str:
"""The signing key as the owning process sees it — read from `key_env`
(default `os.environ`). "" when unset; the caller decides whether that is
fatal (`ControlPlaneProvisioning`) or the open-mode fallback
(`OrchestratorServer`)."""
env = os.environ if environ is None else environ
return env.get(self.key_env, "").strip()
def mint(self, role: str) -> str:
"""A role-scoped token for a delegate, signed with this service's key.
Raises ValueError for a role this service doesn't sign."""
if role not in self.roles:
raise ValueError(f"role {role!r} is not in trust domain {self.name!r}")
return orchestrator_auth.mint(role, self.signing_key(), roles=self.roles)
def verify(self, token: str, key: str) -> str | None:
"""The role `token` carries under `key`, or None. `key` is passed in
(not read from disk) because the verifier — the control-plane process —
holds it in `key_env`, not on disk in its guest."""
return orchestrator_auth.verify(token, key, roles=self.roles)
# The orchestrator's domain: the key the orchestrator (and host CLI) holds, the
# `gateway` token it mints for the data plane, and the `cli` token the CLI mints
# for itself. #468's host controller will add a second, separate domain.
CONTROL_PLANE = TrustDomain(
name="control-plane",
key_filename=ORCHESTRATOR_TOKEN_FILENAME,
roles=orchestrator_auth.ROLES,
key_env=ORCHESTRATOR_TOKEN_ENV,
token_env=ORCHESTRATOR_AUTH_JWT_ENV,
)
@dataclass(frozen=True)
class ControlPlaneProvisioning:
"""The one seam every backend launcher uses to provision control-plane auth,
instead of re-deriving the four invariants that each cost a PR #471 review
round: the orchestrator gets the raw key (`orchestrator_key`), the gateway
gets a minted `gateway` token (`gateway_token`), the host CLI mints its own
`cli` token from the same host-canonical key, and the orchestrator never
starts open."""
domain: TrustDomain = CONTROL_PLANE
def orchestrator_key(self) -> str:
"""The raw signing key the orchestrator process must receive (carry it in
`domain.key_env`). Fail-closed: raises rather than return "", since an
empty key runs the server open — and being on a separate host does not
stop the gateway from reaching the control plane (it must, for
`/resolve`), so an open orchestrator would treat that gateway as `cli`."""
key = self.domain.signing_key()
if not key:
raise ProvisioningError(
f"refusing to start the {self.domain.name} orchestrator without "
"a signing key: an open orchestrator authenticates no one and "
"grants every caller that reaches it full `cli` (#476)"
)
return key
def gateway_token(self) -> str:
"""The `gateway`-role token the gateway receives (carry it in
`domain.token_env`) — minted from the key, never the key itself, so a
compromised gateway cannot forge a `cli` token."""
return self.domain.mint(ROLE_GATEWAY)
__all__ = [
"ProvisioningError",
"TrustDomain",
"CONTROL_PLANE",
"ControlPlaneProvisioning",
]
@@ -0,0 +1,89 @@
# PRD prd-new: Per-service signing keys for control-plane auth
- **Status:** Draft
- **Author:** claude
- **Created:** 2026-07-26
- **Issue:** #476
## Summary
Provision control-plane signing keys **per service**, through one shared seam, so
no service can mint another's credentials. Concretely: the orchestrator holds the
control-plane key and mints the gateway's and CLI's tokens; the host controller
(#468, next) gets a **separate** key the orchestrator never holds — so the
orchestrator cannot forge the credentials it uses to talk to the host controller
that starts and stops it. Landing this seam also retires the per-backend
provisioning duplication that made PR #471 take three review rounds.
## Problem
**1. The orchestrator could forge host-controller credentials.** The
orchestrator's key signs roles `{gateway, cli}`. The tempting way to add the host
controller (#468) is a third role, `host`, on that same key. But then the
orchestrator — which holds the key — can mint `host` tokens, and the host
controller, which owns the orchestrator's lifecycle, must not trust anything the
orchestrator can mint. The two services need separate keys.
**2. Every backend provisioned auth by hand.** Each launcher (docker
gateway/infra, macOS infra, firecracker infra) re-derived how to generate the
signing key, scope it to the orchestrator, mint the gateway JWT, and keep the
host key file canonical. All three PR #471 High-severity findings were this one
integration bug in different launchers: the data plane got the full `cli` token;
the firecracker control plane ran open; the firecracker guest clobbered the host
key.
## Goals / Success Criteria
- The orchestrator and the host controller sign with **different** keys; neither
can mint the other's tokens. (This PR provisions the orchestrator's key and
leaves a drop-in seam for the host controller's.)
- One shared provisioning seam every backend uses — a new backend or daemon
implements it instead of rediscovering these four invariants:
1. the signing key is host-canonical: a guest is handed it, never generates or
overwrites it;
2. only the orchestrator process gets the raw key; the gateway gets a
pre-minted `gateway` token it can't rewrite into `cli`;
3. the host CLI's `cli` token is minted from the same key, so it stays valid
across co-running backends;
4. the orchestrator never runs open — the signing key is mandatory, with no
topology opt-out (a separate host does not stop a caller from reaching the
control-plane listener, so it cannot make open mode safe).
## Non-goals
- The host controller itself (#468) — this only provisions the orchestrator's
key and the seam #468 plugs into.
- Rewriting the HMAC primitive: `orchestrator_auth.mint/verify` gain an optional
`roles=` arg (default unchanged) so a key can carry a different role set;
nothing else changes.
- Network topology, the plane split (#469), or the server's open-mode fallback
for tests.
## Design
A **`TrustDomain`** is one service's signing material: its host-canonical key
file, the roles that key may sign, and the env vars its key and a pre-minted
token ride in. `mint`/`verify` are scoped to that domain's roles, so a token
signed by one service's key neither carries nor verifies another service's role.
- `CONTROL_PLANE` — the orchestrator's domain: key `orchestrator-token`, roles
`{gateway, cli}`. The orchestrator process holds the key; the gateway holds
only a minted `gateway` token; the host CLI mints its own `cli` token.
- The host controller (#468) will add a second `TrustDomain` — its own key file
and role(s) — that the orchestrator never holds.
**`ControlPlaneProvisioning`** is the seam the backends call.
`orchestrator_key()` returns the raw key for the control-plane process
(fail-closed for every backend: it raises rather than hand back an empty key that
would run the server open). `gateway_token()` mints the gateway's token. Each backend applies these through its own transport —
docker/macOS inject env vars, firecracker pushes over SSH — but none re-derives
*which* key or role.
`paths.host_signing_key(filename)` generalizes `host_orchestrator_token()` so each
domain names its own key file.
## Open questions
None blocking. #468 adds its `TrustDomain` and a second
`ControlPlaneProvisioning`-shaped consumer; renaming that class to something
service-neutral is a cosmetic call to make then.
+3 -1
View File
@@ -19,7 +19,9 @@ _ORCH = "bot_bottle.backend.docker.orchestrator"
_RUN = f"{_ORCH}.run_docker"
_SLEEP = f"{_ORCH}.time.sleep"
_MONOTONIC = f"{_ORCH}.time.monotonic"
_TOKEN = f"{_ORCH}.host_orchestrator_token"
# The signing key is read through the shared provisioning contract (#476); patch
# its host-canonical key file read to keep it off the real host file.
_TOKEN = "bot_bottle.trust_domain.host_signing_key"
# The ABC's is_healthy probes /health via urllib in the lifecycle module.
_URLOPEN = "bot_bottle.orchestrator.lifecycle.urllib.request.urlopen"
+1 -1
View File
@@ -44,7 +44,7 @@ class TestEnsureRunning(unittest.TestCase):
vm = infra_vm.InfraVm(guest_ip="10.243.255.1", private_key=Path("/k"), vm=MagicMock())
with patch.object(infra_vm, "boot_vm", return_value=vm) as boot, \
patch.object(infra_vm, "push_secret") as push, \
patch(f"{_ORCH}.host_orchestrator_token", return_value="host-key"), \
patch("bot_bottle.trust_domain.host_signing_key", return_value="host-key"), \
patch.object(orch, "is_running", return_value=False), \
patch.object(orch, "_ensure_registry_volume", return_value=Path("/reg")), \
patch.object(orch, "_wait_for_health"):
+2 -2
View File
@@ -33,7 +33,7 @@ class TestMacosOrchestratorRun(unittest.TestCase):
def _run(self) -> list[str]:
run = Mock(return_value=_ok())
with patch(f"{_ORCH}.container_mod") as mod, \
patch(f"{_ORCH}.host_orchestrator_token", return_value="k"):
patch("bot_bottle.trust_domain.host_signing_key", return_value="k"):
mod.dns_server.return_value = "1.1.1.1"
mod.bind_mount_spec.side_effect = _spec
mod.run_container_argv = run
@@ -65,7 +65,7 @@ class TestMacosOrchestratorRun(unittest.TestCase):
def test_start_failure_raises(self) -> None:
with patch(f"{_ORCH}.container_mod") as mod, \
patch(f"{_ORCH}.host_orchestrator_token", return_value="k"):
patch("bot_bottle.trust_domain.host_signing_key", return_value="k"):
mod.dns_server.return_value = "1.1.1.1"
mod.run_container_argv = Mock(return_value=_fail())
with self.assertRaises(OrchestratorStartError):
+21
View File
@@ -82,5 +82,26 @@ class TestMintVerify(unittest.TestCase):
mint(ROLE_GATEWAY, "")
class TestCustomRoleSet(unittest.TestCase):
"""The `roles=` seam a trust domain other than the control plane uses (#476):
mint/verify are scoped to the passed role set, not the module default."""
_ROLES = frozenset({"host"})
def test_round_trips_a_role_in_the_custom_set(self) -> None:
tok = mint("host", _KEY, roles=self._ROLES)
self.assertEqual("host", verify(tok, _KEY, roles=self._ROLES))
def test_mint_rejects_a_role_outside_the_custom_set(self) -> None:
with self.assertRaises(ValueError):
mint(ROLE_CLI, _KEY, roles=self._ROLES)
def test_verify_rejects_a_role_outside_the_verifiers_set(self) -> None:
# A validly signed token whose role isn't in the verifier's set fails —
# this is what keeps one domain's key from asserting another's role.
tok = mint(ROLE_CLI, _KEY) # a control-plane `cli` token
self.assertIsNone(verify(tok, _KEY, roles=self._ROLES))
if __name__ == "__main__":
unittest.main()
+4 -2
View File
@@ -20,13 +20,15 @@ _URLOPEN = "bot_bottle.orchestrator.client.urllib.request.urlopen"
class TestHostAuthToken(unittest.TestCase):
def test_mints_a_cli_token_from_the_host_key(self) -> None:
with patch("bot_bottle.orchestrator.client.host_orchestrator_token",
# The CLI mints its `cli` token from the control-plane trust domain's
# host-canonical key (#476) — patch the key file read underneath it.
with patch("bot_bottle.trust_domain.host_signing_key",
return_value="signing-key"):
tok = _host_auth_token()
self.assertEqual(ROLE_CLI, verify(tok, "signing-key"))
def test_returns_empty_when_key_unreadable(self) -> None:
with patch("bot_bottle.orchestrator.client.host_orchestrator_token",
with patch("bot_bottle.trust_domain.host_signing_key",
side_effect=OSError("no host root")):
self.assertEqual("", _host_auth_token())
+105
View File
@@ -0,0 +1,105 @@
"""Unit: trust domains + the shared control-plane provisioning contract (#476)."""
from __future__ import annotations
import unittest
from unittest.mock import patch
from bot_bottle import orchestrator_auth
from bot_bottle.orchestrator_auth import ROLE_CLI, ROLE_GATEWAY
from bot_bottle.trust_domain import (
CONTROL_PLANE,
ControlPlaneProvisioning,
ProvisioningError,
TrustDomain,
)
# A second, unrelated domain — the shape #468's host controller would take: its
# own key file, its own role, its own env vars. Distinct from CONTROL_PLANE.
_HOST_CTRL = TrustDomain(
name="host-controller",
key_filename="host-controller-token",
roles=frozenset({"host"}),
key_env="BOT_BOTTLE_HOST_CONTROLLER_TOKEN",
token_env="BOT_BOTTLE_HOST_CONTROLLER_JWT",
)
class TestTrustDomainMintVerify(unittest.TestCase):
def test_mint_verify_round_trips_within_a_domain(self) -> None:
with patch("bot_bottle.trust_domain.host_signing_key", return_value="k"):
tok = CONTROL_PLANE.mint(ROLE_GATEWAY)
self.assertEqual(ROLE_GATEWAY, CONTROL_PLANE.verify(tok, "k"))
def test_mint_rejects_a_role_outside_the_domain(self) -> None:
# `host` is a valid role in _HOST_CTRL but not in the control plane —
# the control-plane key must refuse to mint it.
with patch("bot_bottle.trust_domain.host_signing_key", return_value="k"):
with self.assertRaises(ValueError):
CONTROL_PLANE.mint("host")
def test_a_custom_role_set_verifies_its_own_role(self) -> None:
with patch("bot_bottle.trust_domain.host_signing_key", return_value="k"):
tok = _HOST_CTRL.mint("host")
self.assertEqual("host", _HOST_CTRL.verify(tok, "k"))
def test_key_from_env_reads_the_domains_env_var(self) -> None:
self.assertEqual(
"secret", CONTROL_PLANE.key_from_env({CONTROL_PLANE.key_env: " secret "}))
self.assertEqual("", CONTROL_PLANE.key_from_env({}))
class TestDomainBoundary(unittest.TestCase):
"""The #476/#468 invariant: two domains, two keys, two role sets — one
domain's key can neither mint nor verify the other's tokens."""
def test_a_control_plane_token_does_not_verify_under_another_domains_key(
self,
) -> None:
# Even with an IDENTICAL underlying key, a `cli` token minted under the
# control-plane domain must not verify as a role in the host-controller
# domain — the role isn't in that domain's set.
cli_tok = orchestrator_auth.mint(ROLE_CLI, "shared-bytes")
self.assertIsNone(_HOST_CTRL.verify(cli_tok, "shared-bytes"))
def test_distinct_keys_do_not_cross_verify(self) -> None:
# The realistic case: distinct host-canonical keys per domain. A token
# signed by one key never verifies under the other.
host_tok = orchestrator_auth.mint(
"host", "host-ctrl-key", roles=_HOST_CTRL.roles)
self.assertIsNone(_HOST_CTRL.verify(host_tok, "control-plane-key"))
self.assertEqual("host", _HOST_CTRL.verify(host_tok, "host-ctrl-key"))
class TestControlPlaneProvisioning(unittest.TestCase):
def test_orchestrator_key_returns_the_canonical_key(self) -> None:
prov = ControlPlaneProvisioning()
with patch("bot_bottle.trust_domain.host_signing_key", return_value="key"):
self.assertEqual("key", prov.orchestrator_key())
def test_orchestrator_key_fail_closes_when_empty(self) -> None:
# Invariant 4: the orchestrator must never start without a key — it would
# run OPEN and grant every caller that reaches it full `cli`. There is no
# topology opt-out: a separate host does not stop the gateway (or any
# other caller) from reaching the control-plane listener.
prov = ControlPlaneProvisioning()
with patch("bot_bottle.trust_domain.host_signing_key", return_value=""):
with self.assertRaises(ProvisioningError):
prov.orchestrator_key()
def test_gateway_token_is_a_verifiable_gateway_role_token(self) -> None:
prov = ControlPlaneProvisioning()
with patch("bot_bottle.trust_domain.host_signing_key", return_value="k"):
tok = prov.gateway_token()
self.assertEqual(ROLE_GATEWAY, CONTROL_PLANE.verify(tok, "k"))
def test_gateway_token_cannot_be_reused_as_cli(self) -> None:
# The data plane's token is `gateway`-scoped: it never carries `cli`.
prov = ControlPlaneProvisioning()
with patch("bot_bottle.trust_domain.host_signing_key", return_value="k"):
tok = prov.gateway_token()
self.assertNotEqual(ROLE_CLI, CONTROL_PLANE.verify(tok, "k"))
if __name__ == "__main__":
unittest.main()