Compare commits

...

4 Commits

Author SHA1 Message Date
didericis-claude 8f6148d571 docs(orchestrator): tighten auth-provisioning wording to concrete services
test / integration-docker (pull_request) Successful in 18s
tracker-policy-pr / check-pr (pull_request) Successful in 21s
test / integration-firecracker (pull_request) Successful in 3m51s
test / unit (pull_request) Failing after 11m53s
test / coverage (pull_request) Has been skipped
test / publish-infra (pull_request) Has been skipped
Address review on #482: drop the generic "credential boundary" framing in the
PRD and docstrings and talk about the specific services — the orchestrator holds
the control-plane key; the host controller (#468) gets a separate key the
orchestrator never holds, so the orchestrator can't mint the credentials it uses
to talk to the host controller that owns its lifecycle. Lead with that concrete
win. No code behaviour change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-26 01:32:53 +00:00
didericis-claude 45f3cefbc5 refactor(orchestrator): uniform control-plane auth provisioning per trust domain
Hoist control-plane auth provisioning out of the per-backend launchers into
one shared contract, parameterized per trust domain (#476). Every blocking
finding in PR #471 was the same integration-bug class: each launcher
re-derived, by hand, how to generate the signing key, scope it to the
orchestrator, mint the gateway JWT, and keep the host key canonical.

Introduces `trust_domain.py`:

  * `TrustDomain` — one credential boundary (host-canonical key file + role
    set + env vars). `mint`/`verify` are scoped to the domain's roles, so a
    future host-controller domain (#468) uses its own key/verifier/roles
    rather than a `host` role on the control plane's frozenset (which the
    orchestrator key could then forge).
  * `ControlPlaneProvisioning` — the single seam answering the four
    invariants: host-canonical key, split key-vs-token credential, CLI token
    valid across co-running backends, and fail-closed (no open mode) for any
    co-located topology.
  * `Topology` — the backend declares what it is; the default is co-located +
    fail-closed, so a backend need not redeclare it.

The `Orchestrator` ABC gets `control_plane_key()` (fail-closed) and routes
`mint_gateway_token()` through the contract; docker/macOS/firecracker
orchestrators, the server (verify), and the host CLI client (mint cli) all go
through the domain instead of reading the host key directly. `orchestrator_auth`
gains an optional `roles=` arg (default unchanged) so a domain scopes its own
role set; `paths.host_signing_key(filename)` generalizes host_orchestrator_token.

Adds unit coverage for the domain boundary + provisioning invariants and a PRD
capturing the durable rationale. No change to the auth primitive's HMAC, the
plane split, or the server's documented open-mode fallback.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Closes #476
2026-07-26 01:32:53 +00:00
Quality Badge Bot 0e70d26af4 chore: update quality badges
- Coverage: 83%
- Core coverage: 94%

[skip ci]
2026-07-25 23:45:18 +00:00
didericis-codex f2d8158742 Merge pull request 'fix(supervise): get bot-bottle.db off the data plane (supervise + egress proposals over RPC)' (#471) from fix/db-off-data-plane-469 into main
test / integration-docker (push) Successful in 35s
test / unit (push) Successful in 49s
Update Quality Badges / update-badges (push) Successful in 50s
lint / lint (push) Successful in 57s
test / integration-firecracker (push) Successful in 5m10s
test / coverage (push) Successful in 16s
test / publish-infra (push) Successful in 2m18s
2026-07-25 19:44:20 -04:00
17 changed files with 476 additions and 49 deletions
+1 -1
View File
@@ -5,7 +5,7 @@
# bot-bottle # bot-bottle
[![test](https://gitea.dideric.is/didericis/bot-bottle/actions/workflows/test.yml/badge.svg?branch=main)](https://gitea.dideric.is/didericis/bot-bottle/actions?workflow=test.yml) [![test](https://gitea.dideric.is/didericis/bot-bottle/actions/workflows/test.yml/badge.svg?branch=main)](https://gitea.dideric.is/didericis/bot-bottle/actions?workflow=test.yml)
[![coverage](https://img.shields.io/badge/coverage-84%25-brightgreen)](https://coverage.readthedocs.io/) [![coverage](https://img.shields.io/badge/coverage-83%25-brightgreen)](https://coverage.readthedocs.io/)
[![core coverage](https://img.shields.io/badge/core%20coverage-94%25-brightgreen)](https://gitea.dideric.is/didericis/bot-bottle/src/branch/main/docs/decisions/0004-coverage-policy.md) [![core coverage](https://img.shields.io/badge/core%20coverage-94%25-brightgreen)](https://gitea.dideric.is/didericis/bot-bottle/src/branch/main/docs/decisions/0004-coverage-policy.md)
**Problem:** Developer wants to run a coding agent without supervision, but they don't want a prompt injected or misbehaving agent wrecking their environment or exfiltrating sensitive data. **Problem:** Developer wants to run a coding agent without supervision, but they don't want a prompt injected or misbehaving agent wrecking their environment or exfiltrating sensitive data.
+3 -2
View File
@@ -19,7 +19,6 @@ from .util import run_docker
from ...paths import ( from ...paths import (
ORCHESTRATOR_TOKEN_ENV, ORCHESTRATOR_TOKEN_ENV,
bot_bottle_root, bot_bottle_root,
host_orchestrator_token,
) )
from ...gateway import GatewayError from ...gateway import GatewayError
from ...orchestrator.lifecycle import ( from ...orchestrator.lifecycle import (
@@ -171,7 +170,9 @@ class DockerOrchestrator(Orchestrator):
fixed-name container first).""" fixed-name container first)."""
self._ensure_control_network() self._ensure_control_network()
run_docker(["docker", "rm", "--force", self.name]) run_docker(["docker", "rm", "--force", self.name])
_signing_key = host_orchestrator_token() # The signing key comes through the shared provisioning contract (#476),
# which fail-closes rather than yield an empty key that would run OPEN.
_signing_key = self.control_plane_key()
proc = run_docker([ proc = run_docker([
"docker", "run", "--detach", "docker", "run", "--detach",
"--name", self.name, "--name", self.name,
@@ -21,7 +21,6 @@ import time
from pathlib import Path from pathlib import Path
from ...log import die, info from ...log import die, info
from ...paths import host_orchestrator_token
from ...orchestrator.lifecycle import ( from ...orchestrator.lifecycle import (
DEFAULT_STARTUP_TIMEOUT_SECONDS, DEFAULT_STARTUP_TIMEOUT_SECONDS,
Orchestrator, Orchestrator,
@@ -108,11 +107,13 @@ class FirecrackerOrchestrator(Orchestrator):
data_drive=self._ensure_registry_volume(), data_drive=self._ensure_registry_volume(),
) )
# Push the host-canonical signing key (the init waits for it before # Push the host-canonical signing key (the init waits for it before
# starting the control plane). The host token file stays the single # starting the control plane). It comes through the shared provisioning
# source of truth, so a co-running docker/macOS control plane keeps # contract (#476) — the same host token file every backend uses, so a
# working; the guest verifies tokens with the same key the CLI signs from. # co-running docker/macOS control plane keeps working and the guest
# verifies tokens with the same key the CLI signs from; fail-closed, so
# the guest is never handed an empty key that would run it OPEN.
infra_vm.push_secret( infra_vm.push_secret(
vm, host_orchestrator_token(), infra_vm._GUEST_SIGNING_KEY_PATH, vm, self.control_plane_key(), infra_vm._GUEST_SIGNING_KEY_PATH,
"the control-plane signing key to the orchestrator VM " "the control-plane signing key to the orchestrator VM "
"(its control plane will not start)", "(its control plane will not start)",
) )
@@ -18,10 +18,7 @@ import urllib.request
from pathlib import Path from pathlib import Path
from ... import log from ... import log
from ...paths import ( from ...paths import ORCHESTRATOR_TOKEN_ENV
ORCHESTRATOR_TOKEN_ENV,
host_orchestrator_token,
)
from ...orchestrator.lifecycle import ( from ...orchestrator.lifecycle import (
DEFAULT_HEALTH_TIMEOUT_SECONDS, DEFAULT_HEALTH_TIMEOUT_SECONDS,
DEFAULT_PORT, DEFAULT_PORT,
@@ -134,7 +131,9 @@ class MacosOrchestrator(Orchestrator):
def _run_container(self, current_hash: str) -> None: def _run_container(self, current_hash: str) -> None:
container_mod.force_remove_container(self.name) container_mod.force_remove_container(self.name)
_signing_key = host_orchestrator_token() # The signing key comes through the shared provisioning contract (#476),
# which fail-closes rather than yield an empty key that would run OPEN.
_signing_key = self.control_plane_key()
argv = [ argv = [
"container", "run", "--detach", "container", "run", "--detach",
"--name", self.name, "--name", self.name,
+3 -3
View File
@@ -18,8 +18,8 @@ import urllib.request
from collections.abc import Iterable from collections.abc import Iterable
from dataclasses import dataclass from dataclasses import dataclass
from ..orchestrator_auth import ROLE_CLI, mint from ..orchestrator_auth import ROLE_CLI
from ..paths import host_orchestrator_token from ..trust_domain import CONTROL_PLANE
from .server import ORCHESTRATOR_AUTH_HEADER from .server import ORCHESTRATOR_AUTH_HEADER
DEFAULT_TIMEOUT_SECONDS = 5.0 DEFAULT_TIMEOUT_SECONDS = 5.0
@@ -32,7 +32,7 @@ def _host_auth_token() -> str:
"" means 'send no auth header' — correct against an open (unconfigured) "" means 'send no auth header' — correct against an open (unconfigured)
control plane, and harmlessly rejected by a secured one.""" control plane, and harmlessly rejected by a secured one."""
try: try:
return mint(ROLE_CLI, host_orchestrator_token()) return CONTROL_PLANE.mint(ROLE_CLI)
except (OSError, ValueError): except (OSError, ValueError):
return "" return ""
+20 -5
View File
@@ -21,8 +21,7 @@ import urllib.error
import urllib.request import urllib.request
from pathlib import Path from pathlib import Path
from ..orchestrator_auth import ROLE_GATEWAY, mint from ..trust_domain import ControlPlaneProvisioning
from ..paths import host_orchestrator_token
DEFAULT_PORT = 8099 DEFAULT_PORT = 8099
DEFAULT_STARTUP_TIMEOUT_SECONDS = 45.0 DEFAULT_STARTUP_TIMEOUT_SECONDS = 45.0
@@ -58,6 +57,13 @@ class Orchestrator(abc.ABC):
so it — not the gateway — mints the gateway's role-scoped token. so it — not the gateway — mints the gateway's role-scoped token.
Backend-neutral.""" Backend-neutral."""
# The shared control-plane auth provisioning contract (#476). Every backend
# gets its signing key + gateway token through this one seam rather than
# re-deriving the wiring; the default topology is co-located + fail-closed,
# so a backend need not redeclare it (a truly isolated control plane would
# override this with a different `Topology`).
provisioning: ControlPlaneProvisioning = ControlPlaneProvisioning()
def ensure_built(self) -> None: def ensure_built(self) -> None:
"""Ensure the orchestrator's image / rootfs exists, building it if """Ensure the orchestrator's image / rootfs exists, building it if
needed. Default: nothing to build (e.g. a pre-pulled image). Call before needed. Default: nothing to build (e.g. a pre-pulled image). Call before
@@ -105,9 +111,17 @@ class Orchestrator(abc.ABC):
def mint_gateway_token(self) -> str: def mint_gateway_token(self) -> str:
"""Mint a role-scoped `gateway` JWT from the host signing key for the """Mint a role-scoped `gateway` JWT from the host signing key for the
gateway to present. The orchestrator holds the key; the gateway never gateway to present. The orchestrator holds the key; the gateway never
does (#469). Backend-neutral — the same host token file is the single does (#469). Routed through the shared provisioning contract (#476), so
source of truth across backends.""" the same host token file is the single source of truth across backends."""
return mint(ROLE_GATEWAY, host_orchestrator_token()) return self.provisioning.gateway_token()
def control_plane_key(self) -> str:
"""The raw signing key the control-plane *process* must receive — the ONE
place a backend obtains it (docker/macOS inject it as `key_env`;
firecracker pushes it to the guest). Fail-closed via the provisioning
contract: it raises rather than yield an empty key that would run the
server OPEN (#476)."""
return self.provisioning.orchestrator_key()
__all__ = [ __all__ = [
@@ -117,4 +131,5 @@ __all__ = [
"OrchestratorStartError", "OrchestratorStartError",
"source_hash", "source_hash",
"Orchestrator", "Orchestrator",
"ControlPlaneProvisioning",
] ]
+7 -5
View File
@@ -63,8 +63,8 @@ import sys
import typing import typing
from urllib.parse import urlsplit from urllib.parse import urlsplit
from ..orchestrator_auth import ROLE_CLI, ROLES, verify from ..orchestrator_auth import ROLE_CLI, ROLES
from ..paths import ORCHESTRATOR_TOKEN_ENV from ..trust_domain import CONTROL_PLANE
from ..supervisor.types import TOOLS from ..supervisor.types import TOOLS
from .service import OrchestratorCore from .service import OrchestratorCore
@@ -413,11 +413,13 @@ class OrchestratorServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
def __init__(self, address: tuple[str, int], orchestrator: OrchestratorCore) -> None: def __init__(self, address: tuple[str, int], orchestrator: OrchestratorCore) -> None:
self.orchestrator = orchestrator self.orchestrator = orchestrator
self._signing_key = os.environ.get(ORCHESTRATOR_TOKEN_ENV, "").strip() # The control-plane trust domain's signing key, as injected into THIS
# (the owning) process by the launcher (#476). Unset → open mode below.
self._signing_key = CONTROL_PLANE.key_from_env()
if not self._signing_key: if not self._signing_key:
sys.stderr.write( sys.stderr.write(
"orchestrator: WARNING — no control-plane signing key " "orchestrator: WARNING — no control-plane signing key "
f"(${ORCHESTRATOR_TOKEN_ENV}); running WITHOUT caller " f"(${CONTROL_PLANE.key_env}); running WITHOUT caller "
"authentication. Any client that can reach this port can drive " "authentication. Any client that can reach this port can drive "
"it. Backends that put the control plane on an agent-reachable " "it. Backends that put the control plane on an agent-reachable "
"network MUST set this.\n" "network MUST set this.\n"
@@ -433,7 +435,7 @@ class OrchestratorServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
role (→ per-route 401/403 in `dispatch`).""" role (→ per-route 401/403 in `dispatch`)."""
if not self._signing_key: if not self._signing_key:
return ROLE_CLI return ROLE_CLI
return verify(presented, self._signing_key) return CONTROL_PLANE.verify(presented, self._signing_key)
def make_server( def make_server(
+14 -8
View File
@@ -59,12 +59,17 @@ _HEADER_SEGMENT = _b64url_encode(
) )
def mint(role: str, secret: str) -> str: def mint(role: str, secret: str, *, roles: frozenset[str] = ROLES) -> str:
"""A compact HS256 token asserting `role`, signed with `secret`. """A compact HS256 token asserting `role`, signed with `secret`.
Raises ValueError for an unknown role (mint only what the control plane will `roles` is the set the signing key is allowed to sign (default: the
accept) or an empty signing key (an unsigned credential is never valid).""" orchestrator's `{gateway, cli}`). A separate service (e.g. the host
if role not in ROLES: controller) passes its own key + role set so its tokens can't be forged with
the orchestrator's key — see `trust_domain.py`, issues #476/#468.
Raises ValueError for a role outside `roles`, or an empty signing key (an
unsigned credential is never valid)."""
if role not in roles:
raise ValueError(f"unknown control-plane role {role!r}") raise ValueError(f"unknown control-plane role {role!r}")
if not secret: if not secret:
raise ValueError("cannot mint a control-plane token without a signing key") raise ValueError("cannot mint a control-plane token without a signing key")
@@ -73,10 +78,11 @@ def mint(role: str, secret: str) -> str:
return f"{signing_input}.{_sign(secret, signing_input)}" return f"{signing_input}.{_sign(secret, signing_input)}"
def verify(token: str, secret: str) -> str | None: def verify(token: str, secret: str, *, roles: frozenset[str] = ROLES) -> str | None:
"""The role a valid `token` carries, or None if it is malformed, wrongly """The role a valid `token` carries, or None if it is malformed, wrongly
signed, or names an unknown role. Constant-time signature check; rejects any signed, or names a role outside `roles` (the verifying trust domain's set —
header whose alg isn't HS256 (no alg-confusion / `none`).""" default `{gateway, cli}`). Constant-time signature check; rejects any header
whose alg isn't HS256 (no alg-confusion / `none`)."""
if not token or not secret: if not token or not secret:
return None return None
parts = token.split(".") parts = token.split(".")
@@ -94,7 +100,7 @@ def verify(token: str, secret: str) -> str | None:
if not isinstance(header, dict) or header.get("alg") != _ALG: if not isinstance(header, dict) or header.get("alg") != _ALG:
return None return None
role = payload.get("role") if isinstance(payload, dict) else None role = payload.get("role") if isinstance(payload, dict) else None
return role if isinstance(role, str) and role in ROLES else None return role if isinstance(role, str) and role in roles else None
__all__ = ["ROLE_GATEWAY", "ROLE_CLI", "ROLES", "mint", "verify"] __all__ = ["ROLE_GATEWAY", "ROLE_CLI", "ROLES", "mint", "verify"]
+19 -9
View File
@@ -97,16 +97,17 @@ def host_gateway_ca_dir() -> Path:
return ca_dir return ca_dir
def host_orchestrator_token() -> str: def host_signing_key(filename: str) -> str:
"""The per-host control-plane secret, minted (256-bit, url-safe) and """A per-host signing key at `<root>/<filename>`, minted (256-bit, url-safe)
persisted 0600 on first use, then reused. and persisted 0600 on first use, then reused.
This is the shared secret the launchers inject into the control-plane and The generic form of `host_orchestrator_token()`: each service names its own
gateway containers and that the host CLI presents on every call. It is a key file (`trust_domain.py`), so the orchestrator and a separate service like
*host* artifact — the file lives under the root the agent never mounts, and the host controller (#468) get distinct keys neither can read. It is a *host*
the env var is set only on the trusted containers — so reading it here is artifact — the file lives under the root the agent never mounts, and its value
safe on the host launch path but the value never reaches a bottle.""" is injected only into the trusted control-plane process — so reading it here
path = bot_bottle_root() / ORCHESTRATOR_TOKEN_FILENAME is safe on the launch path but the value never reaches a bottle."""
path = bot_bottle_root() / filename
try: try:
existing = path.read_text().strip() existing = path.read_text().strip()
if existing: if existing:
@@ -128,6 +129,14 @@ def host_orchestrator_token() -> str:
return token return token
def host_orchestrator_token() -> str:
"""The per-host control-plane signing key — the host-canonical key the
launchers inject into the control-plane process and the host CLI mints its
own `cli` token from. The `control-plane` trust domain's specialization of
`host_signing_key()`."""
return host_signing_key(ORCHESTRATOR_TOKEN_FILENAME)
__all__ = [ __all__ = [
"HOST_DB_FILENAME", "HOST_DB_FILENAME",
"ORCHESTRATOR_TOKEN_FILENAME", "ORCHESTRATOR_TOKEN_FILENAME",
@@ -138,5 +147,6 @@ __all__ = [
"host_db_path", "host_db_path",
"host_db_dir", "host_db_dir",
"host_gateway_ca_dir", "host_gateway_ca_dir",
"host_signing_key",
"host_orchestrator_token", "host_orchestrator_token",
] ]
+167
View File
@@ -0,0 +1,167 @@
"""Per-service control-plane signing keys (issue #476).
A `TrustDomain` is one service's signing material: its host-canonical key file,
the roles that key may sign, and the env vars its key and a pre-minted token ride
in. Scoping `mint`/`verify` to a domain's roles keeps one service's key from
signing (or accepting) another service's tokens.
Today there is one domain, `CONTROL_PLANE` — the orchestrator's key (roles
`{gateway, cli}`): the orchestrator holds it and mints the gateway's and CLI's
tokens. The host controller (#468) will add a **second** domain with its own key
the orchestrator never holds. That is the point: the host controller starts and
stops the orchestrator, so the orchestrator must not be able to mint the
credentials it uses to talk to it. Adding a `host` role to `CONTROL_PLANE`
instead would defeat that — the orchestrator holds that key, so it could forge
`host` tokens.
`ControlPlaneProvisioning` is the one seam every backend launcher uses to get the
orchestrator its key and the gateway its token, instead of re-deriving that
wiring per backend (the bug class behind PR #471 — see
`docs/prds/prd-new-control-plane-auth-provisioning.md`).
Stdlib-only: the HMAC lives in `orchestrator_auth`, the key file in `paths`.
"""
from __future__ import annotations
import os
from collections.abc import Mapping
from dataclasses import dataclass, field
from . import orchestrator_auth
from .orchestrator_auth import ROLE_GATEWAY
from .paths import (
ORCHESTRATOR_AUTH_JWT_ENV,
ORCHESTRATOR_TOKEN_ENV,
ORCHESTRATOR_TOKEN_FILENAME,
host_signing_key,
)
class ProvisioningError(RuntimeError):
"""A control-plane auth invariant would be violated (e.g. starting a
co-located control plane without its signing key — which would run OPEN)."""
@dataclass(frozen=True)
class TrustDomain:
"""One service's signing material: a host-canonical key file, the roles that
key may sign, and the env vars its key and a minted token ride in.
The service that *owns* the domain (e.g. the orchestrator) receives the raw
key via `key_env`; a delegate (e.g. the gateway) receives only a pre-minted,
role-scoped token via `token_env` it cannot rewrite. `mint`/`verify` are
scoped to `roles`, so this service's key can neither sign nor accept another
service's role."""
name: str
key_filename: str
roles: frozenset[str]
key_env: str
token_env: str
def signing_key(self) -> str:
"""This service's host-canonical signing key (minted 0600 on first use).
Host-side only — the value is injected into the owning process."""
return host_signing_key(self.key_filename)
def key_from_env(self, environ: Mapping[str, str] | None = None) -> str:
"""The signing key as the owning process sees it — read from `key_env`
(default `os.environ`). "" when unset; the caller decides whether that is
fatal (`ControlPlaneProvisioning`) or the open-mode fallback
(`OrchestratorServer`)."""
env = os.environ if environ is None else environ
return env.get(self.key_env, "").strip()
def mint(self, role: str) -> str:
"""A role-scoped token for a delegate, signed with this service's key.
Raises ValueError for a role this service doesn't sign."""
if role not in self.roles:
raise ValueError(f"role {role!r} is not in trust domain {self.name!r}")
return orchestrator_auth.mint(role, self.signing_key(), roles=self.roles)
def verify(self, token: str, key: str) -> str | None:
"""The role `token` carries under `key`, or None. `key` is passed in
(not read from disk) because the verifier — the control-plane process —
holds it in `key_env`, not on disk in its guest."""
return orchestrator_auth.verify(token, key, roles=self.roles)
# The orchestrator's domain: the key the orchestrator (and host CLI) holds, the
# `gateway` token it mints for the data plane, and the `cli` token the CLI mints
# for itself. #468's host controller will add a second, separate domain.
CONTROL_PLANE = TrustDomain(
name="control-plane",
key_filename=ORCHESTRATOR_TOKEN_FILENAME,
roles=orchestrator_auth.ROLES,
key_env=ORCHESTRATOR_TOKEN_ENV,
token_env=ORCHESTRATOR_AUTH_JWT_ENV,
)
@dataclass(frozen=True)
class Topology:
"""Where a backend runs the two planes, so the provisioning seam can decide
whether an open control plane is dangerous — the backend declares this
instead of hardcoding the decision in its launcher.
`data_plane_shares_control_host` — the gateway runs on the same host/VM as
the orchestrator, so an open orchestrator would hand the co-located gateway
full `cli`. The default, and what makes the signing key mandatory. Every
current backend is co-located (docker/macOS: two containers on one host;
firecracker: two VMs on one host, agents L3-isolated); an isolated control
plane on a separate trusted host may declare False.
`combined_guest` — both planes in one guest (the retired combined infra VM).
Informational; kept so a future combined backend declares it."""
data_plane_shares_control_host: bool = True
combined_guest: bool = False
# The default posture: data plane co-located with the control plane. Fail-closed
# — the signing key is mandatory. A backend opts out only by declaring otherwise.
COLOCATED = Topology(data_plane_shares_control_host=True)
@dataclass(frozen=True)
class ControlPlaneProvisioning:
"""The one seam every backend launcher uses to provision control-plane auth,
instead of re-deriving the four invariants that each cost a PR #471 review
round: the orchestrator gets the raw key (`orchestrator_key`), the gateway
gets a minted `gateway` token (`gateway_token`), the host CLI mints its own
`cli` token from the same host-canonical key, and the orchestrator never
starts open where its gateway is co-located."""
domain: TrustDomain = CONTROL_PLANE
topology: Topology = field(default=COLOCATED)
def orchestrator_key(self) -> str:
"""The raw signing key the orchestrator process must receive (carry it in
`domain.key_env`). Fail-closed: raises rather than return "" for a
co-located topology, since an empty key runs the server open and hands
the co-located gateway full `cli`."""
key = self.domain.signing_key()
if not key and self.topology.data_plane_shares_control_host:
raise ProvisioningError(
f"refusing to start the {self.domain.name} orchestrator without "
"a signing key: its gateway shares this host/VM, so an open "
"orchestrator would grant that gateway full `cli` (#476)"
)
return key
def gateway_token(self) -> str:
"""The `gateway`-role token the gateway receives (carry it in
`domain.token_env`) — minted from the key, never the key itself, so a
compromised gateway cannot forge a `cli` token."""
return self.domain.mint(ROLE_GATEWAY)
__all__ = [
"ProvisioningError",
"TrustDomain",
"CONTROL_PLANE",
"Topology",
"COLOCATED",
"ControlPlaneProvisioning",
]
@@ -0,0 +1,88 @@
# PRD prd-new: Per-service signing keys for control-plane auth
- **Status:** Draft
- **Author:** claude
- **Created:** 2026-07-26
- **Issue:** #476
## Summary
Provision control-plane signing keys **per service**, through one shared seam, so
no service can mint another's credentials. Concretely: the orchestrator holds the
control-plane key and mints the gateway's and CLI's tokens; the host controller
(#468, next) gets a **separate** key the orchestrator never holds — so the
orchestrator cannot forge the credentials it uses to talk to the host controller
that starts and stops it. Landing this seam also retires the per-backend
provisioning duplication that made PR #471 take three review rounds.
## Problem
**1. The orchestrator could forge host-controller credentials.** The
orchestrator's key signs roles `{gateway, cli}`. The tempting way to add the host
controller (#468) is a third role, `host`, on that same key. But then the
orchestrator — which holds the key — can mint `host` tokens, and the host
controller, which owns the orchestrator's lifecycle, must not trust anything the
orchestrator can mint. The two services need separate keys.
**2. Every backend provisioned auth by hand.** Each launcher (docker
gateway/infra, macOS infra, firecracker infra) re-derived how to generate the
signing key, scope it to the orchestrator, mint the gateway JWT, and keep the
host key file canonical. All three PR #471 High-severity findings were this one
integration bug in different launchers: the data plane got the full `cli` token;
the firecracker control plane ran open; the firecracker guest clobbered the host
key.
## Goals / Success Criteria
- The orchestrator and the host controller sign with **different** keys; neither
can mint the other's tokens. (This PR provisions the orchestrator's key and
leaves a drop-in seam for the host controller's.)
- One shared provisioning seam every backend uses — a new backend or daemon
implements it instead of rediscovering these four invariants:
1. the signing key is host-canonical: a guest is handed it, never generates or
overwrites it;
2. only the orchestrator process gets the raw key; the gateway gets a
pre-minted `gateway` token it can't rewrite into `cli`;
3. the host CLI's `cli` token is minted from the same key, so it stays valid
across co-running backends;
4. the control plane never runs open where its data plane shares the host/VM.
## Non-goals
- The host controller itself (#468) — this only provisions the orchestrator's
key and the seam #468 plugs into.
- Rewriting the HMAC primitive: `orchestrator_auth.mint/verify` gain an optional
`roles=` arg (default unchanged) so a key can carry a different role set;
nothing else changes.
- Network topology, the plane split (#469), or the server's open-mode fallback
for tests.
## Design
A **`TrustDomain`** is one service's signing material: its host-canonical key
file, the roles that key may sign, and the env vars its key and a pre-minted
token ride in. `mint`/`verify` are scoped to that domain's roles, so a token
signed by one service's key neither carries nor verifies another service's role.
- `CONTROL_PLANE` — the orchestrator's domain: key `orchestrator-token`, roles
`{gateway, cli}`. The orchestrator process holds the key; the gateway holds
only a minted `gateway` token; the host CLI mints its own `cli` token.
- The host controller (#468) will add a second `TrustDomain` — its own key file
and role(s) — that the orchestrator never holds.
**`ControlPlaneProvisioning`** is the seam the backends call.
`orchestrator_key()` returns the raw key for the control-plane process
(fail-closed: it raises rather than hand back an empty key that would run the
server open where the data plane is co-located). `gateway_token()` mints the
gateway's token. Each backend applies these through its own transport —
docker/macOS inject env vars, firecracker pushes over SSH — but none re-derives
*which* key or role.
`paths.host_signing_key(filename)` generalizes `host_orchestrator_token()` so each
domain names its own key file.
## Open questions
None blocking. #468 adds its `TrustDomain` and a second
`ControlPlaneProvisioning`-shaped consumer; renaming that class to something
service-neutral is a cosmetic call to make then.
+3 -1
View File
@@ -19,7 +19,9 @@ _ORCH = "bot_bottle.backend.docker.orchestrator"
_RUN = f"{_ORCH}.run_docker" _RUN = f"{_ORCH}.run_docker"
_SLEEP = f"{_ORCH}.time.sleep" _SLEEP = f"{_ORCH}.time.sleep"
_MONOTONIC = f"{_ORCH}.time.monotonic" _MONOTONIC = f"{_ORCH}.time.monotonic"
_TOKEN = f"{_ORCH}.host_orchestrator_token" # The signing key is read through the shared provisioning contract (#476); patch
# its host-canonical key file read to keep it off the real host file.
_TOKEN = "bot_bottle.trust_domain.host_signing_key"
# The ABC's is_healthy probes /health via urllib in the lifecycle module. # The ABC's is_healthy probes /health via urllib in the lifecycle module.
_URLOPEN = "bot_bottle.orchestrator.lifecycle.urllib.request.urlopen" _URLOPEN = "bot_bottle.orchestrator.lifecycle.urllib.request.urlopen"
+1 -1
View File
@@ -44,7 +44,7 @@ class TestEnsureRunning(unittest.TestCase):
vm = infra_vm.InfraVm(guest_ip="10.243.255.1", private_key=Path("/k"), vm=MagicMock()) vm = infra_vm.InfraVm(guest_ip="10.243.255.1", private_key=Path("/k"), vm=MagicMock())
with patch.object(infra_vm, "boot_vm", return_value=vm) as boot, \ with patch.object(infra_vm, "boot_vm", return_value=vm) as boot, \
patch.object(infra_vm, "push_secret") as push, \ patch.object(infra_vm, "push_secret") as push, \
patch(f"{_ORCH}.host_orchestrator_token", return_value="host-key"), \ patch("bot_bottle.trust_domain.host_signing_key", return_value="host-key"), \
patch.object(orch, "is_running", return_value=False), \ patch.object(orch, "is_running", return_value=False), \
patch.object(orch, "_ensure_registry_volume", return_value=Path("/reg")), \ patch.object(orch, "_ensure_registry_volume", return_value=Path("/reg")), \
patch.object(orch, "_wait_for_health"): patch.object(orch, "_wait_for_health"):
+2 -2
View File
@@ -33,7 +33,7 @@ class TestMacosOrchestratorRun(unittest.TestCase):
def _run(self) -> list[str]: def _run(self) -> list[str]:
run = Mock(return_value=_ok()) run = Mock(return_value=_ok())
with patch(f"{_ORCH}.container_mod") as mod, \ with patch(f"{_ORCH}.container_mod") as mod, \
patch(f"{_ORCH}.host_orchestrator_token", return_value="k"): patch("bot_bottle.trust_domain.host_signing_key", return_value="k"):
mod.dns_server.return_value = "1.1.1.1" mod.dns_server.return_value = "1.1.1.1"
mod.bind_mount_spec.side_effect = _spec mod.bind_mount_spec.side_effect = _spec
mod.run_container_argv = run mod.run_container_argv = run
@@ -65,7 +65,7 @@ class TestMacosOrchestratorRun(unittest.TestCase):
def test_start_failure_raises(self) -> None: def test_start_failure_raises(self) -> None:
with patch(f"{_ORCH}.container_mod") as mod, \ with patch(f"{_ORCH}.container_mod") as mod, \
patch(f"{_ORCH}.host_orchestrator_token", return_value="k"): patch("bot_bottle.trust_domain.host_signing_key", return_value="k"):
mod.dns_server.return_value = "1.1.1.1" mod.dns_server.return_value = "1.1.1.1"
mod.run_container_argv = Mock(return_value=_fail()) mod.run_container_argv = Mock(return_value=_fail())
with self.assertRaises(OrchestratorStartError): with self.assertRaises(OrchestratorStartError):
+21
View File
@@ -82,5 +82,26 @@ class TestMintVerify(unittest.TestCase):
mint(ROLE_GATEWAY, "") mint(ROLE_GATEWAY, "")
class TestCustomRoleSet(unittest.TestCase):
"""The `roles=` seam a trust domain other than the control plane uses (#476):
mint/verify are scoped to the passed role set, not the module default."""
_ROLES = frozenset({"host"})
def test_round_trips_a_role_in_the_custom_set(self) -> None:
tok = mint("host", _KEY, roles=self._ROLES)
self.assertEqual("host", verify(tok, _KEY, roles=self._ROLES))
def test_mint_rejects_a_role_outside_the_custom_set(self) -> None:
with self.assertRaises(ValueError):
mint(ROLE_CLI, _KEY, roles=self._ROLES)
def test_verify_rejects_a_role_outside_the_verifiers_set(self) -> None:
# A validly signed token whose role isn't in the verifier's set fails —
# this is what keeps one domain's key from asserting another's role.
tok = mint(ROLE_CLI, _KEY) # a control-plane `cli` token
self.assertIsNone(verify(tok, _KEY, roles=self._ROLES))
if __name__ == "__main__": if __name__ == "__main__":
unittest.main() unittest.main()
+4 -2
View File
@@ -20,13 +20,15 @@ _URLOPEN = "bot_bottle.orchestrator.client.urllib.request.urlopen"
class TestHostAuthToken(unittest.TestCase): class TestHostAuthToken(unittest.TestCase):
def test_mints_a_cli_token_from_the_host_key(self) -> None: def test_mints_a_cli_token_from_the_host_key(self) -> None:
with patch("bot_bottle.orchestrator.client.host_orchestrator_token", # The CLI mints its `cli` token from the control-plane trust domain's
# host-canonical key (#476) — patch the key file read underneath it.
with patch("bot_bottle.trust_domain.host_signing_key",
return_value="signing-key"): return_value="signing-key"):
tok = _host_auth_token() tok = _host_auth_token()
self.assertEqual(ROLE_CLI, verify(tok, "signing-key")) self.assertEqual(ROLE_CLI, verify(tok, "signing-key"))
def test_returns_empty_when_key_unreadable(self) -> None: def test_returns_empty_when_key_unreadable(self) -> None:
with patch("bot_bottle.orchestrator.client.host_orchestrator_token", with patch("bot_bottle.trust_domain.host_signing_key",
side_effect=OSError("no host root")): side_effect=OSError("no host root")):
self.assertEqual("", _host_auth_token()) self.assertEqual("", _host_auth_token())
+113
View File
@@ -0,0 +1,113 @@
"""Unit: trust domains + the shared control-plane provisioning contract (#476)."""
from __future__ import annotations
import unittest
from unittest.mock import patch
from bot_bottle import orchestrator_auth
from bot_bottle.orchestrator_auth import ROLE_CLI, ROLE_GATEWAY
from bot_bottle.trust_domain import (
CONTROL_PLANE,
ControlPlaneProvisioning,
ProvisioningError,
Topology,
TrustDomain,
)
# A second, unrelated domain — the shape #468's host controller would take: its
# own key file, its own role, its own env vars. Distinct from CONTROL_PLANE.
_HOST_CTRL = TrustDomain(
name="host-controller",
key_filename="host-controller-token",
roles=frozenset({"host"}),
key_env="BOT_BOTTLE_HOST_CONTROLLER_TOKEN",
token_env="BOT_BOTTLE_HOST_CONTROLLER_JWT",
)
class TestTrustDomainMintVerify(unittest.TestCase):
def test_mint_verify_round_trips_within_a_domain(self) -> None:
with patch("bot_bottle.trust_domain.host_signing_key", return_value="k"):
tok = CONTROL_PLANE.mint(ROLE_GATEWAY)
self.assertEqual(ROLE_GATEWAY, CONTROL_PLANE.verify(tok, "k"))
def test_mint_rejects_a_role_outside_the_domain(self) -> None:
# `host` is a valid role in _HOST_CTRL but not in the control plane —
# the control-plane key must refuse to mint it.
with patch("bot_bottle.trust_domain.host_signing_key", return_value="k"):
with self.assertRaises(ValueError):
CONTROL_PLANE.mint("host")
def test_a_custom_role_set_verifies_its_own_role(self) -> None:
with patch("bot_bottle.trust_domain.host_signing_key", return_value="k"):
tok = _HOST_CTRL.mint("host")
self.assertEqual("host", _HOST_CTRL.verify(tok, "k"))
def test_key_from_env_reads_the_domains_env_var(self) -> None:
self.assertEqual(
"secret", CONTROL_PLANE.key_from_env({CONTROL_PLANE.key_env: " secret "}))
self.assertEqual("", CONTROL_PLANE.key_from_env({}))
class TestDomainBoundary(unittest.TestCase):
"""The #476/#468 invariant: two domains, two keys, two role sets — one
domain's key can neither mint nor verify the other's tokens."""
def test_a_control_plane_token_does_not_verify_under_another_domains_key(
self,
) -> None:
# Even with an IDENTICAL underlying key, a `cli` token minted under the
# control-plane domain must not verify as a role in the host-controller
# domain — the role isn't in that domain's set.
cli_tok = orchestrator_auth.mint(ROLE_CLI, "shared-bytes")
self.assertIsNone(_HOST_CTRL.verify(cli_tok, "shared-bytes"))
def test_distinct_keys_do_not_cross_verify(self) -> None:
# The realistic case: distinct host-canonical keys per domain. A token
# signed by one key never verifies under the other.
host_tok = orchestrator_auth.mint(
"host", "host-ctrl-key", roles=_HOST_CTRL.roles)
self.assertIsNone(_HOST_CTRL.verify(host_tok, "control-plane-key"))
self.assertEqual("host", _HOST_CTRL.verify(host_tok, "host-ctrl-key"))
class TestControlPlaneProvisioning(unittest.TestCase):
def test_orchestrator_key_returns_the_canonical_key(self) -> None:
prov = ControlPlaneProvisioning()
with patch("bot_bottle.trust_domain.host_signing_key", return_value="key"):
self.assertEqual("key", prov.orchestrator_key())
def test_orchestrator_key_fail_closes_when_colocated_and_empty(self) -> None:
# Invariant 4: a co-located backend must never start the control plane
# without a key (it would run OPEN and hand the co-located data plane
# full `cli`).
prov = ControlPlaneProvisioning() # default topology: co-located
with patch("bot_bottle.trust_domain.host_signing_key", return_value=""):
with self.assertRaises(ProvisioningError):
prov.orchestrator_key()
def test_orchestrator_key_allows_empty_when_isolated(self) -> None:
# A genuinely isolated control plane (no co-located data plane) may run
# without a key — the only topology where open mode is permissible.
prov = ControlPlaneProvisioning(
topology=Topology(data_plane_shares_control_host=False))
with patch("bot_bottle.trust_domain.host_signing_key", return_value=""):
self.assertEqual("", prov.orchestrator_key())
def test_gateway_token_is_a_verifiable_gateway_role_token(self) -> None:
prov = ControlPlaneProvisioning()
with patch("bot_bottle.trust_domain.host_signing_key", return_value="k"):
tok = prov.gateway_token()
self.assertEqual(ROLE_GATEWAY, CONTROL_PLANE.verify(tok, "k"))
def test_gateway_token_cannot_be_reused_as_cli(self) -> None:
# The data plane's token is `gateway`-scoped: it never carries `cli`.
prov = ControlPlaneProvisioning()
with patch("bot_bottle.trust_domain.host_signing_key", return_value="k"):
tok = prov.gateway_token()
self.assertNotEqual(ROLE_CLI, CONTROL_PLANE.verify(tok, "k"))
if __name__ == "__main__":
unittest.main()