refactor(orchestrator): rename Sidecar -> Gateway for the consolidated data plane
test / unit (pull_request) Successful in 1m10s
test / integration (pull_request) Successful in 25s
test / coverage (pull_request) Successful in 1m17s
lint / lint (push) Successful in 2m11s

Retire "sidecar" for the consolidated per-host path (PRD 0070 naming
decision): the orchestrator is the umbrella/control plane, and the
egress/git/supervise data-plane unit it runs is the "gateway".

- git mv sidecar.py -> gateway.py and the two integration + one unit test
  files; DockerSidecar->DockerGateway, Sidecar->Gateway,
  SidecarError->GatewayError, SIDECAR_*->GATEWAY_*, ensure_sidecar->
  ensure_gateway, sidecar_status->gateway_status, container name
  bot-bottle-orch-sidecar->bot-bottle-orch-gateway.
- Prose rename across broker/registry/egress/policy_resolver + PRD 0070.
- Preserved: the image name bot-bottle-sidecars, the
  BOT_BOTTLE_SIDECAR_IMAGE env var, Dockerfile.sidecars, and PRD 0069's
  own stage-name cross-references (that doc still uses "sidecar").

No behavior change. Full unit suite green (1679 tests; the 13
test_sidecar_init /bin/sleep errors are pre-existing NixOS-local noise).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
This commit is contained in:
2026-07-13 18:12:17 -04:00
parent d3e08cf039
commit e0b0429cd1
17 changed files with 151 additions and 151 deletions
+2 -2
View File
@@ -1,4 +1,4 @@
"""mitmproxy addon entrypoint for the egress sidecar (PRD 0017, PRD 0053).
"""mitmproxy addon entrypoint for the egress gateway (PRD 0017, PRD 0053).
Loaded by `mitmdump -s /app/egress_addon.py` inside the
egress container."""
@@ -275,7 +275,7 @@ class EgressAddon:
return
# Strip agent-set Authorization after DLP scan so smuggled tokens
# are caught above; the route may inject sidecar-owned auth below.
# are caught above; the route may inject gateway-owned auth below.
flow.request.headers.pop("authorization", None)
# Build headers mapping for match evaluation
+5 -5
View File
@@ -3,7 +3,7 @@
Split out of `egress_addon.py` so the host's unit tests can
exercise the parse + decision functions without depending on the
`mitmproxy` package. The companion module wraps these with the
`mitmproxy.http.HTTPFlow` API and is loaded inside the sidecar
`mitmproxy.http.HTTPFlow` API and is loaded inside the gateway
container.
Imports: stdlib + `yaml_subset` (which is itself stdlib-only and
@@ -22,7 +22,7 @@ except ImportError: # pragma: no cover - host-side path
from .yaml_subset import YamlSubsetError, parse_yaml_subset
# DLP detector-config parsing lives in a sibling module (also flat-bundled
# into the sidecar — see Dockerfile.sidecars). Re-exported below so existing
# into the gateway — see Dockerfile.sidecars). Re-exported below so existing
# `from egress_addon_core import ON_MATCH_*` callers keep working.
try:
from egress_dlp_config import ( # type: ignore[import-not-found]
@@ -120,7 +120,7 @@ class ScanResult:
reason: str
location: str = "" # where the match was found, e.g. "body", "authorization header"
context: str = "" # surrounding text with the match replaced by REDACT
# Raw substring the detector matched. Used inside the sidecar to key the
# Raw substring the detector matched. Used inside the gateway to key the
# supervisor-approved "safe tokens" set (PRD 0062); never logged or written
# to a proposal file. Empty for structural detectors (CRLF) that carry no
# safelist-able value.
@@ -641,7 +641,7 @@ def outbound_scan_headers(
) -> dict[str, str]:
"""Return request headers that should be included in outbound DLP.
Routes that inject sidecar-owned auth always strip the agent's
Routes that inject gateway-owned auth always strip the agent's
Authorization header before forwarding. Scanning that header first
creates false positives for provider clients that insist on sending
their own bearer-shaped placeholder, while still not changing what
@@ -692,7 +692,7 @@ def scan_outbound(
crlf_text: str | None = None,
) -> ScanResult | None:
# Lazy import to avoid circular deps and keep dlp_detectors optional
# at import time (the sidecar copies it flat alongside this file).
# at import time (the gateway copies it flat alongside this file).
try:
from dlp_detectors import ( # type: ignore[import-not-found]
scan_crlf_injection,
+9 -9
View File
@@ -1,6 +1,6 @@
"""Per-host orchestrator service (PRD 0070).
A single persistent per-host service that will run the sidecar functions
A single persistent per-host service that will run the gateway functions
(egress / git-gate / supervise), coordinate with the console, and broker
agent launches. This package is being built bottom-up, starting with the
backend-neutral "consolidation core" that needs no VM packaging:
@@ -10,15 +10,15 @@ backend-neutral "consolidation core" that needs no VM packaging:
* `broker` the signed, structured launch-request contract + a
`LaunchBroker` (stub for the harness) that verifies
provenance before acting.
* `sidecar` the consolidated per-host sidecar: a `Sidecar`
* `gateway` the consolidated per-host gateway: a `Gateway`
lifecycle contract (idempotent singleton) + a
`DockerSidecar` impl. One sidecar shared by all
`DockerGateway` impl. One gateway shared by all
bottles instead of one per bottle.
* `service` the `Orchestrator`: owns the registry, brokers the
launch lifecycle (launch/teardown), manages the
shared sidecar, attributes.
shared gateway, attributes.
* `control_plane` the HTTP control-plane RPC (launch / teardown /
list / attribute / sidecar / health).
list / attribute / gateway / health).
The actual backend-native launch (a real docker/firecracker broker) and
the egress/git/supervise data plane land in later slices once this core is
@@ -38,7 +38,7 @@ from .broker import (
verify_request,
)
from .docker_broker import DockerBroker, DockerBrokerError
from .sidecar import DockerSidecar, Sidecar, SidecarError
from .gateway import DockerGateway, Gateway, GatewayError
from .service import Orchestrator
from .control_plane import ControlPlaneServer, dispatch, make_server
@@ -52,9 +52,9 @@ __all__ = [
"StubBroker",
"DockerBroker",
"DockerBrokerError",
"Sidecar",
"DockerSidecar",
"SidecarError",
"Gateway",
"DockerGateway",
"GatewayError",
"sign_request",
"verify_request",
"Orchestrator",
+8 -8
View File
@@ -21,7 +21,7 @@ from .control_plane import make_server
from .docker_broker import DockerBroker
from .registry import RegistryStore, default_db_path
from .service import Orchestrator
from .sidecar import DockerSidecar, Sidecar
from .gateway import DockerGateway, Gateway
def main(argv: list[str] | None = None) -> int:
@@ -38,7 +38,7 @@ def main(argv: list[str] | None = None) -> int:
help="launch broker: 'stub' records requests; 'docker' runs containers",
)
parser.add_argument(
"--sidecar", action="store_true",
"--gateway", action="store_true",
help="run one consolidated per-host sidecar bundle (build-if-missing)",
)
args = parser.parse_args(argv)
@@ -51,14 +51,14 @@ def main(argv: list[str] | None = None) -> int:
# anything; 'docker' runs real containers (firecracker drops in later).
secret = secrets.token_bytes(32)
broker: LaunchBroker = DockerBroker(secret) if args.broker == "docker" else StubBroker(secret)
sidecar: Sidecar | None = DockerSidecar() if args.sidecar else None
orchestrator = Orchestrator(registry, broker, secret, sidecar)
gateway: Gateway | None = DockerGateway() if args.gateway else None
orchestrator = Orchestrator(registry, broker, secret, gateway)
# One persistent per-host sidecar, shared by every bottle: build the
# One persistent per-host gateway, shared by every bottle: build the
# bundle image if missing, then bring the singleton up (idempotent).
if sidecar is not None:
orchestrator.ensure_sidecar()
log.info("consolidated sidecar ensured", context={"name": sidecar.name})
if gateway is not None:
orchestrator.ensure_gateway()
log.info("consolidated gateway ensured", context={"name": gateway.name})
server = make_server(orchestrator, host=args.host, port=args.port)
bound_host, bound_port = server.server_address[0], server.server_address[1]
+1 -1
View File
@@ -9,7 +9,7 @@ The request it sends is:
request can't be coerced into launching an arbitrary payload; and
* **signed** wrapped as a compact JWS/JWT the broker verifies before
acting, so a *compromised co-located component* (an agent-facing
sidecar, say) can't forge a launch. This is the concrete form of the
gateway, say) can't forge a launch. This is the concrete form of the
"structured requests only" rule in the PRD security review.
Signing is **HS256** over a secret shared by the orchestrator (signer) and
+4 -4
View File
@@ -5,7 +5,7 @@ The backend-agnostic control-plane RPC (CLI / console -> orchestrator) over
vsock / unix-socket portability caveats):
GET /health -> 200 {"status": "ok"}
GET /sidecar -> 200 {"configured", ["name","running"]}
GET /gateway -> 200 {"configured", ["name","running"]}
GET /bottles -> 200 {"bottles": [ <redacted record>, ...]}
POST /bottles -> 201 {"bottle_id","identity_token"} (launch)
body: {"source_ip", ["image_ref"],
@@ -63,8 +63,8 @@ def dispatch( # pylint: disable=too-many-return-statements,too-many-branches
if method == "GET" and route == "/health":
return 200, {"status": "ok"}
if method == "GET" and route == "/sidecar":
return 200, orch.sidecar_status()
if method == "GET" and route == "/gateway":
return 200, orch.gateway_status()
if method == "GET" and route == "/bottles":
return 200, {"bottles": [r.redacted() for r in orch.registry.all()]}
@@ -122,7 +122,7 @@ def dispatch( # pylint: disable=too-many-return-statements,too-many-branches
return 200, {"bottle_id": rec.bottle_id}
if method == "POST" and route == "/resolve":
# The per-request lookup the multi-tenant sidecar makes: returns the
# The per-request lookup the multi-tenant gateway makes: returns the
# bottle's policy. identity_token is OPTIONAL — absent means resolve
# by source IP alone (network-layer attribution).
try:
@@ -1,18 +1,18 @@
"""The consolidated per-host sidecar (PRD 0070).
"""The consolidated per-host gateway (PRD 0070).
The core consolidation win: **one** persistent sidecar per host, shared by
The core consolidation win: **one** persistent gateway per host, shared by
every bottle, instead of a sidecar bundle per bottle. It's safe to share
because the attribution invariant (source IP + identity token, see
`registry`) lets the sidecar attribute each request to the right bottle
`registry`) lets the gateway attribute each request to the right bottle
so per-bottle policy lives in one long-lived process keyed on who's calling.
`Sidecar` is the backend-neutral lifecycle contract (mirrors `LaunchBroker`):
ensure the single instance is up, report it, tear it down. `DockerSidecar`
is the docker implementation; a firecracker sidecar VM slots in later.
`Gateway` is the backend-neutral lifecycle contract (mirrors `LaunchBroker`):
ensure the single instance is up, report it, tear it down. `DockerGateway`
is the docker implementation; a firecracker gateway VM slots in later.
The defining behaviour is **idempotent singleton**: `ensure_running` starts
the instance if absent and is a no-op if it's already up, so N bottle
launches never spawn N sidecars.
launches never spawn N gateways.
"""
from __future__ import annotations
@@ -23,49 +23,49 @@ from pathlib import Path
from ..docker_cmd import run_docker
SIDECAR_NAME = "bot-bottle-orch-sidecar"
SIDECAR_LABEL = "bot-bottle-orch-sidecar=1"
GATEWAY_NAME = "bot-bottle-orch-gateway"
GATEWAY_LABEL = "bot-bottle-orch-gateway=1"
# The real sidecar-bundle image + its Dockerfile. Kept as a local constant
# rather than imported from backend.docker.sidecar_bundle, which would drag
# the whole backend layer into the lean orchestrator (see #359); unify when
# that lands. Env override matches the backend's BOT_BOTTLE_SIDECAR_IMAGE.
SIDECAR_BUNDLE_IMAGE = os.environ.get("BOT_BOTTLE_SIDECAR_IMAGE", "bot-bottle-sidecars:latest")
SIDECAR_DOCKERFILE = "Dockerfile.sidecars"
GATEWAY_IMAGE = os.environ.get("BOT_BOTTLE_SIDECAR_IMAGE", "bot-bottle-sidecars:latest")
GATEWAY_DOCKERFILE = "Dockerfile.sidecars"
_REPO_ROOT = Path(__file__).resolve().parents[2]
class SidecarError(Exception):
"""The shared sidecar failed to build/start/stop (non-zero `docker` exit)."""
class GatewayError(Exception):
"""The shared gateway failed to build/start/stop (non-zero `docker` exit)."""
class Sidecar(abc.ABC):
"""Lifecycle of the single per-host sidecar. Backend-neutral."""
class Gateway(abc.ABC):
"""Lifecycle of the single per-host gateway. Backend-neutral."""
name: str
def ensure_built(self) -> None:
"""Ensure the sidecar's image / rootfs exists, building it if needed.
"""Ensure the gateway's image / rootfs exists, building it if needed.
Default: nothing to build (e.g. a stub or a pre-pulled image)."""
return
@abc.abstractmethod
def ensure_running(self) -> None:
"""Start the sidecar if it isn't already up. Idempotent: a no-op
"""Start the gateway if it isn't already up. Idempotent: a no-op
when it's already running (that's the whole point one per host).
Assumes the image exists call `ensure_built()` first."""
@abc.abstractmethod
def is_running(self) -> bool:
"""True iff the sidecar instance is currently up."""
"""True iff the gateway instance is currently up."""
@abc.abstractmethod
def stop(self) -> None:
"""Remove the sidecar. Idempotent — absent is success."""
"""Remove the gateway. Idempotent — absent is success."""
class DockerSidecar(Sidecar):
"""The consolidated sidecar as a single, fixed-name Docker container.
class DockerGateway(Gateway):
"""The consolidated gateway as a single, fixed-name Docker container.
`image_ref` defaults to the real sidecar-bundle image; `ensure_built`
builds it from `Dockerfile.sidecars` when it's missing. (Note: slice 5
@@ -74,11 +74,11 @@ class DockerSidecar(Sidecar):
def __init__(
self,
image_ref: str = SIDECAR_BUNDLE_IMAGE,
image_ref: str = GATEWAY_IMAGE,
*,
name: str = SIDECAR_NAME,
name: str = GATEWAY_NAME,
build_context: Path | None = None,
dockerfile: str | None = SIDECAR_DOCKERFILE,
dockerfile: str | None = GATEWAY_DOCKERFILE,
) -> None:
self.image_ref = image_ref
self.name = name
@@ -101,7 +101,7 @@ class DockerSidecar(Sidecar):
str(self._build_context),
])
if proc.returncode != 0:
raise SidecarError(f"sidecar image build failed: {proc.stderr.strip()}")
raise GatewayError(f"gateway image build failed: {proc.stderr.strip()}")
def is_running(self) -> bool:
proc = run_docker([
@@ -121,19 +121,19 @@ class DockerSidecar(Sidecar):
proc = run_docker([
"docker", "run", "--detach",
"--name", self.name,
"--label", SIDECAR_LABEL,
"--label", GATEWAY_LABEL,
self.image_ref,
])
if proc.returncode != 0:
raise SidecarError(f"sidecar failed to start: {proc.stderr.strip()}")
raise GatewayError(f"gateway failed to start: {proc.stderr.strip()}")
def stop(self) -> None:
proc = run_docker(["docker", "rm", "--force", self.name])
if proc.returncode != 0 and "No such container" not in proc.stderr:
raise SidecarError(f"sidecar failed to stop: {proc.stderr.strip()}")
raise GatewayError(f"gateway failed to stop: {proc.stderr.strip()}")
__all__ = [
"Sidecar", "DockerSidecar", "SidecarError",
"SIDECAR_NAME", "SIDECAR_LABEL", "SIDECAR_BUNDLE_IMAGE",
"Gateway", "DockerGateway", "GatewayError",
"GATEWAY_NAME", "GATEWAY_LABEL", "GATEWAY_IMAGE",
]
+5 -5
View File
@@ -67,9 +67,9 @@ class BottleRecord:
# Opaque JSON blob for forward-compat (pool slot, ...) — the registry
# doesn't interpret it, so new fields need no migration.
metadata: str = ""
# The bottle's sidecar policy (opaque JSON — egress allowlist / routes /
# The bottle's gateway policy (opaque JSON — egress allowlist / routes /
# git config). The registry stores and serves it verbatim, keyed by
# source IP; the multi-tenant sidecar interprets it.
# source IP; the multi-tenant gateway interprets it.
policy: str = ""
def redacted(self) -> dict[str, object]:
@@ -102,9 +102,9 @@ _MIGRATIONS = TableMigrations(
# the "one active bottle per source IP" check cheap.
"CREATE INDEX IF NOT EXISTS idx_orchestrator_bottles_source_ip "
"ON orchestrator_bottles (source_ip)",
# v3 — per-bottle policy (opaque JSON the sidecar interprets): the
# v3 — per-bottle policy (opaque JSON the gateway interprets): the
# egress allowlist / routes / git config selected by source IP. The
# multi-tenant sidecar resolves it per request via `attribute`.
# multi-tenant gateway resolves it per request via `attribute`.
"ALTER TABLE orchestrator_bottles ADD COLUMN policy TEXT NOT NULL DEFAULT ''",
],
)
@@ -217,7 +217,7 @@ class RegistryStore(DbStore):
IP, or None if unknown or ambiguous (more than one a
misconfiguration). Safe as the *sole* attributor only where the
source IP is unspoofable (Firecracker `/31` + nft) and the control
plane is reachable only by the trusted sidecar; pair with the
plane is reachable only by the trusted gateway; pair with the
identity token (`attribute`) elsewhere."""
with self._connect() as conn:
rows = conn.execute(
+20 -20
View File
@@ -19,12 +19,12 @@ from __future__ import annotations
from .broker import LaunchBroker, LaunchRequest, sign_request
from .registry import BottleRecord, RegistryStore
from .sidecar import Sidecar
from .gateway import Gateway
class Orchestrator:
"""Owns the registry + brokers launches, and manages the single
consolidated per-host sidecar. Backend-neutral (broker and sidecar
consolidated per-host gateway. Backend-neutral (broker and gateway
abstract the backend-native pieces)."""
def __init__(
@@ -32,12 +32,12 @@ class Orchestrator:
registry: RegistryStore,
broker: LaunchBroker,
sign_secret: bytes,
sidecar: Sidecar | None = None,
gateway: Gateway | None = None,
) -> None:
self.registry = registry
self._broker = broker
self._secret = sign_secret
self._sidecar = sidecar
self._gateway = gateway
def launch_bottle(
self,
@@ -48,7 +48,7 @@ class Orchestrator:
metadata: str = "",
policy: str = "",
) -> BottleRecord:
"""Register a bottle (with its sidecar policy) and broker its launch.
"""Register a bottle (with its gateway policy) and broker its launch.
Rolls the registry entry back if the launch doesn't take, so a
failure leaves no orphan."""
rec = self.registry.register(source_ip, metadata=metadata, policy=policy)
@@ -84,37 +84,37 @@ class Orchestrator:
def resolve(self, source_ip: str, identity_token: str = "") -> BottleRecord | None:
"""Resolve the bottle behind a request — the source-IP-keyed lookup
the multi-tenant sidecar makes per request; the returned record
the multi-tenant gateway makes per request; the returned record
carries its `policy`. With a token, full attribution (source IP +
token); without, network-layer attribution by source IP alone
(valid where the IP is unspoofable and the control plane is
sidecar-only)."""
gateway-only)."""
if identity_token:
return self.registry.attribute(source_ip, identity_token)
return self.registry.by_source_ip(source_ip)
def set_policy(self, bottle_id: str, policy: str) -> bool:
"""Update a bottle's sidecar policy in place (live reload). False if
"""Update a bottle's gateway policy in place (live reload). False if
the bottle is unknown."""
return self.registry.set_policy(bottle_id, policy)
# --- consolidated sidecar ----------------------------------------------
# --- consolidated gateway ----------------------------------------------
def ensure_sidecar(self) -> None:
"""Ensure the single per-host sidecar is built and up (idempotent).
No-op when no sidecar is configured."""
if self._sidecar is not None:
self._sidecar.ensure_built()
self._sidecar.ensure_running()
def ensure_gateway(self) -> None:
"""Ensure the single per-host gateway is built and up (idempotent).
No-op when no gateway is configured."""
if self._gateway is not None:
self._gateway.ensure_built()
self._gateway.ensure_running()
def sidecar_status(self) -> dict[str, object]:
"""Report the shared sidecar for the control plane / console."""
if self._sidecar is None:
def gateway_status(self) -> dict[str, object]:
"""Report the shared gateway for the control plane / console."""
if self._gateway is None:
return {"configured": False}
return {
"configured": True,
"name": self._sidecar.name,
"running": self._sidecar.is_running(),
"name": self._gateway.name,
"running": self._gateway.is_running(),
}
+2 -2
View File
@@ -1,6 +1,6 @@
"""Sidecar-side per-client policy resolver (PRD 0070).
"""Gateway-side per-client policy resolver (PRD 0070).
The consolidated sidecar serves every bottle from one process, so for each
The consolidated gateway serves every bottle from one process, so for each
request it must apply the *calling* bottle's policy, selected by the source
IP the attribution invariant makes unspoofable. This resolves that policy
from the orchestrator's control plane (`POST /resolve`), keyed on the
+20 -20
View File
@@ -10,8 +10,8 @@
## Summary
Replace the **per-bottle sidecar bundle** with a single **persistent,
per-host orchestrator**: one long-lived service that runs the sidecar
Replace the **per-bottle gateway bundle** with a single **persistent,
per-host orchestrator**: one long-lived service that runs the gateway
functions (egress / git-gate / supervise), coordinates with the console,
and brokers agent launches and teardown. It is **virtualized from the
start** using each backend's native isolation primitive — a Firecracker
@@ -22,13 +22,13 @@ container on the legacy backend — and is fronted by a single
## Motivation
Today each bottle spins up its own sidecar bundle (egress mitmproxy +
Today each bottle spins up its own gateway bundle (egress mitmproxy +
git-gate + supervise). That costs:
- **Resources.** N bottles → N heavy bundles booting and idling.
- **Operational churn.** Per-launch container/VM lifecycle for the
sidecars, a control path baked at launch and torn down at exit.
- **A blurry contract.** "How a bottle talks to its sidecar" is
gateways, a control path baked at launch and torn down at exit.
- **A blurry contract.** "How a bottle talks to its gateway" is
re-implemented per backend instead of being one agreed interface.
A per-host orchestrator collapses the first two and forces the third to
@@ -52,11 +52,11 @@ weaken, and the weakened ones must be mitigated by design, not hand-waved.
structured, auditable privileged core instead of a fat socket.
- **Attribution is enforced, not assumed.** Making source-IP identity a
first-class contract invariant (below) means each backend must *prove*
it, rather than the sidecar implicitly trusting network position.
it, rather than the gateway implicitly trusting network position.
### What gets weaker, and the mitigation
1. **Secret concentration.** Per-bottle sidecars isolate secrets at the
1. **Secret concentration.** Per-bottle gateways isolate secrets at the
process boundary — each holds only its bottle's tokens/keys. A host
orchestrator concentrates **every bottle's** egress tokens, git deploy
keys, and the console credential in one long-lived process. A single
@@ -75,7 +75,7 @@ weaken, and the weakened ones must be mitigated by design, not hand-waved.
fleet, plus launch authority, plus the console token.
- *Mitigation:* the orchestrator is itself confined (its own VM/container
with its own fail-closed egress); make it **restartable without killing
running agent VMs** (agents keep running; they briefly lose sidecar
running agent VMs** (agents keep running; they briefly lose gateway
connectivity until it's back); persist state to a host volume so a
restart re-adopts live bottles rather than losing them.
@@ -114,13 +114,13 @@ responsibility** and part of the contract:
- **Apple** — the host-only network.
If a backend can't honor the invariant, source-IP consolidation is not
safe there and that backend keeps per-bottle sidecars. The invariant is a
safe there and that backend keeps per-bottle gateways. The invariant is a
hard precondition, not an aspiration.
**Defense-in-depth — a per-bottle identity token.** On top of the
network-layer invariant, inject a per-bottle secret token into every
request the agent makes to the orchestrator (the agent already egresses
through the sidecar proxy, so this is cheap to add). It gives an
through the gateway proxy, so this is cheap to add). It gives an
**application-layer** proof of identity independent of the network layer:
- On **Firecracker** the `/31` + nft already make source IP unspoofable
@@ -133,7 +133,7 @@ Requirements: the token is **per-bottle, unguessable, and
non-cross-leakable** — a bottle can only ever prove it is *itself* (it
can't learn another bottle's token, given bottle isolation), so a hostile
agent gains nothing by presenting it. The orchestrator provisions the
token at launch and the sidecar requires it to attribute + authorize.
token at launch and the gateway requires it to attribute + authorize.
Note this hardens *attribution*, not *secret exposure*: it's app-layer, so
a compromised orchestrator still sees every token (that's the
concentration problem, addressed separately under Secret handling).
@@ -178,7 +178,7 @@ manifest anywhere a raw token value is accepted, discovered from
`AgentProvider`s are — because maintaining every forge/cloud/OAuth
provider in-tree is untenable. The orchestrator's vault mints via these
providers; but the abstraction is shippable independently and today's
per-bottle sidecars can use it too.
per-bottle gateways can use it too.
## Design
@@ -195,7 +195,7 @@ Three surfaces; only one is per-backend.
every host (no vsock/unix-socket portability caveats); a local unix
socket is a fine optimization, but HTTP is the wire contract.
2. **Data plane (agent → orchestrator)** — the egress / git / supervise
endpoints. Already agnostic today (agents dial `http://sidecar:9099`);
endpoints. Already agnostic today (agents dial `http://gateway:9099`);
only the *address* and *how packets get there* are per-backend.
3. **Launch / wire (orchestrator → backend)** — the irreducibly
backend-specific part; lives on `BottleBackend`.
@@ -249,7 +249,7 @@ forged one from a compromised co-located component. The orchestrator signs;
the broker verifies with the orchestrator's public key (provisioned at
broker install). This is the concrete form of security #3's "structured
requests only." Same JSON-with-ids + JWT shape for all
orchestrator↔broker/sidecar communication; cases that don't fit get
orchestrator↔broker/gateway communication; cases that don't fit get
handled as they arise, with this as the default.
If `BottleBackend` bloats, the pressure valve is composition one level
@@ -299,10 +299,10 @@ and integrate a console against.
the data plane and the console reach state through the control-plane RPC,
never a direct file handle.** No agent-facing component gets the file, so
none can forge attribution. (This supersedes the earlier `ro`-mount idea.)
- *Transitional caveat:* today the per-bottle **supervise sidecar
- *Transitional caveat:* today the per-bottle **supervise gateway
rw-bind-mounts `bot-bottle.db`** to write proposals — exactly the
pattern the orchestrator removes (supervise consolidates into the
orchestrator; sidecar writes become RPC calls). Until that lands, don't
orchestrator; gateway writes become RPC calls). Until that lands, don't
put the attribution registry behind a data-plane-writable mount.
Implementation note for the VM slices: SQLite **WAL** over a guest share
@@ -327,7 +327,7 @@ orchestrator becomes a VM. Decouple the two risks instead:
Backend order (cheapest proof → hardest → last):
1. **Docker orchestrator** — nearly free (the sidecar bundle is already
1. **Docker orchestrator** — nearly free (the gateway bundle is already
containers; collapse N bundles into one persistent container). Proves
consolidation + the `BottleBackend` seam with the least moving parts.
2. **Firecracker orchestrator** — the real work: the shim + VM-to-VM
@@ -336,13 +336,13 @@ Backend order (cheapest proof → hardest → last):
the dev-harness so the app logic is already proven.
3. **macOS (Apple container)** — last (container-to-container networking).
Keep the sidecar **service one shared thing** throughout.
Keep the gateway **service one shared thing** throughout.
## Non-goals
- Removing OCI/Dockerfile support for agent images (0069's concern).
- A single database for *all* config (see the three-tier table).
- Changing the per-bottle isolation of agent workloads — only the sidecar
- Changing the per-bottle isolation of agent workloads — only the gateway
is consolidated; agents stay one-VM/container-each.
## Relationship to other work
@@ -393,7 +393,7 @@ Keep the sidecar **service one shared thing** throughout.
`BottleBackend.wire()` (DNAT+forward for fc, shared-net for
docker/apple), not orchestrator logic. **Not a blocker** — resolvable at
implementation time (may require a bit of host modification, as the pool
setup already does on NixOS); an earlier "sidecars in VMs" spike showed
setup already does on NixOS); an earlier "gateways in VMs" spike showed
it's feasible.
- **Live-reload protocol** for per-bottle policy over the HTTP control
plane (add/remove routes/keys/proposals without a restart).
@@ -1,7 +1,7 @@
"""Integration: the consolidated Docker sidecar is an idempotent singleton.
"""Integration: the consolidated Docker gateway is an idempotent singleton.
Gated on a reachable Docker daemon. Uses a unique name so it never
collides with a real per-host sidecar or a leftover from another run.
collides with a real per-host gateway or a leftover from another run.
"""
from __future__ import annotations
@@ -10,17 +10,17 @@ import secrets
import subprocess
import unittest
from bot_bottle.orchestrator.sidecar import DockerSidecar
from bot_bottle.orchestrator.gateway import DockerGateway
from tests._docker import skip_unless_docker
IMAGE = "busybox"
@skip_unless_docker()
class TestDockerSidecarIntegration(unittest.TestCase):
class TestDockerGatewayIntegration(unittest.TestCase):
def setUp(self) -> None:
self.name = "bot-bottle-orch-sidecar-itest-" + secrets.token_hex(4)
self.sc = DockerSidecar(IMAGE, name=self.name)
self.name = "bot-bottle-orch-gateway-itest-" + secrets.token_hex(4)
self.sc = DockerGateway(IMAGE, name=self.name)
self.addCleanup(self.sc.stop)
def _count(self) -> int:
@@ -1,4 +1,4 @@
"""Integration: DockerSidecar.image_exists reflects real docker state.
"""Integration: DockerGateway.image_exists reflects real docker state.
Gated on a reachable Docker daemon. Deliberately does NOT build the full
sidecar bundle (a heavy, slow image build) it exercises the `image_exists`
@@ -11,14 +11,14 @@ from __future__ import annotations
import subprocess
import unittest
from bot_bottle.orchestrator.sidecar import DockerSidecar
from bot_bottle.orchestrator.gateway import DockerGateway
from tests._docker import skip_unless_docker
IMAGE = "busybox"
@skip_unless_docker()
class TestDockerSidecarImageExists(unittest.TestCase):
class TestDockerGatewayImageExists(unittest.TestCase):
def test_image_exists_true_for_present_false_for_absent(self) -> None:
# Ensure the tiny image is present (build_if_missing is disabled here
# so this never triggers a Dockerfile.sidecars build).
@@ -26,9 +26,9 @@ class TestDockerSidecarImageExists(unittest.TestCase):
["docker", "pull", IMAGE],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
)
self.assertTrue(DockerSidecar(IMAGE, dockerfile=None).image_exists())
self.assertTrue(DockerGateway(IMAGE, dockerfile=None).image_exists())
self.assertFalse(
DockerSidecar("bot-bottle-nonexistent:doesnotexist", dockerfile=None).image_exists()
DockerGateway("bot-bottle-nonexistent:doesnotexist", dockerfile=None).image_exists()
)
@@ -118,8 +118,8 @@ class TestDispatch(unittest.TestCase):
status, _ = dispatch(self.orch, "GET", "/health/", b"")
self.assertEqual(200, status)
def test_sidecar_status_unconfigured(self) -> None:
status, payload = dispatch(self.orch, "GET", "/sidecar", b"")
def test_gateway_status_unconfigured(self) -> None:
status, payload = dispatch(self.orch, "GET", "/gateway", b"")
self.assertEqual(200, status)
self.assertEqual(False, payload["configured"])
@@ -1,29 +1,29 @@
"""Unit tests for the consolidated Docker sidecar (PRD 0070). Docker mocked."""
"""Unit tests for the consolidated Docker gateway (PRD 0070). Docker mocked."""
from __future__ import annotations
import unittest
from unittest.mock import Mock, patch
from bot_bottle.orchestrator.sidecar import (
SIDECAR_NAME,
DockerSidecar,
SidecarError,
from bot_bottle.orchestrator.gateway import (
GATEWAY_NAME,
DockerGateway,
GatewayError,
)
_RUN_DOCKER = "bot_bottle.orchestrator.sidecar.run_docker"
_RUN_DOCKER = "bot_bottle.orchestrator.gateway.run_docker"
def _proc(returncode: int = 0, stdout: str = "", stderr: str = "") -> Mock:
return Mock(returncode=returncode, stdout=stdout, stderr=stderr)
class TestDockerSidecar(unittest.TestCase):
class TestDockerGateway(unittest.TestCase):
def setUp(self) -> None:
self.sc = DockerSidecar("bot-bottle-sidecars:latest")
self.sc = DockerGateway("bot-bottle-sidecars:latest")
def test_default_name(self) -> None:
self.assertEqual(SIDECAR_NAME, self.sc.name)
self.assertEqual(GATEWAY_NAME, self.sc.name)
def test_is_running_reads_docker_ps(self) -> None:
with patch(_RUN_DOCKER, return_value=_proc(stdout=self.sc.name + "\n")):
@@ -61,7 +61,7 @@ class TestDockerSidecar(unittest.TestCase):
return _proc()
with patch(_RUN_DOCKER, side_effect=fake):
with self.assertRaises(SidecarError):
with self.assertRaises(GatewayError):
self.sc.ensure_running()
def test_stop_is_idempotent_on_missing(self) -> None:
@@ -71,13 +71,13 @@ class TestDockerSidecar(unittest.TestCase):
def test_stop_raises_on_other_failure(self) -> None:
with patch(_RUN_DOCKER, return_value=_proc(returncode=1, stderr="daemon down")):
with self.assertRaises(SidecarError):
with self.assertRaises(GatewayError):
self.sc.stop()
class TestDockerSidecarBuild(unittest.TestCase):
class TestDockerGatewayBuild(unittest.TestCase):
def setUp(self) -> None:
self.sc = DockerSidecar() # defaults to the real bundle image + dockerfile
self.sc = DockerGateway() # defaults to the real bundle image + dockerfile
def test_image_exists_reads_docker_inspect(self) -> None:
with patch(_RUN_DOCKER, return_value=_proc(returncode=0)):
@@ -109,7 +109,7 @@ class TestDockerSidecarBuild(unittest.TestCase):
self.assertTrue(any(a.endswith("Dockerfile.sidecars") for a in builds[0]))
def test_ensure_built_noop_when_no_dockerfile(self) -> None:
sc = DockerSidecar("busybox", dockerfile=None)
sc = DockerGateway("busybox", dockerfile=None)
with patch(_RUN_DOCKER) as m:
sc.ensure_built()
m.assert_not_called()
@@ -121,7 +121,7 @@ class TestDockerSidecarBuild(unittest.TestCase):
return _proc(returncode=1, stderr="build boom")
with patch(_RUN_DOCKER, side_effect=fake):
with self.assertRaises(SidecarError):
with self.assertRaises(GatewayError):
self.sc.ensure_built()
+16 -16
View File
@@ -10,7 +10,7 @@ from pathlib import Path
from bot_bottle.orchestrator.broker import LaunchBroker, LaunchRequest, StubBroker
from bot_bottle.orchestrator.registry import RegistryStore
from bot_bottle.orchestrator.service import Orchestrator
from bot_bottle.orchestrator.sidecar import Sidecar
from bot_bottle.orchestrator.gateway import Gateway
class _FailingBroker(LaunchBroker):
@@ -24,11 +24,11 @@ class _FailingBroker(LaunchBroker):
pass
class _FakeSidecar(Sidecar):
"""In-memory sidecar for wiring tests."""
class _FakeGateway(Gateway):
"""In-memory gateway for wiring tests."""
def __init__(self) -> None:
self.name = "fake-sidecar"
self.name = "fake-gateway"
self.ensured = 0
self.built = 0
self._running = False
@@ -120,23 +120,23 @@ class TestOrchestrator(unittest.TestCase):
orch.launch_bottle("10.243.0.9")
self.assertEqual([], self.store.all()) # no orphan
def test_sidecar_unconfigured_by_default(self) -> None:
self.assertEqual({"configured": False}, self.orch.sidecar_status())
self.orch.ensure_sidecar() # no-op, must not raise
def test_gateway_unconfigured_by_default(self) -> None:
self.assertEqual({"configured": False}, self.orch.gateway_status())
self.orch.ensure_gateway() # no-op, must not raise
def test_sidecar_wired_and_ensured(self) -> None:
sc = _FakeSidecar()
orch = Orchestrator(self.store, self.broker, self.secret, sidecar=sc)
def test_gateway_wired_and_ensured(self) -> None:
sc = _FakeGateway()
orch = Orchestrator(self.store, self.broker, self.secret, gateway=sc)
self.assertEqual(
{"configured": True, "name": "fake-sidecar", "running": False},
orch.sidecar_status(),
{"configured": True, "name": "fake-gateway", "running": False},
orch.gateway_status(),
)
orch.ensure_sidecar()
self.assertEqual(1, sc.built) # ensure_sidecar builds first,
orch.ensure_gateway()
self.assertEqual(1, sc.built) # ensure_gateway builds first,
self.assertEqual(1, sc.ensured) # then runs
self.assertEqual(
{"configured": True, "name": "fake-sidecar", "running": True},
orch.sidecar_status(),
{"configured": True, "name": "fake-gateway", "running": True},
orch.gateway_status(),
)
+1 -1
View File
@@ -1,4 +1,4 @@
"""Unit tests for the sidecar-side PolicyResolver (PRD 0070). HTTP mocked."""
"""Unit tests for the gateway-side PolicyResolver (PRD 0070). HTTP mocked."""
from __future__ import annotations