refactor(de-sidecar): purge the "sidecar" name from live code, tests, and current docs
Completes the de-sidecar cleanup: no live code, test, current doc, script, or nix file mentions or is named "sidecar" any more. Only the dated PRD/research docs keep the term as historical record (agreed on the #385 thread). - Rename `sidecar_init.py`→`gateway_init.py` was done earlier; this pass sweeps the remaining descriptive uses: the egress / git-gate / supervise components are the gateway's *daemons*, the shared container is the *gateway*, the old per-bottle container was the *companion container*. - Rename `tests/integration/test_sidecar_bundle_image.py`→`test_gateway_image.py` and its class; update `docs/ci.md` + `tests/README.md` for the renamed/ removed integration tests. - `SIDECAR_PORTS` shell var in `scripts/firecracker-netpool.sh`→`GATEWAY_PORTS`. Full unit suite green (bar the pre-existing `/bin/sleep`-missing env errors in test_gateway_init); docker integration — gateway singleton, broker, real two-bottle multitenant isolation, and the gateway-image build — all pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
This commit is contained in:
@@ -8,10 +8,10 @@ broad permissions inside a sandbox, so a misbehaving agent cannot reach the
|
|||||||
host. A Python CLI (entry point `cli.py`, package `bot_bottle/`) orchestrates
|
host. A Python CLI (entry point `cli.py`, package `bot_bottle/`) orchestrates
|
||||||
the runtime lifecycle and the copying of skills and env vars into it.
|
the runtime lifecycle and the copying of skills and env vars into it.
|
||||||
The default backend on compatible macOS hosts is macos-container:
|
The default backend on compatible macOS hosts is macos-container:
|
||||||
agents and sidecar bundles run through Apple's `container` CLI without
|
agents and gateways run through Apple's `container` CLI without
|
||||||
requiring Docker. On KVM-capable Linux hosts the default is firecracker:
|
requiring Docker. On KVM-capable Linux hosts the default is firecracker:
|
||||||
agents run in a Firecracker microVM reached over SSH on a point-to-point
|
agents run in a Firecracker microVM reached over SSH on a point-to-point
|
||||||
TAP, while the sidecar bundle still uses Docker. The legacy Docker
|
TAP, while the gateway still uses Docker. The legacy Docker
|
||||||
backend remains available with `BOT_BOTTLE_BACKEND=docker` or
|
backend remains available with `BOT_BOTTLE_BACKEND=docker` or
|
||||||
`--backend=docker`.
|
`--backend=docker`.
|
||||||
|
|
||||||
@@ -59,7 +59,7 @@ backend remains available with `BOT_BOTTLE_BACKEND=docker` or
|
|||||||
in a PRD, research note, or decision record.
|
in a PRD, research note, or decision record.
|
||||||
- Low dependencies by default. The project is Python, stdlib-first (no
|
- Low dependencies by default. The project is Python, stdlib-first (no
|
||||||
runtime pip dependencies in the package itself; the only language
|
runtime pip dependencies in the package itself; the only language
|
||||||
runtime is the Python 3.13 used by the CLI + sidecars). Ask before
|
runtime is the Python 3.13 used by the CLI + companion containers). Ask before
|
||||||
adding new tools, runtimes, or package managers.
|
adding new tools, runtimes, or package managers.
|
||||||
- Commit messages follow [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/):
|
- Commit messages follow [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/):
|
||||||
`<type>[(scope)][!]: <description>`, where `<type>` is one of `feat`, `fix`,
|
`<type>[(scope)][!]: <description>`, where `<type>` is one of `feat`, `fix`,
|
||||||
|
|||||||
@@ -16,7 +16,7 @@
|
|||||||
|
|
||||||
- **Per-bottle egress allowlist** — TLS-bumped HTTP/HTTPS chokepoint with a per-manifest host allowlist; per-route path/method/header `matches` filtering; outbound DLP scanning for known tokens and secrets, inbound DLP scanning for prompt-injection attempts; DoH and arbitrary hosts blocked by default.
|
- **Per-bottle egress allowlist** — TLS-bumped HTTP/HTTPS chokepoint with a per-manifest host allowlist; per-route path/method/header `matches` filtering; outbound DLP scanning for known tokens and secrets, inbound DLP scanning for prompt-injection attempts; DoH and arbitrary hosts blocked by default.
|
||||||
- **Per-route token-match policy** — each egress route picks what happens when the outbound DLP catches a token via `dlp.outbound_on_match`: `supervise` (default) holds the request and surfaces it in `./cli.py supervise` for approval (an approved value is remembered for the life of the proxy); `redact` scrubs the value and forwards; `block` is a hard `403`. Cuts false-positive friction without weakening default-deny.
|
- **Per-route token-match policy** — each egress route picks what happens when the outbound DLP catches a token via `dlp.outbound_on_match`: `supervise` (default) holds the request and surfaces it in `./cli.py supervise` for approval (an approved value is remembered for the life of the proxy); `redact` scrubs the value and forwards; `block` is a hard `403`. Cuts false-positive friction without weakening default-deny.
|
||||||
- **Tokens the agent never sees** — host secrets live in a sidecar; the agent dials `http://sidecar:9099/<path>` and the proxy strips inbound `Authorization` and injects the real token before forwarding. `printenv` in the agent shows proxy URLs only.
|
- **Tokens the agent never sees** — host secrets live in a gateway; the agent dials `http://gateway:9099/<path>` and the proxy strips inbound `Authorization` and injects the real token before forwarding. `printenv` in the agent shows proxy URLs only.
|
||||||
- **Gitleaks-scanned push (git-gate)** — `bottle.git` remotes route through a per-bottle `git daemon` that gitleaks-scans incoming refs pre-receive and forwards clean refs upstream over SSH. The agent never holds the upstream credential.
|
- **Gitleaks-scanned push (git-gate)** — `bottle.git` remotes route through a per-bottle `git daemon` that gitleaks-scans incoming refs pre-receive and forwards clean refs upstream over SSH. The agent never holds the upstream credential.
|
||||||
- **Manifest-scoped skills + secrets** — each bottle declares its skills, env, git identity, remotes, and egress routes; unknown keys die at load.
|
- **Manifest-scoped skills + secrets** — each bottle declares its skills, env, git identity, remotes, and egress routes; unknown keys die at load.
|
||||||
- **Trust boundary at `$HOME`** — bottles (credentials, egress, remotes) live only under `~/.bot-bottle/bottles/`. Repos may ship agents but not bottles, so a cloned repo can't redirect an env var to an attacker host.
|
- **Trust boundary at `$HOME`** — bottles (credentials, egress, remotes) live only under `~/.bot-bottle/bottles/`. Repos may ship agents but not bottles, so a cloned repo can't redirect an env var to an attacker host.
|
||||||
@@ -24,17 +24,17 @@
|
|||||||
- **Parallel, isolated bottles** — each bottle runs in its own backend-owned isolation boundary; bottles don't share state or talk to each other.
|
- **Parallel, isolated bottles** — each bottle runs in its own backend-owned isolation boundary; bottles don't share state or talk to each other.
|
||||||
- **Provider templates (Claude, Codex)** — `Dockerfile.claude` / `Dockerfile.codex`, or a bottle-supplied Dockerfile. Claude auth via long-lived OAuth token; Codex via opt-in host device-auth forwarding.
|
- **Provider templates (Claude, Codex)** — `Dockerfile.claude` / `Dockerfile.codex`, or a bottle-supplied Dockerfile. Claude auth via long-lived OAuth token; Codex via opt-in host device-auth forwarding.
|
||||||
- **gVisor auto-detect** — on Linux hosts where `runsc` is registered with Docker, every bottle launches under it for a userspace syscall barrier; no manifest config required.
|
- **gVisor auto-detect** — on Linux hosts where `runsc` is registered with Docker, every bottle launches under it for a userspace syscall barrier; no manifest config required.
|
||||||
- **Apple Container backend (macOS default when available)** — runs the agent and sidecar bundle with Apple's `container` CLI, using a host-only agent network plus a separate sidecar egress network.
|
- **Apple Container backend (macOS default when available)** — runs the agent and gateway with Apple's `container` CLI, using a host-only agent network plus a separate gateway egress network.
|
||||||
- **Firecracker backend (Linux default when available)** — runs the agent in a KVM Firecracker microVM reached over SSH on a point-to-point TAP, with the sidecar bundle in Docker. A dedicated, fail-closed `nftables` table isolates the guest, closing the raw DNS/IP exfiltration gap that exists in the legacy Docker backend. Requires KVM (`/dev/kvm`) and a one-time privileged network-pool setup.
|
- **Firecracker backend (Linux default when available)** — runs the agent in a KVM Firecracker microVM reached over SSH on a point-to-point TAP, with the gateway in Docker. A dedicated, fail-closed `nftables` table isolates the guest, closing the raw DNS/IP exfiltration gap that exists in the legacy Docker backend. Requires KVM (`/dev/kvm`) and a one-time privileged network-pool setup.
|
||||||
- **Legacy Docker backend** — still available for examples, CI, and hosts without Apple Container or KVM via `BOT_BOTTLE_BACKEND=docker` or `--backend=docker`.
|
- **Legacy Docker backend** — still available for examples, CI, and hosts without Apple Container or KVM via `BOT_BOTTLE_BACKEND=docker` or `--backend=docker`.
|
||||||
|
|
||||||
## Architecture
|
## Architecture
|
||||||
|
|
||||||
On the default macOS Apple Container backend, a bottle is an agent container on a host-only internal network plus a sidecar bundle attached to both that internal network and a NAT egress network. The agent gets HTTP(S)_PROXY and CA bundle env vars pointing at the sidecar's internal-network IP, so HTTP/HTTPS traffic flows through the sidecar instead of direct egress. `bottle.git` / git-gate is intentionally deferred on this backend until a safe Apple Container key-delivery path exists.
|
On the default macOS Apple Container backend, a bottle is an agent container on a host-only internal network plus a gateway attached to both that internal network and a NAT egress network. The agent gets HTTP(S)_PROXY and CA bundle env vars pointing at the gateway's internal-network IP, so HTTP/HTTPS traffic flows through the gateway instead of direct egress. `bottle.git` / git-gate is intentionally deferred on this backend until a safe Apple Container key-delivery path exists.
|
||||||
|
|
||||||
On the Firecracker backend, a bottle is an agent microVM plus a Docker sidecar bundle for egress, git-gate, and supervise. The VM reaches the sidecars over a per-bottle point-to-point TAP link; a dedicated fail-closed `nftables` table (`inet bot_bottle_fc`) confines the guest to that link, so nothing leaves the box except through the sidecars. The TAP pool and nft table are provisioned once (root); per-launch needs no privilege.
|
On the Firecracker backend, a bottle is an agent microVM plus a Docker gateway for egress, git-gate, and supervise. The VM reaches the gateway over a per-bottle point-to-point TAP link; a dedicated fail-closed `nftables` table (`inet bot_bottle_fc`) confines the guest to that link, so nothing leaves the box except through the gateway. The TAP pool and nft table are provisioned once (root); per-launch needs no privilege.
|
||||||
|
|
||||||
On the legacy Docker backend, the same logical bottle is two containers per agent: an `agent` container and a `sidecars` container. They share a per-agent Docker `--internal` network; the agent has no default route off-box.
|
On the legacy Docker backend, the same logical bottle is two containers per agent: an `agent` container and a `companion containers` container. They share a per-agent Docker `--internal` network; the agent has no default route off-box.
|
||||||
|
|
||||||
The Docker topology looks like this:
|
The Docker topology looks like this:
|
||||||
|
|
||||||
@@ -67,11 +67,11 @@ The Docker topology looks like this:
|
|||||||
└─────────────────────────────────────────────────────────────────────┘
|
└─────────────────────────────────────────────────────────────────────┘
|
||||||
```
|
```
|
||||||
|
|
||||||
When the agent exits, `cli.py` tears down every sidecar and both networks; nothing about a bottle persists between runs.
|
When the agent exits, `cli.py` tears down every gateway and both networks; nothing about a bottle persists between runs.
|
||||||
|
|
||||||
## Quickstart
|
## Quickstart
|
||||||
|
|
||||||
On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The Firecracker backend (Linux) requires Docker on the host for the sidecar bundle plus the `firecracker` binary and KVM. The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
|
On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The Firecracker backend (Linux) requires Docker on the host for the gateway plus the `firecracker` binary and KVM. The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
|
||||||
|
|
||||||
Use `BOT_BOTTLE_BACKEND=docker ./cli.py start <agent>` on hosts where neither Apple Container nor KVM is available and Docker is the desired backend.
|
Use `BOT_BOTTLE_BACKEND=docker ./cli.py start <agent>` on hosts where neither Apple Container nor KVM is available and Docker is the desired backend.
|
||||||
|
|
||||||
@@ -81,7 +81,7 @@ On Linux, a KVM-capable host defaults to the Firecracker backend. It needs:
|
|||||||
|
|
||||||
- **`/dev/kvm`** present and accessible. Load `kvm-intel` or `kvm-amd` (and enable virtualization in BIOS/firmware). The invoking user must be in the `kvm` group: `sudo usermod -aG kvm "$USER"` then re-login. bot-bottle preflights this and reports exactly what's missing.
|
- **`/dev/kvm`** present and accessible. Load `kvm-intel` or `kvm-amd` (and enable virtualization in BIOS/firmware). The invoking user must be in the `kvm` group: `sudo usermod -aG kvm "$USER"` then re-login. bot-bottle preflights this and reports exactly what's missing.
|
||||||
- **`firecracker`** on `PATH`: grab a release from <https://github.com/firecracker-microvm/firecracker/releases>. Start flows print this pointer when the binary is missing.
|
- **`firecracker`** on `PATH`: grab a release from <https://github.com/firecracker-microvm/firecracker/releases>. Start flows print this pointer when the binary is missing.
|
||||||
- **Docker** for the sidecar bundle and image build.
|
- **Docker** for the gateway and image build.
|
||||||
- **A one-time privileged network setup** — the per-bottle TAP pool plus the fail-closed `nftables` isolation table. Run `./cli.py backend setup --backend=firecracker` for the host-appropriate config (a NixOS module, a `sudo` script elsewhere); `./cli.py backend status --backend=firecracker` reports what's present, including whether the pool range collides with an existing route. The pool defaults to `10.243.0.0/16` (an obscure RFC-1918 block that dodges docker/libvirt/LAN and, deliberately, Tailscale's `100.64.0.0/10` CGNAT range); override with `BOT_BOTTLE_FC_IP_BASE` if it clashes on your host.
|
- **A one-time privileged network setup** — the per-bottle TAP pool plus the fail-closed `nftables` isolation table. Run `./cli.py backend setup --backend=firecracker` for the host-appropriate config (a NixOS module, a `sudo` script elsewhere); `./cli.py backend status --backend=firecracker` reports what's present, including whether the pool range collides with an existing route. The pool defaults to `10.243.0.0/16` (an obscure RFC-1918 block that dodges docker/libvirt/LAN and, deliberately, Tailscale's `100.64.0.0/10` CGNAT range); override with `BOT_BOTTLE_FC_IP_BASE` if it clashes on your host.
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
|
|||||||
@@ -204,9 +204,9 @@ class AgentProvider(ABC):
|
|||||||
bottle: "Bottle",
|
bottle: "Bottle",
|
||||||
supervise_url: str,
|
supervise_url: str,
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Register the per-bottle supervise sidecar as an MCP server
|
"""Register the per-bottle supervise daemon as an MCP server
|
||||||
in the provider's in-guest config. Called by the backend after
|
in the provider's in-guest config. Called by the backend after
|
||||||
the supervise sidecar is reachable. No-op when
|
the supervise daemon is reachable. No-op when
|
||||||
`plan.supervise_plan is None`."""
|
`plan.supervise_plan is None`."""
|
||||||
|
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
|
|||||||
@@ -199,7 +199,7 @@ class ActiveAgent:
|
|||||||
bottle is the container, the agent is what runs in it.)
|
bottle is the container, the agent is what runs in it.)
|
||||||
|
|
||||||
Fields are deliberately backend-neutral. `services` is the set
|
Fields are deliberately backend-neutral. `services` is the set
|
||||||
of sidecar daemons currently up for this bottle (`egress`,
|
of gateway daemons currently up for this bottle (`egress`,
|
||||||
`git-gate`, `supervise`); the dashboard uses it to
|
`git-gate`, `supervise`); the dashboard uses it to
|
||||||
gate edit verbs. `backend_name` is the matching key in
|
gate edit verbs. `backend_name` is the matching key in
|
||||||
`_BACKENDS` (`docker` / `firecracker` / `macos-container`) — used by the active-
|
`_BACKENDS` (`docker` / `firecracker` / `macos-container`) — used by the active-
|
||||||
@@ -251,7 +251,7 @@ class Bottle(ABC):
|
|||||||
`user` (default `node`, matching the agent image's USER
|
`user` (default `node`, matching the agent image's USER
|
||||||
directive) and return the captured stdout/stderr/returncode.
|
directive) and return the captured stdout/stderr/returncode.
|
||||||
The bottle's environment (including HTTPS_PROXY pointing at
|
The bottle's environment (including HTTPS_PROXY pointing at
|
||||||
the egress sidecar) is inherited by the child. Non-zero
|
the egress daemon) is inherited by the child. Non-zero
|
||||||
exit does not raise — callers inspect `returncode`
|
exit does not raise — callers inspect `returncode`
|
||||||
themselves.
|
themselves.
|
||||||
|
|
||||||
@@ -454,7 +454,7 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
|
|||||||
declarative provision-plan apply, supervise MCP registration)
|
declarative provision-plan apply, supervise MCP registration)
|
||||||
live on the `AgentProvider` plugin. The backend only owns the
|
live on the `AgentProvider` plugin. The backend only owns the
|
||||||
steps that are about backend infrastructure (CA, workspace,
|
steps that are about backend infrastructure (CA, workspace,
|
||||||
git) and surfaces the supervise sidecar URL its launch step
|
git) and surfaces the supervise daemon URL its launch step
|
||||||
knows about via `supervise_mcp_url`.
|
knows about via `supervise_mcp_url`.
|
||||||
|
|
||||||
PRD 0017: cred-proxy's agent-side dotfile rewrites (~/.npmrc,
|
PRD 0017: cred-proxy's agent-side dotfile rewrites (~/.npmrc,
|
||||||
@@ -502,7 +502,7 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
|
|||||||
|
|
||||||
def supervise_mcp_url(self, plan: PlanT) -> str:
|
def supervise_mcp_url(self, plan: PlanT) -> str:
|
||||||
"""Return the agent-side URL of the per-bottle supervise
|
"""Return the agent-side URL of the per-bottle supervise
|
||||||
sidecar, or "" when this bottle has no sidecar. The provider
|
gateway, or "" when this bottle has no gateway. The provider
|
||||||
plugin's `provision_supervise_mcp` uses it to register the
|
plugin's `provision_supervise_mcp` uses it to register the
|
||||||
MCP entry inside the guest.
|
MCP entry inside the guest.
|
||||||
|
|
||||||
@@ -524,7 +524,7 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
|
|||||||
def enumerate_active(self) -> Sequence[ActiveAgent]:
|
def enumerate_active(self) -> Sequence[ActiveAgent]:
|
||||||
"""Return every currently-running agent on this backend.
|
"""Return every currently-running agent on this backend.
|
||||||
Empty when none. Backend-specific: docker queries `docker
|
Empty when none. Backend-specific: docker queries `docker
|
||||||
compose ls`; firecracker cross-references its running sidecar
|
compose ls`; firecracker cross-references its running gateway
|
||||||
containers against per-bottle metadata."""
|
containers against per-bottle metadata."""
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
|
|||||||
@@ -106,7 +106,7 @@ class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanup
|
|||||||
yield bottle
|
yield bottle
|
||||||
|
|
||||||
def supervise_mcp_url(self, plan: DockerBottlePlan) -> str:
|
def supervise_mcp_url(self, plan: DockerBottlePlan) -> str:
|
||||||
"""Docker bottles reach the supervise sidecar via the
|
"""Docker bottles reach the supervise daemon via the
|
||||||
compose-network alias `supervise:9100`. No per-bottle URL
|
compose-network alias `supervise:9100`. No per-bottle URL
|
||||||
plumbing needed; the alias resolves inside the bridge."""
|
plumbing needed; the alias resolves inside the bridge."""
|
||||||
if plan.supervise_plan is None:
|
if plan.supervise_plan is None:
|
||||||
|
|||||||
@@ -1,8 +1,8 @@
|
|||||||
"""Agent-only compose for the consolidated docker backend (PRD 0070).
|
"""Agent-only compose for the consolidated docker backend (PRD 0070).
|
||||||
|
|
||||||
The per-bottle model rendered a compose project with the agent *and* a
|
The per-bottle model rendered a compose project with the agent *and* a
|
||||||
sidecar bundle on two per-bottle networks. In the consolidated model the
|
gateway on two per-bottle networks. In the consolidated model the
|
||||||
sidecars are gone — one shared gateway serves every bottle — so this renders
|
per-bottle companion containers are gone — one shared gateway serves every bottle — so this renders
|
||||||
just the agent, attached to the **external shared gateway network** with the
|
just the agent, attached to the **external shared gateway network** with the
|
||||||
pinned source IP the orchestrator allocated, and pointed at the gateway's
|
pinned source IP the orchestrator allocated, and pointed at the gateway's
|
||||||
address for egress (and, around the proxy, for git-http / supervise).
|
address for egress (and, around the proxy, for git-http / supervise).
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
"""Consolidated bottle launch sequence for the docker backend (PRD 0070).
|
"""Consolidated bottle launch sequence for the docker backend (PRD 0070).
|
||||||
|
|
||||||
Composes the orchestrator primitives into the register/teardown sequence that
|
Composes the orchestrator primitives into the register/teardown sequence that
|
||||||
replaces the per-bottle sidecar bundle:
|
replaces the per-bottle gateway:
|
||||||
|
|
||||||
1. ensure the orchestrator control plane + shared gateway are up;
|
1. ensure the orchestrator control plane + shared gateway are up;
|
||||||
2. allocate the bottle a pinned source IP on the gateway network (the
|
2. allocate the bottle a pinned source IP on the gateway network (the
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ prepare-time routes-yaml rendering itself lives on the
|
|||||||
platform-neutral `Egress` ABC — backends instantiate it directly.
|
platform-neutral `Egress` ABC — backends instantiate it directly.
|
||||||
|
|
||||||
The per-container `.start()` / `.stop()` lifecycle was removed in
|
The per-container `.start()` / `.stop()` lifecycle was removed in
|
||||||
PRD 0024 chunk 3; the sidecar bundle (PRD 0024) runs egress
|
PRD 0024 chunk 3; the gateway (PRD 0024) runs egress
|
||||||
under its python init supervisor."""
|
under its python init supervisor."""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
"""Host-side egress route-apply for the docker backend.
|
"""Host-side egress route-apply for the docker backend.
|
||||||
|
|
||||||
The per-bottle companion container this used to signal (`docker kill
|
The per-bottle companion container this used to signal (`docker kill
|
||||||
--signal HUP <container>`) was removed in the de-sidecar cleanup (#385).
|
--signal HUP <container>`) was removed in the companion-container removal (#385).
|
||||||
In the consolidated model the shared gateway resolves egress policy
|
In the consolidated model the shared gateway resolves egress policy
|
||||||
per-request against the orchestrator rather than reloading a per-bottle
|
per-request against the orchestrator rather than reloading a per-bottle
|
||||||
routes file, so the live per-bottle reload is not supported here and
|
routes file, so the live per-bottle reload is not supported here and
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
bind-mounts target + the listening port. The prepare-time entrypoint
|
bind-mounts target + the listening port. The prepare-time entrypoint
|
||||||
/ hook render lives on the platform-neutral `GitGate` ABC — backends
|
/ hook render lives on the platform-neutral `GitGate` ABC — backends
|
||||||
instantiate it directly. The git-gate daemon's container lifecycle
|
instantiate it directly. The git-gate daemon's container lifecycle
|
||||||
is owned by the sidecar bundle (PRD 0024)."""
|
is owned by the gateway (PRD 0024)."""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
|||||||
@@ -5,7 +5,7 @@ PRD 0018 chunk 3: each instance is one `docker compose` project.
|
|||||||
The flow is:
|
The flow is:
|
||||||
|
|
||||||
1. Build the agent image from the provider Dockerfile (compose
|
1. Build the agent image from the provider Dockerfile (compose
|
||||||
builds the sidecar images via the `build:` directive on first up).
|
builds the gateway image on first up).
|
||||||
2. Mint the per-bottle egress CA (chunk 2 writes it under
|
2. Mint the per-bottle egress CA (chunk 2 writes it under
|
||||||
state/<slug>/egress/).
|
state/<slug>/egress/).
|
||||||
3. Populate the inner plans with launch-time fields so the
|
3. Populate the inner plans with launch-time fields so the
|
||||||
|
|||||||
@@ -76,7 +76,7 @@ def network_create_internal(slug: str) -> str:
|
|||||||
|
|
||||||
def network_create_egress(slug: str) -> str:
|
def network_create_egress(slug: str) -> str:
|
||||||
"""Create a per-agent user-defined bridge (NOT the legacy `bridge`)
|
"""Create a per-agent user-defined bridge (NOT the legacy `bridge`)
|
||||||
so the egress sidecar has working DNS for upstream hostnames."""
|
so the egress daemon has working DNS for upstream hostnames."""
|
||||||
return _network_create_with_prefix(network_egress_name_for_slug(slug), internal=False)
|
return _network_create_with_prefix(network_egress_name_for_slug(slug), internal=False)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
"""Host setup + status for the Docker backend.
|
"""Host setup + status for the Docker backend.
|
||||||
|
|
||||||
Unlike Firecracker, the Docker backend needs no privileged one-time
|
Unlike Firecracker, the Docker backend needs no privileged one-time
|
||||||
host provisioning (no TAP pool / nft table) — networks and the sidecar
|
host provisioning (no TAP pool / nft table) — networks and the gateway
|
||||||
bundle are created per-launch. So `setup()` is mostly an install/daemon
|
bundle are created per-launch. So `setup()` is mostly an install/daemon
|
||||||
pointer, and `status()` reports whether docker is usable.
|
pointer, and `status()` reports whether docker is usable.
|
||||||
|
|
||||||
@@ -45,7 +45,7 @@ def setup() -> int:
|
|||||||
return 1
|
return 1
|
||||||
sys.stderr.write(
|
sys.stderr.write(
|
||||||
"Docker backend: no privileged host setup required — networks and "
|
"Docker backend: no privileged host setup required — networks and "
|
||||||
"the sidecar bundle are created per-launch.\n"
|
"the gateway are created per-launch.\n"
|
||||||
)
|
)
|
||||||
if not _daemon_reachable():
|
if not _daemon_reachable():
|
||||||
sys.stderr.write(
|
sys.stderr.write(
|
||||||
@@ -64,7 +64,7 @@ def setup() -> int:
|
|||||||
def teardown() -> int:
|
def teardown() -> int:
|
||||||
sys.stderr.write(
|
sys.stderr.write(
|
||||||
"Docker backend: nothing to undo — it provisions no privileged host "
|
"Docker backend: nothing to undo — it provisions no privileged host "
|
||||||
"state (networks and the sidecar bundle are per-launch and are "
|
"state (networks and the gateway are per-launch and are "
|
||||||
"removed by `./cli.py cleanup`). Docker itself is left installed.\n"
|
"removed by `./cli.py cleanup`). Docker itself is left installed.\n"
|
||||||
)
|
)
|
||||||
return 0
|
return 0
|
||||||
|
|||||||
@@ -7,7 +7,7 @@ session. `ssh -t` forwards the host terminal's SIGWINCH to the remote
|
|||||||
PTY natively, so no separate resize bridge is needed.
|
PTY natively, so no separate resize bridge is needed.
|
||||||
|
|
||||||
Commands run as the image's `node` user via `runuser`, with HOME/USER/
|
Commands run as the image's `node` user via `runuser`, with HOME/USER/
|
||||||
PATH and the bottle env (HTTPS_PROXY at the sidecar, CA paths, …) set
|
PATH and the bottle env (HTTPS_PROXY at the gateway, CA paths, …) set
|
||||||
per-invocation through `env` (the VM itself just runs the init; it has
|
per-invocation through `env` (the VM itself just runs the init; it has
|
||||||
no baked-in process env like a `docker run` container would).
|
no baked-in process env like a `docker run` container would).
|
||||||
"""
|
"""
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ from .. import BottlePlan
|
|||||||
class FirecrackerBottlePlan(BottlePlan):
|
class FirecrackerBottlePlan(BottlePlan):
|
||||||
slug: str
|
slug: str
|
||||||
forwarded_env: dict[str, str] = field(repr=False)
|
forwarded_env: dict[str, str] = field(repr=False)
|
||||||
# Stamped by launch once the sidecar is up and its ports are
|
# Stamped by launch once the gateway is up and its ports are
|
||||||
# published on the host-side TAP IP (empty at prepare time).
|
# published on the host-side TAP IP (empty at prepare time).
|
||||||
agent_proxy_url: str = ""
|
agent_proxy_url: str = ""
|
||||||
agent_git_gate_url: str = ""
|
agent_git_gate_url: str = ""
|
||||||
@@ -21,7 +21,7 @@ class FirecrackerBottlePlan(BottlePlan):
|
|||||||
|
|
||||||
@property
|
@property
|
||||||
def container_name(self) -> str:
|
def container_name(self) -> str:
|
||||||
"""Instance name, reused for the sidecar container + VM run dir.
|
"""Instance name, reused for the gateway container + VM run dir.
|
||||||
Matches the `bot-bottle-<slug>` convention the other backends
|
Matches the `bot-bottle-<slug>` convention the other backends
|
||||||
use so cleanup/enumerate discovery-by-prefix keeps working."""
|
use so cleanup/enumerate discovery-by-prefix keeps working."""
|
||||||
return self.agent_provision.instance_name
|
return self.agent_provision.instance_name
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
"""Active-agent enumeration for the Firecracker backend.
|
"""Active-agent enumeration for the Firecracker backend.
|
||||||
|
|
||||||
The backend is disabled during the de-sidecar cleanup (#385) — it can't
|
The backend is disabled during the companion-container removal (#385) — it can't
|
||||||
launch bottles, so there are none to enumerate. Real enumeration returns
|
launch bottles, so there are none to enumerate. Real enumeration returns
|
||||||
with the backend's consolidated relaunch (#354).
|
with the backend's consolidated relaunch (#354).
|
||||||
"""
|
"""
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
The firecracker backend launched a per-bottle companion container (the
|
The firecracker backend launched a per-bottle companion container (the
|
||||||
egress / git-gate / supervise data plane) alongside each microVM. That
|
egress / git-gate / supervise data plane) alongside each microVM. That
|
||||||
per-bottle-companion architecture was removed in the de-sidecar cleanup;
|
per-bottle-companion architecture was removed in the companion-container removal;
|
||||||
firecracker's replacement — the consolidated per-host gateway — lands in
|
firecracker's replacement — the consolidated per-host gateway — lands in
|
||||||
its own cutover (#354).
|
its own cutover (#354).
|
||||||
|
|
||||||
|
|||||||
@@ -18,7 +18,7 @@ Topology (per slot i):
|
|||||||
* a /31 host<->guest link: host = base + 2i (the gateway the VM
|
* a /31 host<->guest link: host = base + 2i (the gateway the VM
|
||||||
routes through), guest = base + 2i + 1 (the VM's address).
|
routes through), guest = base + 2i + 1 (the VM's address).
|
||||||
* isolation via ``table inet bot_bottle_fc``: a VM reaches only its
|
* isolation via ``table inet bot_bottle_fc``: a VM reaches only its
|
||||||
own sidecar (DNAT'd from the host TAP IP) and nothing else.
|
own gateway (DNAT'd from the host TAP IP) and nothing else.
|
||||||
|
|
||||||
The default IP block is ``10.243.0.0/16`` — an intentionally obscure
|
The default IP block is ``10.243.0.0/16`` — an intentionally obscure
|
||||||
corner of RFC-1918 private space. RFC-1918 is the range *designated*
|
corner of RFC-1918 private space. RFC-1918 is the range *designated*
|
||||||
@@ -88,7 +88,7 @@ def ip_base() -> str:
|
|||||||
return _cfg("BOT_BOTTLE_FC_IP_BASE")
|
return _cfg("BOT_BOTTLE_FC_IP_BASE")
|
||||||
|
|
||||||
|
|
||||||
# Sidecar ports the VM reaches at its host-side TAP IP. Kept in sync
|
# Gateway ports the VM reaches at its host-side TAP IP. Kept in sync
|
||||||
# with the backend constants (egress 9099, supervise 9100, git-http
|
# with the backend constants (egress 9099, supervise 9100, git-http
|
||||||
# 9420); rendered into the setup output for operator visibility.
|
# 9420); rendered into the setup output for operator visibility.
|
||||||
GATEWAY_PORTS = (9099, 9100, 9420)
|
GATEWAY_PORTS = (9099, 9100, 9420)
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
Selectable via `BOT_BOTTLE_BACKEND=macos-container`. This package owns
|
Selectable via `BOT_BOTTLE_BACKEND=macos-container`. This package owns
|
||||||
the Apple `container` CLI integration; launch remains gated until the
|
the Apple `container` CLI integration; launch remains gated until the
|
||||||
sidecar network enforcement shape is implemented.
|
gateway network enforcement shape is implemented.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from .backend import MacosContainerBottleBackend
|
from .backend import MacosContainerBottleBackend
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
"""Host-side egress route-apply for the macos-container backend.
|
"""Host-side egress route-apply for the macos-container backend.
|
||||||
|
|
||||||
The per-bottle companion container this used to signal (`container kill
|
The per-bottle companion container this used to signal (`container kill
|
||||||
--signal HUP <container>`) was removed in the de-sidecar cleanup (#385),
|
--signal HUP <container>`) was removed in the companion-container removal (#385),
|
||||||
along with the disabled macOS launch path. Fails closed until the macOS
|
along with the disabled macOS launch path. Fails closed until the macOS
|
||||||
backend grows the consolidated gateway.
|
backend grows the consolidated gateway.
|
||||||
"""
|
"""
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
"""Active-agent enumeration for the macOS Apple Container backend.
|
"""Active-agent enumeration for the macOS Apple Container backend.
|
||||||
|
|
||||||
The backend is disabled during the de-sidecar cleanup (#385) — it can't
|
The backend is disabled during the companion-container removal (#385) — it can't
|
||||||
launch bottles, so there are none to enumerate. Enumeration returns when
|
launch bottles, so there are none to enumerate. Enumeration returns when
|
||||||
the backend grows the consolidated gateway.
|
the backend grows the consolidated gateway.
|
||||||
"""
|
"""
|
||||||
|
|||||||
@@ -3,7 +3,7 @@
|
|||||||
This backend launched a per-bottle companion container (the egress /
|
This backend launched a per-bottle companion container (the egress /
|
||||||
git-gate / supervise data plane) alongside the agent container, with the
|
git-gate / supervise data plane) alongside the agent container, with the
|
||||||
agent's proxy env pointed at the companion's host-only IP. That
|
agent's proxy env pointed at the companion's host-only IP. That
|
||||||
per-bottle-companion architecture was removed in the de-sidecar cleanup;
|
per-bottle-companion architecture was removed in the companion-container removal;
|
||||||
the macOS backend will be re-enabled once it grows the consolidated
|
the macOS backend will be re-enabled once it grows the consolidated
|
||||||
per-host gateway the docker backend already uses.
|
per-host gateway the docker backend already uses.
|
||||||
|
|
||||||
|
|||||||
@@ -94,7 +94,7 @@ def prepare_egress(
|
|||||||
|
|
||||||
|
|
||||||
def prepare_supervise(bottle: ManifestBottle, slug: str) -> SupervisePlan | None:
|
def prepare_supervise(bottle: ManifestBottle, slug: str) -> SupervisePlan | None:
|
||||||
"""Prepare the supervise sidecar state dir. Returns None when
|
"""Prepare the supervise daemon state dir. Returns None when
|
||||||
bottle.supervise is falsy."""
|
bottle.supervise is falsy."""
|
||||||
if not bottle.supervise:
|
if not bottle.supervise:
|
||||||
return None
|
return None
|
||||||
|
|||||||
@@ -44,16 +44,16 @@ _STATE_SUBDIR = "state"
|
|||||||
_PER_BOTTLE_DOCKERFILE_NAME = "Dockerfile"
|
_PER_BOTTLE_DOCKERFILE_NAME = "Dockerfile"
|
||||||
_COMMITTED_IMAGE_NAME = "committed-image"
|
_COMMITTED_IMAGE_NAME = "committed-image"
|
||||||
_TRANSCRIPT_SUBDIR = "transcript"
|
_TRANSCRIPT_SUBDIR = "transcript"
|
||||||
# Per-sidecar scratch subdirs. PRD 0018 chunk 2: bind-mount sources
|
# Per-daemon scratch subdirs. PRD 0018 chunk 2: bind-mount sources
|
||||||
# live here so chunk 3's `docker compose up` can find them at stable
|
# live here so chunk 3's `docker compose up` can find them at stable
|
||||||
# paths. Each sidecar's `prepare()` writes config + CAs into its own
|
# paths. Each daemon's `prepare()` writes config + CAs into its own
|
||||||
# subdir; the launch step is unchanged today (still `docker cp`).
|
# subdir; the launch step is unchanged today (still `docker cp`).
|
||||||
_EGRESS_SUBDIR = "egress"
|
_EGRESS_SUBDIR = "egress"
|
||||||
_GIT_GATE_SUBDIR = "git-gate"
|
_GIT_GATE_SUBDIR = "git-gate"
|
||||||
_SUPERVISE_SUBDIR = "supervise"
|
_SUPERVISE_SUBDIR = "supervise"
|
||||||
_AGENT_SUBDIR = "agent"
|
_AGENT_SUBDIR = "agent"
|
||||||
_METADATA_NAME = "metadata.json"
|
_METADATA_NAME = "metadata.json"
|
||||||
# Live-config dir bind-mounted into the supervise sidecar (read-only).
|
# Live-config dir bind-mounted into the supervise daemon (read-only).
|
||||||
# Host's apply paths keep these files fresh so supervise's
|
# Host's apply paths keep these files fresh so supervise's
|
||||||
# `list-egress-routes` MCP tool returns the current state —
|
# `list-egress-routes` MCP tool returns the current state —
|
||||||
# not a snapshot from launch time.
|
# not a snapshot from launch time.
|
||||||
@@ -222,7 +222,7 @@ def per_bottle_image_tag(identity: str) -> str:
|
|||||||
|
|
||||||
def live_config_dir(identity: str) -> Path:
|
def live_config_dir(identity: str) -> Path:
|
||||||
"""Per-bottle live-config dir. Bind-mounted read-only into the
|
"""Per-bottle live-config dir. Bind-mounted read-only into the
|
||||||
supervise sidecar; the host's apply paths refresh the files on
|
supervise daemon; the host's apply paths refresh the files on
|
||||||
every operator approval so the agent's `list-*` MCP tools always
|
every operator approval so the agent's `list-*` MCP tools always
|
||||||
return current state."""
|
return current state."""
|
||||||
return bottle_state_dir(identity) / _LIVE_CONFIG_SUBDIR
|
return bottle_state_dir(identity) / _LIVE_CONFIG_SUBDIR
|
||||||
@@ -260,9 +260,9 @@ def transcript_snapshot_dir(identity: str) -> Path:
|
|||||||
return bottle_state_dir(identity) / _TRANSCRIPT_SUBDIR
|
return bottle_state_dir(identity) / _TRANSCRIPT_SUBDIR
|
||||||
|
|
||||||
|
|
||||||
# --- Per-sidecar scratch subdirs (PRD 0018 chunk 2) ------------------------
|
# --- Per-daemon scratch subdirs (PRD 0018 chunk 2) ------------------------
|
||||||
#
|
#
|
||||||
# Each sidecar gets its own subdir under the bottle's state dir for
|
# Each daemon gets its own subdir under the bottle's state dir for
|
||||||
# bind-mount sources (config, CAs, hooks, etc.). Prepare-time writes
|
# bind-mount sources (config, CAs, hooks, etc.). Prepare-time writes
|
||||||
# land here; the state dir's normal cleanup (`cleanup_state`) reaps
|
# land here; the state dir's normal cleanup (`cleanup_state`) reaps
|
||||||
# them along with everything else when the bottle session ends and
|
# them along with everything else when the bottle session ends and
|
||||||
@@ -270,20 +270,20 @@ def transcript_snapshot_dir(identity: str) -> Path:
|
|||||||
|
|
||||||
|
|
||||||
def egress_state_dir(identity: str) -> Path:
|
def egress_state_dir(identity: str) -> Path:
|
||||||
"""State subdir for the egress sidecar: routes.yaml + the
|
"""State subdir for the egress daemon: routes.yaml + the
|
||||||
per-bottle mitmproxy CA. Bind-mount source from chunk 3 onward."""
|
per-bottle mitmproxy CA. Bind-mount source from chunk 3 onward."""
|
||||||
return bottle_state_dir(identity) / _EGRESS_SUBDIR
|
return bottle_state_dir(identity) / _EGRESS_SUBDIR
|
||||||
|
|
||||||
|
|
||||||
def git_gate_state_dir(identity: str) -> Path:
|
def git_gate_state_dir(identity: str) -> Path:
|
||||||
"""State subdir for the git-gate sidecar: entrypoint + hooks +
|
"""State subdir for the git-gate daemon: entrypoint + hooks +
|
||||||
per-upstream known_hosts. Bind-mount source from chunk 3
|
per-upstream known_hosts. Bind-mount source from chunk 3
|
||||||
onward."""
|
onward."""
|
||||||
return bottle_state_dir(identity) / _GIT_GATE_SUBDIR
|
return bottle_state_dir(identity) / _GIT_GATE_SUBDIR
|
||||||
|
|
||||||
|
|
||||||
def supervise_state_dir(identity: str) -> Path:
|
def supervise_state_dir(identity: str) -> Path:
|
||||||
"""State subdir reserved for supervise sidecar bind-mount sources.
|
"""State subdir reserved for supervise daemon bind-mount sources.
|
||||||
Runtime queue/audit rows live in the host-level bot-bottle SQLite
|
Runtime queue/audit rows live in the host-level bot-bottle SQLite
|
||||||
database, so they survive state-dir cleanup."""
|
database, so they survive state-dir cleanup."""
|
||||||
return bottle_state_dir(identity) / _SUPERVISE_SUBDIR
|
return bottle_state_dir(identity) / _SUPERVISE_SUBDIR
|
||||||
|
|||||||
@@ -2,8 +2,8 @@
|
|||||||
|
|
||||||
Walks every registered backend (docker, firecracker, macos-container)
|
Walks every registered backend (docker, firecracker, macos-container)
|
||||||
so a single `./cli.py cleanup` reaps every backend's leftovers — a
|
so a single `./cli.py cleanup` reaps every backend's leftovers — a
|
||||||
firecracker bottle's sidecars won't survive a docker-only cleanup pass
|
firecracker bottle's VM processes and run dirs won't survive a
|
||||||
(issue addressed alongside #77).
|
docker-only cleanup pass (issue addressed alongside #77).
|
||||||
|
|
||||||
Each backend's `prepare_cleanup` enumerates its own resources;
|
Each backend's `prepare_cleanup` enumerates its own resources;
|
||||||
docker's `_list_orphan_state_dirs` consults
|
docker's `_list_orphan_state_dirs` consults
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ The Claude-specific behavior previously inlined under
|
|||||||
`agent_provider.agent_provision_plan` (claude.json trust marker,
|
`agent_provider.agent_provision_plan` (claude.json trust marker,
|
||||||
api.anthropic.com egress route, OAuth-token placeholder), plus
|
api.anthropic.com egress route, OAuth-token placeholder), plus
|
||||||
the `claude mcp add` invocation that registers the supervise
|
the `claude mcp add` invocation that registers the supervise
|
||||||
sidecar in claude-code's user config (PRD 0013)."""
|
gateway in claude-code's user config (PRD 0013)."""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
@@ -293,7 +293,7 @@ class ClaudeAgentProvider(AgentProvider):
|
|||||||
supervise_url: str,
|
supervise_url: str,
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Run `claude mcp add` inside the agent guest to register the
|
"""Run `claude mcp add` inside the agent guest to register the
|
||||||
supervise sidecar in claude-code's user config (~/.claude.json).
|
supervise daemon in claude-code's user config (~/.claude.json).
|
||||||
|
|
||||||
Failure is logged but not fatal — the bottle still works without
|
Failure is logged but not fatal — the bottle still works without
|
||||||
the entry; the operator can register it manually."""
|
the entry; the operator can register it manually."""
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ The Codex-specific behavior previously inlined under
|
|||||||
`agent_provider.agent_provision_plan` (config.toml trust marker,
|
`agent_provider.agent_provision_plan` (config.toml trust marker,
|
||||||
chatgpt.com / api.openai.com egress routes, optional host-credential
|
chatgpt.com / api.openai.com egress routes, optional host-credential
|
||||||
forwarding with dummy-auth.json + verify), plus the `codex mcp add`
|
forwarding with dummy-auth.json + verify), plus the `codex mcp add`
|
||||||
invocation that registers the supervise sidecar in Codex's
|
invocation that registers the supervise daemon in Codex's
|
||||||
~/.codex/config.toml (PRD 0050)."""
|
~/.codex/config.toml (PRD 0050)."""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
@@ -266,7 +266,7 @@ class CodexAgentProvider(AgentProvider):
|
|||||||
supervise_url: str,
|
supervise_url: str,
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Run `codex mcp add` inside the agent guest to register the
|
"""Run `codex mcp add` inside the agent guest to register the
|
||||||
supervise sidecar in Codex's user config (~/.codex/config.toml).
|
supervise daemon in Codex's user config (~/.codex/config.toml).
|
||||||
|
|
||||||
Mirrors the Claude provider's `claude mcp add` flow — failure
|
Mirrors the Claude provider's `claude mcp add` flow — failure
|
||||||
is logged but not fatal."""
|
is logged but not fatal."""
|
||||||
|
|||||||
@@ -3,7 +3,7 @@
|
|||||||
Pure Python, no mitmproxy dependency. Each detector is a module-level
|
Pure Python, no mitmproxy dependency. Each detector is a module-level
|
||||||
function returning `ScanResult | None`.
|
function returning `ScanResult | None`.
|
||||||
|
|
||||||
Ships flat into the sidecar bundle image alongside
|
Ships flat into the gateway image alongside
|
||||||
`egress_addon_core.py` — both this file and the package source use
|
`egress_addon_core.py` — both this file and the package source use
|
||||||
the same try/except import shim pattern.
|
the same try/except import shim pattern.
|
||||||
"""
|
"""
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
This module defines the abstract proxy (`Egress`), its plan
|
This module defines the abstract proxy (`Egress`), its plan
|
||||||
dataclass (`EgressPlan`), and the resolved per-route shape
|
dataclass (`EgressPlan`), and the resolved per-route shape
|
||||||
(`EgressRoute`). The sidecar's start/stop lifecycle is backend-
|
(`EgressRoute`). The gateway's start/stop lifecycle is backend-
|
||||||
specific and lives on concrete subclasses (see
|
specific and lives on concrete subclasses (see
|
||||||
`bot_bottle/backend/docker/egress.py`).
|
`bot_bottle/backend/docker/egress.py`).
|
||||||
"""
|
"""
|
||||||
@@ -87,7 +87,7 @@ class EgressRoute(Route):
|
|||||||
|
|
||||||
Inherits `host`, `matches`, `auth_scheme`, and `token_env`
|
Inherits `host`, `matches`, `auth_scheme`, and `token_env`
|
||||||
from `egress_addon_core.Route` — those are the fields that cross the
|
from `egress_addon_core.Route` — those are the fields that cross the
|
||||||
YAML wire into the sidecar. The fields below are host-only and
|
YAML wire into the gateway. The fields below are host-only and
|
||||||
are never serialised to the addon.
|
are never serialised to the addon.
|
||||||
|
|
||||||
`token_ref` is the host env var the CLI reads at launch and forwards
|
`token_ref` is the host env var the CLI reads at launch and forwards
|
||||||
@@ -386,7 +386,7 @@ class Egress(ABC):
|
|||||||
routes_path.write_text(egress_render_routes(routes, log=log))
|
routes_path.write_text(egress_render_routes(routes, log=log))
|
||||||
routes_path.chmod(0o600)
|
routes_path.chmod(0o600)
|
||||||
# Generate a per-session fake secret under a plausible random env name.
|
# Generate a per-session fake secret under a plausible random env name.
|
||||||
# The sidecar marks that exact env name as sensitive for known-secret
|
# The gateway marks that exact env name as sensitive for known-secret
|
||||||
# scanning; the agent receives the same name/value as exfil bait.
|
# scanning; the agent receives the same name/value as exfil bait.
|
||||||
canary = secrets.token_urlsafe(32)
|
canary = secrets.token_urlsafe(32)
|
||||||
return EgressPlan(
|
return EgressPlan(
|
||||||
|
|||||||
@@ -240,7 +240,7 @@ class EgressAddon:
|
|||||||
tokens, resolved by source IP in one round-trip (fail-closed to deny-all
|
tokens, resolved by source IP in one round-trip (fail-closed to deny-all
|
||||||
+ empty slug if unattributed); `env` is the process env overlaid with
|
+ empty slug if unattributed); `env` is the process env overlaid with
|
||||||
the bottle's tokens, so upstream-auth injection (and DLP) use *this*
|
the bottle's tokens, so upstream-auth injection (and DLP) use *this*
|
||||||
bottle's credentials — exactly what the per-bottle sidecar's env did.
|
bottle's credentials — exactly what the per-bottle gateway daemon's env did.
|
||||||
The identity token, if the agent injected one, is read then stripped so
|
The identity token, if the agent injected one, is read then stripped so
|
||||||
it never leaks upstream."""
|
it never leaks upstream."""
|
||||||
if self._resolver is None:
|
if self._resolver is None:
|
||||||
|
|||||||
@@ -7,7 +7,7 @@ exercise the parse + decision functions without depending on the
|
|||||||
container.
|
container.
|
||||||
|
|
||||||
Imports: stdlib + `yaml_subset` (which is itself stdlib-only and
|
Imports: stdlib + `yaml_subset` (which is itself stdlib-only and
|
||||||
ships flat into the sidecar bundle image alongside this file —
|
ships flat into the gateway image alongside this file —
|
||||||
see `Dockerfile.gateway`)."""
|
see `Dockerfile.gateway`)."""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|||||||
@@ -6,7 +6,7 @@ and what the proxy does when an outbound detector matches a token
|
|||||||
kept apart from the request-time scan/decision flow in `egress_addon_core`
|
kept apart from the request-time scan/decision flow in `egress_addon_core`
|
||||||
so each half reads top-to-bottom without scrolling past the other.
|
so each half reads top-to-bottom without scrolling past the other.
|
||||||
|
|
||||||
Stdlib-only; ships flat into the sidecar bundle image alongside
|
Stdlib-only; ships flat into the gateway image alongside
|
||||||
`egress_addon_core.py` — see `Dockerfile.gateway`."""
|
`egress_addon_core.py` — see `Dockerfile.gateway`."""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|||||||
@@ -1,8 +1,8 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# Egress daemon entrypoint inside the sidecar bundle (PRD 0024).
|
# Egress daemon entrypoint inside the gateway (PRD 0024).
|
||||||
#
|
#
|
||||||
# Extracted verbatim from Dockerfile.egress's prior inline `sh -c`
|
# Extracted verbatim from Dockerfile.egress's prior inline `sh -c`
|
||||||
# ENTRYPOINT so the supervisor in bot_bottle/sidecar_init.py can
|
# ENTRYPOINT so the supervisor in bot_bottle/gateway_init.py can
|
||||||
# call it as a normal child. Behavior is unchanged:
|
# call it as a normal child. Behavior is unchanged:
|
||||||
#
|
#
|
||||||
# * Upstream proxy: when EGRESS_UPSTREAM_PROXY is set, switch
|
# * Upstream proxy: when EGRESS_UPSTREAM_PROXY is set, switch
|
||||||
@@ -22,7 +22,7 @@ set -e
|
|||||||
|
|
||||||
# Pin mitmproxy's config dir to the bind-mount location of its CA
|
# Pin mitmproxy's config dir to the bind-mount location of its CA
|
||||||
# regardless of which user mitmdump runs as. In the legacy
|
# regardless of which user mitmdump runs as. In the legacy
|
||||||
# four-sidecar setup (Dockerfile.egress, USER mitmproxy) this
|
# four-daemon setup (Dockerfile.egress, USER mitmproxy) this
|
||||||
# resolved naturally to `~mitmproxy/.mitmproxy`. In the PRD 0024
|
# resolved naturally to `~mitmproxy/.mitmproxy`. In the PRD 0024
|
||||||
# bundle (USER root) `~root/.mitmproxy` is empty, so without this
|
# bundle (USER root) `~root/.mitmproxy` is empty, so without this
|
||||||
# flag mitmdump would generate a fresh CA on the wrong path and
|
# flag mitmdump would generate a fresh CA on the wrong path and
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
"""Per-agent git-gate (PRD 0008).
|
"""Per-agent git-gate (PRD 0008).
|
||||||
|
|
||||||
A third per-agent sidecar that fronts the bottle's declared git
|
A third per-agent daemon that fronts the bottle's declared git
|
||||||
upstreams as a transparent mirror. Each `bottle.git` entry maps to
|
upstreams as a transparent mirror. Each `bottle.git` entry maps to
|
||||||
a bare repo on the gate; `git daemon` serves the bare repos over
|
a bare repo on the gate; `git daemon` serves the bare repos over
|
||||||
`git://<gate>/<name>.git`. Two hooks make the mirror bidirectional:
|
`git://<gate>/<name>.git`. Two hooks make the mirror bidirectional:
|
||||||
@@ -15,7 +15,7 @@ a bare repo on the gate; `git daemon` serves the bare repos over
|
|||||||
|
|
||||||
The agent never sees the upstream credential under either path.
|
The agent never sees the upstream credential under either path.
|
||||||
|
|
||||||
Why a separate sidecar (not folded into egress or ssh-gate): the
|
Why a separate daemon (not folded into egress or ssh-gate): the
|
||||||
gate is the only one of the three that holds upstream push
|
gate is the only one of the three that holds upstream push
|
||||||
credentials. Mixing it with egress would put push creds in the
|
credentials. Mixing it with egress would put push creds in the
|
||||||
same blast radius as internet-facing TLS interception; mixing it
|
same blast radius as internet-facing TLS interception; mixing it
|
||||||
@@ -23,7 +23,7 @@ with ssh-gate would force ssh-gate above L4 and into git-protocol
|
|||||||
land. See `docs/prds/0008-git-gate.md`.
|
land. See `docs/prds/0008-git-gate.md`.
|
||||||
|
|
||||||
This module defines the abstract gate (`GitGate`) and its plan
|
This module defines the abstract gate (`GitGate`) and its plan
|
||||||
dataclass (`GitGatePlan`). The sidecar's start/stop lifecycle is
|
dataclass (`GitGatePlan`). The gateway's start/stop lifecycle is
|
||||||
backend-specific and lives on concrete subclasses (see
|
backend-specific and lives on concrete subclasses (see
|
||||||
`bot_bottle/backend/docker/git_gate.py`)."""
|
`bot_bottle/backend/docker/git_gate.py`)."""
|
||||||
|
|
||||||
@@ -86,7 +86,7 @@ class GitGatePlan:
|
|||||||
|
|
||||||
class GitGate(ABC):
|
class GitGate(ABC):
|
||||||
"""The per-agent git-gate. Encapsulates the host-side prepare
|
"""The per-agent git-gate. Encapsulates the host-side prepare
|
||||||
(upstream lift + entrypoint/hook render); the sidecar's
|
(upstream lift + entrypoint/hook render); the gateway's
|
||||||
start/stop lifecycle is backend-specific and lives on concrete
|
start/stop lifecycle is backend-specific and lives on concrete
|
||||||
subclasses."""
|
subclasses."""
|
||||||
|
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
"""Pure host-side rendering for the per-agent git-gate (PRD 0008).
|
"""Pure host-side rendering for the per-agent git-gate (PRD 0008).
|
||||||
|
|
||||||
Builds the agent's `.gitconfig` insteadOf rewrites, the known_hosts
|
Builds the agent's `.gitconfig` insteadOf rewrites, the known_hosts
|
||||||
line, and the entrypoint / pre-receive / access-hook scripts the sidecar
|
line, and the entrypoint / pre-receive / access-hook scripts the gateway
|
||||||
runs. No docker or forge calls — exposed for tests and reuse across
|
runs. No docker or forge calls — exposed for tests and reuse across
|
||||||
backends. Split out of `git_gate.py` so the control surface (`GitGate`)
|
backends. Split out of `git_gate.py` so the control surface (`GitGate`)
|
||||||
and the deploy-key lifecycle (`git_gate_provision`) each read on their
|
and the deploy-key lifecycle (`git_gate_provision`) each read on their
|
||||||
@@ -16,7 +16,7 @@ from pathlib import Path
|
|||||||
|
|
||||||
from .manifest import ManifestBottle, ManifestGitEntry
|
from .manifest import ManifestBottle, ManifestGitEntry
|
||||||
|
|
||||||
# Short network alias for git-gate inside the sidecar bundle. The
|
# Short network alias for git-gate inside the gateway. The
|
||||||
# agent's `.gitconfig` insteadOf rewrites resolve through this name.
|
# agent's `.gitconfig` insteadOf rewrites resolve through this name.
|
||||||
GIT_GATE_HOSTNAME = "git-gate"
|
GIT_GATE_HOSTNAME = "git-gate"
|
||||||
# Shared timeout (seconds) for all git-gate subprocess and CGI calls:
|
# Shared timeout (seconds) for all git-gate subprocess and CGI calls:
|
||||||
@@ -38,7 +38,7 @@ class GitGateUpstream:
|
|||||||
KnownHostKey string from the manifest; the gate's start step
|
KnownHostKey string from the manifest; the gate's start step
|
||||||
materialises it into a known_hosts file if non-empty.
|
materialises it into a known_hosts file if non-empty.
|
||||||
|
|
||||||
the gate credential paths inside the running sidecar."""
|
the gate credential paths inside the running gateway."""
|
||||||
|
|
||||||
name: str
|
name: str
|
||||||
upstream_url: str
|
upstream_url: str
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
Used where `git://` push traffic over a host-published Docker port can
|
Used where `git://` push traffic over a host-published Docker port can
|
||||||
hang before receive-pack reaches hooks (e.g. the firecracker backend,
|
hang before receive-pack reaches hooks (e.g. the firecracker backend,
|
||||||
where the guest reaches the sidecar over the point-to-point TAP). The
|
where the guest reaches the gateway over the point-to-point TAP). The
|
||||||
wrapper serves the same `/git/*.git` bare repos through
|
wrapper serves the same `/git/*.git` bare repos through
|
||||||
`git http-backend`, so pre-receive and upstream forwarding remain the
|
`git http-backend`, so pre-receive and upstream forwarding remain the
|
||||||
git-gate enforcement point.
|
git-gate enforcement point.
|
||||||
@@ -27,7 +27,7 @@ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from urllib.parse import urlsplit
|
from urllib.parse import urlsplit
|
||||||
|
|
||||||
# policy_resolver ships flat alongside this file in the sidecar bundle
|
# policy_resolver ships flat alongside this file in the gateway
|
||||||
# image (see Dockerfile.gateway); the bot_bottle.* fallback is the
|
# image (see Dockerfile.gateway); the bot_bottle.* fallback is the
|
||||||
# host-side / test path. Mirrors egress_addon's import shape.
|
# host-side / test path. Mirrors egress_addon's import shape.
|
||||||
try:
|
try:
|
||||||
@@ -103,7 +103,7 @@ def resolve_sandbox_root(
|
|||||||
return namespace
|
return namespace
|
||||||
|
|
||||||
# Mirrors git_gate_render.GIT_GATE_TIMEOUT_SECS. Duplicated rather than
|
# Mirrors git_gate_render.GIT_GATE_TIMEOUT_SECS. Duplicated rather than
|
||||||
# imported: this module ships as a flat top-level sibling in the sidecar
|
# imported: this module ships as a flat top-level sibling in the gateway
|
||||||
# bundle image (see Dockerfile.gateway), not as part of the bot_bottle
|
# bundle image (see Dockerfile.gateway), not as part of the bot_bottle
|
||||||
# package, so `bot_bottle.git_gate` and its dependency chain aren't
|
# package, so `bot_bottle.git_gate` and its dependency chain aren't
|
||||||
# available at runtime.
|
# available at runtime.
|
||||||
|
|||||||
@@ -17,7 +17,7 @@ class ManifestAgentProvider:
|
|||||||
|
|
||||||
`template` selects a built-in launch/runtime contract. `dockerfile`
|
`template` selects a built-in launch/runtime contract. `dockerfile`
|
||||||
optionally points at a custom agent-image Dockerfile while leaving
|
optionally points at a custom agent-image Dockerfile while leaving
|
||||||
bot-bottle's sidecar infrastructure intact.
|
bot-bottle's gateway infrastructure intact.
|
||||||
|
|
||||||
`auth_token` names the host env var that holds the provider's OAuth
|
`auth_token` names the host env var that holds the provider's OAuth
|
||||||
token (Claude only). The provisioner injects a provider-owned egress
|
token (Claude only). The provisioner injects a provider-owned egress
|
||||||
@@ -26,7 +26,7 @@ class ManifestAgentProvider:
|
|||||||
so the Claude Code CLI starts.
|
so the Claude Code CLI starts.
|
||||||
|
|
||||||
`forward_host_credentials` forwards the host Codex auth token into
|
`forward_host_credentials` forwards the host Codex auth token into
|
||||||
the egress sidecar (Codex only).
|
the egress daemon (Codex only).
|
||||||
"""
|
"""
|
||||||
|
|
||||||
template: str = "claude"
|
template: str = "claude"
|
||||||
|
|||||||
@@ -39,10 +39,10 @@ class ManifestBottle:
|
|||||||
# identity without any git-gate.repos upstreams, and vice versa.
|
# identity without any git-gate.repos upstreams, and vice versa.
|
||||||
git_user: ManifestGitUser = field(default_factory=ManifestGitUser)
|
git_user: ManifestGitUser = field(default_factory=ManifestGitUser)
|
||||||
egress: ManifestEgressConfig = field(default_factory=ManifestEgressConfig)
|
egress: ManifestEgressConfig = field(default_factory=ManifestEgressConfig)
|
||||||
# Per-bottle stuck-recovery sidecar (PRD 0013). When true (the
|
# Per-bottle stuck-recovery daemon (PRD 0013). When true (the
|
||||||
# default, issue #249), the launch step brings up a supervise
|
# default, issue #249), the launch step brings up a supervise
|
||||||
# sidecar that exposes egress MCP tools to the agent. Set
|
# daemon that exposes egress MCP tools to the agent. Set
|
||||||
# `supervise: false` to skip the sidecar.
|
# `supervise: false` to skip the gateway.
|
||||||
supervise: bool = True
|
supervise: bool = True
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
@@ -61,7 +61,7 @@ class ManifestBottle:
|
|||||||
raise ManifestError(
|
raise ManifestError(
|
||||||
f"bottle '{name}' has an 'ssh' field, which has been removed "
|
f"bottle '{name}' has an 'ssh' field, which has been removed "
|
||||||
f"(PRD 0009). Declare upstreams under 'git-gate.repos' with "
|
f"(PRD 0009). Declare upstreams under 'git-gate.repos' with "
|
||||||
f"url + identity + host_key; the git-gate sidecar (PRD 0008) "
|
f"url + identity + host_key; the git-gate daemon (PRD 0008) "
|
||||||
f"holds the credential and gitleaks-scans pushes."
|
f"holds the credential and gitleaks-scans pushes."
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
@@ -39,7 +39,7 @@ def main(argv: list[str] | None = None) -> int:
|
|||||||
)
|
)
|
||||||
parser.add_argument(
|
parser.add_argument(
|
||||||
"--gateway", action="store_true",
|
"--gateway", action="store_true",
|
||||||
help="run one consolidated per-host sidecar bundle (build-if-missing)",
|
help="run one consolidated per-host gateway (build-if-missing)",
|
||||||
)
|
)
|
||||||
args = parser.parse_args(argv)
|
args = parser.parse_args(argv)
|
||||||
|
|
||||||
|
|||||||
@@ -2,12 +2,12 @@
|
|||||||
|
|
||||||
On a verified launch request it starts a Docker container; on teardown it
|
On a verified launch request it starts a Docker container; on teardown it
|
||||||
removes it. This proves the orchestrator -> backend seam on the cheapest
|
removes it. This proves the orchestrator -> backend seam on the cheapest
|
||||||
backend (the sidecar bundle is already containers). Only the request's
|
backend (the gateway is already containers). Only the request's
|
||||||
static ids/flags reach `docker`, so nothing free-form crosses the boundary.
|
static ids/flags reach `docker`, so nothing free-form crosses the boundary.
|
||||||
|
|
||||||
Slice 3 launches a single container from the request's `image_ref`, named
|
Slice 3 launches a single container from the request's `image_ref`, named
|
||||||
after the bottle id and labelled for cleanup. Wiring the full agent +
|
after the bottle id and labelled for cleanup. Wiring the full agent +
|
||||||
sidecar bundle (networks, mounts, the consolidated sidecar) is a later
|
gateway (networks, mounts, the consolidated gateway) is a later
|
||||||
slice — this is the seam, not the finished launcher.
|
slice — this is the seam, not the finished launcher.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
"""The consolidated per-host gateway (PRD 0070).
|
"""The consolidated per-host gateway (PRD 0070).
|
||||||
|
|
||||||
The core consolidation win: **one** persistent gateway per host, shared by
|
The core consolidation win: **one** persistent gateway per host, shared by
|
||||||
every bottle, instead of a sidecar bundle per bottle. It's safe to share
|
every bottle, instead of a gateway per bottle. It's safe to share
|
||||||
because the attribution invariant (source IP + identity token, see
|
because the attribution invariant (source IP + identity token, see
|
||||||
`registry`) lets the gateway attribute each request to the right bottle —
|
`registry`) lets the gateway attribute each request to the right bottle —
|
||||||
so per-bottle policy lives in one long-lived process keyed on who's calling.
|
so per-bottle policy lives in one long-lived process keyed on who's calling.
|
||||||
@@ -46,7 +46,7 @@ GATEWAY_CA_VOLUME = "bot-bottle-gateway-mitmproxy"
|
|||||||
GATEWAY_CA_CERT = f"{MITMPROXY_HOME}/mitmproxy-ca-cert.pem"
|
GATEWAY_CA_CERT = f"{MITMPROXY_HOME}/mitmproxy-ca-cert.pem"
|
||||||
|
|
||||||
# The gateway data-plane image + its Dockerfile. Kept as a local constant
|
# The gateway data-plane image + its Dockerfile. Kept as a local constant
|
||||||
# rather than imported from backend.docker.sidecar_bundle, which would drag
|
# rather than imported from the backend layer, which would drag
|
||||||
# the whole backend layer into the lean orchestrator (see #359); unify when
|
# the whole backend layer into the lean orchestrator (see #359); unify when
|
||||||
# that lands. Env override matches the backend's BOT_BOTTLE_GATEWAY_IMAGE.
|
# that lands. Env override matches the backend's BOT_BOTTLE_GATEWAY_IMAGE.
|
||||||
GATEWAY_IMAGE = os.environ.get("BOT_BOTTLE_GATEWAY_IMAGE", "bot-bottle-gateway:latest")
|
GATEWAY_IMAGE = os.environ.get("BOT_BOTTLE_GATEWAY_IMAGE", "bot-bottle-gateway:latest")
|
||||||
|
|||||||
@@ -5,7 +5,7 @@ registry: turns a prepared bottle's egress plan into the backend-neutral
|
|||||||
inputs `Orchestrator.launch_bottle` takes — the egress **policy** blob and
|
inputs `Orchestrator.launch_bottle` takes — the egress **policy** blob and
|
||||||
launch **metadata**.
|
launch **metadata**.
|
||||||
|
|
||||||
The policy blob is the exact routes YAML the per-bottle egress sidecar used
|
The policy blob is the exact routes YAML the per-bottle egress daemon used
|
||||||
to read from a file; in the consolidated model the multi-tenant gateway's
|
to read from a file; in the consolidated model the multi-tenant gateway's
|
||||||
`PolicyResolver` fetches it from the registry per request (keyed by source
|
`PolicyResolver` fetches it from the registry per request (keyed by source
|
||||||
IP) instead. Same render, so consolidated and single-tenant egress apply
|
IP) instead. Same render, so consolidated and single-tenant egress apply
|
||||||
|
|||||||
+2
-2
@@ -10,7 +10,7 @@ suite points it at a throwaway dir instead of monkey-patching the function
|
|||||||
override covers them all), and operators can relocate the root if needed.
|
override covers them all), and operators can relocate the root if needed.
|
||||||
|
|
||||||
This module has no bot-bottle imports, so it is safe to import from any
|
This module has no bot-bottle imports, so it is safe to import from any
|
||||||
layer (and to COPY flat into the sidecar bundle).
|
layer (and to COPY flat into the gateway).
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
@@ -35,7 +35,7 @@ def host_db_path() -> Path:
|
|||||||
|
|
||||||
Kept in its own `db/` subdirectory (not directly under the root) so a
|
Kept in its own `db/` subdirectory (not directly under the root) so a
|
||||||
backend that can only bind-mount *directories* can share this one file
|
backend that can only bind-mount *directories* can share this one file
|
||||||
with a sidecar without exposing the root's other contents (git-gate
|
with a gateway without exposing the root's other contents (git-gate
|
||||||
keys, per-bottle state, ...)."""
|
keys, per-bottle state, ...)."""
|
||||||
return bot_bottle_root() / "db" / HOST_DB_FILENAME
|
return bot_bottle_root() / "db" / HOST_DB_FILENAME
|
||||||
|
|
||||||
|
|||||||
@@ -23,7 +23,7 @@ closed too rather than silently serving stale or empty policy.
|
|||||||
The resolved value is the policy blob the orchestrator stores verbatim; the
|
The resolved value is the policy blob the orchestrator stores verbatim; the
|
||||||
consumer parses it (e.g. the egress addon's `load_config`). This module is
|
consumer parses it (e.g. the egress addon's `load_config`). This module is
|
||||||
stdlib-only and free of bot-bottle imports so it can be COPYed flat into
|
stdlib-only and free of bot-bottle imports so it can be COPYed flat into
|
||||||
the sidecar bundle.
|
the gateway.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|||||||
@@ -26,7 +26,7 @@ class QueueStore(DbStore):
|
|||||||
if db_path is not None:
|
if db_path is not None:
|
||||||
resolved = db_path
|
resolved = db_path
|
||||||
else:
|
else:
|
||||||
# In the sidecar container SUPERVISE_DB_PATH points at the
|
# In the gateway container SUPERVISE_DB_PATH points at the
|
||||||
# bind-mounted host DB. On the host this env var is never set,
|
# bind-mounted host DB. On the host this env var is never set,
|
||||||
# so we always fall through to host_db_path().
|
# so we always fall through to host_db_path().
|
||||||
env_path = os.environ.get("SUPERVISE_DB_PATH", "").strip()
|
env_path = os.environ.get("SUPERVISE_DB_PATH", "").strip()
|
||||||
|
|||||||
+14
-14
@@ -1,25 +1,25 @@
|
|||||||
"""Per-bottle supervise plane (PRD 0013).
|
"""Per-bottle supervise plane (PRD 0013).
|
||||||
|
|
||||||
The supervise plane is the per-bottle MCP sidecar plus its host-side
|
The supervise plane is the per-bottle MCP daemon plus its host-side
|
||||||
queue/audit support. The sidecar (bot_bottle.supervise_server)
|
queue/audit support. The daemon (bot_bottle.supervise_server)
|
||||||
sits on the bottle's internal network and exposes MCP tools the agent
|
sits on the bottle's internal network and exposes MCP tools the agent
|
||||||
calls when it needs an operator-reviewed egress change:
|
calls when it needs an operator-reviewed egress change:
|
||||||
|
|
||||||
* egress-block / allow — agent proposes a new routes.yaml
|
* egress-block / allow — agent proposes a new routes.yaml
|
||||||
|
|
||||||
Each tool call: the agent passes the full proposed file plus a
|
Each tool call: the agent passes the full proposed file plus a
|
||||||
justification text. The sidecar validates the proposal syntactically,
|
justification text. The gateway validates the proposal syntactically,
|
||||||
writes it to the host SQLite queue table, and holds the tool-call
|
writes it to the host SQLite queue table, and holds the tool-call
|
||||||
connection open. The operator's supervise TUI
|
connection open. The operator's supervise TUI
|
||||||
(bot_bottle.cli.supervise) sees the proposal, accepts
|
(bot_bottle.cli.supervise) sees the proposal, accepts
|
||||||
approve / modify / reject, and writes a response row. The sidecar sees
|
approve / modify / reject, and writes a response row. The gateway sees
|
||||||
the response and returns `{status, notes}` to the agent.
|
the response and returns `{status, notes}` to the agent.
|
||||||
|
|
||||||
This module defines the host-side library: dataclasses for the queue
|
This module defines the host-side library: dataclasses for the queue
|
||||||
record shapes, queue read/write helpers, the audit log writer, and the
|
record shapes, queue read/write helpers, the audit log writer, and the
|
||||||
diff renderer. The in-container sidecar lives in
|
diff renderer. The in-gateway daemon lives in
|
||||||
bot_bottle/supervise_server.py; the supervise daemon's container
|
bot_bottle/supervise_server.py; the supervise daemon's container
|
||||||
lifecycle is owned by the sidecar bundle (PRD 0024).
|
lifecycle is owned by the gateway (PRD 0024).
|
||||||
|
|
||||||
For 0013 the supervisor's approval handlers are deliberately no-ops:
|
For 0013 the supervisor's approval handlers are deliberately no-ops:
|
||||||
on approval the audit log is written and the response file is
|
on approval the audit log is written and the response file is
|
||||||
@@ -75,18 +75,18 @@ except ImportError:
|
|||||||
|
|
||||||
try:
|
try:
|
||||||
from .paths import bot_bottle_root
|
from .paths import bot_bottle_root
|
||||||
except ImportError: # flat imports inside the sidecar bundle
|
except ImportError: # flat imports inside the gateway
|
||||||
from paths import bot_bottle_root # type: ignore[import-not-found,no-redef] # pylint: disable=import-error,no-name-in-module
|
from paths import bot_bottle_root # type: ignore[import-not-found,no-redef] # pylint: disable=import-error,no-name-in-module
|
||||||
|
|
||||||
|
|
||||||
SUPERVISE_HOSTNAME = "supervise"
|
SUPERVISE_HOSTNAME = "supervise"
|
||||||
SUPERVISE_PORT = 9100
|
SUPERVISE_PORT = 9100
|
||||||
|
|
||||||
# The supervise sidecar uses these to query egress's
|
# The supervise daemon uses these to query egress's
|
||||||
# introspection endpoint for the `list-egress-routes` MCP
|
# introspection endpoint for the `list-egress-routes` MCP
|
||||||
# tool. The hostname + port match egress's docker network
|
# tool. The hostname + port match egress's docker network
|
||||||
# listen port (see backend.docker.egress.EGRESS_PORT). The supervise
|
# listen port (see backend.docker.egress.EGRESS_PORT). The supervise
|
||||||
# daemon runs inside the sidecar bundle alongside egress, so loopback
|
# daemon runs inside the gateway alongside egress, so loopback
|
||||||
# is the stable address across docker, firecracker, and Apple
|
# is the stable address across docker, firecracker, and Apple
|
||||||
# Container backends.
|
# Container backends.
|
||||||
EGRESS_FORWARD_PROXY = "http://127.0.0.1:9099"
|
EGRESS_FORWARD_PROXY = "http://127.0.0.1:9099"
|
||||||
@@ -117,7 +117,7 @@ try:
|
|||||||
from .audit_store import AuditStore
|
from .audit_store import AuditStore
|
||||||
from .store_manager import StoreManager
|
from .store_manager import StoreManager
|
||||||
except ImportError:
|
except ImportError:
|
||||||
# Sidecar bundle: files are flat-copied under /app, not a package.
|
# Gateway: files are flat-copied under /app, not a package.
|
||||||
from queue_store import QueueStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
|
from queue_store import QueueStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
|
||||||
from audit_store import AuditStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
|
from audit_store import AuditStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
|
||||||
from store_manager import StoreManager # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
|
from store_manager import StoreManager # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
|
||||||
@@ -222,14 +222,14 @@ def sha256_hex(content: str) -> str:
|
|||||||
return hashlib.sha256(content.encode("utf-8")).hexdigest()
|
return hashlib.sha256(content.encode("utf-8")).hexdigest()
|
||||||
|
|
||||||
|
|
||||||
# --- Sidecar plan + abstract lifecycle -------------------------------------
|
# --- Gateway plan + abstract lifecycle -------------------------------------
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class SupervisePlan:
|
class SupervisePlan:
|
||||||
"""Output of Supervise.prepare; consumed by .start.
|
"""Output of Supervise.prepare; consumed by .start.
|
||||||
|
|
||||||
`db_path` is the host database bind-mounted into the sidecar at
|
`db_path` is the host database bind-mounted into the gateway at
|
||||||
/run/supervise/bot-bottle.db. `internal_network` is empty at
|
/run/supervise/bot-bottle.db. `internal_network` is empty at
|
||||||
prepare time; the backend's launch step fills it via
|
prepare time; the backend's launch step fills it via
|
||||||
dataclasses.replace before calling .start."""
|
dataclasses.replace before calling .start."""
|
||||||
@@ -240,8 +240,8 @@ class SupervisePlan:
|
|||||||
|
|
||||||
|
|
||||||
class Supervise(ABC):
|
class Supervise(ABC):
|
||||||
"""Per-bottle supervise sidecar. Encapsulates host-side database
|
"""Per-bottle supervise daemon. Encapsulates host-side database
|
||||||
staging; the sidecar's start/stop lifecycle is backend-specific."""
|
staging; the gateway's start/stop lifecycle is backend-specific."""
|
||||||
|
|
||||||
def prepare(
|
def prepare(
|
||||||
self,
|
self,
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
"""Supervise sidecar HTTP server (PRD 0013).
|
"""Supervise daemon HTTP server (PRD 0013).
|
||||||
|
|
||||||
Per-bottle MCP server exposing tools the agent calls to propose egress
|
Per-bottle MCP server exposing tools the agent calls to propose egress
|
||||||
config changes when stuck. The tools are `egress-allow`,
|
config changes when stuck. The tools are `egress-allow`,
|
||||||
|
|||||||
@@ -71,7 +71,7 @@ class YamlSubsetError(ValueError):
|
|||||||
that want fatal-exit semantics (manifest loader, egress-apply,
|
that want fatal-exit semantics (manifest loader, egress-apply,
|
||||||
etc.) catch this at their own boundary and forward to `die`;
|
etc.) catch this at their own boundary and forward to `die`;
|
||||||
callers running outside the bot-bottle CLI process (the
|
callers running outside the bot-bottle CLI process (the
|
||||||
egress sidecar's addon) handle it as a normal exception."""
|
egress daemon's addon) handle it as a normal exception."""
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
+1
-2
@@ -22,8 +22,7 @@ mounted in. That topology breaks two assumptions those tests make:
|
|||||||
`http://127.0.0.1:<host_port>` from inside the job time out.
|
`http://127.0.0.1:<host_port>` from inside the job time out.
|
||||||
|
|
||||||
The affected tests (`test_orphan_cleanup.test_create_and_remove`,
|
The affected tests (`test_orphan_cleanup.test_create_and_remove`,
|
||||||
`test_sidecar_bundle_image.TestSidecarBundleImage`,
|
`test_gateway_image.TestGatewayImage`) still run
|
||||||
`test_sidecar_bundle_compose.TestSidecarBundleCompose`) still run
|
|
||||||
locally where the test process and Docker daemon share a host.
|
locally where the test process and Docker daemon share a host.
|
||||||
Making them work in CI is a follow-up: either re-write them to
|
Making them work in CI is a follow-up: either re-write them to
|
||||||
discover container IPs via `docker inspect`, or reconfigure the
|
discover container IPs via `docker inspect`, or reconfigure the
|
||||||
|
|||||||
+2
-2
@@ -38,7 +38,7 @@ Type "y"
|
|||||||
Enter
|
Enter
|
||||||
|
|
||||||
# Wait for the bottle to launch: networks created, pipelock + git-gate
|
# Wait for the bottle to launch: networks created, pipelock + git-gate
|
||||||
# sidecars started, agent container started, claude boots.
|
# companion containers started, agent container started, claude boots.
|
||||||
Sleep 22s
|
Sleep 22s
|
||||||
|
|
||||||
# Probe 1 — warm-up. A reply at all proves api.anthropic.com is
|
# Probe 1 — warm-up. A reply at all proves api.anthropic.com is
|
||||||
@@ -72,7 +72,7 @@ Type "init /tmp/r, commit AKIAQRJHK7N5ZPM2VXTL to leak.txt, push to ssh://git@up
|
|||||||
Enter
|
Enter
|
||||||
Sleep 30s
|
Sleep 30s
|
||||||
|
|
||||||
# Leave claude. The launcher tears down the container, sidecars, and
|
# Leave claude. The launcher tears down the container, companion containers, and
|
||||||
# networks on session end.
|
# networks on session end.
|
||||||
Ctrl+D
|
Ctrl+D
|
||||||
Sleep 4s
|
Sleep 4s
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
#
|
#
|
||||||
# The one-time privileged setup the Firecracker backend needs: a pool of
|
# The one-time privileged setup the Firecracker backend needs: a pool of
|
||||||
# user- (or group-) owned point-to-point TAP devices plus a fail-closed
|
# user- (or group-) owned point-to-point TAP devices plus a fail-closed
|
||||||
# nftables table that confines every microVM to its own sidecar.
|
# nftables table that confines every microVM to its own gateway.
|
||||||
#
|
#
|
||||||
# NON-INVASIVE BY DESIGN. It does NOT flip `networking.nftables.enable`
|
# NON-INVASIVE BY DESIGN. It does NOT flip `networking.nftables.enable`
|
||||||
# (which would switch your whole host firewall backend) or
|
# (which would switch your whole host firewall backend) or
|
||||||
@@ -150,7 +150,7 @@ in
|
|||||||
}
|
}
|
||||||
];
|
];
|
||||||
|
|
||||||
# VM->sidecar traffic is DNAT'd and forwarded, so forwarding must be on.
|
# VM->gateway traffic is DNAT'd and forwarded, so forwarding must be on.
|
||||||
boot.kernel.sysctl."net.ipv4.ip_forward" = 1;
|
boot.kernel.sysctl."net.ipv4.ip_forward" = 1;
|
||||||
|
|
||||||
# One oneshot brings up the whole pool (TAPs + independent nft table)
|
# One oneshot brings up the whole pool (TAPs + independent nft table)
|
||||||
|
|||||||
@@ -4,7 +4,7 @@
|
|||||||
# Creates a pool of point-to-point TAP devices (owned by the invoking
|
# Creates a pool of point-to-point TAP devices (owned by the invoking
|
||||||
# user so the backend can open them without root at launch) and a
|
# user so the backend can open them without root at launch) and a
|
||||||
# dedicated nftables table that isolates every VM: a bottle VM can
|
# dedicated nftables table that isolates every VM: a bottle VM can
|
||||||
# reach only its own sidecar (published on the host-side TAP IP) and
|
# reach only its own gateway (published on the host-side TAP IP) and
|
||||||
# nothing else on the host or network.
|
# nothing else on the host or network.
|
||||||
#
|
#
|
||||||
# Why a pool + one-time setup: creating a TAP and assigning it an IP
|
# Why a pool + one-time setup: creating a TAP and assigning it an IP
|
||||||
@@ -72,9 +72,9 @@ for _v in POOL_SIZE IP_BASE PREFIX TABLE; do
|
|||||||
[ -n "${!_v}" ] || { echo "error: $_v unresolved (set BOT_BOTTLE_FC_* or fix $_DEFAULTS)" >&2; exit 1; }
|
[ -n "${!_v}" ] || { echo "error: $_v unresolved (set BOT_BOTTLE_FC_* or fix $_DEFAULTS)" >&2; exit 1; }
|
||||||
done
|
done
|
||||||
|
|
||||||
# Sidecar ports (must match the backend). egress=9099, supervise=9100,
|
# Gateway ports (must match the backend). egress=9099, supervise=9100,
|
||||||
# git-http=9420. Reached by the VM at its host-side TAP IP.
|
# git-http=9420. Reached by the VM at its host-side TAP IP.
|
||||||
SIDECAR_PORTS="9099,9100,9420"
|
GATEWAY_PORTS="9099,9100,9420"
|
||||||
|
|
||||||
# --- IP math ---------------------------------------------------------
|
# --- IP math ---------------------------------------------------------
|
||||||
# Slot i occupies the /31 {base+2i, base+2i+1}: host = base+2i (the
|
# Slot i occupies the /31 {base+2i, base+2i+1}: host = base+2i (the
|
||||||
@@ -107,7 +107,7 @@ cmd_up() {
|
|||||||
fi
|
fi
|
||||||
echo "firecracker net pool: $POOL_SIZE slots, base $IP_BASE, $own_desc"
|
echo "firecracker net pool: $POOL_SIZE slots, base $IP_BASE, $own_desc"
|
||||||
|
|
||||||
# VM->sidecar traffic is DNAT'd to the sidecar container and
|
# VM->gateway traffic is DNAT'd to the gateway container and
|
||||||
# forwarded, so forwarding must be enabled (Docker also sets this).
|
# forwarded, so forwarding must be enabled (Docker also sets this).
|
||||||
sysctl -qw net.ipv4.ip_forward=1
|
sysctl -qw net.ipv4.ip_forward=1
|
||||||
|
|
||||||
@@ -135,11 +135,11 @@ _install_nft() {
|
|||||||
# tool's traffic is affected. Priority -10 runs before Docker's
|
# tool's traffic is affected. Priority -10 runs before Docker's
|
||||||
# filter hooks (priority 0); a drop here is terminal for the packet.
|
# filter hooks (priority 0); a drop here is terminal for the packet.
|
||||||
#
|
#
|
||||||
# forward: VM egress is DNAT'd to the sidecar (established via
|
# forward: VM egress is DNAT'd to the gateway (established via
|
||||||
# `ct status dnat`); return traffic via `ct state established`.
|
# `ct status dnat`); return traffic via `ct state established`.
|
||||||
# Anything else from a VM is dropped -> no route to the internet
|
# Anything else from a VM is dropped -> no route to the internet
|
||||||
# or the rest of the host except through the sidecar proxy.
|
# or the rest of the host except through the gateway proxy.
|
||||||
# input: a VM never needs host-local delivery (its sidecar is
|
# input: a VM never needs host-local delivery (its gateway is
|
||||||
# reached via DNAT->forward), so drop all direct input from VMs
|
# reached via DNAT->forward), so drop all direct input from VMs
|
||||||
# -> host services bound on 0.0.0.0 are unreachable from the VM.
|
# -> host services bound on 0.0.0.0 are unreachable from the VM.
|
||||||
nft -f - <<EOF
|
nft -f - <<EOF
|
||||||
|
|||||||
+3
-7
@@ -18,8 +18,7 @@ tests/
|
|||||||
test_manifest_runtime.py
|
test_manifest_runtime.py
|
||||||
... # many others; see unit/ directory
|
... # many others; see unit/ directory
|
||||||
integration/
|
integration/
|
||||||
test_sidecar_bundle_image.py
|
test_gateway_image.py
|
||||||
test_sidecar_bundle_compose.py
|
|
||||||
test_dry_run_plan.py
|
test_dry_run_plan.py
|
||||||
test_orphan_cleanup.py
|
test_orphan_cleanup.py
|
||||||
...
|
...
|
||||||
@@ -48,12 +47,9 @@ Discovery is invoked with `-t .` (top-level dir = repo root) so the
|
|||||||
the bottle's runtime, and creates zero Docker resources.
|
the bottle's runtime, and creates zero Docker resources.
|
||||||
- `test_orphan_cleanup.py` — `network_remove` is idempotent against
|
- `test_orphan_cleanup.py` — `network_remove` is idempotent against
|
||||||
missing resources, so the EXIT trap can call it unconditionally.
|
missing resources, so the EXIT trap can call it unconditionally.
|
||||||
- `test_sidecar_bundle_image.py` — builds Dockerfile.gateway and
|
- `test_gateway_image.py` — builds Dockerfile.gateway and
|
||||||
probes that gitleaks / mitmdump / supervise are all reachable
|
probes that gitleaks / mitmdump / supervise are all reachable
|
||||||
inside the bundle.
|
inside the gateway image.
|
||||||
- `test_sidecar_bundle_compose.py` — end-to-end compose-up of an
|
|
||||||
agent + bundle pair; verifies the agent reaches the bundle via
|
|
||||||
the legacy network aliases.
|
|
||||||
|
|
||||||
## Canaries
|
## Canaries
|
||||||
|
|
||||||
|
|||||||
+5
-5
@@ -1,4 +1,4 @@
|
|||||||
"""Integration: PRD 0024 chunk 1 — the sidecar bundle image builds
|
"""Integration: PRD 0024 chunk 1 — the gateway image builds
|
||||||
and the daemon binaries are present + executable inside it.
|
and the daemon binaries are present + executable inside it.
|
||||||
|
|
||||||
This test does NOT exercise the daemons running against real
|
This test does NOT exercise the daemons running against real
|
||||||
@@ -10,7 +10,7 @@ the chunk-1 contract:
|
|||||||
pull, COPYs resolve).
|
pull, COPYs resolve).
|
||||||
- gitleaks, mitmdump are at the documented paths and answer
|
- gitleaks, mitmdump are at the documented paths and answer
|
||||||
`--version`.
|
`--version`.
|
||||||
- The Python init at /app/sidecar_init.py runs and prints the
|
- The Python init at /app/gateway_init.py runs and prints the
|
||||||
expected "no daemons selected" line when the supervisor is
|
expected "no daemons selected" line when the supervisor is
|
||||||
pointed at an empty daemon set.
|
pointed at an empty daemon set.
|
||||||
|
|
||||||
@@ -36,10 +36,10 @@ _DOCKERFILE = "Dockerfile.gateway"
|
|||||||
@unittest.skipIf(
|
@unittest.skipIf(
|
||||||
os.environ.get("GITEA_ACTIONS") == "true",
|
os.environ.get("GITEA_ACTIONS") == "true",
|
||||||
"skipped under act_runner: multi-stage build pulls a 200+MB "
|
"skipped under act_runner: multi-stage build pulls a 200+MB "
|
||||||
"mitmproxy base + two upstream sidecar images; runner storage "
|
"mitmproxy base + two upstream gateway images; runner storage "
|
||||||
"+ time budget make this an interactive-only test",
|
"+ time budget make this an interactive-only test",
|
||||||
)
|
)
|
||||||
class TestSidecarBundleImage(unittest.TestCase):
|
class TestGatewayImage(unittest.TestCase):
|
||||||
"""Builds the image once for the class, then runs a few
|
"""Builds the image once for the class, then runs a few
|
||||||
`docker run` probes against it."""
|
`docker run` probes against it."""
|
||||||
|
|
||||||
@@ -103,7 +103,7 @@ class TestSidecarBundleImage(unittest.TestCase):
|
|||||||
# ENTRYPOINT wiring works.
|
# ENTRYPOINT wiring works.
|
||||||
proc = subprocess.run(
|
proc = subprocess.run(
|
||||||
["docker", "run", "--rm",
|
["docker", "run", "--rm",
|
||||||
"-e", "BOT_BOTTLE_SIDECAR_DAEMONS=nothing",
|
"-e", "BOT_BOTTLE_GATEWAY_DAEMONS=nothing",
|
||||||
_IMAGE],
|
_IMAGE],
|
||||||
stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
|
stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
|
||||||
timeout=10.0,
|
timeout=10.0,
|
||||||
@@ -1,7 +1,7 @@
|
|||||||
"""Integration: DockerGateway.image_exists reflects real docker state.
|
"""Integration: DockerGateway.image_exists reflects real docker state.
|
||||||
|
|
||||||
Gated on a reachable Docker daemon. Deliberately does NOT build the full
|
Gated on a reachable Docker daemon. Deliberately does NOT build the full
|
||||||
sidecar bundle (a heavy, slow image build) — it exercises the `image_exists`
|
gateway (a heavy, slow image build) — it exercises the `image_exists`
|
||||||
primitive that `ensure_built` gates on, against a tiny pulled image and a
|
primitive that `ensure_built` gates on, against a tiny pulled image and a
|
||||||
name that can't exist.
|
name that can't exist.
|
||||||
"""
|
"""
|
||||||
|
|||||||
@@ -6,7 +6,7 @@ resources.
|
|||||||
|
|
||||||
The PipelockProxy.stop idempotency case that used to live here was
|
The PipelockProxy.stop idempotency case that used to live here was
|
||||||
removed in PRD 0024 chunk 3 when the per-container .stop method
|
removed in PRD 0024 chunk 3 when the per-container .stop method
|
||||||
went away — sidecar teardown is now compose's responsibility, and
|
went away — gateway teardown is now compose's responsibility, and
|
||||||
`compose down` already no-ops on missing containers."""
|
`compose down` already no-ops on missing containers."""
|
||||||
|
|
||||||
import os
|
import os
|
||||||
|
|||||||
@@ -72,7 +72,7 @@ _DUMMY_HOST_KEY = (
|
|||||||
os.environ.get("GITEA_ACTIONS") == "true",
|
os.environ.get("GITEA_ACTIONS") == "true",
|
||||||
"skipped under act_runner: egress_tls_init uses a host bind mount "
|
"skipped under act_runner: egress_tls_init uses a host bind mount "
|
||||||
"the runner container can't see, and the network topology hides "
|
"the runner container can't see, and the network topology hides "
|
||||||
"sibling-sidecar visibility — same constraint as the other "
|
"sibling-gateway visibility — same constraint as the other "
|
||||||
"bottle-bringup integration tests",
|
"bottle-bringup integration tests",
|
||||||
)
|
)
|
||||||
class TestSandboxEscape(unittest.TestCase):
|
class TestSandboxEscape(unittest.TestCase):
|
||||||
@@ -90,8 +90,8 @@ class TestSandboxEscape(unittest.TestCase):
|
|||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def setUpClass(cls) -> None:
|
def setUpClass(cls) -> None:
|
||||||
# Docker is always required (the agent + sidecars run under it,
|
# Docker is always required (the agent + companion containers run under it,
|
||||||
# and VM backends still use it for the sidecar bundle); the
|
# and VM backends still use it for the gateway); the
|
||||||
# class-level @skip_unless_docker already covers that. Pin
|
# class-level @skip_unless_docker already covers that. Pin
|
||||||
# Docker when BOT_BOTTLE_BACKEND is unset to preserve the
|
# Docker when BOT_BOTTLE_BACKEND is unset to preserve the
|
||||||
# Docker-backed CI path.
|
# Docker-backed CI path.
|
||||||
@@ -121,7 +121,7 @@ class TestSandboxEscape(unittest.TestCase):
|
|||||||
"egress": {
|
"egress": {
|
||||||
"routes": [{"host": "api.anthropic.com"}],
|
"routes": [{"host": "api.anthropic.com"}],
|
||||||
},
|
},
|
||||||
# git-gate sidecar so attack 5 can push. Upstream
|
# git-gate daemon so attack 5 can push. Upstream
|
||||||
# is intentionally unreachable — the pre-receive
|
# is intentionally unreachable — the pre-receive
|
||||||
# gitleaks hook must reject BEFORE git-gate
|
# gitleaks hook must reject BEFORE git-gate
|
||||||
# attempts the upstream push. A preset `host_key`
|
# attempts the upstream push. A preset `host_key`
|
||||||
@@ -275,7 +275,7 @@ class TestSandboxEscape(unittest.TestCase):
|
|||||||
|
|
||||||
def _assert_sandbox_block(self, label: str, r: object) -> None: # type: ignore
|
def _assert_sandbox_block(self, label: str, r: object) -> None: # type: ignore
|
||||||
"""A real sandbox block produces an HTTP 403 with a
|
"""A real sandbox block produces an HTTP 403 with a
|
||||||
recognizable sandbox sidecar marker in the body. ANY
|
recognizable sandbox gateway marker in the body. ANY
|
||||||
other outcome (200 from upstream, 401/404 from upstream,
|
other outcome (200 from upstream, 401/404 from upstream,
|
||||||
non-marker 5xx) means the request escaped — the secret
|
non-marker 5xx) means the request escaped — the secret
|
||||||
reached the network."""
|
reached the network."""
|
||||||
|
|||||||
@@ -5,7 +5,7 @@ no test ever reads or writes the real ``~/.bot-bottle`` (state, queue,
|
|||||||
and audit dirs all derive from ``paths.bot_bottle_root()`` →
|
and audit dirs all derive from ``paths.bot_bottle_root()`` →
|
||||||
``Path.home()``). Without this, a test that takes a ``flock`` on the
|
``Path.home()``). Without this, a test that takes a ``flock`` on the
|
||||||
real audit log can **block indefinitely** when a live bottle's supervise
|
real audit log can **block indefinitely** when a live bottle's supervise
|
||||||
sidecar holds that lock — observed as a hung ``coverage run`` at 0% CPU —
|
gateway holds that lock — observed as a hung ``coverage run`` at 0% CPU —
|
||||||
and unisolated tests otherwise pollute the developer's home dir.
|
and unisolated tests otherwise pollute the developer's home dir.
|
||||||
|
|
||||||
Individual tests that need their own ``HOME`` still override
|
Individual tests that need their own ``HOME`` still override
|
||||||
|
|||||||
@@ -19,8 +19,8 @@ class TestConsolidatedAgentCompose(unittest.TestCase):
|
|||||||
plan = type(plan)(**{**vars(plan), "use_runsc": True}) # type: ignore[arg-type]
|
plan = type(plan)(**{**vars(plan), "use_runsc": True}) # type: ignore[arg-type]
|
||||||
return consolidated_agent_compose(plan, gateway_ip=_GW, source_ip=_IP, network=_NET)
|
return consolidated_agent_compose(plan, gateway_ip=_GW, source_ip=_IP, network=_NET)
|
||||||
|
|
||||||
def test_only_agent_service_no_sidecars(self) -> None:
|
def test_only_agent_service_no_companion_container(self) -> None:
|
||||||
# The whole point of consolidation: no per-bottle sidecar bundle.
|
# The whole point of consolidation: no per-bottle gateway.
|
||||||
self.assertEqual(["agent"], list(self._spec()["services"]))
|
self.assertEqual(["agent"], list(self._spec()["services"]))
|
||||||
|
|
||||||
def test_agent_pinned_on_external_gateway_network(self) -> None:
|
def test_agent_pinned_on_external_gateway_network(self) -> None:
|
||||||
@@ -35,7 +35,7 @@ class TestConsolidatedAgentCompose(unittest.TestCase):
|
|||||||
# git-http + supervise on the gateway must bypass the egress proxy.
|
# git-http + supervise on the gateway must bypass the egress proxy.
|
||||||
self.assertTrue(any(e.startswith("NO_PROXY=") and _GW in e for e in env))
|
self.assertTrue(any(e.startswith("NO_PROXY=") and _GW in e for e in env))
|
||||||
|
|
||||||
def test_no_sidecar_dependency(self) -> None:
|
def test_no_companion_container_dependency(self) -> None:
|
||||||
self.assertNotIn("depends_on", self._spec()["services"]["agent"])
|
self.assertNotIn("depends_on", self._spec()["services"]["agent"])
|
||||||
|
|
||||||
def test_runsc_runtime_when_enabled(self) -> None:
|
def test_runsc_runtime_when_enabled(self) -> None:
|
||||||
|
|||||||
@@ -586,7 +586,7 @@ class TestCanaryGeneration(unittest.TestCase):
|
|||||||
|
|
||||||
|
|
||||||
class TestEgressEnvEntries(unittest.TestCase):
|
class TestEgressEnvEntries(unittest.TestCase):
|
||||||
def test_sidecar_entries_include_route_tokens_and_canary_scan_prefix(self):
|
def test_gateway_entries_include_route_tokens_and_canary_scan_prefix(self):
|
||||||
plan = EgressPlan(
|
plan = EgressPlan(
|
||||||
slug="s",
|
slug="s",
|
||||||
routes_path=Path("/tmp/r.yaml"),
|
routes_path=Path("/tmp/r.yaml"),
|
||||||
|
|||||||
@@ -951,7 +951,7 @@ class TestScanOutbound(unittest.TestCase):
|
|||||||
body='{"jsonrpc":"2.0","method":"initialize"}',
|
body='{"jsonrpc":"2.0","method":"initialize"}',
|
||||||
)
|
)
|
||||||
self.assertIsNone(scan_outbound(route, text, {
|
self.assertIsNone(scan_outbound(route, text, {
|
||||||
"EGRESS_TOKEN_0": "sidecar-owned-secret",
|
"EGRESS_TOKEN_0": "gateway-owned-secret",
|
||||||
}))
|
}))
|
||||||
|
|
||||||
def test_token_in_body_blocked(self):
|
def test_token_in_body_blocked(self):
|
||||||
@@ -1302,7 +1302,7 @@ class TestScanOutboundEnhanced(unittest.TestCase):
|
|||||||
self.assertEqual("warn", result.severity)
|
self.assertEqual("warn", result.severity)
|
||||||
|
|
||||||
def test_bot_bottle_sensitive_prefixes_env_var(self):
|
def test_bot_bottle_sensitive_prefixes_env_var(self):
|
||||||
# When the sidecar env contains BOT_BOTTLE_SENSITIVE_PREFIXES,
|
# When the gateway env contains BOT_BOTTLE_SENSITIVE_PREFIXES,
|
||||||
# scan_outbound should scan those additional prefixes.
|
# scan_outbound should scan those additional prefixes.
|
||||||
secret = "extra-sensitive-value-abc"
|
secret = "extra-sensitive-value-abc"
|
||||||
env = {
|
env = {
|
||||||
@@ -1324,7 +1324,7 @@ class TestScanOutboundEnhanced(unittest.TestCase):
|
|||||||
self.assertIsNotNone(result)
|
self.assertIsNotNone(result)
|
||||||
|
|
||||||
def test_canary_detected_via_random_secret_env_name(self):
|
def test_canary_detected_via_random_secret_env_name(self):
|
||||||
# The fake secret uses a randomized env name that the sidecar marks
|
# The fake secret uses a randomized env name that the gateway marks
|
||||||
# as sensitive through BOT_BOTTLE_SENSITIVE_PREFIXES.
|
# as sensitive through BOT_BOTTLE_SENSITIVE_PREFIXES.
|
||||||
canary = "canaryvalue12345abcdef"
|
canary = "canaryvalue12345abcdef"
|
||||||
env = {
|
env = {
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
"""Unit: LOG_FULL credential redaction in _log_request / _log_response (issue #257).
|
"""Unit: LOG_FULL credential redaction in _log_request / _log_response (issue #257).
|
||||||
|
|
||||||
egress_addon.py is sidecar-only code that depends on mitmproxy, which is
|
egress_addon.py is gateway-only code that depends on mitmproxy, which is
|
||||||
not installed on the host. This file pre-populates sys.modules with the
|
not installed on the host. This file pre-populates sys.modules with the
|
||||||
minimum mocks needed so EgressAddon can be imported and tested without the
|
minimum mocks needed so EgressAddon can be imported and tested without the
|
||||||
real mitmproxy package."""
|
real mitmproxy package."""
|
||||||
@@ -17,7 +17,7 @@ from unittest.mock import patch
|
|||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Sidecar-import shims — must run before importing egress_addon
|
# Gateway-import shims — must run before importing egress_addon
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
def _ensure_shims() -> None:
|
def _ensure_shims() -> None:
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
"""Unit: EgressAddon request/response decision flow (issue #286).
|
"""Unit: EgressAddon request/response decision flow (issue #286).
|
||||||
|
|
||||||
`egress_addon.py` is the sidecar-only mitmproxy adapter that wires the
|
`egress_addon.py` is the gateway-only mitmproxy adapter that wires the
|
||||||
host-importable decision logic in `egress_addon_core` into mitmproxy's
|
host-importable decision logic in `egress_addon_core` into mitmproxy's
|
||||||
request/response hooks. The core logic is exercised directly by
|
request/response hooks. The core logic is exercised directly by
|
||||||
`test_egress_addon_core.py`; the redaction logging by
|
`test_egress_addon_core.py`; the redaction logging by
|
||||||
@@ -13,7 +13,7 @@ from coverage.
|
|||||||
mitmproxy is not installed on the host, so we pre-populate `sys.modules`
|
mitmproxy is not installed on the host, so we pre-populate `sys.modules`
|
||||||
with the minimum stubs needed to import the adapter (a `mitmproxy.http`
|
with the minimum stubs needed to import the adapter (a `mitmproxy.http`
|
||||||
module exposing a `Response` with `.make`, plus the flat
|
module exposing a `Response` with `.make`, plus the flat
|
||||||
`egress_addon_core` name the sidecar uses)."""
|
`egress_addon_core` name the gateway uses)."""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
@@ -158,7 +158,7 @@ class _WebSocketData:
|
|||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Sidecar-import shims — must run before importing egress_addon
|
# Gateway-import shims — must run before importing egress_addon
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
@@ -303,9 +303,9 @@ class TestAuthInjection(unittest.TestCase):
|
|||||||
route = Route(host="api.example.com", auth_scheme="Bearer", token_env="EGRESS_TOKEN_0")
|
route = Route(host="api.example.com", auth_scheme="Bearer", token_env="EGRESS_TOKEN_0")
|
||||||
addon = _addon(Config(routes=(route,)))
|
addon = _addon(Config(routes=(route,)))
|
||||||
flow = _Flow(_Request(host="api.example.com", headers={"authorization": "Bearer agent-faked"}))
|
flow = _Flow(_Request(host="api.example.com", headers={"authorization": "Bearer agent-faked"}))
|
||||||
with patch.dict("os.environ", {"EGRESS_TOKEN_0": "real-sidecar-token"}):
|
with patch.dict("os.environ", {"EGRESS_TOKEN_0": "real-gateway-token"}):
|
||||||
_run_request(addon, flow)
|
_run_request(addon, flow)
|
||||||
self.assertEqual("Bearer real-sidecar-token", flow.request.headers.get("authorization"))
|
self.assertEqual("Bearer real-gateway-token", flow.request.headers.get("authorization"))
|
||||||
self.assertIsNone(flow.response)
|
self.assertIsNone(flow.response)
|
||||||
|
|
||||||
def test_auth_route_with_unset_env_blocks(self) -> None:
|
def test_auth_route_with_unset_env_blocks(self) -> None:
|
||||||
|
|||||||
@@ -72,7 +72,7 @@ class TestApplyRoutesChange(unittest.TestCase):
|
|||||||
|
|
||||||
def test_apply_routes_change_fails_closed_after_companion_removal(self):
|
def test_apply_routes_change_fails_closed_after_companion_removal(self):
|
||||||
# The per-bottle companion container that live route-apply used to
|
# The per-bottle companion container that live route-apply used to
|
||||||
# signal was removed in the de-sidecar cleanup (#385); apply now
|
# signal was removed in the companion-container removal (#385); apply now
|
||||||
# fails closed until the gateway-side apply lands.
|
# fails closed until the gateway-side apply lands.
|
||||||
with self.assertRaises(EgressApplyError) as cm:
|
with self.assertRaises(EgressApplyError) as cm:
|
||||||
applicator.apply_routes_change(
|
applicator.apply_routes_change(
|
||||||
|
|||||||
@@ -218,9 +218,9 @@ class TestHookRender(unittest.TestCase):
|
|||||||
self.assertIn("supervisor approved # gitleaks:allow", hook)
|
self.assertIn("supervisor approved # gitleaks:allow", hook)
|
||||||
self.assertIn("supervisor rejected # gitleaks:allow", hook)
|
self.assertIn("supervisor rejected # gitleaks:allow", hook)
|
||||||
|
|
||||||
def test_inline_gitleaks_allow_python_imports_work_in_sidecar_layout(self):
|
def test_inline_gitleaks_allow_python_imports_work_in_gateway_layout(self):
|
||||||
hook = git_gate_render_hook()
|
hook = git_gate_render_hook()
|
||||||
# The sidecar image copies supervise.py flat under /app, while
|
# The gateway image copies supervise.py flat under /app, while
|
||||||
# host-side tests import it through the bot_bottle package.
|
# host-side tests import it through the bot_bottle package.
|
||||||
# Hooks execute from the bare repo directory, so the embedded
|
# Hooks execute from the bare repo directory, so the embedded
|
||||||
# Python must include /app and support both import layouts.
|
# Python must include /app and support both import layouts.
|
||||||
|
|||||||
@@ -220,7 +220,7 @@ class TestGitHttpBackend(unittest.TestCase):
|
|||||||
|
|
||||||
def test_subprocess_calls_include_timeout(self):
|
def test_subprocess_calls_include_timeout(self):
|
||||||
"""Both subprocess.run calls (access-hook and git http-backend) must
|
"""Both subprocess.run calls (access-hook and git http-backend) must
|
||||||
pass timeout= so a hung upstream cannot wedge the sidecar."""
|
pass timeout= so a hung upstream cannot wedge the gateway."""
|
||||||
from http.server import ThreadingHTTPServer
|
from http.server import ThreadingHTTPServer
|
||||||
|
|
||||||
with tempfile.TemporaryDirectory() as tmp:
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
|||||||
@@ -16,12 +16,12 @@ class TestMacosContainerCleanup(unittest.TestCase):
|
|||||||
completed = cleanup.subprocess.CompletedProcess(
|
completed = cleanup.subprocess.CompletedProcess(
|
||||||
args=[],
|
args=[],
|
||||||
returncode=0,
|
returncode=0,
|
||||||
stdout="bot-bottle-a\nbot-bottle-sidecars-a\nother\n",
|
stdout="bot-bottle-a\nbot-bottle-b\nother\n",
|
||||||
stderr="",
|
stderr="",
|
||||||
)
|
)
|
||||||
with patch.object(cleanup.subprocess, "run", return_value=completed):
|
with patch.object(cleanup.subprocess, "run", return_value=completed):
|
||||||
self.assertEqual(
|
self.assertEqual(
|
||||||
["bot-bottle-a", "bot-bottle-sidecars-a"],
|
["bot-bottle-a", "bot-bottle-b"],
|
||||||
cleanup._list_prefixed_containers(),
|
cleanup._list_prefixed_containers(),
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -44,7 +44,7 @@ class TestMacosContainerCleanup(unittest.TestCase):
|
|||||||
|
|
||||||
class TestMacosContainerEnumerate(unittest.TestCase):
|
class TestMacosContainerEnumerate(unittest.TestCase):
|
||||||
def test_enumerate_active_is_empty_while_disabled(self):
|
def test_enumerate_active_is_empty_while_disabled(self):
|
||||||
# The macOS backend is disabled during the de-sidecar cleanup
|
# The macOS backend is disabled during the companion-container removal cleanup
|
||||||
# (#385); it launches nothing, so there is nothing to enumerate.
|
# (#385); it launches nothing, so there is nothing to enumerate.
|
||||||
self.assertEqual([], enum_mod.enumerate_active())
|
self.assertEqual([], enum_mod.enumerate_active())
|
||||||
|
|
||||||
|
|||||||
@@ -267,7 +267,7 @@ resolver #2
|
|||||||
self.assertEqual(
|
self.assertEqual(
|
||||||
"192.168.128.2",
|
"192.168.128.2",
|
||||||
util.container_ipv4_on_network(
|
util.container_ipv4_on_network(
|
||||||
"bot-bottle-sidecars-demo",
|
"bot-bottle-demo",
|
||||||
"bot-bottle-net-demo",
|
"bot-bottle-net-demo",
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -28,7 +28,7 @@ def _plan(routes: tuple[EgressRoute, ...], *, slug: str = "demo", log: int = 0)
|
|||||||
class TestEgressPolicy(unittest.TestCase):
|
class TestEgressPolicy(unittest.TestCase):
|
||||||
def test_policy_round_trips_through_load_config(self) -> None:
|
def test_policy_round_trips_through_load_config(self) -> None:
|
||||||
# The policy the gateway serves must parse back to the same allow-list
|
# The policy the gateway serves must parse back to the same allow-list
|
||||||
# the per-bottle sidecar applied — moving onto the shared gateway must
|
# the per-bottle gateway applied — moving onto the shared gateway must
|
||||||
# not change a bottle's egress.
|
# not change a bottle's egress.
|
||||||
routes = (EgressRoute(host="api.example.com"), EgressRoute(host="pypi.org"))
|
routes = (EgressRoute(host="api.example.com"), EgressRoute(host="pypi.org"))
|
||||||
cfg = load_config(egress_policy(_plan(routes)))
|
cfg = load_config(egress_policy(_plan(routes)))
|
||||||
|
|||||||
@@ -26,7 +26,7 @@ class TestGitGateGitconfigRender(unittest.TestCase):
|
|||||||
bottle = fixture_with_git().bottles["dev"]
|
bottle = fixture_with_git().bottles["dev"]
|
||||||
out = git_gate_render_gitconfig(bottle.git, GIT_GATE_HOSTNAME)
|
out = git_gate_render_gitconfig(bottle.git, GIT_GATE_HOSTNAME)
|
||||||
# Both entries map to a [url ...] block keyed on the gate's
|
# Both entries map to a [url ...] block keyed on the gate's
|
||||||
# short network alias (`git-gate`) inside the sidecar bundle.
|
# short network alias (`git-gate`) inside the gateway.
|
||||||
self.assertIn(
|
self.assertIn(
|
||||||
'[url "git://git-gate/bot-bottle.git"]',
|
'[url "git://git-gate/bot-bottle.git"]',
|
||||||
out,
|
out,
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
"""Unit: supervise sidecar MCP server (PRD 0013)."""
|
"""Unit: supervise daemon MCP server (PRD 0013)."""
|
||||||
|
|
||||||
import http.client
|
import http.client
|
||||||
import json
|
import json
|
||||||
|
|||||||
Reference in New Issue
Block a user