Compare commits
5 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| dd2e83b8a9 | |||
| ce3fad9320 | |||
| c07ebca867 | |||
| c276f7b0b1 | |||
| 814c7338a1 |
@@ -9,11 +9,11 @@ host. A Python CLI (entry point `cli.py`, package `bot_bottle/`) orchestrates
|
||||
the runtime lifecycle and the copying of skills and env vars into it.
|
||||
The default backend on compatible macOS hosts is macos-container:
|
||||
agents and sidecar bundles run through Apple's `container` CLI without
|
||||
requiring Docker. The smolmachines backend remains available with
|
||||
`BOT_BOTTLE_BACKEND=smolmachines` or `--backend=smolmachines`; agents
|
||||
run in a libkrun micro-VM, while the sidecar bundle still uses Docker.
|
||||
The legacy Docker backend remains available with `BOT_BOTTLE_BACKEND=docker`
|
||||
or `--backend=docker`.
|
||||
requiring Docker. On KVM-capable Linux hosts the default is firecracker:
|
||||
agents run in a Firecracker microVM reached over SSH on a point-to-point
|
||||
TAP, while the sidecar bundle still uses Docker. The legacy Docker
|
||||
backend remains available with `BOT_BOTTLE_BACKEND=docker` or
|
||||
`--backend=docker`.
|
||||
|
||||
## Goals
|
||||
|
||||
|
||||
+1
-1
@@ -24,7 +24,7 @@
|
||||
# Exposed ports inside the container:
|
||||
# 9099 egress (mitmproxy, agent-facing HTTPS proxy)
|
||||
# 9418 git-gate (git-daemon)
|
||||
# 9420 git-gate smart HTTP (smolmachines agent-facing transport)
|
||||
# 9420 git-gate smart HTTP (VM-backend agent-facing transport)
|
||||
# 9100 supervise (MCP HTTP)
|
||||
|
||||
# Stage 1: gitleaks binary. The upstream gitleaks image is alpine
|
||||
|
||||
@@ -25,14 +25,14 @@
|
||||
- **Provider templates (Claude, Codex)** — `Dockerfile.claude` / `Dockerfile.codex`, or a bottle-supplied Dockerfile. Claude auth via long-lived OAuth token; Codex via opt-in host device-auth forwarding.
|
||||
- **gVisor auto-detect** — on Linux hosts where `runsc` is registered with Docker, every bottle launches under it for a userspace syscall barrier; no manifest config required.
|
||||
- **Apple Container backend (macOS default when available)** — runs the agent and sidecar bundle with Apple's `container` CLI, using a host-only agent network plus a separate sidecar egress network.
|
||||
- **Smolmachines backend** — runs the agent in a libkrun micro-VM while the sidecar bundle stays in Docker. TSI and smolmachines DNS filtering close the raw DNS exfiltration gap that exists in the legacy Docker backend. Runs on macOS (Hypervisor.framework) and Linux (KVM, `/dev/kvm`).
|
||||
- **Legacy Docker backend** — still available for examples, CI, and hosts without Apple Container via `BOT_BOTTLE_BACKEND=docker` or `--backend=docker`.
|
||||
- **Firecracker backend (Linux default when available)** — runs the agent in a KVM Firecracker microVM reached over SSH on a point-to-point TAP, with the sidecar bundle in Docker. A dedicated, fail-closed `nftables` table isolates the guest, closing the raw DNS/IP exfiltration gap that exists in the legacy Docker backend. Requires KVM (`/dev/kvm`) and a one-time privileged network-pool setup.
|
||||
- **Legacy Docker backend** — still available for examples, CI, and hosts without Apple Container or KVM via `BOT_BOTTLE_BACKEND=docker` or `--backend=docker`.
|
||||
|
||||
## Architecture
|
||||
|
||||
On the default macOS Apple Container backend, a bottle is an agent container on a host-only internal network plus a sidecar bundle attached to both that internal network and a NAT egress network. The agent gets HTTP(S)_PROXY and CA bundle env vars pointing at the sidecar's internal-network IP, so HTTP/HTTPS traffic flows through the sidecar instead of direct egress. `bottle.git` / git-gate is intentionally deferred on this backend until a safe Apple Container key-delivery path exists.
|
||||
|
||||
On the smolmachines backend, a bottle is an agent micro-VM plus a Docker sidecar bundle for egress, git-gate, and supervise. The VM reaches the sidecars through a per-bottle loopback alias allowed by TSI; smolmachines handles DNS filtering below the guest OS.
|
||||
On the Firecracker backend, a bottle is an agent microVM plus a Docker sidecar bundle for egress, git-gate, and supervise. The VM reaches the sidecars over a per-bottle point-to-point TAP link; a dedicated fail-closed `nftables` table (`inet bot_bottle_fc`) confines the guest to that link, so nothing leaves the box except through the sidecars. The TAP pool and nft table are provisioned once (root); per-launch needs no privilege.
|
||||
|
||||
On the legacy Docker backend, the same logical bottle is two containers per agent: an `agent` container and a `sidecars` container. They share a per-agent Docker `--internal` network; the agent has no default route off-box.
|
||||
|
||||
@@ -71,25 +71,24 @@ When the agent exits, `cli.py` tears down every sidecar and both networks; nothi
|
||||
|
||||
## Quickstart
|
||||
|
||||
On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The smolmachines backend requires Docker on the host for the sidecar bundle plus `smolvm` (macOS or Linux). The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
|
||||
On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The Firecracker backend (Linux) requires Docker on the host for the sidecar bundle plus the `firecracker` binary and KVM. The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
|
||||
|
||||
Use `BOT_BOTTLE_BACKEND=docker ./cli.py start <agent>` on hosts where Apple Container is not installed and Docker is the desired backend.
|
||||
Use `BOT_BOTTLE_BACKEND=docker ./cli.py start <agent>` on hosts where neither Apple Container nor KVM is available and Docker is the desired backend.
|
||||
|
||||
### smolmachines on Linux
|
||||
### Firecracker on Linux
|
||||
|
||||
The smolmachines backend runs on Linux as well as macOS. On Linux, `smolvm`/libkrun use KVM, so the host needs:
|
||||
On Linux, a KVM-capable host defaults to the Firecracker backend. It needs:
|
||||
|
||||
- **`/dev/kvm`** present and accessible. Load `kvm-intel` or `kvm-amd` (and enable virtualization in BIOS/firmware). The invoking user must be in the `kvm` group: `sudo usermod -aG kvm "$USER"` then re-login. bot-bottle preflights this and reports exactly what's missing.
|
||||
- **`smolvm`** on `PATH`: `curl -sSL https://smolmachines.com/install.sh | sh`.
|
||||
- **Docker** for the sidecar bundle and image build, same as macOS.
|
||||
|
||||
Per-bottle isolation works the same as macOS without any `ifconfig`/sudo step — all of `127.0.0.0/8` is already loopback on Linux, so each bottle's sidecar bundle is published on its own `127.0.0.<N>` and TSI's allowlist is scoped to that `/32`.
|
||||
- **`firecracker`** on `PATH`: grab a release from <https://github.com/firecracker-microvm/firecracker/releases>. Start flows print this pointer when the binary is missing.
|
||||
- **Docker** for the sidecar bundle and image build.
|
||||
- **A one-time privileged network setup** — the per-bottle TAP pool plus the fail-closed `nftables` isolation table. Run `./cli.py backend setup --backend=firecracker` for the host-appropriate config (a NixOS module, a `sudo` script elsewhere); `./cli.py backend status --backend=firecracker` reports what's present, including whether the pool range collides with an existing route. The pool defaults to `10.243.0.0/16` (an obscure RFC-1918 block that dodges docker/libvirt/LAN and, deliberately, Tailscale's `100.64.0.0/10` CGNAT range); override with `BOT_BOTTLE_FC_IP_BASE` if it clashes on your host.
|
||||
|
||||
```sh
|
||||
BOT_BOTTLE_BACKEND=smolmachines ./cli.py start <agent>
|
||||
BOT_BOTTLE_BACKEND=firecracker ./cli.py start <agent>
|
||||
```
|
||||
|
||||
> **NixOS:** enable `virtualisation.docker`, ensure the KVM module is loaded (`boot.kernelModules = [ "kvm-intel" ];` or `kvm-amd`), and add your user to the `kvm` and `docker` groups. If you run bottles from a Gitea Actions runner, use a `host`-label runner so Docker, `smolvm`, and `/dev/kvm` are all reachable from the job. `smolvm` isn't in nixpkgs — install the release binary (pin the version) and put it on the runner's `PATH`.
|
||||
> **NixOS:** enable `virtualisation.docker`, ensure the KVM module is loaded (`boot.kernelModules = [ "kvm-intel" ];` or `kvm-amd`), and add your user to the `kvm` and `docker` groups. For the network pool, consume the flake module — `imports = [ inputs.bot-bottle.nixosModules.firecracker-netpool ]; services.bot-bottle-firecracker = { enable = true; owner = "you"; };` — then `nixos-rebuild switch` (imperative nft/TAP rules don't survive a rebuild; channel users can `imports = [ <bot-bottle>/nix/firecracker-netpool.nix ]`). `firecracker` isn't in nixpkgs by default as a user binary — install the release binary (pin the version) and put it on `PATH`.
|
||||
|
||||
```sh
|
||||
./cli.py start <agent> # builds the image on first run, drops you into claude
|
||||
|
||||
@@ -227,9 +227,9 @@ class AgentProvider(ABC):
|
||||
from .backend.util import AGENT_CA_PATH, log_ca_fingerprint, select_ca_cert
|
||||
from .log import die
|
||||
cert_host_path, label = select_ca_cert(plan.egress_plan)
|
||||
# Ensure the target directory exists. smolvm's pack step may not
|
||||
# preserve the empty /usr/local/share/ca-certificates/ directory
|
||||
# on Linux; mkdir -p is idempotent and safe for all backends.
|
||||
# Ensure the target directory exists. A backend's rootfs build
|
||||
# may not preserve the empty /usr/local/share/ca-certificates/
|
||||
# directory; mkdir -p is idempotent and safe for all backends.
|
||||
bottle.exec("mkdir -p /usr/local/share/ca-certificates", user="root")
|
||||
bottle.cp_in(str(cert_host_path), AGENT_CA_PATH)
|
||||
r = bottle.exec(
|
||||
|
||||
@@ -26,8 +26,9 @@ backend exposes five methods:
|
||||
|
||||
Selection is driven by `--backend` on `start` or BOT_BOTTLE_BACKEND
|
||||
(env var). When neither is set, compatible macOS hosts default to
|
||||
`macos-container`; other hosts default to `smolmachines`. Per PRD 0003
|
||||
the manifest does not carry a backend field; the host picks.
|
||||
`macos-container`; Linux hosts with KVM default to `firecracker`;
|
||||
otherwise `docker`. Per PRD 0003 the manifest does not carry a
|
||||
backend field; the host picks.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -97,14 +98,14 @@ class BottlePlan(ABC):
|
||||
@property
|
||||
def git_gate_insteadof_host(self) -> str:
|
||||
"""Host (and optional port) used in git-gate insteadOf URLs.
|
||||
Docker uses the compose-network DNS alias; smolmachines
|
||||
overrides with a loopback IP:port since TSI has no DNS."""
|
||||
Docker uses the compose-network DNS alias; VM backends may
|
||||
override with an IP:port when the guest has no DNS."""
|
||||
return "git-gate"
|
||||
|
||||
@property
|
||||
def git_gate_insteadof_scheme(self) -> str:
|
||||
"""URL scheme for git-gate insteadOf rewrites. 'git' for
|
||||
Docker (git daemon); 'http' for smolmachines (HTTP proxy
|
||||
Docker (git daemon); VM backends may override (e.g. 'http'
|
||||
over a published host port)."""
|
||||
return "git"
|
||||
egress_plan: EgressPlan
|
||||
@@ -182,8 +183,8 @@ class BottleCleanupPlan(ABC):
|
||||
class ExecResult:
|
||||
"""Captured result of `Bottle.exec`. Backend-neutral: the Docker
|
||||
impl populates it from a `subprocess.CompletedProcess`, but a
|
||||
future fly/smolmachines backend could populate it from any source
|
||||
that produces a returncode + captured streams."""
|
||||
VM backend could populate it from any source that produces a
|
||||
returncode + captured streams."""
|
||||
|
||||
returncode: int
|
||||
stdout: str
|
||||
@@ -201,7 +202,7 @@ class ActiveAgent:
|
||||
of sidecar daemons currently up for this bottle (`egress`,
|
||||
`git-gate`, `supervise`); the dashboard uses it to
|
||||
gate edit verbs. `backend_name` is the matching key in
|
||||
`_BACKENDS` (`docker` / `smolmachines` / `macos-container`) — used by the active-
|
||||
`_BACKENDS` (`docker` / `firecracker` / `macos-container`) — used by the active-
|
||||
list rendering to disambiguate and by the dashboard's
|
||||
re-attach path."""
|
||||
|
||||
@@ -506,7 +507,7 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
|
||||
MCP entry inside the guest.
|
||||
|
||||
Default returns "" so backends without supervise support
|
||||
don't have to implement it. Docker and smolmachines override."""
|
||||
don't have to implement it. Docker and firecracker override."""
|
||||
del plan
|
||||
return ""
|
||||
|
||||
@@ -523,27 +524,49 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
|
||||
def enumerate_active(self) -> Sequence[ActiveAgent]:
|
||||
"""Return every currently-running agent on this backend.
|
||||
Empty when none. Backend-specific: docker queries `docker
|
||||
compose ls`; smolmachines queries `smolvm machine ls --json`
|
||||
+ cross-references its bundle container."""
|
||||
compose ls`; firecracker cross-references its running sidecar
|
||||
containers against per-bottle metadata."""
|
||||
|
||||
@classmethod
|
||||
@abstractmethod
|
||||
def is_available(cls) -> bool:
|
||||
"""Whether this backend's runtime prerequisites are satisfied
|
||||
on the current host. Docker → `docker` on PATH; smolmachines
|
||||
→ `smolvm` on PATH. Used by the cross-backend
|
||||
on the current host. Docker → `docker` on PATH; firecracker →
|
||||
Linux + KVM. Used by the cross-backend
|
||||
`enumerate_active_agents` / `cmd_cleanup` to skip backends
|
||||
the operator hasn't installed, so a docker-only host
|
||||
doesn't fail when `cli.py list active` walks past
|
||||
smolmachines."""
|
||||
firecracker."""
|
||||
|
||||
@classmethod
|
||||
@abstractmethod
|
||||
def setup(cls) -> int:
|
||||
"""Emit this backend's one-time host setup — privileged network
|
||||
pool, daemon bring-up, install pointers, etc. — as
|
||||
host-appropriate config or commands. Prints to stdout/stderr and
|
||||
returns a shell exit code (0 = nothing to report / success). A
|
||||
backend that needs no host setup prints a short note and returns
|
||||
0. Invoked generically by `./cli.py backend setup [--backend=…]`
|
||||
so operators can provision any backend without a
|
||||
backend-specific command. Classmethod (like `is_available`) —
|
||||
it's a host query, not per-bottle state."""
|
||||
|
||||
@classmethod
|
||||
@abstractmethod
|
||||
def status(cls) -> int:
|
||||
"""Report whether this backend's prerequisites are satisfied on
|
||||
the host — binaries, daemon reachability, network pool, range
|
||||
conflicts, etc. Prints a human-readable summary; returns 0 when
|
||||
the backend is ready to launch and non-zero when something is
|
||||
missing. Invoked by `./cli.py backend status [--backend=…]`."""
|
||||
|
||||
|
||||
# Import concrete backend classes AFTER the base types are defined, so
|
||||
# each backend module can pull BottleSpec / BottlePlan / BottleBackend
|
||||
# via `from . import ...` without hitting a partially-initialized module.
|
||||
from .docker import DockerBottleBackend # noqa: E402 # pylint: disable=wrong-import-position
|
||||
from .firecracker import FirecrackerBottleBackend # noqa: E402 # pylint: disable=wrong-import-position
|
||||
from .macos_container import MacosContainerBottleBackend # noqa: E402 # pylint: disable=wrong-import-position
|
||||
from .smolmachines import SmolmachinesBottleBackend # noqa: E402 # pylint: disable=wrong-import-position
|
||||
|
||||
# Freezer is imported after the backend classes for the same reason:
|
||||
# Freezer.commit_slug constructs ActiveAgent, which must be fully
|
||||
@@ -557,8 +580,8 @@ from .freeze import CommitCancelled, Freezer, get_freezer # noqa: E402 # pylin
|
||||
# unparameterized methods (prepare → plan → launch(plan), cleanup, etc.).
|
||||
_BACKENDS: dict[str, BottleBackend[Any, Any]] = {
|
||||
"docker": DockerBottleBackend(),
|
||||
"firecracker": FirecrackerBottleBackend(),
|
||||
"macos-container": MacosContainerBottleBackend(),
|
||||
"smolmachines": SmolmachinesBottleBackend(),
|
||||
}
|
||||
|
||||
|
||||
@@ -571,7 +594,8 @@ def get_bottle_backend(
|
||||
1. explicit arg (CLI `--backend=<name>` passes through here)
|
||||
2. BOT_BOTTLE_BACKEND env var
|
||||
3. `macos-container` on compatible macOS hosts
|
||||
4. default `smolmachines`
|
||||
4. `firecracker` on KVM-capable Linux hosts
|
||||
5. default `docker`
|
||||
|
||||
Dies with a pointer at the known backends if the chosen name
|
||||
isn't implemented."""
|
||||
@@ -585,7 +609,13 @@ def get_bottle_backend(
|
||||
def _default_backend_name() -> str:
|
||||
if has_backend("macos-container"):
|
||||
return "macos-container"
|
||||
return "smolmachines"
|
||||
# A KVM-capable Linux host defaults to firecracker even when the
|
||||
# `firecracker` binary isn't installed yet: selecting it here routes
|
||||
# start through firecracker's preflight, which prints an install
|
||||
# pointer, instead of silently falling back to docker.
|
||||
if FirecrackerBottleBackend.is_host_capable():
|
||||
return "firecracker"
|
||||
return "docker"
|
||||
|
||||
|
||||
def known_backend_names() -> tuple[str, ...]:
|
||||
@@ -599,7 +629,7 @@ def has_backend(name: str) -> bool:
|
||||
"""Whether the named backend's runtime prerequisites are
|
||||
available on the current host. Cross-backend callers (list,
|
||||
cleanup) skip unavailable backends so a docker-only host
|
||||
doesn't fail when the smolmachines backend isn't installed,
|
||||
doesn't fail when the firecracker backend isn't usable,
|
||||
and vice versa.
|
||||
|
||||
Returns False for unknown names so callers can pass
|
||||
|
||||
@@ -54,6 +54,16 @@ class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanup
|
||||
launch."""
|
||||
return shutil.which("docker") is not None
|
||||
|
||||
@classmethod
|
||||
def setup(cls) -> int:
|
||||
from . import setup as _setup
|
||||
return _setup.setup()
|
||||
|
||||
@classmethod
|
||||
def status(cls) -> int:
|
||||
from . import setup as _setup
|
||||
return _setup.status()
|
||||
|
||||
def _preflight(self) -> None:
|
||||
_resolve_plan.preflight()
|
||||
|
||||
|
||||
@@ -18,8 +18,7 @@ scan, just as a fallback bucket alongside the project list.
|
||||
|
||||
`cleanup` removes everything in the plan.
|
||||
|
||||
Active-agent enumeration lives in `backend/docker/enumerate.py`
|
||||
(mirror of `backend/smolmachines/enumerate.py`).
|
||||
Active-agent enumeration lives in `backend/docker/enumerate.py`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -93,8 +92,8 @@ def _list_orphan_state_dirs(
|
||||
|
||||
`protected_identities` is the set of slugs that are live in
|
||||
ANY backend — used so this docker-side check doesn't reap a
|
||||
running smolmachines bottle's state dir (the layout is shared
|
||||
across both backends)."""
|
||||
running non-docker bottle's state dir (the layout is shared
|
||||
across backends)."""
|
||||
state_root = _supervise.bot_bottle_root() / "state"
|
||||
if not state_root.is_dir():
|
||||
return []
|
||||
@@ -119,7 +118,7 @@ def prepare_cleanup() -> DockerBottleCleanupPlan:
|
||||
|
||||
Pulls the union of live identities across backends via
|
||||
`enumerate_active_agents()` so the orphan-state-dir bucket
|
||||
doesn't include slugs whose smolmachines VM is still up."""
|
||||
doesn't include slugs whose non-docker bottle is still up."""
|
||||
docker_mod.require_docker()
|
||||
projects = list_compose_projects()
|
||||
project_set = set(projects)
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
"""Active-agent enumeration for the docker backend.
|
||||
|
||||
Mirrors `backend/smolmachines/enumerate.py`: returns
|
||||
`ActiveAgent` records the CLI `list active` command and the
|
||||
Returns `ActiveAgent` records the CLI `list active` command and the
|
||||
dashboard agents pane consume. Empty when docker isn't reachable
|
||||
— gated by `has_backend('docker')` at the cross-backend caller
|
||||
so this module trusts that docker is available when called.
|
||||
|
||||
@@ -0,0 +1,80 @@
|
||||
"""Host setup + status for the Docker backend.
|
||||
|
||||
Unlike Firecracker, the Docker backend needs no privileged one-time
|
||||
host provisioning (no TAP pool / nft table) — networks and the sidecar
|
||||
bundle are created per-launch. So `setup()` is mostly an install/daemon
|
||||
pointer, and `status()` reports whether docker is usable.
|
||||
|
||||
This is intentionally minimal; a richer version (daemon config checks,
|
||||
gVisor/runsc install guidance, rootless-docker hints) is tracked
|
||||
separately. Reached via `DockerBottleBackend.setup` / `.status`, which
|
||||
the generic `./cli.py backend {setup,status}` dispatches to.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
from . import util as _util
|
||||
|
||||
|
||||
def _docker_on_path() -> bool:
|
||||
return shutil.which("docker") is not None
|
||||
|
||||
|
||||
def _daemon_reachable() -> bool:
|
||||
if not _docker_on_path():
|
||||
return False
|
||||
return subprocess.run(
|
||||
["docker", "info"],
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
|
||||
).returncode == 0
|
||||
|
||||
|
||||
def _print_install_pointer() -> None:
|
||||
sys.stderr.write("Docker is required but was not found on PATH.\n")
|
||||
sys.stderr.write(" macOS: Docker Desktop https://docs.docker.com/desktop/install/mac-install/\n")
|
||||
sys.stderr.write(" Linux: Docker Engine https://docs.docker.com/engine/install/\n")
|
||||
|
||||
|
||||
def setup() -> int:
|
||||
if not _docker_on_path():
|
||||
_print_install_pointer()
|
||||
return 1
|
||||
sys.stderr.write(
|
||||
"Docker backend: no privileged host setup required — networks and "
|
||||
"the sidecar bundle are created per-launch.\n"
|
||||
)
|
||||
if not _daemon_reachable():
|
||||
sys.stderr.write(
|
||||
"The Docker daemon isn't reachable; start it (e.g. Docker "
|
||||
"Desktop, or `systemctl start docker`).\n"
|
||||
)
|
||||
return 1
|
||||
if not _util.runsc_available():
|
||||
sys.stderr.write(
|
||||
"Optional: register the gVisor (`runsc`) runtime with Docker for "
|
||||
"a userspace syscall barrier; bottles auto-detect and use it.\n"
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
def status() -> int:
|
||||
ok = True
|
||||
if _docker_on_path():
|
||||
sys.stderr.write("docker on PATH: yes\n")
|
||||
else:
|
||||
sys.stderr.write("docker on PATH: NO\n")
|
||||
ok = False
|
||||
if ok and _daemon_reachable():
|
||||
sys.stderr.write("docker daemon: reachable\n")
|
||||
elif ok:
|
||||
sys.stderr.write("docker daemon: UNREACHABLE\n")
|
||||
ok = False
|
||||
runsc = _docker_on_path() and _util.runsc_available()
|
||||
sys.stderr.write(f"gVisor runsc runtime: {'registered' if runsc else 'not registered (optional)'}\n")
|
||||
if not ok:
|
||||
sys.stderr.write("\nRun: ./cli.py backend setup --backend=docker\n")
|
||||
return 0 if ok else 1
|
||||
@@ -167,36 +167,6 @@ def commit_container(container_name: str, image_tag: str) -> None:
|
||||
info(f"committed {container_name!r} → {image_tag!r}")
|
||||
|
||||
|
||||
def image_id(ref: str) -> str:
|
||||
"""Return the content-addressed image ID (e.g.
|
||||
`sha256:abcd...`) for `ref`. The smolmachines backend keys its
|
||||
`.smolmachine` artifact cache on this, so a Dockerfile change
|
||||
that produces a new image automatically invalidates the cache."""
|
||||
r = subprocess.run(
|
||||
["docker", "image", "inspect", "--format", "{{.Id}}", ref],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if r.returncode != 0:
|
||||
die(
|
||||
f"docker image inspect for {ref!r} failed: "
|
||||
f"{(r.stderr or '').strip() or '<no stderr>'}"
|
||||
)
|
||||
return r.stdout.strip()
|
||||
|
||||
|
||||
def save(ref: str, output: str) -> None:
|
||||
"""`docker save REF -o OUTPUT`. Writes a tarball of the image
|
||||
layers + manifest to the host path. Used by smolmachines
|
||||
prepare to hand the agent image to a containerized crane that
|
||||
pushes it to the ephemeral registry — bypassing the docker
|
||||
daemon's `docker push` (which on Docker Desktop can't reach a
|
||||
host-loopback registry and refuses plain-HTTP pushes to
|
||||
non-loopback hosts)."""
|
||||
subprocess.run(["docker", "save", ref, "-o", output], check=True)
|
||||
|
||||
|
||||
def _silent_run(cmd: Iterable[str]) -> int:
|
||||
return subprocess.run(
|
||||
list(cmd),
|
||||
|
||||
@@ -0,0 +1,7 @@
|
||||
"""Firecracker backend: Linux KVM microVM isolation (issue #342)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from .backend import FirecrackerBottleBackend
|
||||
|
||||
__all__ = ["FirecrackerBottleBackend"]
|
||||
+41
-35
@@ -1,11 +1,10 @@
|
||||
"""SmolmachinesBottleBackend — the smolmachines implementation of
|
||||
BottleBackend (PRD 0023).
|
||||
"""FirecrackerBottleBackend — Linux microVM implementation.
|
||||
|
||||
Per PRD 0050 the per-provider provisioning steps (prompt, skills,
|
||||
the declarative provision-plan apply, supervise MCP registration)
|
||||
live on the `AgentProvider` plugin under `bot_bottle/contrib/`. The
|
||||
smolmachines backend only owns the steps that are about backend
|
||||
infrastructure: CA install (no-op for now), workspace, git copy-in."""
|
||||
Replaces smolmachines on Linux (issue #342): mature KVM-based
|
||||
isolation via Firecracker, SSH control over a point-to-point TAP, and a
|
||||
fail-closed nftables egress boundary. Selected by
|
||||
`BOT_BOTTLE_BACKEND=firecracker` or `--backend=firecracker`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -17,34 +16,45 @@ from ...agent_provider import AgentProvisionPlan
|
||||
from ...egress import EgressPlan
|
||||
from ...env import ResolvedEnv
|
||||
from ...git_gate import GitGatePlan
|
||||
from ...supervise import SupervisePlan
|
||||
from ...manifest import Manifest
|
||||
from ...supervise import SupervisePlan
|
||||
from .. import ActiveAgent, BottleBackend, BottleSpec
|
||||
from . import cleanup as _cleanup
|
||||
from . import enumerate as _enumerate
|
||||
from . import launch as _launch
|
||||
from . import resolve_plan as _resolve_plan
|
||||
from . import smolvm as _smolvm
|
||||
from .bottle import SmolmachinesBottle
|
||||
from .bottle_cleanup_plan import SmolmachinesBottleCleanupPlan
|
||||
from .bottle_plan import SmolmachinesBottlePlan
|
||||
from . import util as _util
|
||||
from .bottle import FirecrackerBottle
|
||||
from .bottle_cleanup_plan import FirecrackerBottleCleanupPlan
|
||||
from .bottle_plan import FirecrackerBottlePlan
|
||||
|
||||
|
||||
class SmolmachinesBottleBackend(
|
||||
BottleBackend["SmolmachinesBottlePlan", "SmolmachinesBottleCleanupPlan"]
|
||||
class FirecrackerBottleBackend(
|
||||
BottleBackend["FirecrackerBottlePlan", "FirecrackerBottleCleanupPlan"]
|
||||
):
|
||||
"""smolmachines backend. Selected by
|
||||
`BOT_BOTTLE_BACKEND=smolmachines`."""
|
||||
|
||||
name = "smolmachines"
|
||||
name = "firecracker"
|
||||
|
||||
@classmethod
|
||||
def is_available(cls) -> bool:
|
||||
"""`smolvm` on PATH. The backend additionally needs macOS
|
||||
for libkrun + TSI, but `enumerate_active` / `cleanup` are
|
||||
host-shell ops that gracefully no-op on Linux too — the
|
||||
runtime check happens at `prepare`."""
|
||||
return _smolvm.is_available()
|
||||
return _util.is_available()
|
||||
|
||||
@classmethod
|
||||
def is_host_capable(cls) -> bool:
|
||||
"""Linux + KVM, regardless of whether the `firecracker` binary
|
||||
is installed. Drives default-backend selection so a capable host
|
||||
without the binary still lands on firecracker and gets an
|
||||
install pointer at launch."""
|
||||
return _util.is_host_capable()
|
||||
|
||||
@classmethod
|
||||
def setup(cls) -> int:
|
||||
from . import setup as _setup
|
||||
return _setup.setup()
|
||||
|
||||
@classmethod
|
||||
def status(cls) -> int:
|
||||
from . import setup as _setup
|
||||
return _setup.status()
|
||||
|
||||
def _preflight(self) -> None:
|
||||
_resolve_plan.preflight()
|
||||
@@ -64,7 +74,7 @@ class SmolmachinesBottleBackend(
|
||||
git_gate_plan: GitGatePlan,
|
||||
supervise_plan: SupervisePlan | None,
|
||||
stage_dir: Path,
|
||||
) -> SmolmachinesBottlePlan:
|
||||
) -> FirecrackerBottlePlan:
|
||||
return _resolve_plan.resolve_plan(
|
||||
spec,
|
||||
manifest=manifest,
|
||||
@@ -79,23 +89,19 @@ class SmolmachinesBottleBackend(
|
||||
|
||||
@contextmanager
|
||||
def launch(
|
||||
self, plan: SmolmachinesBottlePlan
|
||||
) -> Generator[SmolmachinesBottle, None, None]:
|
||||
self, plan: FirecrackerBottlePlan
|
||||
) -> Generator[FirecrackerBottle, None, None]:
|
||||
with _launch.launch(plan, provision=self.provision) as bottle:
|
||||
yield bottle
|
||||
|
||||
def supervise_mcp_url(self, plan: SmolmachinesBottlePlan) -> str:
|
||||
"""The smolmachines guest reaches the supervise sidecar via a
|
||||
host-published random port the launch step pinned earlier
|
||||
(`http://<loopback_ip>:<random_port>/`). `agent_supervise_url`
|
||||
on the plan is "" when the bottle has no sidecar."""
|
||||
return plan.agent_supervise_url
|
||||
|
||||
def prepare_cleanup(self) -> SmolmachinesBottleCleanupPlan:
|
||||
def prepare_cleanup(self) -> FirecrackerBottleCleanupPlan:
|
||||
return _cleanup.prepare_cleanup()
|
||||
|
||||
def cleanup(self, plan: SmolmachinesBottleCleanupPlan) -> None:
|
||||
def cleanup(self, plan: FirecrackerBottleCleanupPlan) -> None:
|
||||
_cleanup.cleanup(plan)
|
||||
|
||||
def enumerate_active(self) -> Sequence[ActiveAgent]:
|
||||
return _enumerate.enumerate_active()
|
||||
|
||||
def supervise_mcp_url(self, plan: FirecrackerBottlePlan) -> str:
|
||||
return plan.agent_supervise_url
|
||||
@@ -0,0 +1,169 @@
|
||||
"""Bottle handle for a Firecracker microVM, over SSH.
|
||||
|
||||
Every operation routes through SSH to the guest (dropbear, listening on
|
||||
the TAP link). `exec` pipes a script over stdin and captures output;
|
||||
`cp_in` uses scp; `exec_agent` uses `ssh -t` for an interactive PTY
|
||||
session. `ssh -t` forwards the host terminal's SIGWINCH to the remote
|
||||
PTY natively, so no separate resize bridge is needed.
|
||||
|
||||
Commands run as the image's `node` user via `runuser`, with HOME/USER/
|
||||
PATH and the bottle env (HTTPS_PROXY at the sidecar, CA paths, …) set
|
||||
per-invocation through `env` (the VM itself just runs the init; it has
|
||||
no baked-in process env like a `docker run` container would).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shlex
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Mapping, cast
|
||||
|
||||
from ...agent_provider import PromptMode, prompt_args
|
||||
from .. import Bottle, ExecResult
|
||||
from ..terminal import exec_shell_script
|
||||
from . import util
|
||||
|
||||
|
||||
_HOME_FOR = {"node": "/home/node", "root": "/root"}
|
||||
_DEFAULT_PATH_FOR = {
|
||||
"node": (
|
||||
"/home/node/.local/bin:"
|
||||
"/home/node/.codex/packages/standalone/current/bin:"
|
||||
"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
|
||||
),
|
||||
"root": "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
|
||||
}
|
||||
|
||||
|
||||
def _env_assignments_for(user: str, env: Mapping[str, str]) -> list[str]:
|
||||
home = _HOME_FOR.get(user, f"/home/{user}")
|
||||
out = [f"HOME={home}", f"USER={user}"]
|
||||
if "PATH" not in env:
|
||||
out.append(f"PATH={_DEFAULT_PATH_FOR.get(user, _DEFAULT_PATH_FOR['root'])}")
|
||||
for k, v in env.items():
|
||||
out.append(f"{k}={v}")
|
||||
return out
|
||||
|
||||
|
||||
class FirecrackerBottle(Bottle):
|
||||
def __init__(
|
||||
self,
|
||||
name: str,
|
||||
*,
|
||||
private_key: Path,
|
||||
guest_ip: str,
|
||||
guest_env: Mapping[str, str] | None = None,
|
||||
prompt_path_in_guest: str | None = None,
|
||||
agent_command: str = "claude",
|
||||
agent_prompt_mode: PromptMode = "append_file",
|
||||
agent_provider_template: str = "claude",
|
||||
terminal_title: str = "",
|
||||
terminal_color: str = "",
|
||||
agent_workdir: str = "/home/node",
|
||||
) -> None:
|
||||
self.name = name
|
||||
self._private_key = private_key
|
||||
self._guest_ip = guest_ip
|
||||
self._guest_env = dict(guest_env or {})
|
||||
self.prompt_path = prompt_path_in_guest
|
||||
self._agent_prompt_mode = agent_prompt_mode
|
||||
self.agent_command = agent_command
|
||||
self.terminal_title = terminal_title
|
||||
self.terminal_color = terminal_color
|
||||
self.agent_provider_template = agent_provider_template
|
||||
self.agent_workdir = agent_workdir
|
||||
|
||||
def _ssh(self, *, tty: bool) -> list[str]:
|
||||
argv = util.ssh_base_argv(self._private_key, self._guest_ip)
|
||||
if tty:
|
||||
# -t: allocate a remote PTY and forward window-size changes.
|
||||
argv.insert(1, "-t")
|
||||
return argv
|
||||
|
||||
def _agent_remote_argv(self, argv: list[str]) -> list[str]:
|
||||
"""The `runuser … <agent> …` command run inside the guest."""
|
||||
full_argv = list(argv)
|
||||
full_argv.extend(
|
||||
prompt_args(
|
||||
cast(PromptMode, self._agent_prompt_mode),
|
||||
self.prompt_path,
|
||||
argv=full_argv,
|
||||
)
|
||||
)
|
||||
remote = ["runuser", "-u", "node", "--",
|
||||
"env", *_env_assignments_for("node", self._guest_env)]
|
||||
if self.agent_workdir and self.agent_workdir != _HOME_FOR["node"]:
|
||||
remote += ["sh", "-lc", f"cd {self.agent_workdir} && exec \"$@\"",
|
||||
"bot-bottle-agent"]
|
||||
remote += [self.agent_command, *full_argv]
|
||||
return remote
|
||||
|
||||
def agent_argv(self, argv: list[str], *, tty: bool = True) -> list[str]:
|
||||
return [*self._ssh(tty=tty), "--", *self._agent_remote_argv(argv)]
|
||||
|
||||
def exec_agent(self, argv: list[str], *, tty: bool = True) -> int:
|
||||
agent_argv = self.agent_argv(argv, tty=tty)
|
||||
script = (
|
||||
exec_shell_script(agent_argv, self.terminal_title, self.terminal_color)
|
||||
if tty else None
|
||||
)
|
||||
if script is None:
|
||||
return subprocess.run(agent_argv, check=False).returncode
|
||||
return subprocess.run(["sh", "-lc", script], check=False).returncode
|
||||
|
||||
def exec(self, script: str, *, user: str = "node") -> ExecResult:
|
||||
# Pipe the script over stdin (`sh -s`) so nothing needs shell
|
||||
# quoting through the SSH command line.
|
||||
remote = ["runuser", "-u", user, "--",
|
||||
"env", *_env_assignments_for(user, self._guest_env),
|
||||
"/bin/sh", "-s"]
|
||||
result = subprocess.run(
|
||||
[*self._ssh(tty=False), "--", *remote],
|
||||
input=script, capture_output=True, text=True, check=False,
|
||||
)
|
||||
return ExecResult(
|
||||
returncode=result.returncode,
|
||||
stdout=result.stdout,
|
||||
stderr=result.stderr,
|
||||
)
|
||||
|
||||
def cp_in(self, host_path: str, container_path: str) -> None:
|
||||
# Stream a tar over the SSH exec channel rather than scp: the
|
||||
# guest runs dropbear, which ships no sftp-server/scp, but every
|
||||
# agent image has `tar`. Copy so `container_path` becomes a copy
|
||||
# of `host_path` (docker cp semantics — callers rm -rf the
|
||||
# destination first).
|
||||
host_parent = os.path.dirname(host_path.rstrip("/")) or "/"
|
||||
host_base = os.path.basename(host_path.rstrip("/"))
|
||||
guest_parent = os.path.dirname(container_path.rstrip("/")) or "/"
|
||||
guest_base = os.path.basename(container_path.rstrip("/"))
|
||||
remote = (
|
||||
f"mkdir -p {shlex.quote(guest_parent)} && "
|
||||
f"cd {shlex.quote(guest_parent)} && "
|
||||
f"rm -rf {shlex.quote(guest_base)} && tar -xf - && "
|
||||
f"{{ [ {shlex.quote(host_base)} = {shlex.quote(guest_base)} ] || "
|
||||
f"mv {shlex.quote(host_base)} {shlex.quote(guest_base)}; }}"
|
||||
)
|
||||
tar = subprocess.Popen(
|
||||
["tar", "-C", host_parent, "-cf", "-", host_base],
|
||||
stdout=subprocess.PIPE,
|
||||
)
|
||||
ssh = subprocess.run(
|
||||
[*self._ssh(tty=False), "--", "sh", "-c", remote],
|
||||
stdin=tar.stdout, capture_output=True, text=True, check=False,
|
||||
)
|
||||
tar.wait()
|
||||
if tar.returncode != 0 or ssh.returncode != 0:
|
||||
sys.stderr.write(ssh.stderr)
|
||||
raise subprocess.CalledProcessError(
|
||||
ssh.returncode or tar.returncode,
|
||||
["cp_in", host_path, container_path],
|
||||
)
|
||||
|
||||
def close(self) -> None:
|
||||
# Real teardown (VM terminate, TAP release) lives on the launch
|
||||
# ExitStack; this is the idempotent alias the ABC expects.
|
||||
pass
|
||||
@@ -0,0 +1,32 @@
|
||||
"""Cleanup plan for the Firecracker backend."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
from ...log import info
|
||||
from .. import BottleCleanupPlan
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class FirecrackerBottleCleanupPlan(BottleCleanupPlan):
|
||||
# PIDs of orphaned firecracker VMM processes and the sidecar
|
||||
# containers left behind by previous bottles.
|
||||
vm_pids: tuple[int, ...] = ()
|
||||
containers: tuple[str, ...] = ()
|
||||
run_dirs: tuple[str, ...] = ()
|
||||
|
||||
def print(self) -> None:
|
||||
if self.empty:
|
||||
info("firecracker cleanup: nothing to remove")
|
||||
return
|
||||
for pid in self.vm_pids:
|
||||
info(f"firecracker VM process: pid {pid}")
|
||||
for name in self.containers:
|
||||
info(f"firecracker sidecar container: {name}")
|
||||
for path in self.run_dirs:
|
||||
info(f"firecracker run dir: {path}")
|
||||
|
||||
@property
|
||||
def empty(self) -> bool:
|
||||
return not (self.vm_pids or self.containers or self.run_dirs)
|
||||
@@ -0,0 +1,63 @@
|
||||
"""Plan type for the Firecracker backend."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
from ...agent_provider import PromptMode
|
||||
from .. import BottlePlan
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class FirecrackerBottlePlan(BottlePlan):
|
||||
slug: str
|
||||
forwarded_env: dict[str, str] = field(repr=False)
|
||||
# Stamped by launch once the sidecar is up and its ports are
|
||||
# published on the host-side TAP IP (empty at prepare time).
|
||||
agent_proxy_url: str = ""
|
||||
agent_git_gate_url: str = ""
|
||||
agent_supervise_url: str = ""
|
||||
|
||||
@property
|
||||
def container_name(self) -> str:
|
||||
"""Instance name, reused for the sidecar container + VM run dir.
|
||||
Matches the `bot-bottle-<slug>` convention the other backends
|
||||
use so cleanup/enumerate discovery-by-prefix keeps working."""
|
||||
return self.agent_provision.instance_name
|
||||
|
||||
@property
|
||||
def image(self) -> str:
|
||||
return self.agent_provision.image
|
||||
|
||||
@property
|
||||
def dockerfile_path(self) -> str:
|
||||
return self.agent_provision.dockerfile
|
||||
|
||||
@property
|
||||
def prompt_file(self) -> Path:
|
||||
return self.agent_provision.prompt_file
|
||||
|
||||
@property
|
||||
def agent_command(self) -> str:
|
||||
return self.agent_provision.command
|
||||
|
||||
@property
|
||||
def agent_prompt_mode(self) -> PromptMode:
|
||||
return self.agent_provision.prompt_mode
|
||||
|
||||
@property
|
||||
def agent_provider_template(self) -> str:
|
||||
return self.agent_provision.template
|
||||
|
||||
@property
|
||||
def git_gate_insteadof_host(self) -> str:
|
||||
if self.agent_git_gate_url.startswith("http://"):
|
||||
return self.agent_git_gate_url.removeprefix("http://").rstrip("/")
|
||||
return super().git_gate_insteadof_host
|
||||
|
||||
@property
|
||||
def git_gate_insteadof_scheme(self) -> str:
|
||||
if self.agent_git_gate_url.startswith("http://"):
|
||||
return "http"
|
||||
return super().git_gate_insteadof_scheme
|
||||
@@ -0,0 +1,90 @@
|
||||
"""Cleanup for the Firecracker backend.
|
||||
|
||||
Orphans are: firecracker VMM processes whose config lives under our run
|
||||
dir, the `bot-bottle-sidecars-*` containers, and the per-bottle run
|
||||
dirs. TAP slots free themselves (the flock drops when the launcher
|
||||
exits), so there is nothing to reclaim there.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import signal
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
from ...log import info
|
||||
from . import util
|
||||
from .bottle_cleanup_plan import FirecrackerBottleCleanupPlan
|
||||
|
||||
_SIDECAR_PREFIX = "bot-bottle-sidecars-"
|
||||
|
||||
|
||||
def _run_root() -> Path:
|
||||
return util.cache_dir() / "run"
|
||||
|
||||
|
||||
def _orphan_vm_pids() -> list[int]:
|
||||
"""firecracker processes whose --config-file is under our run dir."""
|
||||
run_root = str(_run_root())
|
||||
result = subprocess.run(
|
||||
["pgrep", "-a", "firecracker"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return []
|
||||
pids: list[int] = []
|
||||
for line in result.stdout.splitlines():
|
||||
parts = line.split(None, 1)
|
||||
if len(parts) != 2 or run_root not in parts[1]:
|
||||
continue
|
||||
try:
|
||||
pids.append(int(parts[0]))
|
||||
except ValueError:
|
||||
continue
|
||||
return pids
|
||||
|
||||
|
||||
def _sidecar_containers() -> list[str]:
|
||||
result = subprocess.run(
|
||||
["docker", "ps", "-a", "--format", "{{.Names}}",
|
||||
"--filter", f"name={_SIDECAR_PREFIX}"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return []
|
||||
return sorted(n.strip() for n in result.stdout.splitlines() if n.strip())
|
||||
|
||||
|
||||
def _run_dirs() -> list[str]:
|
||||
run_root = _run_root()
|
||||
if not run_root.is_dir():
|
||||
return []
|
||||
return sorted(str(p) for p in run_root.iterdir() if p.is_dir())
|
||||
|
||||
|
||||
def prepare_cleanup() -> FirecrackerBottleCleanupPlan:
|
||||
return FirecrackerBottleCleanupPlan(
|
||||
vm_pids=tuple(_orphan_vm_pids()),
|
||||
containers=tuple(_sidecar_containers()),
|
||||
run_dirs=tuple(_run_dirs()),
|
||||
)
|
||||
|
||||
|
||||
def cleanup(plan: FirecrackerBottleCleanupPlan) -> None:
|
||||
for pid in plan.vm_pids:
|
||||
info(f"kill firecracker VM pid {pid}")
|
||||
try:
|
||||
os.kill(pid, signal.SIGTERM)
|
||||
except ProcessLookupError:
|
||||
pass
|
||||
for name in plan.containers:
|
||||
info(f"docker rm -f {name}")
|
||||
subprocess.run(
|
||||
["docker", "rm", "-f", name],
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
|
||||
)
|
||||
for path in plan.run_dirs:
|
||||
info(f"rm -rf {path}")
|
||||
shutil.rmtree(path, ignore_errors=True)
|
||||
@@ -0,0 +1,43 @@
|
||||
"""Active-agent enumeration for the Firecracker backend.
|
||||
|
||||
The agent runs in a VM (no container to list), so a live bottle is
|
||||
identified by its running sidecar container `bot-bottle-sidecars-<slug>`
|
||||
— the same discovery-by-prefix the other backends use.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
|
||||
from ...bottle_state import read_metadata
|
||||
from .. import ActiveAgent
|
||||
|
||||
_SIDECAR_PREFIX = "bot-bottle-sidecars-"
|
||||
|
||||
|
||||
def enumerate_active() -> list[ActiveAgent]:
|
||||
result = subprocess.run(
|
||||
["docker", "ps", "--format", "{{.Names}}",
|
||||
"--filter", f"name={_SIDECAR_PREFIX}"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return []
|
||||
out: list[ActiveAgent] = []
|
||||
for name in sorted(n.strip() for n in result.stdout.splitlines() if n.strip()):
|
||||
slug = name[len(_SIDECAR_PREFIX):]
|
||||
metadata = read_metadata(slug)
|
||||
if metadata is None or metadata.backend != "firecracker":
|
||||
# Skip sidecars owned by another backend (docker shares the
|
||||
# container-name prefix).
|
||||
continue
|
||||
out.append(ActiveAgent(
|
||||
backend_name="firecracker",
|
||||
slug=slug,
|
||||
agent_name=metadata.agent_name,
|
||||
started_at=metadata.started_at,
|
||||
services=(),
|
||||
label=metadata.label,
|
||||
color=metadata.color,
|
||||
))
|
||||
return out
|
||||
@@ -0,0 +1,173 @@
|
||||
"""Firecracker microVM process lifecycle.
|
||||
|
||||
A bottle VM is one `firecracker` process booted from a JSON config
|
||||
(kernel + writable rootfs ext4 + one TAP network interface). We run it
|
||||
`--no-api`: all configuration is in the config file, and teardown is a
|
||||
process signal — the microVM dies with its VMM, which is exactly the
|
||||
ephemeral-bottle semantics we want. No API socket, no runtime
|
||||
reconfiguration.
|
||||
|
||||
The guest gets its IP from the kernel `ip=` cmdline (kernel-level
|
||||
autoconfig, no in-guest iproute2) and its SSH pubkey from a `bb_pubkey=`
|
||||
cmdline arg the init decodes.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import json
|
||||
import signal
|
||||
import subprocess
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
from ...log import die, info
|
||||
from . import util
|
||||
|
||||
|
||||
# Serial console off the critical path but captured to a log for
|
||||
# debugging boot failures. `reboot=k panic=1 pci=off` are the standard
|
||||
# Firecracker guest args; `i8042.*` skip the (absent) PS/2 probe to
|
||||
# shave boot time.
|
||||
_BASE_BOOT_ARGS = (
|
||||
"console=ttyS0 reboot=k panic=1 pci=off "
|
||||
"i8042.noaux i8042.nomux i8042.nopnp i8042.dumbkbd "
|
||||
"root=/dev/vda rw init=/bb-init"
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class VmHandle:
|
||||
"""A running microVM: its VMM process, guest IP, and console log."""
|
||||
|
||||
process: subprocess.Popen[bytes]
|
||||
guest_ip: str
|
||||
console_log: Path
|
||||
|
||||
def is_alive(self) -> bool:
|
||||
return self.process.poll() is None
|
||||
|
||||
def terminate(self) -> None:
|
||||
"""Stop the VMM (and thus the guest). SIGTERM, then SIGKILL."""
|
||||
if self.process.poll() is not None:
|
||||
return
|
||||
self.process.send_signal(signal.SIGTERM)
|
||||
try:
|
||||
self.process.wait(timeout=5)
|
||||
except subprocess.TimeoutExpired:
|
||||
self.process.kill()
|
||||
self.process.wait(timeout=5)
|
||||
|
||||
|
||||
def _boot_args(guest_ip: str, host_ip: str, pubkey: str) -> str:
|
||||
# ip=<client>::<gw>:<netmask>::<dev>:off — /31 point-to-point link,
|
||||
# so netmask is 255.255.255.254 and the gateway is the host TAP IP.
|
||||
ip_arg = f"ip={guest_ip}::{host_ip}:255.255.255.254::eth0:off"
|
||||
pub_b64 = base64.b64encode(pubkey.encode()).decode()
|
||||
return f"{_BASE_BOOT_ARGS} {ip_arg} bb_pubkey={pub_b64}"
|
||||
|
||||
|
||||
def _config(
|
||||
*,
|
||||
rootfs: Path,
|
||||
tap: str,
|
||||
guest_ip: str,
|
||||
host_ip: str,
|
||||
pubkey: str,
|
||||
vcpus: int,
|
||||
mem_mib: int,
|
||||
guest_mac: str,
|
||||
) -> dict[str, object]:
|
||||
return {
|
||||
"boot-source": {
|
||||
"kernel_image_path": str(util.kernel_path()),
|
||||
"boot_args": _boot_args(guest_ip, host_ip, pubkey),
|
||||
},
|
||||
"drives": [
|
||||
{
|
||||
"drive_id": "rootfs",
|
||||
"path_on_host": str(rootfs),
|
||||
"is_root_device": True,
|
||||
"is_read_only": False,
|
||||
}
|
||||
],
|
||||
"network-interfaces": [
|
||||
{
|
||||
"iface_id": "eth0",
|
||||
"host_dev_name": tap,
|
||||
"guest_mac": guest_mac,
|
||||
}
|
||||
],
|
||||
"machine-config": {
|
||||
"vcpu_count": vcpus,
|
||||
"mem_size_mib": mem_mib,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def boot(
|
||||
*,
|
||||
name: str,
|
||||
rootfs: Path,
|
||||
tap: str,
|
||||
guest_ip: str,
|
||||
host_ip: str,
|
||||
pubkey: str,
|
||||
run_dir: Path,
|
||||
vcpus: int = 2,
|
||||
mem_mib: int = 2048,
|
||||
guest_mac: str = "06:00:AC:10:00:02",
|
||||
) -> VmHandle:
|
||||
"""Write the config and launch the VMM. Returns once the process is
|
||||
spawned; callers wait for SSH readiness separately."""
|
||||
run_dir.mkdir(parents=True, exist_ok=True)
|
||||
config_path = run_dir / "config.json"
|
||||
console_log = run_dir / "console.log"
|
||||
config_path.write_text(json.dumps(
|
||||
_config(
|
||||
rootfs=rootfs, tap=tap, guest_ip=guest_ip, host_ip=host_ip,
|
||||
pubkey=pubkey, vcpus=vcpus, mem_mib=mem_mib, guest_mac=guest_mac,
|
||||
),
|
||||
indent=2,
|
||||
))
|
||||
|
||||
info(f"booting microVM {name} on {tap} (guest {guest_ip})")
|
||||
log_fh = console_log.open("wb")
|
||||
process = subprocess.Popen(
|
||||
["firecracker", "--no-api", "--config-file", str(config_path)],
|
||||
stdout=log_fh, stderr=subprocess.STDOUT, stdin=subprocess.DEVNULL,
|
||||
)
|
||||
return VmHandle(process=process, guest_ip=guest_ip, console_log=console_log)
|
||||
|
||||
|
||||
def wait_for_ssh(
|
||||
vm: VmHandle, private_key: Path, *, timeout: float = 30.0,
|
||||
) -> None:
|
||||
"""Poll SSH until the guest accepts a command or the deadline
|
||||
passes. Dies (with the console tail) if the VMM exits early or the
|
||||
guest never comes up — a boot failure must be loud, not a hang."""
|
||||
deadline = time.monotonic() + timeout
|
||||
probe = util.ssh_base_argv(private_key, vm.guest_ip) + ["true"]
|
||||
while time.monotonic() < deadline:
|
||||
if not vm.is_alive():
|
||||
die(f"microVM exited during boot (rc={vm.process.returncode}).\n"
|
||||
f"{_console_tail(vm.console_log)}")
|
||||
result = subprocess.run(
|
||||
probe, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode == 0:
|
||||
return
|
||||
time.sleep(0.5)
|
||||
die(f"microVM {vm.guest_ip} did not accept SSH within {timeout:.0f}s.\n"
|
||||
f"{_console_tail(vm.console_log)}")
|
||||
|
||||
|
||||
def _console_tail(console_log: Path, lines: int = 25) -> str:
|
||||
try:
|
||||
text = console_log.read_text(errors="replace").splitlines()
|
||||
except OSError:
|
||||
return "(no console log)"
|
||||
tail = "\n".join(text[-lines:])
|
||||
return f"--- guest console (last {lines} lines) ---\n{tail}"
|
||||
@@ -0,0 +1,76 @@
|
||||
"""FirecrackerFreezer — snapshot a running microVM to a Docker image.
|
||||
|
||||
The VM is live and can't be block-copied safely, so — like the macOS
|
||||
backend — we stream the guest root filesystem out over the control
|
||||
channel (SSH here) and rebuild an image from it. The bottle keeps
|
||||
running after the snapshot.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
from ...log import die, info
|
||||
from .. import ActiveAgent
|
||||
from ..freeze import Freezer
|
||||
from . import util
|
||||
|
||||
|
||||
class FirecrackerFreezer(Freezer):
|
||||
backend_name = "firecracker"
|
||||
|
||||
def _freeze(self, agent: ActiveAgent) -> str:
|
||||
run_dir = util.cache_dir() / "run" / agent.slug
|
||||
private_key = run_dir / "bottle_id_ed25519"
|
||||
guest_ip = _guest_ip_from_config(run_dir / "config.json")
|
||||
if not private_key.is_file() or not guest_ip:
|
||||
die(f"cannot freeze {agent.slug}: run dir {run_dir} is missing the "
|
||||
f"SSH key or VM config (is the bottle still running?)")
|
||||
image_tag = f"bot-bottle-committed-{agent.slug}:latest"
|
||||
_commit_via_ssh(private_key, guest_ip, image_tag)
|
||||
info(f"committed {agent.slug} -> {image_tag!r}")
|
||||
return image_tag
|
||||
|
||||
def _export_hint(self, slug: str, image_ref: str) -> None:
|
||||
info(f"to export for migration: docker image save {image_ref} "
|
||||
f"-o {slug}.tar")
|
||||
|
||||
|
||||
def _guest_ip_from_config(config_path: Path) -> str:
|
||||
try:
|
||||
cfg = json.loads(config_path.read_text())
|
||||
except (OSError, json.JSONDecodeError):
|
||||
return ""
|
||||
boot_args = cfg.get("boot-source", {}).get("boot_args", "")
|
||||
for token in boot_args.split():
|
||||
if token.startswith("ip="):
|
||||
# ip=<client>::<gw>:<mask>::<dev>:off
|
||||
return token[len("ip="):].split(":", 1)[0]
|
||||
return ""
|
||||
|
||||
|
||||
def _commit_via_ssh(private_key: Path, guest_ip: str, image_tag: str) -> None:
|
||||
with tempfile.TemporaryDirectory(prefix="bot-bottle-fc-commit.") as tmp:
|
||||
rootfs_tar = os.path.join(tmp, "rootfs.tar")
|
||||
ssh = util.ssh_base_argv(private_key, guest_ip)
|
||||
with open(rootfs_tar, "wb") as tar_out:
|
||||
result = subprocess.run(
|
||||
[*ssh, "--", "tar", "--create", "--one-file-system",
|
||||
"--exclude=./proc", "--exclude=./sys", "--exclude=./dev",
|
||||
"--exclude=./run", "--file=-", "--directory=/", "."],
|
||||
stdout=tar_out, stderr=subprocess.PIPE, check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
die(f"ssh tar for {guest_ip} failed: "
|
||||
f"{(result.stderr or b'').decode().strip() or '<no stderr>'}")
|
||||
with open(os.path.join(tmp, "Dockerfile"), "w", encoding="utf-8") as f:
|
||||
f.write("FROM scratch\nADD rootfs.tar /\nUSER node\nWORKDIR /home/node\n")
|
||||
build = subprocess.run(
|
||||
["docker", "build", "-t", image_tag, tmp], check=False,
|
||||
)
|
||||
if build.returncode != 0:
|
||||
die(f"docker build for {image_tag!r} failed")
|
||||
@@ -0,0 +1,116 @@
|
||||
"""Empirical, post-boot isolation probe — the authoritative fail-closed
|
||||
egress-boundary check.
|
||||
|
||||
The pre-boot nft check can't be trusted on every host (listing an
|
||||
nftables table typically needs root; the launcher runs unprivileged).
|
||||
So before the agent ever runs, we prove the boundary directly: open a
|
||||
canary TCP listener on a host IP the VM must NOT be able to reach (the
|
||||
host's primary non-TAP address), then have the guest try to connect. If
|
||||
the isolation rules are in force the connection is dropped and times
|
||||
out; if it succeeds, the VM can reach host services it shouldn't — a
|
||||
sandbox escape — and we refuse to continue.
|
||||
|
||||
The listener is load-bearing: without it a "blocked" result could just
|
||||
be connection-refused. With it, a reachable canary means a real leak
|
||||
and a timeout means a real drop.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import socket
|
||||
import subprocess
|
||||
import threading
|
||||
from pathlib import Path
|
||||
|
||||
from ...log import die, info, warn
|
||||
from . import util
|
||||
|
||||
|
||||
def _host_primary_ip() -> str:
|
||||
"""The source IP the host uses for off-box traffic (its LAN
|
||||
address) — a canary the VM must not reach. Empty if the host has no
|
||||
such route (isolated box)."""
|
||||
result = subprocess.run(
|
||||
["ip", "-o", "route", "get", "1.1.1.1"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return ""
|
||||
tokens = result.stdout.split()
|
||||
# `... src <addr> ...` — the address the host would use as source.
|
||||
if "src" in tokens:
|
||||
idx = tokens.index("src")
|
||||
if idx + 1 < len(tokens):
|
||||
return tokens[idx + 1]
|
||||
return ""
|
||||
|
||||
|
||||
# Guest-side connect test: exit 0 = reached the canary (LEAK), 1 =
|
||||
# blocked (good), 2 = no usable tool (inconclusive -> fail-closed).
|
||||
_GUEST_PROBE = r"""
|
||||
h="$1"; p="$2"
|
||||
if command -v python3 >/dev/null 2>&1; then
|
||||
python3 - "$h" "$p" <<'PY'
|
||||
import socket, sys
|
||||
s = socket.socket(); s.settimeout(3)
|
||||
sys.exit(0 if s.connect_ex((sys.argv[1], int(sys.argv[2]))) == 0 else 1)
|
||||
PY
|
||||
elif command -v bash >/dev/null 2>&1; then
|
||||
timeout 3 bash -c "exec 3<>/dev/tcp/$h/$p" >/dev/null 2>&1
|
||||
elif command -v nc >/dev/null 2>&1; then
|
||||
nc -w3 -z "$h" "$p" >/dev/null 2>&1
|
||||
else
|
||||
exit 2
|
||||
fi
|
||||
"""
|
||||
|
||||
|
||||
def verify_isolation(private_key: Path, guest_ip: str) -> None:
|
||||
"""Prove the VM cannot reach the host's primary IP. Dies
|
||||
(fail-closed) on a confirmed leak or if the guest has no tool to
|
||||
run the test. Warns (does not fail) when the host has no canary
|
||||
address to test against."""
|
||||
canary_ip = _host_primary_ip()
|
||||
if not canary_ip:
|
||||
warn("isolation probe skipped: host has no non-TAP route to test "
|
||||
"against (isolated box).")
|
||||
return
|
||||
|
||||
listener = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
|
||||
listener.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
|
||||
listener.bind((canary_ip, 0))
|
||||
listener.listen(1)
|
||||
listener.settimeout(6)
|
||||
canary_port = listener.getsockname()[1]
|
||||
|
||||
accepted: list[bool] = []
|
||||
|
||||
def _accept() -> None:
|
||||
try:
|
||||
conn, _ = listener.accept()
|
||||
accepted.append(True)
|
||||
conn.close()
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
thread = threading.Thread(target=_accept, daemon=True)
|
||||
thread.start()
|
||||
|
||||
ssh = util.ssh_base_argv(private_key, guest_ip)
|
||||
result = subprocess.run(
|
||||
[*ssh, "--", "sh", "-s", "--", canary_ip, str(canary_port)],
|
||||
input=_GUEST_PROBE, capture_output=True, text=True, check=False,
|
||||
)
|
||||
thread.join(timeout=7)
|
||||
listener.close()
|
||||
|
||||
if result.returncode == 0 or accepted:
|
||||
die(f"ISOLATION FAILURE: the VM reached the host canary "
|
||||
f"{canary_ip}:{canary_port}. The egress boundary is not in "
|
||||
f"force — refusing to run the agent (fail-closed). Verify the "
|
||||
f"nft table with: ./cli.py backend setup --backend=firecracker")
|
||||
if result.returncode == 2:
|
||||
die("isolation probe inconclusive: the guest has no python3/bash/nc "
|
||||
"to run the connectivity test. Refusing to continue "
|
||||
"(fail-closed).")
|
||||
info(f"isolation verified: VM cannot reach host {canary_ip}")
|
||||
@@ -0,0 +1,404 @@
|
||||
"""Launch flow for the Firecracker backend.
|
||||
|
||||
Per bottle:
|
||||
1. mint the egress CA, build the agent image (docker), export it to a
|
||||
cached ext4 rootfs;
|
||||
2. claim a free TAP pool slot (rootless flock);
|
||||
3. bring up the Docker sidecar bundle, publishing egress / git-gate /
|
||||
supervise on the slot's host-side TAP IP at fixed ports;
|
||||
4. boot the microVM on that TAP; wait for SSH;
|
||||
5. provision (CA, prompt, skills, workspace, git, supervise) over SSH.
|
||||
|
||||
Isolation is enforced by the operator-provisioned nft table (checked
|
||||
fail-closed in preflight): a VM reaches only its sidecar (DNAT'd from
|
||||
the host TAP IP) and nothing else. The agent's HTTPS_PROXY therefore
|
||||
points at `http://<host_tap_ip>:9099`, its only route to the world.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import dataclasses
|
||||
import os
|
||||
import subprocess
|
||||
from contextlib import ExitStack, contextmanager
|
||||
from pathlib import Path
|
||||
from typing import Callable, Generator
|
||||
|
||||
from ...bottle_state import (
|
||||
egress_state_dir,
|
||||
git_gate_state_dir,
|
||||
read_committed_image,
|
||||
)
|
||||
from ...egress import (
|
||||
EGRESS_ROUTES_IN_CONTAINER,
|
||||
egress_agent_env_entries,
|
||||
egress_resolve_token_values,
|
||||
egress_sidecar_env_entries,
|
||||
)
|
||||
from ...git_gate import (
|
||||
provision_git_gate_dynamic_keys,
|
||||
revoke_git_gate_provisioned_keys,
|
||||
)
|
||||
from ...log import die, info, warn
|
||||
from ...supervise import DB_PATH_IN_CONTAINER, SUPERVISE_PORT
|
||||
from ...util import expand_tilde
|
||||
from ..docker.egress import (
|
||||
EGRESS_CA_IN_CONTAINER,
|
||||
EGRESS_PORT,
|
||||
egress_tls_init,
|
||||
)
|
||||
from ..docker.git_gate import (
|
||||
GIT_GATE_ACCESS_HOOK_IN_CONTAINER,
|
||||
GIT_GATE_CREDS_DIR_IN_CONTAINER,
|
||||
GIT_GATE_ENTRYPOINT_IN_CONTAINER,
|
||||
GIT_GATE_HOOK_IN_CONTAINER,
|
||||
)
|
||||
from ..docker.sidecar_bundle import (
|
||||
SIDECAR_BUNDLE_DOCKERFILE,
|
||||
SIDECAR_BUNDLE_IMAGE,
|
||||
)
|
||||
from ..util import AGENT_CA_BUNDLE, AGENT_CA_PATH
|
||||
from . import firecracker_vm, isolation_probe, netpool, util
|
||||
from .bottle import FirecrackerBottle
|
||||
from .bottle_plan import FirecrackerBottlePlan
|
||||
|
||||
|
||||
_REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent)
|
||||
_GIT_HTTP_PORT = 9420
|
||||
_GIT_GATE_READY_FILE = "/run/git-gate/ready"
|
||||
|
||||
|
||||
def sidecar_container_name(slug: str) -> str:
|
||||
return f"bot-bottle-sidecars-{slug}"
|
||||
|
||||
|
||||
@contextmanager
|
||||
def launch(
|
||||
plan: FirecrackerBottlePlan,
|
||||
*,
|
||||
provision: Callable[[FirecrackerBottlePlan, "FirecrackerBottle"], str | None],
|
||||
) -> Generator[FirecrackerBottle, None, None]:
|
||||
stack = ExitStack()
|
||||
bottle_for_revoke = plan.manifest.bottle
|
||||
git_gate_dir_for_revoke = git_gate_state_dir(plan.slug)
|
||||
|
||||
def teardown() -> None:
|
||||
teardown_exc: BaseException | None = None
|
||||
try:
|
||||
stack.close()
|
||||
except BaseException as exc: # noqa: W0718 - teardown must continue
|
||||
teardown_exc = exc
|
||||
warn(f"firecracker teardown failed: {exc!r}")
|
||||
revoke_git_gate_provisioned_keys(bottle_for_revoke, git_gate_dir_for_revoke)
|
||||
if teardown_exc is not None:
|
||||
raise teardown_exc
|
||||
|
||||
try:
|
||||
plan = _mint_certs(plan)
|
||||
plan = _build_agent_image(plan)
|
||||
|
||||
# Claim a TAP slot; the flock is held until teardown closes it.
|
||||
slot, lock = netpool.allocate(plan.slug)
|
||||
stack.callback(lock.close)
|
||||
info(f"firecracker slot {slot.iface}: host={slot.host_ip} "
|
||||
f"guest={slot.guest_ip}")
|
||||
|
||||
plan = _provision_git_gate_keys(plan)
|
||||
|
||||
sidecar_name = sidecar_container_name(plan.slug)
|
||||
_force_remove_container(sidecar_name)
|
||||
_start_sidecar_bundle(plan, sidecar_name, slot.host_ip)
|
||||
stack.callback(_force_remove_container, sidecar_name)
|
||||
_stage_git_gate(plan, sidecar_name)
|
||||
|
||||
plan = _stamp_agent_urls(plan, slot.host_ip)
|
||||
|
||||
# Build the per-bottle rootfs + SSH key, then boot.
|
||||
base_dir = util.build_base_rootfs_dir(plan.image)
|
||||
run_dir = util.cache_dir() / "run" / plan.slug
|
||||
run_dir.mkdir(parents=True, exist_ok=True)
|
||||
rootfs = run_dir / "rootfs.ext4"
|
||||
util.build_rootfs_ext4(base_dir, rootfs)
|
||||
private_key, pubkey = util.generate_keypair(run_dir)
|
||||
|
||||
vm = firecracker_vm.boot(
|
||||
name=plan.container_name,
|
||||
rootfs=rootfs,
|
||||
tap=slot.iface,
|
||||
guest_ip=slot.guest_ip,
|
||||
host_ip=slot.host_ip,
|
||||
pubkey=pubkey,
|
||||
run_dir=run_dir,
|
||||
)
|
||||
stack.callback(vm.terminate)
|
||||
firecracker_vm.wait_for_ssh(vm, private_key)
|
||||
|
||||
# Authoritative fail-closed egress-boundary check, before the
|
||||
# agent runs: prove the VM cannot reach the host directly.
|
||||
isolation_probe.verify_isolation(private_key, slot.guest_ip)
|
||||
|
||||
bottle = FirecrackerBottle(
|
||||
plan.container_name,
|
||||
private_key=private_key,
|
||||
guest_ip=slot.guest_ip,
|
||||
guest_env=_agent_guest_env(plan, slot.host_ip),
|
||||
agent_command=plan.agent_command,
|
||||
agent_prompt_mode=plan.agent_prompt_mode,
|
||||
agent_provider_template=plan.agent_provider_template,
|
||||
terminal_title=(
|
||||
f"{plan.spec.label} ({plan.spec.agent_name})"
|
||||
if plan.spec.label else plan.spec.agent_name
|
||||
),
|
||||
terminal_color=plan.spec.color,
|
||||
agent_workdir=plan.workspace_plan.workdir,
|
||||
)
|
||||
bottle.prompt_path = provision(plan, bottle)
|
||||
|
||||
yield bottle
|
||||
finally:
|
||||
teardown()
|
||||
|
||||
|
||||
def _mint_certs(plan: FirecrackerBottlePlan) -> FirecrackerBottlePlan:
|
||||
egress_ca_host, egress_ca_cert_only = egress_tls_init(egress_state_dir(plan.slug))
|
||||
egress_plan = dataclasses.replace(
|
||||
plan.egress_plan,
|
||||
mitmproxy_ca_host_path=egress_ca_host,
|
||||
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
|
||||
)
|
||||
return dataclasses.replace(plan, egress_plan=egress_plan)
|
||||
|
||||
|
||||
def _build_agent_image(plan: FirecrackerBottlePlan) -> FirecrackerBottlePlan:
|
||||
_docker_build(SIDECAR_BUNDLE_IMAGE, _REPO_DIR, dockerfile=SIDECAR_BUNDLE_DOCKERFILE)
|
||||
committed = read_committed_image(plan.slug)
|
||||
if committed and _image_exists(committed):
|
||||
info(f"using committed image {committed!r}")
|
||||
return dataclasses.replace(
|
||||
plan,
|
||||
agent_provision=dataclasses.replace(plan.agent_provision, image=committed),
|
||||
)
|
||||
_docker_build(plan.image, _REPO_DIR, dockerfile=plan.dockerfile_path)
|
||||
return plan
|
||||
|
||||
|
||||
def _provision_git_gate_keys(plan: FirecrackerBottlePlan) -> FirecrackerBottlePlan:
|
||||
if not plan.git_gate_plan.upstreams:
|
||||
return plan
|
||||
git_gate_plan = provision_git_gate_dynamic_keys(
|
||||
plan.manifest.bottle, plan.git_gate_plan, git_gate_state_dir(plan.slug),
|
||||
)
|
||||
return dataclasses.replace(plan, git_gate_plan=git_gate_plan)
|
||||
|
||||
|
||||
def _stamp_agent_urls(
|
||||
plan: FirecrackerBottlePlan, host_ip: str,
|
||||
) -> FirecrackerBottlePlan:
|
||||
proxy_url = f"http://{host_ip}:{EGRESS_PORT}"
|
||||
supervise_url = (
|
||||
f"http://{host_ip}:{SUPERVISE_PORT}/" if plan.supervise_plan is not None else ""
|
||||
)
|
||||
git_gate_url = (
|
||||
f"http://{host_ip}:{_GIT_HTTP_PORT}" if plan.git_gate_plan.upstreams else ""
|
||||
)
|
||||
return dataclasses.replace(
|
||||
plan,
|
||||
agent_proxy_url=proxy_url,
|
||||
agent_git_gate_url=git_gate_url,
|
||||
agent_supervise_url=supervise_url,
|
||||
)
|
||||
|
||||
|
||||
# --- sidecar bundle (Docker) ----------------------------------------
|
||||
|
||||
def _start_sidecar_bundle(
|
||||
plan: FirecrackerBottlePlan, sidecar_name: str, host_ip: str,
|
||||
) -> None:
|
||||
argv = ["docker", "run", "--name", sidecar_name, "--detach", "--rm",
|
||||
"-e", f"BOT_BOTTLE_SIDECAR_DAEMONS={','.join(_sidecar_daemons(plan))}"]
|
||||
for entry in _sidecar_env_entries(plan):
|
||||
argv += ["-e", entry]
|
||||
for host_path, container_path, read_only in _sidecar_mounts(plan):
|
||||
argv += ["-v", f"{host_path}:{container_path}{':ro' if read_only else ''}"]
|
||||
# Publish on the slot's host TAP IP at fixed ports — each bottle
|
||||
# has a distinct host_ip, so fixed ports never collide, and the VM
|
||||
# reaches them at a stable, well-known address (its only route out).
|
||||
for port in _sidecar_ports(plan):
|
||||
argv += ["-p", f"{host_ip}:{port}:{port}"]
|
||||
argv.append(SIDECAR_BUNDLE_IMAGE)
|
||||
|
||||
effective_env = {**dict(os.environ), **plan.agent_provision.provisioned_env}
|
||||
token_values = egress_resolve_token_values(
|
||||
plan.egress_plan.token_env_map, effective_env,
|
||||
)
|
||||
env = {**os.environ, **token_values}
|
||||
info(f"docker run sidecar bundle {sidecar_name} (published on {host_ip})")
|
||||
result = subprocess.run(argv, capture_output=True, text=True, env=env, check=False)
|
||||
if result.returncode != 0:
|
||||
die(f"docker run for sidecar bundle {sidecar_name} failed: "
|
||||
f"{(result.stderr or '').strip() or '<no stderr>'}")
|
||||
|
||||
|
||||
def _sidecar_daemons(plan: FirecrackerBottlePlan) -> tuple[str, ...]:
|
||||
daemons = ["egress"]
|
||||
if plan.git_gate_plan.upstreams:
|
||||
daemons += ["git-gate", "git-http"]
|
||||
if plan.supervise_plan is not None:
|
||||
daemons.append("supervise")
|
||||
return tuple(daemons)
|
||||
|
||||
|
||||
def _sidecar_ports(plan: FirecrackerBottlePlan) -> tuple[int, ...]:
|
||||
ports = [EGRESS_PORT]
|
||||
if plan.git_gate_plan.upstreams:
|
||||
ports.append(_GIT_HTTP_PORT)
|
||||
if plan.supervise_plan is not None:
|
||||
ports.append(SUPERVISE_PORT)
|
||||
return tuple(ports)
|
||||
|
||||
|
||||
def _sidecar_env_entries(plan: FirecrackerBottlePlan) -> tuple[str, ...]:
|
||||
env: list[str] = list(egress_sidecar_env_entries(plan.egress_plan))
|
||||
if plan.git_gate_plan.upstreams:
|
||||
env.append(f"BOT_BOTTLE_GIT_GATE_READY_FILE={_GIT_GATE_READY_FILE}")
|
||||
if plan.supervise_plan is not None:
|
||||
env += [
|
||||
f"SUPERVISE_BOTTLE_SLUG={plan.slug}",
|
||||
f"SUPERVISE_DB_PATH={DB_PATH_IN_CONTAINER}",
|
||||
f"SUPERVISE_PORT={SUPERVISE_PORT}",
|
||||
]
|
||||
return tuple(env)
|
||||
|
||||
|
||||
def _sidecar_mounts(
|
||||
plan: FirecrackerBottlePlan,
|
||||
) -> tuple[tuple[str, str, bool], ...]:
|
||||
mounts: list[tuple[str, str, bool]] = []
|
||||
ep = plan.egress_plan
|
||||
mounts.append((str(ep.mitmproxy_ca_host_path.parent),
|
||||
str(Path(EGRESS_CA_IN_CONTAINER).parent), False))
|
||||
if ep.routes:
|
||||
mounts.append((str(ep.routes_path.parent),
|
||||
str(Path(EGRESS_ROUTES_IN_CONTAINER).parent), True))
|
||||
sp = plan.supervise_plan
|
||||
if sp is not None:
|
||||
mounts.append((str(sp.db_path.parent),
|
||||
str(Path(DB_PATH_IN_CONTAINER).parent), False))
|
||||
return tuple(mounts)
|
||||
|
||||
|
||||
def _stage_git_gate(plan: FirecrackerBottlePlan, sidecar_name: str) -> None:
|
||||
gp = plan.git_gate_plan
|
||||
if not gp.upstreams:
|
||||
return
|
||||
_docker_exec(sidecar_name, [
|
||||
"mkdir", "-p",
|
||||
str(Path(GIT_GATE_HOOK_IN_CONTAINER).parent),
|
||||
GIT_GATE_CREDS_DIR_IN_CONTAINER, "/git",
|
||||
str(Path(_GIT_GATE_READY_FILE).parent),
|
||||
])
|
||||
for host_path, container_path in _git_gate_files(plan):
|
||||
_docker_cp(host_path, f"{sidecar_name}:{container_path}")
|
||||
_docker_exec(sidecar_name, [
|
||||
"sh", "-c",
|
||||
f"chmod 755 {GIT_GATE_ENTRYPOINT_IN_CONTAINER} "
|
||||
f"{GIT_GATE_HOOK_IN_CONTAINER} {GIT_GATE_ACCESS_HOOK_IN_CONTAINER} && "
|
||||
f"chmod 600 {GIT_GATE_CREDS_DIR_IN_CONTAINER}/* && "
|
||||
f"touch {_GIT_GATE_READY_FILE}",
|
||||
])
|
||||
|
||||
|
||||
def _git_gate_files(plan: FirecrackerBottlePlan) -> tuple[tuple[str, str], ...]:
|
||||
gp = plan.git_gate_plan
|
||||
files: list[tuple[str, str]] = [
|
||||
(str(gp.entrypoint_script), GIT_GATE_ENTRYPOINT_IN_CONTAINER),
|
||||
(str(gp.hook_script), GIT_GATE_HOOK_IN_CONTAINER),
|
||||
(str(gp.access_hook_script), GIT_GATE_ACCESS_HOOK_IN_CONTAINER),
|
||||
]
|
||||
for upstream in gp.upstreams:
|
||||
files.append((expand_tilde(upstream.identity_file),
|
||||
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{upstream.name}-key"))
|
||||
if upstream.known_hosts_file:
|
||||
files.append((str(upstream.known_hosts_file),
|
||||
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{upstream.name}-known_hosts"))
|
||||
return tuple(files)
|
||||
|
||||
|
||||
# --- agent guest env -------------------------------------------------
|
||||
|
||||
def _agent_guest_env(plan: FirecrackerBottlePlan, host_ip: str) -> dict[str, str]:
|
||||
"""Env injected into every agent/exec call over SSH. The VM has no
|
||||
baked process env (it just runs init), so the proxy/CA/git/supervise
|
||||
wiring is applied per-invocation."""
|
||||
proxy_url = f"http://{host_ip}:{EGRESS_PORT}"
|
||||
no_proxy = f"localhost,127.0.0.1,{host_ip}"
|
||||
env: dict[str, str] = {
|
||||
"HTTPS_PROXY": proxy_url, "HTTP_PROXY": proxy_url,
|
||||
"https_proxy": proxy_url, "http_proxy": proxy_url,
|
||||
"NO_PROXY": no_proxy, "no_proxy": no_proxy,
|
||||
"NODE_EXTRA_CA_CERTS": AGENT_CA_PATH,
|
||||
"SSL_CERT_FILE": AGENT_CA_BUNDLE,
|
||||
"REQUESTS_CA_BUNDLE": AGENT_CA_BUNDLE,
|
||||
}
|
||||
if plan.agent_git_gate_url:
|
||||
env["GIT_GATE_URL"] = plan.agent_git_gate_url
|
||||
if plan.agent_supervise_url:
|
||||
env["MCP_SUPERVISE_URL"] = plan.agent_supervise_url
|
||||
for entry in egress_agent_env_entries(plan.egress_plan):
|
||||
key, _, value = entry.partition("=")
|
||||
env[key] = value
|
||||
env.update(plan.agent_provision.guest_env)
|
||||
# Forwarded (bare-name) env: resolve host values now, since the VM
|
||||
# can't inherit them from a `docker run --env NAME`.
|
||||
for name in plan.forwarded_env:
|
||||
value = os.environ.get(name)
|
||||
if value is not None:
|
||||
env[name] = value
|
||||
return env
|
||||
|
||||
|
||||
# --- docker helpers --------------------------------------------------
|
||||
|
||||
def _docker_build(ref: str, context: str, *, dockerfile: str = "") -> None:
|
||||
info(f"docker build {ref}")
|
||||
args = ["docker", "build", "-t", ref]
|
||||
if dockerfile:
|
||||
if not os.path.isabs(dockerfile):
|
||||
dockerfile = os.path.join(context, dockerfile)
|
||||
args += ["-f", dockerfile]
|
||||
args.append(context)
|
||||
result = subprocess.run(args, check=False)
|
||||
if result.returncode != 0:
|
||||
die(f"docker build for {ref!r} failed")
|
||||
|
||||
|
||||
def _image_exists(ref: str) -> bool:
|
||||
return subprocess.run(
|
||||
["docker", "image", "inspect", ref],
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
|
||||
).returncode == 0
|
||||
|
||||
|
||||
def _force_remove_container(name: str) -> None:
|
||||
subprocess.run(
|
||||
["docker", "rm", "-f", name],
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
|
||||
)
|
||||
|
||||
|
||||
def _docker_exec(name: str, argv: list[str]) -> None:
|
||||
result = subprocess.run(
|
||||
["docker", "exec", name, *argv], capture_output=True, text=True, check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
die(f"docker exec in {name} failed: "
|
||||
f"{(result.stderr or '').strip() or '<no stderr>'}")
|
||||
|
||||
|
||||
def _docker_cp(host_path: str, dest: str) -> None:
|
||||
result = subprocess.run(
|
||||
["docker", "cp", host_path, dest], capture_output=True, text=True, check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
die(f"docker cp {host_path} -> {dest} failed: "
|
||||
f"{(result.stderr or '').strip() or '<no stderr>'}")
|
||||
@@ -0,0 +1,318 @@
|
||||
"""Firecracker network pool: constants, IP math, allocation, and the
|
||||
config renderers (shell command + NixOS module) shown to operators.
|
||||
|
||||
The Firecracker backend needs a privileged one-time network setup:
|
||||
a pool of point-to-point TAP devices (owned by the invoking user, so
|
||||
`./cli.py start` never needs root) and a dedicated nftables table that
|
||||
isolates every VM. This module is the single source of truth for the
|
||||
pool parameters — the shell script (`scripts/firecracker-netpool.sh`),
|
||||
the printed NixOS snippet, and the backend's fail-closed preflight all
|
||||
derive from the constants here so they can't drift.
|
||||
|
||||
Topology (per slot i):
|
||||
* TAP ``bbfc{i}`` — no shared bridge, so no docker0 / virbr0 / cni0
|
||||
/ br-* collisions.
|
||||
* a /31 host<->guest link: host = base + 2i (the gateway the VM
|
||||
routes through), guest = base + 2i + 1 (the VM's address).
|
||||
* isolation via ``table inet bot_bottle_fc``: a VM reaches only its
|
||||
own sidecar (DNAT'd from the host TAP IP) and nothing else.
|
||||
|
||||
The default IP block is ``10.243.0.0/16`` — an intentionally obscure
|
||||
corner of RFC-1918 private space. RFC-1918 is the range *designated*
|
||||
for private links; we pick a high, unusual /16 to steer clear of the
|
||||
common occupants (Docker's 172.17-31, libvirt's 192.168.122, k8s
|
||||
10.42/10.244, and typical home LANs on 192.168.0/1.x or 10.0.0.x).
|
||||
We deliberately avoid ``100.64.0.0/10`` (RFC-6598 CGNAT) because
|
||||
Tailscale hands out node addresses from exactly that range. No default
|
||||
is collision-proof, so `overlapping_routes()` is the real guard —
|
||||
`backend setup`/`status` and the launch preflight call it to catch
|
||||
a base that clashes with something already on the host.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import fcntl
|
||||
import ipaddress
|
||||
import os
|
||||
import subprocess
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import IO
|
||||
|
||||
from ...log import die
|
||||
|
||||
|
||||
# Interface names are capped at 15 chars (IFNAMSIZ-1); "bbfc" + a small
|
||||
# index stays well under that and is distinctive enough to grep for.
|
||||
IFACE_PREFIX = os.environ.get("BOT_BOTTLE_FC_IFACE_PREFIX", "bbfc")
|
||||
NFT_TABLE = "bot_bottle_fc"
|
||||
|
||||
|
||||
def pool_size() -> int:
|
||||
return int(os.environ.get("BOT_BOTTLE_FC_POOL_SIZE", "8"))
|
||||
|
||||
|
||||
def ip_base() -> str:
|
||||
return os.environ.get("BOT_BOTTLE_FC_IP_BASE", "10.243.0.0")
|
||||
|
||||
|
||||
# Sidecar ports the VM reaches at its host-side TAP IP. Kept in sync
|
||||
# with the backend constants (egress 9099, supervise 9100, git-http
|
||||
# 9420); rendered into the setup output for operator visibility.
|
||||
SIDECAR_PORTS = (9099, 9100, 9420)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Slot:
|
||||
"""One pool slot: a TAP device and its host/guest /31 addresses."""
|
||||
|
||||
index: int
|
||||
iface: str
|
||||
host_ip: str
|
||||
guest_ip: str
|
||||
|
||||
@property
|
||||
def guest_cidr(self) -> str:
|
||||
"""Guest address with the /31 prefix, for the kernel `ip=` arg."""
|
||||
return f"{self.guest_ip}/31"
|
||||
|
||||
|
||||
def slot(index: int) -> Slot:
|
||||
base = int(ipaddress.IPv4Address(ip_base()))
|
||||
return Slot(
|
||||
index=index,
|
||||
iface=f"{IFACE_PREFIX}{index}",
|
||||
host_ip=str(ipaddress.IPv4Address(base + 2 * index)),
|
||||
guest_ip=str(ipaddress.IPv4Address(base + 2 * index + 1)),
|
||||
)
|
||||
|
||||
|
||||
def all_slots() -> list[Slot]:
|
||||
return [slot(i) for i in range(pool_size())]
|
||||
|
||||
|
||||
# --- fail-closed verification ---------------------------------------
|
||||
|
||||
def _run_ok(argv: list[str]) -> bool:
|
||||
"""Run a probe command, treating a missing binary as failure
|
||||
(rather than crashing) so callers can stay fail-closed."""
|
||||
try:
|
||||
return subprocess.run(
|
||||
argv, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
).returncode == 0
|
||||
except FileNotFoundError:
|
||||
return False
|
||||
|
||||
|
||||
def nft_table_present() -> bool:
|
||||
"""True iff the isolation table exists in the active nftables
|
||||
backend. The backend's preflight treats absence as fatal — the VM
|
||||
must not boot without its egress boundary in place.
|
||||
|
||||
Listing may require root; when it can't be confirmed here the
|
||||
launch path falls back to an empirical post-boot isolation probe.
|
||||
Returns False (fail-closed) if `nft` is unavailable."""
|
||||
return _run_ok(["nft", "list", "table", "inet", NFT_TABLE])
|
||||
|
||||
|
||||
def tap_present(iface: str) -> bool:
|
||||
# `ip link show` is unprivileged, so the TAP-pool check is reliable
|
||||
# for the non-root launcher.
|
||||
return _run_ok(["ip", "link", "show", iface])
|
||||
|
||||
|
||||
def missing_taps() -> list[str]:
|
||||
"""Pool TAPs that the one-time setup has not created yet."""
|
||||
return [s.iface for s in all_slots() if not tap_present(s.iface)]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RouteConflict:
|
||||
"""An existing host route whose destination overlaps the pool range
|
||||
but isn't one of our own bbfc TAPs — i.e. the chosen IP base clashes
|
||||
with something already on the host (a Tailscale CGNAT peer, a
|
||||
docker/libvirt bridge, the LAN…)."""
|
||||
|
||||
dst: str
|
||||
dev: str
|
||||
|
||||
|
||||
def _pool_span() -> tuple[int, int]:
|
||||
"""Inclusive [first, last] integer bounds of every address the pool
|
||||
occupies: base .. base + 2*pool_size - 1."""
|
||||
base = int(ipaddress.IPv4Address(ip_base()))
|
||||
return base, base + 2 * pool_size() - 1
|
||||
|
||||
|
||||
def overlapping_routes() -> list[RouteConflict]:
|
||||
"""Existing routes whose destination overlaps the pool's address
|
||||
range, excluding our own `bbfc*` TAP routes and the default route.
|
||||
|
||||
A non-empty result means `BOT_BOTTLE_FC_IP_BASE` collides with
|
||||
something already configured on this host, so the pool would shadow
|
||||
(or be shadowed by) it. Empty on success — or when `ip` is missing
|
||||
or unparseable, since this is an advisory guard, not the fail-closed
|
||||
isolation check (that's the nft table + post-boot probe)."""
|
||||
import json
|
||||
|
||||
try:
|
||||
proc = subprocess.run(
|
||||
["ip", "-json", "route", "show", "table", "all"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
except FileNotFoundError:
|
||||
return []
|
||||
if proc.returncode != 0 or not proc.stdout.strip():
|
||||
return []
|
||||
try:
|
||||
routes = json.loads(proc.stdout)
|
||||
except json.JSONDecodeError:
|
||||
return []
|
||||
|
||||
lo, hi = _pool_span()
|
||||
conflicts: list[RouteConflict] = []
|
||||
for r in routes:
|
||||
if not isinstance(r, dict):
|
||||
continue
|
||||
dst = r.get("dst")
|
||||
dev = r.get("dev", "")
|
||||
if not isinstance(dst, str) or dst in ("", "default"):
|
||||
continue
|
||||
if isinstance(dev, str) and dev.startswith(IFACE_PREFIX):
|
||||
continue # our own pool link
|
||||
try:
|
||||
net = ipaddress.ip_network(dst, strict=False)
|
||||
except ValueError:
|
||||
continue
|
||||
if net.version != 4:
|
||||
continue
|
||||
r_lo = int(net.network_address)
|
||||
r_hi = int(net.broadcast_address)
|
||||
if r_lo <= hi and lo <= r_hi: # ranges intersect
|
||||
conflicts.append(RouteConflict(dst=dst, dev=str(dev)))
|
||||
return conflicts
|
||||
|
||||
|
||||
# --- allocation ------------------------------------------------------
|
||||
|
||||
def _lock_dir() -> Path:
|
||||
d = Path.home() / ".cache" / "bot-bottle" / "firecracker" / "pool"
|
||||
d.mkdir(parents=True, exist_ok=True)
|
||||
return d
|
||||
|
||||
|
||||
def allocate(slug: str) -> tuple[Slot, IO[str]]:
|
||||
"""Claim a free pool slot for one bottle. Returns the slot and the
|
||||
held lock file — the caller keeps it open for the VM's lifetime and
|
||||
closes it on teardown; the flock auto-releases if the launcher
|
||||
crashes, so a slot is never leaked. Dies when the pool is
|
||||
exhausted.
|
||||
|
||||
A per-slot `flock` (rather than inspecting running processes) makes
|
||||
allocation race-free across concurrent launches without a central
|
||||
registry: the first launcher to grab the lock owns the slot."""
|
||||
del slug # logged by the caller; allocation is purely lock-driven
|
||||
for s in all_slots():
|
||||
lock_path = _lock_dir() / f"{s.iface}.lock"
|
||||
handle = open(lock_path, "w", encoding="utf-8")
|
||||
try:
|
||||
fcntl.flock(handle, fcntl.LOCK_EX | fcntl.LOCK_NB)
|
||||
except OSError:
|
||||
handle.close()
|
||||
continue
|
||||
return s, handle
|
||||
die(f"Firecracker TAP pool exhausted ({pool_size()} slots, all in "
|
||||
f"use). Stop a running bottle or raise BOT_BOTTLE_FC_POOL_SIZE "
|
||||
f"and re-run `./cli.py backend setup --backend=firecracker`.")
|
||||
raise AssertionError("unreachable")
|
||||
|
||||
|
||||
# --- config renderers (shown by `./cli.py backend setup`) -----------
|
||||
|
||||
def render_shell_setup() -> str:
|
||||
"""The imperative command for non-NixOS hosts."""
|
||||
env = _nondefault_env()
|
||||
prefix = f"{env} " if env else ""
|
||||
return f"sudo {prefix}./scripts/firecracker-netpool.sh up"
|
||||
|
||||
|
||||
def render_nixos_module() -> str:
|
||||
"""A paste-ready NixOS snippet, derived from the live constants.
|
||||
|
||||
Uses systemd-networkd tap netdevs (so TAPs are only reconfigured
|
||||
when this config changes, not torn down on every rebuild — which
|
||||
would drop running VMs) and `networking.nftables.tables.<name>`
|
||||
(an independent table that coexists with Docker's iptables rules
|
||||
rather than replacing the whole ruleset)."""
|
||||
slots = all_slots()
|
||||
owner = os.environ.get("USER", "youruser")
|
||||
|
||||
netdevs = "\n".join(
|
||||
f''' "10-{s.iface}" = {{
|
||||
netdevConfig = {{ Name = "{s.iface}"; Kind = "tap"; }};
|
||||
tapConfig = {{ User = "{owner}"; }};
|
||||
}};'''
|
||||
for s in slots
|
||||
)
|
||||
networks = "\n".join(
|
||||
f''' "10-{s.iface}" = {{
|
||||
matchConfig.Name = "{s.iface}";
|
||||
address = [ "{s.host_ip}/31" ];
|
||||
networkConfig.ConfigureWithoutCarrier = true;
|
||||
}};'''
|
||||
for s in slots
|
||||
)
|
||||
|
||||
return f"""# bot-bottle Firecracker network pool ({len(slots)} slots, base {ip_base()}).
|
||||
# Generated by `./cli.py backend setup --backend=firecracker`; owner user = {owner!r}.
|
||||
{{ ... }}:
|
||||
{{
|
||||
boot.kernel.sysctl."net.ipv4.ip_forward" = 1;
|
||||
|
||||
systemd.network.enable = true;
|
||||
systemd.network.netdevs = {{
|
||||
{netdevs}
|
||||
}};
|
||||
systemd.network.networks = {{
|
||||
{networks}
|
||||
}};
|
||||
|
||||
# Independent table — does not touch Docker/ufw/firewalld rules.
|
||||
# A bottle VM (traffic from a bbfc* TAP) may reach only its own
|
||||
# sidecar (DNAT'd from the host TAP IP); everything else is dropped,
|
||||
# so there is no egress except through the sidecar proxy.
|
||||
networking.nftables.enable = true;
|
||||
networking.nftables.tables."{NFT_TABLE}" = {{
|
||||
family = "inet";
|
||||
content = ''
|
||||
chain forward {{
|
||||
type filter hook forward priority -10; policy accept;
|
||||
iifname != "{IFACE_PREFIX}*" return
|
||||
ct state established,related accept
|
||||
ct status dnat accept
|
||||
drop
|
||||
}}
|
||||
chain input {{
|
||||
type filter hook input priority -10; policy accept;
|
||||
iifname != "{IFACE_PREFIX}*" return
|
||||
ct state established,related accept
|
||||
drop
|
||||
}}
|
||||
'';
|
||||
}};
|
||||
}}
|
||||
"""
|
||||
|
||||
|
||||
def _nondefault_env() -> str:
|
||||
"""Render any non-default pool env overrides so the printed shell
|
||||
command reproduces the operator's current settings."""
|
||||
pairs = []
|
||||
if os.environ.get("BOT_BOTTLE_FC_POOL_SIZE"):
|
||||
pairs.append(f"BOT_BOTTLE_FC_POOL_SIZE={pool_size()}")
|
||||
if os.environ.get("BOT_BOTTLE_FC_IP_BASE"):
|
||||
pairs.append(f"BOT_BOTTLE_FC_IP_BASE={ip_base()}")
|
||||
if os.environ.get("BOT_BOTTLE_FC_IFACE_PREFIX"):
|
||||
pairs.append(f"BOT_BOTTLE_FC_IFACE_PREFIX={IFACE_PREFIX}")
|
||||
return " ".join(pairs)
|
||||
@@ -0,0 +1,47 @@
|
||||
"""Prepare step for the Firecracker backend."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from ...agent_provider import AgentProvisionPlan
|
||||
from ...egress import EgressPlan
|
||||
from ...env import ResolvedEnv
|
||||
from ...git_gate import GitGatePlan
|
||||
from ...manifest import Manifest
|
||||
from ...supervise import SupervisePlan
|
||||
from .. import BottleSpec
|
||||
from . import util
|
||||
from .bottle_plan import FirecrackerBottlePlan
|
||||
|
||||
|
||||
def preflight() -> None:
|
||||
util.require_firecracker()
|
||||
|
||||
|
||||
def build_guest_env(resolved_env: ResolvedEnv) -> dict[str, str]:
|
||||
return dict(resolved_env.literals)
|
||||
|
||||
|
||||
def resolve_plan(
|
||||
spec: BottleSpec,
|
||||
manifest: Manifest,
|
||||
slug: str,
|
||||
resolved_env: ResolvedEnv,
|
||||
agent_provision_plan: AgentProvisionPlan,
|
||||
egress_plan: EgressPlan,
|
||||
supervise_plan: SupervisePlan | None,
|
||||
git_gate_plan: GitGatePlan,
|
||||
stage_dir: Path,
|
||||
) -> FirecrackerBottlePlan:
|
||||
return FirecrackerBottlePlan(
|
||||
spec=spec,
|
||||
manifest=manifest,
|
||||
stage_dir=stage_dir,
|
||||
slug=slug,
|
||||
forwarded_env=dict(resolved_env.forwarded),
|
||||
git_gate_plan=git_gate_plan,
|
||||
egress_plan=egress_plan,
|
||||
supervise_plan=supervise_plan,
|
||||
agent_provision=agent_provision_plan,
|
||||
)
|
||||
@@ -0,0 +1,109 @@
|
||||
"""Host setup + status for the Firecracker backend.
|
||||
|
||||
`setup()` prints the host-appropriate config for the privileged,
|
||||
one-time network pool (TAP devices + isolation nftables table) the
|
||||
backend needs. On NixOS it points at the flake module (and prints a
|
||||
paste-able fallback); elsewhere it prints the sudo command for the
|
||||
bundled setup script. `status()` reports what's present, including
|
||||
whether the pool range collides with an existing route.
|
||||
|
||||
Called through `FirecrackerBottleBackend.setup` / `.status`, which the
|
||||
generic `./cli.py backend {setup,status}` command dispatches to.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from . import netpool
|
||||
|
||||
|
||||
def _owner() -> str:
|
||||
return os.environ.get("USER", "youruser")
|
||||
|
||||
|
||||
def _is_nixos() -> bool:
|
||||
if Path("/etc/NIXOS").exists():
|
||||
return True
|
||||
try:
|
||||
return "ID=nixos" in Path("/etc/os-release").read_text()
|
||||
except OSError:
|
||||
return False
|
||||
|
||||
|
||||
def _warn_overlaps() -> None:
|
||||
"""Warn if the chosen pool range collides with an existing host route
|
||||
(Tailscale CGNAT peer, a docker/libvirt bridge, the LAN…)."""
|
||||
conflicts = netpool.overlapping_routes()
|
||||
if not conflicts:
|
||||
return
|
||||
detail = "\n".join(f" {c.dst} dev {c.dev}" for c in conflicts)
|
||||
sys.stderr.write(
|
||||
f"WARNING: pool range (base {netpool.ip_base()}, "
|
||||
f"{netpool.pool_size()} slots) overlaps existing routes:\n"
|
||||
f"{detail}\n"
|
||||
"Set BOT_BOTTLE_FC_IP_BASE to a free range before setup, or the "
|
||||
"pool may shadow / be shadowed by the above.\n\n"
|
||||
)
|
||||
|
||||
|
||||
def setup() -> int:
|
||||
slots = netpool.all_slots()
|
||||
sys.stderr.write(
|
||||
f"Firecracker network pool: {len(slots)} slots "
|
||||
f"({slots[0].iface}..{slots[-1].iface}), base {netpool.ip_base()}.\n"
|
||||
f"This is a one-time privileged setup (needs root once).\n\n"
|
||||
)
|
||||
_warn_overlaps()
|
||||
if _is_nixos():
|
||||
sys.stderr.write(
|
||||
"Detected NixOS. Preferred: consume the flake module (versioned, "
|
||||
"no copy-paste drift):\n\n"
|
||||
" # flake inputs (point at wherever you host bot-bottle):\n"
|
||||
" inputs.bot-bottle.url = \"git+ssh://<your-bot-bottle-remote>\";\n"
|
||||
" # host module:\n"
|
||||
" imports = [ inputs.bot-bottle.nixosModules.firecracker-netpool ];\n"
|
||||
f" services.bot-bottle-firecracker = {{ enable = true; owner = \"{_owner()}\"; }};\n\n"
|
||||
"Then `nixos-rebuild switch`. Channel (non-flake) users can "
|
||||
"`imports = [ <bot-bottle>/nix/firecracker-netpool.nix ];` instead.\n\n"
|
||||
"Fallback — paste this generated module directly:\n\n"
|
||||
)
|
||||
sys.stdout.write(netpool.render_nixos_module())
|
||||
else:
|
||||
sys.stderr.write("Run the one-time setup as root:\n\n")
|
||||
sys.stdout.write(netpool.render_shell_setup() + "\n")
|
||||
sys.stderr.write(
|
||||
"\n(On NixOS, use the declarative module instead — this host "
|
||||
"was not detected as NixOS.)\n"
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
def status() -> int:
|
||||
ok = True
|
||||
if netpool.nft_table_present():
|
||||
sys.stderr.write(f"nft table inet {netpool.NFT_TABLE}: present\n")
|
||||
else:
|
||||
sys.stderr.write(f"nft table inet {netpool.NFT_TABLE}: MISSING\n")
|
||||
ok = False
|
||||
missing = netpool.missing_taps()
|
||||
total = netpool.pool_size()
|
||||
if missing:
|
||||
sys.stderr.write(f"TAP pool: {total - len(missing)}/{total} present "
|
||||
f"(missing: {', '.join(missing)})\n")
|
||||
ok = False
|
||||
else:
|
||||
sys.stderr.write(f"TAP pool: {total}/{total} present\n")
|
||||
conflicts = netpool.overlapping_routes()
|
||||
if conflicts:
|
||||
detail = ", ".join(f"{c.dst} dev {c.dev}" for c in conflicts)
|
||||
sys.stderr.write(f"range overlap: base {netpool.ip_base()} CLASHES "
|
||||
f"with {detail}\n")
|
||||
ok = False
|
||||
else:
|
||||
sys.stderr.write(f"range overlap: none (base {netpool.ip_base()})\n")
|
||||
if not ok:
|
||||
sys.stderr.write("\nRun: ./cli.py backend setup --backend=firecracker\n")
|
||||
return 0 if ok else 1
|
||||
@@ -0,0 +1,311 @@
|
||||
"""Host-side primitives for the Firecracker backend.
|
||||
|
||||
Covers the pieces that don't need root at launch time: locating the
|
||||
firecracker binary / guest kernel / injected dropbear, the fail-closed
|
||||
preflight (KVM + kernel + isolation table + TAP pool must all be
|
||||
present before a VM boots), the rootless rootfs pipeline
|
||||
(`docker export` -> `mke2fs -d`, no mount), and per-bottle SSH key
|
||||
generation.
|
||||
|
||||
The privileged network setup (TAP pool + nft table) is a one-time
|
||||
operator step — see `netpool.py`, `scripts/firecracker-netpool.sh`,
|
||||
and `./cli.py backend setup --backend=firecracker`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import platform
|
||||
import shutil
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
from ...log import die, info, warn
|
||||
from . import netpool
|
||||
|
||||
|
||||
# Guest agent images are Debian-family with USER node; the VM is
|
||||
# reached over SSH as root (init drops the pubkey into both root's and
|
||||
# node's authorized_keys) and commands `runuser` down to node.
|
||||
GUEST_SSH_USER = "root"
|
||||
|
||||
# `/dev/kvm` must exist and be openable by the invoking user.
|
||||
_KVM_DEVICE = "/dev/kvm"
|
||||
|
||||
|
||||
def cache_dir() -> Path:
|
||||
d = Path(
|
||||
os.environ.get(
|
||||
"BOT_BOTTLE_FC_CACHE",
|
||||
str(Path.home() / ".cache" / "bot-bottle" / "firecracker"),
|
||||
)
|
||||
)
|
||||
d.mkdir(parents=True, exist_ok=True)
|
||||
return d
|
||||
|
||||
|
||||
def kernel_path() -> Path:
|
||||
return Path(os.environ.get("BOT_BOTTLE_FC_KERNEL", str(cache_dir() / "vmlinux")))
|
||||
|
||||
|
||||
def dropbear_path() -> Path:
|
||||
"""The static dropbear injected into every guest rootfs as its SSH
|
||||
server. Must be statically linked — the guest has none of the
|
||||
host's shared libraries."""
|
||||
return Path(
|
||||
os.environ.get("BOT_BOTTLE_FC_DROPBEAR", str(cache_dir() / "dropbear"))
|
||||
)
|
||||
|
||||
|
||||
# --- availability + preflight ---------------------------------------
|
||||
|
||||
def is_linux() -> bool:
|
||||
return platform.system() == "Linux"
|
||||
|
||||
|
||||
def is_host_capable() -> bool:
|
||||
"""Whether this host *could* run firecracker — Linux with KVM —
|
||||
regardless of whether the `firecracker` binary is installed. Used
|
||||
for default-backend selection so a KVM Linux host that hasn't
|
||||
installed firecracker yet still selects it and gets an install
|
||||
pointer at launch (see `require_firecracker`), rather than silently
|
||||
falling back to docker."""
|
||||
return is_linux() and os.path.exists(_KVM_DEVICE)
|
||||
|
||||
|
||||
def is_available() -> bool:
|
||||
"""Cheap capability probe used by cross-backend enumeration —
|
||||
firecracker on PATH, on Linux, with KVM. Does not check the
|
||||
(operator-provisioned) kernel / pool, so an available-but-unset
|
||||
host still shows up and gets an actionable error at launch."""
|
||||
return is_host_capable() and shutil.which("firecracker") is not None
|
||||
|
||||
|
||||
def require_firecracker() -> None:
|
||||
"""Fail-closed preflight. Every check that gates the security
|
||||
boundary (the isolation table, the TAP pool) errors rather than
|
||||
booting a VM without it."""
|
||||
if not is_linux():
|
||||
die("firecracker backend is only supported on Linux (KVM). "
|
||||
"On macOS use --backend=macos-container.")
|
||||
if shutil.which("firecracker") is None:
|
||||
info("Firecracker is required but was not found on PATH.")
|
||||
info("Install: https://github.com/firecracker-microvm/firecracker/releases")
|
||||
die("firecracker not found on PATH")
|
||||
_require_kvm()
|
||||
if not kernel_path().is_file():
|
||||
die(f"guest kernel not found at {kernel_path()}. Set "
|
||||
"BOT_BOTTLE_FC_KERNEL or place a vmlinux there.")
|
||||
if not dropbear_path().is_file():
|
||||
die(f"static dropbear not found at {dropbear_path()}. Set "
|
||||
"BOT_BOTTLE_FC_DROPBEAR or cache one there.")
|
||||
if shutil.which("mke2fs") is None:
|
||||
die("mke2fs (e2fsprogs) not found — required to build guest rootfs")
|
||||
_require_network_pool()
|
||||
|
||||
|
||||
def _require_kvm() -> None:
|
||||
if not os.path.exists(_KVM_DEVICE):
|
||||
die(f"{_KVM_DEVICE} is missing. Enable KVM (load kvm-intel/kvm-amd; "
|
||||
"confirm virtualization is on in firmware).")
|
||||
if not os.access(_KVM_DEVICE, os.R_OK | os.W_OK):
|
||||
die(f"{_KVM_DEVICE} exists but is not accessible. Add your user to "
|
||||
"the `kvm` group and re-login.")
|
||||
|
||||
|
||||
def _require_network_pool() -> None:
|
||||
"""Fail-closed on the parts we can check unprivileged; defer the
|
||||
rest to the post-boot isolation probe.
|
||||
|
||||
The TAP pool is verified here (`ip link show` is unprivileged). The
|
||||
nft table can only be *confirmed present* here when `nft` is
|
||||
queryable — which usually needs root — so a missing/absent nft is
|
||||
not treated as fatal at this stage: the authoritative check is the
|
||||
empirical isolation probe run after boot, before the agent starts
|
||||
(see isolation_probe.verify_isolation). What we must never do is
|
||||
boot without the TAP pool."""
|
||||
conflicts = netpool.overlapping_routes()
|
||||
if conflicts:
|
||||
detail = "; ".join(f"{c.dst} dev {c.dev}" for c in conflicts)
|
||||
warn(f"Firecracker pool range ({netpool.ip_base()}, "
|
||||
f"{netpool.pool_size()} slots) overlaps existing routes: "
|
||||
f"{detail}. This can shadow or be shadowed by that route; "
|
||||
f"set BOT_BOTTLE_FC_IP_BASE to a free range and re-run "
|
||||
f"./cli.py backend setup --backend=firecracker.")
|
||||
missing = netpool.missing_taps()
|
||||
if missing:
|
||||
die(f"network pool incomplete — missing TAP devices: "
|
||||
f"{', '.join(missing)}.\n ./cli.py backend setup --backend=firecracker")
|
||||
if shutil.which("nft") is not None and not netpool.nft_table_present():
|
||||
# nft is queryable and says the table is absent — that's a
|
||||
# definite, catchable misconfiguration; fail early.
|
||||
warn(f"isolation table `inet {netpool.NFT_TABLE}` not found via nft. "
|
||||
"If this is a permissions issue it will be re-checked "
|
||||
"empirically after boot; otherwise run: "
|
||||
"./cli.py backend setup --backend=firecracker")
|
||||
|
||||
|
||||
# --- rootfs pipeline (rootless) -------------------------------------
|
||||
|
||||
def docker_image_id(ref: str) -> str:
|
||||
"""The image's content digest, used as the rootfs cache key."""
|
||||
result = subprocess.run(
|
||||
["docker", "image", "inspect", "--format", "{{.Id}}", ref],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if result.returncode != 0 or not result.stdout.strip():
|
||||
die(f"docker image inspect for {ref!r} failed: "
|
||||
f"{(result.stderr or '').strip() or '<no output>'}")
|
||||
return result.stdout.strip().replace("sha256:", "")[:16]
|
||||
|
||||
|
||||
def build_base_rootfs_dir(image_ref: str) -> Path:
|
||||
"""Export the agent image's filesystem and inject the guest init +
|
||||
static dropbear. Cached by image digest — the per-bottle bits
|
||||
(authorized_keys, IP) are passed at boot via the kernel cmdline, so
|
||||
this tree carries nothing bottle-specific and is safely shared.
|
||||
|
||||
Returns the prepared directory (read as the `mke2fs -d` source)."""
|
||||
digest = docker_image_id(image_ref)
|
||||
base = cache_dir() / "rootfs" / digest
|
||||
ready = base / ".bb-ready"
|
||||
if ready.is_file():
|
||||
return base
|
||||
|
||||
if base.exists():
|
||||
shutil.rmtree(base, ignore_errors=True)
|
||||
base.mkdir(parents=True)
|
||||
|
||||
info(f"exporting {image_ref} rootfs -> {base}")
|
||||
cid = subprocess.run(
|
||||
["docker", "create", image_ref, "sleep", "infinity"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if cid.returncode != 0 or not cid.stdout.strip():
|
||||
die(f"docker create {image_ref!r} failed: {cid.stderr.strip()}")
|
||||
container = cid.stdout.strip()
|
||||
try:
|
||||
export = subprocess.Popen(
|
||||
["docker", "export", container], stdout=subprocess.PIPE,
|
||||
)
|
||||
untar = subprocess.run(
|
||||
["tar", "-x", "-C", str(base)], stdin=export.stdout, check=False,
|
||||
)
|
||||
export.wait()
|
||||
if export.returncode != 0 or untar.returncode != 0:
|
||||
die(f"exporting rootfs for {image_ref!r} failed")
|
||||
finally:
|
||||
subprocess.run(
|
||||
["docker", "rm", "-f", container],
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
|
||||
)
|
||||
|
||||
_inject_guest_boot(base)
|
||||
ready.write_text("ok\n")
|
||||
return base
|
||||
|
||||
|
||||
def _inject_guest_boot(rootfs: Path) -> None:
|
||||
"""Drop the static dropbear and the PID-1 init into the rootfs."""
|
||||
shutil.copy2(dropbear_path(), rootfs / "bb-dropbear")
|
||||
os.chmod(rootfs / "bb-dropbear", 0o755)
|
||||
init = rootfs / "bb-init"
|
||||
init.write_text(_GUEST_INIT)
|
||||
os.chmod(init, 0o755)
|
||||
|
||||
|
||||
def build_rootfs_ext4(base_dir: Path, out_path: Path, *, slack_mib: int = 1024) -> None:
|
||||
"""Build a fresh, writable ext4 for one bottle from the cached base
|
||||
dir. Rootless: `mke2fs -d` populates the image from a directory
|
||||
without mounting. Each call produces an independent disk, so the
|
||||
shared base dir stays untouched and concurrent bottles don't race."""
|
||||
used_mib = _dir_size_mib(base_dir)
|
||||
size_mib = used_mib + slack_mib
|
||||
out_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
result = subprocess.run(
|
||||
["mke2fs", "-q", "-t", "ext4", "-d", str(base_dir), "-F",
|
||||
str(out_path), f"{size_mib}M"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
die(f"mke2fs for {out_path} failed: {result.stderr.strip()}")
|
||||
|
||||
|
||||
def _dir_size_mib(path: Path) -> int:
|
||||
result = subprocess.run(
|
||||
["du", "-sm", str(path)], capture_output=True, text=True, check=False,
|
||||
)
|
||||
try:
|
||||
return int(result.stdout.split()[0])
|
||||
except (ValueError, IndexError):
|
||||
return 2048
|
||||
|
||||
|
||||
# --- per-bottle SSH keys --------------------------------------------
|
||||
|
||||
def generate_keypair(dest_dir: Path) -> tuple[Path, str]:
|
||||
"""Mint a per-bottle ed25519 keypair. Returns (private_key_path,
|
||||
public_key_line). The public line is injected into the guest via
|
||||
the boot cmdline; the private key authenticates host->guest SSH."""
|
||||
dest_dir.mkdir(parents=True, exist_ok=True)
|
||||
key = dest_dir / "bottle_id_ed25519"
|
||||
if key.exists():
|
||||
key.unlink()
|
||||
(dest_dir / "bottle_id_ed25519.pub").unlink(missing_ok=True)
|
||||
subprocess.run(
|
||||
["ssh-keygen", "-t", "ed25519", "-N", "", "-q", "-f", str(key),
|
||||
"-C", "bot-bottle-firecracker"],
|
||||
check=True,
|
||||
)
|
||||
pub = (dest_dir / "bottle_id_ed25519.pub").read_text().strip()
|
||||
return key, pub
|
||||
|
||||
|
||||
def ssh_base_argv(private_key: Path, guest_ip: str) -> list[str]:
|
||||
"""Common SSH options for host->guest control. The VM is ephemeral
|
||||
and per-bottle, so host-key TOFU is meaningless — pin no known_hosts
|
||||
and skip the check rather than accumulate churn."""
|
||||
return [
|
||||
"ssh",
|
||||
"-i", str(private_key),
|
||||
"-o", "StrictHostKeyChecking=no",
|
||||
"-o", "UserKnownHostsFile=/dev/null",
|
||||
"-o", "LogLevel=ERROR",
|
||||
"-o", "ConnectTimeout=5",
|
||||
f"{GUEST_SSH_USER}@{guest_ip}",
|
||||
]
|
||||
|
||||
|
||||
# PID-1 init injected into every guest. Kept dependency-light: relies
|
||||
# only on coreutils + a POSIX shell (present in the Debian-family agent
|
||||
# images). The kernel `ip=` cmdline configures eth0 before init runs,
|
||||
# so no iproute2 is needed. The per-bottle SSH pubkey arrives base64 on
|
||||
# the cmdline; dropbear generates ephemeral host keys with -R.
|
||||
_GUEST_INIT = r"""#!/bin/sh
|
||||
# bot-bottle Firecracker guest init (PID 1).
|
||||
mount -t proc proc /proc 2>/dev/null
|
||||
mount -t sysfs sys /sys 2>/dev/null
|
||||
mount -t devtmpfs dev /dev 2>/dev/null
|
||||
mkdir -p /dev/pts && mount -t devpts devpts /dev/pts 2>/dev/null
|
||||
mount -o remount,rw / 2>/dev/null
|
||||
|
||||
# Install the per-bottle SSH pubkey from the kernel cmdline.
|
||||
KEY=$(sed -n 's/.*bb_pubkey=\([^ ]*\).*/\1/p' /proc/cmdline | base64 -d 2>/dev/null)
|
||||
if [ -n "$KEY" ]; then
|
||||
for home in /root /home/node; do
|
||||
mkdir -p "$home/.ssh"
|
||||
printf '%s\n' "$KEY" > "$home/.ssh/authorized_keys"
|
||||
chmod 700 "$home/.ssh"
|
||||
chmod 600 "$home/.ssh/authorized_keys"
|
||||
done
|
||||
chown -R node:node /home/node/.ssh 2>/dev/null || true
|
||||
fi
|
||||
|
||||
mkdir -p /etc/dropbear /run
|
||||
# -R: generate host keys on demand. -E: log to stderr (guest console).
|
||||
/bb-dropbear -R -E -p 22 2>/dev/null &
|
||||
|
||||
# Reap zombies as PID 1. dropbear is always a child, so `wait` blocks
|
||||
# rather than busy-looping.
|
||||
while : ; do wait ; done
|
||||
"""
|
||||
@@ -90,11 +90,11 @@ def get_freezer(backend_name: str) -> Freezer:
|
||||
if resolved == "macos-container":
|
||||
from .macos_container.freezer import MacosContainerFreezer
|
||||
return MacosContainerFreezer()
|
||||
if resolved == "smolmachines":
|
||||
from .smolmachines.freezer import SmolmachinesFreezer
|
||||
return SmolmachinesFreezer()
|
||||
if resolved == "firecracker":
|
||||
from .firecracker.freezer import FirecrackerFreezer
|
||||
return FirecrackerFreezer()
|
||||
die(
|
||||
f"commit is only supported for docker, macos-container, and "
|
||||
f"smolmachines; backend {backend_name!r} has no freezer"
|
||||
f"firecracker; backend {backend_name!r} has no freezer"
|
||||
)
|
||||
raise AssertionError("unreachable")
|
||||
|
||||
@@ -36,6 +36,16 @@ class MacosContainerBottleBackend(
|
||||
def is_available(cls) -> bool:
|
||||
return _container.is_available()
|
||||
|
||||
@classmethod
|
||||
def setup(cls) -> int:
|
||||
from . import setup as _setup
|
||||
return _setup.setup()
|
||||
|
||||
@classmethod
|
||||
def status(cls) -> int:
|
||||
from . import setup as _setup
|
||||
return _setup.status()
|
||||
|
||||
def _preflight(self) -> None:
|
||||
_resolve_plan.preflight()
|
||||
|
||||
|
||||
@@ -0,0 +1,69 @@
|
||||
"""Host setup + status for the macOS Apple Container backend.
|
||||
|
||||
Like Docker, this backend needs no privileged network-pool provisioning
|
||||
— it wants Apple's `container` CLI installed and its system service
|
||||
running. `setup()` points at the install/`container system start` steps;
|
||||
`status()` reports readiness. Reached via
|
||||
`MacosContainerBottleBackend.setup` / `.status`, dispatched from the
|
||||
generic `./cli.py backend {setup,status}`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
from . import util as _container
|
||||
|
||||
|
||||
def _service_running() -> bool:
|
||||
if shutil.which("container") is None:
|
||||
return False
|
||||
return subprocess.run(
|
||||
["container", "system", "status"],
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
|
||||
).returncode == 0
|
||||
|
||||
|
||||
def setup() -> int:
|
||||
if not _container.is_macos():
|
||||
sys.stderr.write("macos-container backend requires macOS.\n")
|
||||
return 1
|
||||
if shutil.which("container") is None:
|
||||
sys.stderr.write("Apple Container is required but was not found on PATH.\n")
|
||||
sys.stderr.write("Install: https://github.com/apple/container/releases\n")
|
||||
return 1
|
||||
if not _service_running():
|
||||
sys.stderr.write(
|
||||
"Apple Container is installed but its system service isn't "
|
||||
"running. Start it with: container system start\n"
|
||||
)
|
||||
return 1
|
||||
sys.stderr.write(
|
||||
"macos-container backend: ready — no privileged host setup required.\n"
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
def status() -> int:
|
||||
ok = True
|
||||
if _container.is_macos():
|
||||
sys.stderr.write("host: macOS\n")
|
||||
else:
|
||||
sys.stderr.write("host: NOT macOS (backend unsupported here)\n")
|
||||
ok = False
|
||||
if shutil.which("container") is not None:
|
||||
sys.stderr.write("container CLI on PATH: yes\n")
|
||||
else:
|
||||
sys.stderr.write("container CLI on PATH: NO\n")
|
||||
ok = False
|
||||
if ok:
|
||||
sys.stderr.write(
|
||||
f"container system service: {'running' if _service_running() else 'NOT running'}\n"
|
||||
)
|
||||
if not _service_running():
|
||||
ok = False
|
||||
if not ok:
|
||||
sys.stderr.write("\nRun: ./cli.py backend setup --backend=macos-container\n")
|
||||
return 0 if ok else 1
|
||||
@@ -1,9 +1,8 @@
|
||||
"""Shared print helpers for BottlePlan.print implementations.
|
||||
|
||||
Lifts the multi-value label printer out of DockerBottlePlan so the
|
||||
smolmachines backend (and any future backend) renders the same
|
||||
two-column scannable preflight without duplicating the indent
|
||||
math."""
|
||||
Lifts the multi-value label printer out of DockerBottlePlan so every
|
||||
backend (and any future backend) renders the same two-column
|
||||
scannable preflight without duplicating the indent math."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
"""Shared helpers used by both backends' resolve_plan steps.
|
||||
"""Shared helpers used across backends' resolve_plan steps.
|
||||
|
||||
Each helper owns one well-defined step of the per-bottle plan
|
||||
resolution so docker and smolmachines don't repeat the same logic.
|
||||
resolution so the backends don't repeat the same logic.
|
||||
Backend-specific steps (container names, env-file, per-bottle
|
||||
Dockerfile overrides, subnet allocation) stay in the backend's own
|
||||
resolve_plan.py.
|
||||
|
||||
@@ -1,15 +0,0 @@
|
||||
"""smolmachines bottle backend (PRD 0023).
|
||||
|
||||
Selectable via `BOT_BOTTLE_BACKEND=smolmachines`. Runs each
|
||||
bottle inside a per-agent microVM (libkrun / Hypervisor.framework
|
||||
on macOS) with a userspace gvproxy gateway as the egress
|
||||
primitive. The sidecar bundle (PRD 0024) runs as a host-side
|
||||
docker container reached only through gvproxy's port-forward list.
|
||||
|
||||
Chunk 1 (this commit) ships the backend skeleton + Smolfile +
|
||||
gvproxy renderers + preflight check. VM lifecycle, sidecar
|
||||
bringup, and provisioning land in later chunks."""
|
||||
|
||||
from .backend import SmolmachinesBottleBackend # noqa: F401
|
||||
|
||||
__all__ = ["SmolmachinesBottleBackend"]
|
||||
@@ -1,217 +0,0 @@
|
||||
"""SmolmachinesBottle — running-instance handle (PRD 0023 chunk 2d).
|
||||
|
||||
Routes `exec_agent` / `exec` / `cp_in` through `smolvm machine
|
||||
exec` / `smolvm machine cp`. The handle is yielded by `launch`
|
||||
and torn down via the surrounding ExitStack on context exit;
|
||||
`close` is a no-op idempotent alias so the BottleBackend ABC's
|
||||
context-manager contract is satisfied.
|
||||
|
||||
User context: `smolvm machine exec` runs commands as root in the
|
||||
VM, but the agent image's USER is `node` and agent CLIs may refuse
|
||||
to run as root in bypass modes. Both
|
||||
`exec_agent` and `exec` switch to the requested user (default
|
||||
`node`) via `runuser -u <user> --` and set `HOME` / `USER`
|
||||
through `smolvm -e` — avoiding `runuser -l`'s login-shell wiring
|
||||
(PAM session setup, /etc/profile sourcing) which can hang on a
|
||||
minimal Debian VM with no PAM session config."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
import shlex
|
||||
from typing import Mapping, cast
|
||||
|
||||
from ...agent_provider import PromptMode, prompt_args
|
||||
from .. import Bottle, ExecResult
|
||||
from ..terminal import exec_shell_script
|
||||
from . import pty_resize as _pty_resize
|
||||
from . import smolvm as _smolvm
|
||||
|
||||
|
||||
# Absolute path to the pty_resize wrapper. Invoke as
|
||||
# `python <path>` rather than `python -m <dotted-path>` so the
|
||||
# wrapper runs regardless of cwd / sys.path — it has no
|
||||
# bot_bottle.* imports, so it's self-contained.
|
||||
_PTY_RESIZE_SCRIPT = _pty_resize.__file__
|
||||
|
||||
|
||||
# Per-user env the agent image's USER (node) expects. Some providers
|
||||
# write session state under the user's home directory;
|
||||
# bare `runuser -u` inherits root's HOME=/root, which claude
|
||||
# can't write to. Set HOME / USER explicitly through smolvm -e
|
||||
# so the child process sees them.
|
||||
_HOME_FOR = {
|
||||
"node": "/home/node",
|
||||
"root": "/root",
|
||||
}
|
||||
|
||||
_DEFAULT_PATH_FOR = {
|
||||
# Committed smolmachine snapshots are rebuilt from a rootfs tarball and
|
||||
# lose Docker image ENV metadata. Restore the provider CLI path here so
|
||||
# resumed Codex bottles can still find the per-user install.
|
||||
"node": (
|
||||
"/home/node/.local/bin:"
|
||||
"/home/node/.codex/packages/standalone/current/bin:"
|
||||
"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
|
||||
),
|
||||
"root": "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
|
||||
}
|
||||
|
||||
|
||||
def _env_assignments_for(user: str, env: Mapping[str, str]) -> list[str]:
|
||||
home = _HOME_FOR.get(user, f"/home/{user}")
|
||||
out = [f"HOME={home}", f"USER={user}"]
|
||||
if "PATH" not in env:
|
||||
out.append(f"PATH={_DEFAULT_PATH_FOR.get(user, _DEFAULT_PATH_FOR['root'])}")
|
||||
for k, v in env.items():
|
||||
out.append(f"{k}={v}")
|
||||
return out
|
||||
|
||||
|
||||
class SmolmachinesBottle(Bottle):
|
||||
"""Handle returned by `SmolmachinesBottleBackend.launch`. The
|
||||
underlying VM lifecycle (create / start / stop / delete) lives
|
||||
on the launch ExitStack — this class only routes runtime
|
||||
operations to the right `smolvm machine ...` subcommand."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
machine_name: str,
|
||||
*,
|
||||
prompt_path: str | None = None,
|
||||
guest_env: Mapping[str, str] | None = None,
|
||||
agent_command: str = "claude",
|
||||
agent_prompt_mode: PromptMode = "append_file",
|
||||
agent_provider_template: str = "claude",
|
||||
terminal_title: str = "",
|
||||
terminal_color: str = "",
|
||||
agent_workdir: str = "/home/node",
|
||||
) -> None:
|
||||
self.name = machine_name
|
||||
# In-VM path to the agent's prompt file. None when the
|
||||
# agent declared no prompt (file still exists; we just
|
||||
# don't pass --append-system-prompt-file).
|
||||
self.prompt_path = prompt_path
|
||||
# Env vars the agent process needs (HTTPS_PROXY,
|
||||
# CLAUDE_CODE_OAUTH_TOKEN, manifest-declared bottle env, …).
|
||||
# Forwarded on every `smolvm machine exec` via `-e K=V`
|
||||
# because exec doesn't inherit from machine_create's env.
|
||||
self._guest_env = dict(guest_env or {})
|
||||
self._agent_prompt_mode = agent_prompt_mode
|
||||
self.agent_command = agent_command
|
||||
self.terminal_title = terminal_title
|
||||
self.terminal_color = terminal_color
|
||||
self.agent_provider_template = agent_provider_template
|
||||
self.agent_workdir = agent_workdir
|
||||
|
||||
def agent_argv(
|
||||
self, argv: list[str], *, tty: bool = True,
|
||||
) -> list[str]:
|
||||
flags = ["smolvm", "machine", "exec", "--name", self.name]
|
||||
if tty:
|
||||
flags += ["-i", "-t"]
|
||||
agent_tail = ["env", *_env_assignments_for("node", self._guest_env)]
|
||||
if self.agent_workdir and self.agent_workdir != _HOME_FOR["node"]:
|
||||
agent_tail += [
|
||||
"sh", "-lc",
|
||||
f"cd {shlex.quote(self.agent_workdir)} && exec \"$@\"",
|
||||
"bot-bottle-agent",
|
||||
]
|
||||
agent_tail.append(self.agent_command)
|
||||
provider_prompt_args = prompt_args(
|
||||
cast(PromptMode, self._agent_prompt_mode), self.prompt_path, argv=argv,
|
||||
)
|
||||
if cast(PromptMode, self._agent_prompt_mode) == "read_prompt_file":
|
||||
agent_tail += argv
|
||||
agent_tail += provider_prompt_args
|
||||
else:
|
||||
agent_tail += provider_prompt_args
|
||||
agent_tail += argv
|
||||
flags += ["--", "runuser", "-u", "node", "--", *agent_tail]
|
||||
if not tty:
|
||||
# No PTY allocated — no SIGWINCH to forward, no resize
|
||||
# bridge needed. Skip the wrapper so non-interactive
|
||||
# exec paths (e.g., provisioning shell-outs that
|
||||
# happen to go through this method) stay light.
|
||||
return flags
|
||||
return [
|
||||
sys.executable, _PTY_RESIZE_SCRIPT,
|
||||
self.name, "--", *flags,
|
||||
]
|
||||
|
||||
def exec_agent(self, argv: list[str], *, tty: bool = True) -> int:
|
||||
"""Run the selected agent interactively inside the VM as the `node`
|
||||
user. Inherits the operator's terminal (stdin / stdout /
|
||||
stderr) so the session feels native. Blocks until the agent
|
||||
exits; returns the in-VM exit code.
|
||||
|
||||
We bypass the captured-output `machine_exec` helper here
|
||||
because that one wraps stdout/stderr in pipes — fine for
|
||||
scripted exec, wrong for an interactive shell. Drop down
|
||||
to `subprocess.run` with the TTY inherited.
|
||||
|
||||
UID switches via `runuser -u node --` (not `-l`) so we
|
||||
avoid login-shell wiring. HOME / USER come from `smolvm
|
||||
-e` instead, which sets them on the process env."""
|
||||
agent_argv = self.agent_argv(argv, tty=tty)
|
||||
script = exec_shell_script(agent_argv, self.terminal_title, self.terminal_color) if tty else None
|
||||
if script is None:
|
||||
return subprocess.run(agent_argv, check=False).returncode
|
||||
# Use sh -c (not -lc) so the script inherits PATH from the calling
|
||||
# process. sh -l sources login-shell init files (e.g. /etc/profile)
|
||||
# which may NOT include smolvm's location when it was installed via
|
||||
# homebrew. The calling process (./cli.py) already has smolvm on PATH
|
||||
# (provision steps succeed), so -c is sufficient.
|
||||
return subprocess.run(["sh", "-c", script], check=False).returncode
|
||||
|
||||
# smolvm/libkrun can SIGKILL an otherwise-normal exec during
|
||||
# early-VM provisioning. Retry once after a short settle so
|
||||
# callers (provision_ca, etc.) don't have to handle it themselves.
|
||||
_SIGKILL_EXIT = 128 + 9
|
||||
|
||||
def exec(self, script: str, *, user: str = "node") -> ExecResult:
|
||||
"""Run a POSIX shell script as `user` (default `node`) and
|
||||
capture the result. Matches the docker backend's `exec`,
|
||||
which defaults to the image's USER (also node) — so test
|
||||
helpers / provision shell-outs run with the same identity
|
||||
on both backends. Pass `user="root"` for tests that need
|
||||
root.
|
||||
|
||||
`runuser -u <user> -- env ... /bin/sh -c <script>` switches UID
|
||||
without invoking a login shell, then sets HOME / USER and the
|
||||
bottle env in the child process.
|
||||
|
||||
Retries once on SIGKILL (exit 137) — libkrun occasionally
|
||||
kills short-lived execs during VM bring-up."""
|
||||
r = self._exec_raw(script, user=user)
|
||||
if r.returncode == self._SIGKILL_EXIT:
|
||||
time.sleep(1.0)
|
||||
r = self._exec_raw(script, user=user)
|
||||
return r
|
||||
|
||||
def _exec_raw(self, script: str, *, user: str = "node") -> ExecResult:
|
||||
argv = [
|
||||
"--", "runuser", "-u", user, "--",
|
||||
"env", *_env_assignments_for(user, self._guest_env),
|
||||
"/bin/sh", "-c", script,
|
||||
]
|
||||
r = subprocess.run(
|
||||
["smolvm", "machine", "exec", "--name", self.name] + argv,
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
return ExecResult(
|
||||
returncode=r.returncode,
|
||||
stdout=r.stdout or "",
|
||||
stderr=r.stderr or "",
|
||||
)
|
||||
|
||||
def cp_in(self, host_path: str, container_path: str) -> None:
|
||||
"""Copy a host path into the guest at `container_path`."""
|
||||
_smolvm.machine_cp(host_path, f"{self.name}:{container_path}")
|
||||
|
||||
def close(self) -> None:
|
||||
# Real teardown lives on the launch ExitStack; this is just
|
||||
# the idempotent alias the BottleBackend ABC expects.
|
||||
pass
|
||||
@@ -1,55 +0,0 @@
|
||||
"""SmolmachinesBottleCleanupPlan — concrete BottleCleanupPlan (issue #77).
|
||||
|
||||
Tracks the resources `SmolmachinesBottleBackend.cleanup` will
|
||||
remove:
|
||||
|
||||
- machines: smolvm machines whose name starts with
|
||||
`bot-bottle-` (running or stopped). Stopped +
|
||||
deleted via `smolvm machine stop` + `machine delete -f`.
|
||||
- bundles: docker containers `bot-bottle-sidecars-<slug>`
|
||||
left over from a smolmachines bottle (the bundle's
|
||||
port-forwards stay published on lo0 aliases until
|
||||
the container is gone). Removed via `docker rm -f`.
|
||||
- networks: docker networks `bot-bottle-bundle-<slug>`
|
||||
attached to the bundles. Removed via
|
||||
`docker network rm`.
|
||||
|
||||
Smolmachines state dirs live under the same `~/.bot-bottle/state/`
|
||||
path the docker backend uses; the docker backend's
|
||||
`prepare_cleanup` already enumerates orphan state dirs and is the
|
||||
single source of truth for that bucket (consults
|
||||
`enumerate_active_bottles()` so it doesn't reap a live
|
||||
smolmachines bottle's dir)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from dataclasses import dataclass
|
||||
|
||||
from ...log import info
|
||||
from .. import BottleCleanupPlan
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SmolmachinesBottleCleanupPlan(BottleCleanupPlan):
|
||||
"""Resources SmolmachinesBottleBackend.cleanup will remove.
|
||||
Produced by `prepare_cleanup`; sorted so the y/N output is
|
||||
stable."""
|
||||
|
||||
machines: tuple[str, ...] = ()
|
||||
bundles: tuple[str, ...] = ()
|
||||
networks: tuple[str, ...] = ()
|
||||
|
||||
@property
|
||||
def empty(self) -> bool:
|
||||
return not self.machines and not self.bundles and not self.networks
|
||||
|
||||
def print(self) -> None:
|
||||
print(file=sys.stderr)
|
||||
for name in self.machines:
|
||||
info(f"smolvm machine: {name}")
|
||||
for name in self.bundles:
|
||||
info(f"bundle container:{name}")
|
||||
for name in self.networks:
|
||||
info(f"bundle network: {name}")
|
||||
print(file=sys.stderr)
|
||||
@@ -1,109 +0,0 @@
|
||||
"""SmolmachinesBottlePlan — concrete BottlePlan for the smolmachines
|
||||
backend (PRD 0023).
|
||||
|
||||
Slug + bundle docker subnet / gateway / pinned IP + smolvm
|
||||
machine name + agent `.smolmachine` artifact + per-bottle guest
|
||||
env. Provisioning fields (CA cert path, prompt path, etc.) land
|
||||
in chunk 4."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
from ...agent_provider import PromptMode
|
||||
from .. import BottlePlan
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SmolmachinesBottlePlan(BottlePlan):
|
||||
"""Resolved fields the launch step needs to bring up the bottle.
|
||||
|
||||
Inherits `spec`, `stage_dir`, `git_gate_plan`, `egress_plan`,
|
||||
`supervise_plan`, and `agent_provision` from BottlePlan."""
|
||||
|
||||
slug: str
|
||||
# Per-bottle docker subnet for the sidecar bundle container.
|
||||
# The bundle runs at `bundle_ip` (always `.2`); the gateway is
|
||||
# at `.1`. smolvm's TSI allowlist is set to `bundle_ip/32`.
|
||||
bundle_subnet: str
|
||||
bundle_gateway: str
|
||||
bundle_ip: str
|
||||
# In-guest env vars (HTTPS_PROXY etc) — IP-literal URLs since
|
||||
# the guest has no DNS resolver inside the TSI allowlist.
|
||||
# Passed to `smolvm machine create` as `-e K=V` flags.
|
||||
# Smolfile-rendering is gone (smolvm 0.8.0's
|
||||
# `--smolfile` is mutually exclusive with `--from`, and
|
||||
# `--from` is the path that avoids the registry-pull race).
|
||||
guest_env: dict[str, str]
|
||||
# Inner Plans for the sidecar bundle daemons. The same shape the
|
||||
# docker backend uses — same `.prepare()` calls produced
|
||||
# them — but our launch step doesn't populate the
|
||||
# docker-specific network fields (internal_network,
|
||||
# egress_network) because the smolmachines bundle isn't on
|
||||
# docker's `--internal` + egress bridge topology; it's on a
|
||||
# per-bottle bridge with a pinned IP. The unused fields stay
|
||||
# at their dataclass defaults.
|
||||
# Agent-side endpoints. On Docker Desktop the docker bridge
|
||||
# IPs aren't reachable from the smolvm guest (TSI uses macOS
|
||||
# networking; docker container IPs live in the daemon's VM),
|
||||
# so the agent dials the bundle via host loopback +
|
||||
# docker-published random ports. Empty at prepare time;
|
||||
# launch populates these after bundle bringup via
|
||||
# `dataclasses.replace`. Format: a `host:port` for git-gate
|
||||
# (insteadOf URL prefix) + full URLs for proxy / supervise.
|
||||
agent_proxy_url: str = ""
|
||||
agent_git_gate_host: str = ""
|
||||
agent_supervise_url: str = ""
|
||||
|
||||
@property
|
||||
def machine_name(self) -> str:
|
||||
"""smolvm machine name. `machine_create` boots from a packed
|
||||
`.smolmachine` artifact (pre-baked at prepare time via
|
||||
`smolvm pack create`); using `--from` instead of `--image`
|
||||
avoids the registry-pull race we hit when machine_start tried
|
||||
to fetch on-demand and the libkrun agent's network attempt
|
||||
got refused by macOS."""
|
||||
return self.agent_provision.instance_name
|
||||
|
||||
@property
|
||||
def agent_image(self) -> str:
|
||||
"""Agent image ref (docker tag). `launch` runs the
|
||||
build → save → registry push → smolvm pack pipeline against
|
||||
this and feeds the resulting `.smolmachine` artifact to
|
||||
`machine_create --from`. The pipeline runs at launch time
|
||||
(not prepare time) so the docker build output doesn't garble
|
||||
the dashboard's preflight modal."""
|
||||
return self.agent_provision.image
|
||||
|
||||
@property
|
||||
def prompt_file(self) -> Path:
|
||||
"""Path to the agent's prompt file on the host. Always written
|
||||
(mode 0o600) so the in-VM path always exists; the file is
|
||||
empty when the agent has no prompt — claude-code reads it
|
||||
via --append-system-prompt-file only when non-empty."""
|
||||
return self.agent_provision.prompt_file
|
||||
|
||||
@property
|
||||
def git_gate_insteadof_host(self) -> str:
|
||||
return self.agent_git_gate_host
|
||||
|
||||
@property
|
||||
def git_gate_insteadof_scheme(self) -> str:
|
||||
return "http"
|
||||
|
||||
@property
|
||||
def agent_command(self) -> str:
|
||||
return self.agent_provision.command
|
||||
|
||||
@property
|
||||
def agent_prompt_mode(self) -> PromptMode:
|
||||
return self.agent_provision.prompt_mode
|
||||
|
||||
@property
|
||||
def agent_provider_template(self) -> str:
|
||||
return self.agent_provision.template
|
||||
|
||||
@property
|
||||
def agent_dockerfile_path(self) -> str:
|
||||
return self.agent_provision.dockerfile
|
||||
@@ -1,159 +0,0 @@
|
||||
"""Cleanup + active-listing for the smolmachines backend (issue #77).
|
||||
|
||||
`prepare_cleanup` enumerates leftover smolmachines resources:
|
||||
|
||||
- smolvm machines (`smolvm machine ls --json`) whose name starts
|
||||
with `bot-bottle-`.
|
||||
- bundle docker containers (`bot-bottle-sidecars-<slug>`).
|
||||
- bundle docker networks (`bot-bottle-bundle-<slug>`).
|
||||
|
||||
State dirs live under `~/.bot-bottle/state/<identity>/` —
|
||||
shared layout with the docker backend, which has the single
|
||||
orphan-state-dir enumerator (it already consults
|
||||
`enumerate_active_agents()` so a live smolmachines bottle's dir
|
||||
is preserved).
|
||||
|
||||
`cleanup` removes everything in the plan: stop + delete each VM,
|
||||
force-rm each container, rm each network. Each step is
|
||||
best-effort — a failure on one resource doesn't block the others."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
|
||||
from ...log import info, warn
|
||||
from . import sidecar_bundle as _bundle
|
||||
from . import smolvm as _smolvm
|
||||
from .bottle_cleanup_plan import SmolmachinesBottleCleanupPlan
|
||||
|
||||
|
||||
# Both names start with the same prefix the launcher uses.
|
||||
_VM_PREFIX = "bot-bottle-"
|
||||
_BUNDLE_PREFIX = _bundle.bundle_container_name("") # `bot-bottle-sidecars-`
|
||||
_NETWORK_PREFIX = _bundle.bundle_network_name("") # `bot-bottle-bundle-`
|
||||
|
||||
|
||||
def prepare_cleanup() -> SmolmachinesBottleCleanupPlan:
|
||||
"""Enumerate every smolmachines-owned resource on the host.
|
||||
No side effects. Returns an empty plan when smolvm isn't on
|
||||
PATH (no machines to reap) — `cleanup` is a no-op in that
|
||||
case too."""
|
||||
machines = _list_bot_bottle_machines()
|
||||
bundles = _list_bundle_containers()
|
||||
networks = _list_bundle_networks()
|
||||
return SmolmachinesBottleCleanupPlan(
|
||||
machines=tuple(sorted(machines)),
|
||||
bundles=tuple(sorted(bundles)),
|
||||
networks=tuple(sorted(networks)),
|
||||
)
|
||||
|
||||
|
||||
def cleanup(plan: SmolmachinesBottleCleanupPlan) -> None:
|
||||
"""Remove everything in the plan. Order matters: stop VMs
|
||||
first (they hold ports on lo0 aliases via libkrun), then the
|
||||
bundle containers (which hold the host port-forwards), then
|
||||
the networks (which docker won't reap until the containers
|
||||
are gone)."""
|
||||
for name in plan.machines:
|
||||
info(f"stopping smolvm machine {name}")
|
||||
subprocess.run(
|
||||
["smolvm", "machine", "stop", "--name", name],
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
)
|
||||
info(f"deleting smolvm machine {name}")
|
||||
r = subprocess.run(
|
||||
["smolvm", "machine", "delete", "-f", name],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if r.returncode != 0:
|
||||
warn(
|
||||
f"smolvm machine delete -f {name} failed: "
|
||||
f"{(r.stderr or '').strip()}"
|
||||
)
|
||||
|
||||
for name in plan.bundles:
|
||||
info(f"removing bundle container {name}")
|
||||
subprocess.run(
|
||||
["docker", "rm", "-f", name],
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
)
|
||||
|
||||
for name in plan.networks:
|
||||
info(f"removing bundle network {name}")
|
||||
r = subprocess.run(
|
||||
["docker", "network", "rm", name],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if r.returncode != 0 and "no such network" not in (r.stderr or "").lower():
|
||||
warn(
|
||||
f"docker network rm {name} failed: "
|
||||
f"{(r.stderr or '').strip()}"
|
||||
)
|
||||
|
||||
|
||||
def _list_bot_bottle_machines() -> list[str]:
|
||||
"""All smolvm machines named `bot-bottle-*`, regardless of
|
||||
state (running / stopped / created). Empty when smolvm isn't
|
||||
installed."""
|
||||
if not _smolvm.is_available():
|
||||
return []
|
||||
r = subprocess.run(
|
||||
["smolvm", "machine", "ls", "--json"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if r.returncode != 0:
|
||||
return []
|
||||
try:
|
||||
machines = json.loads(r.stdout or "[]")
|
||||
except json.JSONDecodeError:
|
||||
return []
|
||||
return [
|
||||
m["name"] for m in machines
|
||||
if isinstance(m, dict)
|
||||
and m.get("name", "").startswith(_VM_PREFIX)
|
||||
]
|
||||
|
||||
|
||||
def _list_bundle_containers() -> list[str]:
|
||||
"""All docker containers named `bot-bottle-sidecars-*`,
|
||||
running or stopped. Empty when docker isn't installed."""
|
||||
# Late import: `backend/__init__` imports this module
|
||||
# transitively via the smolmachines backend.
|
||||
from .. import has_backend
|
||||
if not has_backend("docker"):
|
||||
return []
|
||||
r = subprocess.run(
|
||||
["docker", "ps", "-a",
|
||||
"--filter", f"name=^{_BUNDLE_PREFIX}",
|
||||
"--format", "{{.Names}}"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if r.returncode != 0:
|
||||
return []
|
||||
return [
|
||||
line for line in (r.stdout or "").splitlines()
|
||||
if line and line.startswith(_BUNDLE_PREFIX)
|
||||
]
|
||||
|
||||
|
||||
def _list_bundle_networks() -> list[str]:
|
||||
"""All docker networks named `bot-bottle-bundle-*`. Empty
|
||||
when docker isn't installed."""
|
||||
from .. import has_backend
|
||||
if not has_backend("docker"):
|
||||
return []
|
||||
r = subprocess.run(
|
||||
["docker", "network", "ls",
|
||||
"--filter", f"name={_NETWORK_PREFIX}",
|
||||
"--format", "{{.Name}}"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if r.returncode != 0:
|
||||
return []
|
||||
return [
|
||||
line for line in (r.stdout or "").splitlines()
|
||||
if line and line.startswith(_NETWORK_PREFIX)
|
||||
]
|
||||
@@ -1,21 +0,0 @@
|
||||
"""Egress apply for the smolmachines backend.
|
||||
|
||||
The smolmachines sidecar bundle runs as a host-side Docker container,
|
||||
so egress signalling is identical to the docker backend.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from ..docker.egress_apply import ( # noqa: F401
|
||||
DockerEgressApplicator,
|
||||
EgressApplyError,
|
||||
applicator,
|
||||
fetch_current_routes,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"DockerEgressApplicator",
|
||||
"EgressApplyError",
|
||||
"applicator",
|
||||
"fetch_current_routes",
|
||||
]
|
||||
@@ -1,123 +0,0 @@
|
||||
"""Active-agent enumeration for the smolmachines backend (PRD
|
||||
0023 chunk 4 follow-up + issue #77).
|
||||
|
||||
Returns a list of `ActiveAgent` records — same shape the docker
|
||||
backend produces — so CLI `list active` and the dashboard agents
|
||||
pane render both backends through one code path.
|
||||
|
||||
A smolmachines agent is "active" when its smolvm guest is
|
||||
running. We cross-reference against the per-bottle sidecar
|
||||
bundle container to populate the `services` field (which daemons
|
||||
are up in the bundle); without a bundle we still surface the VM
|
||||
so the operator can see + clean it up.
|
||||
|
||||
The cross-backend caller gates on `has_backend("smolmachines")`
|
||||
and `has_backend("docker")`, so this module assumes both are
|
||||
available when called. Both subprocess calls below still
|
||||
tolerate "command not on PATH" defensively, but the gate is the
|
||||
intended access pattern."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
|
||||
from .. import ActiveAgent
|
||||
from ...bottle_state import read_metadata
|
||||
from . import sidecar_bundle as _bundle
|
||||
|
||||
|
||||
# Smolvm VM names produced by prepare are `bot-bottle-<slug>`,
|
||||
# matching the bundle container name pattern. We use the prefix
|
||||
# both as a filter and to strip back to the slug.
|
||||
_VM_NAME_PREFIX = "bot-bottle-"
|
||||
|
||||
|
||||
def enumerate_active() -> list[ActiveAgent]:
|
||||
"""All currently-running smolmachines-backed agents. Empty
|
||||
list when no matching VMs are running. Caller is responsible
|
||||
for gating on `has_backend('smolmachines')` if needed; if
|
||||
smolvm is missing the `smolvm machine ls` call below returns
|
||||
nothing silently."""
|
||||
result = subprocess.run(
|
||||
["smolvm", "machine", "ls", "--json"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return []
|
||||
try:
|
||||
machines = json.loads(result.stdout or "[]")
|
||||
except json.JSONDecodeError:
|
||||
return []
|
||||
services_by_slug = _query_bundle_services()
|
||||
out: list[ActiveAgent] = []
|
||||
for m in machines:
|
||||
name = m.get("name") or ""
|
||||
state = m.get("state") or ""
|
||||
if state != "running" or not name.startswith(_VM_NAME_PREFIX):
|
||||
continue
|
||||
slug = name[len(_VM_NAME_PREFIX):]
|
||||
metadata = read_metadata(slug)
|
||||
out.append(ActiveAgent(
|
||||
backend_name="smolmachines",
|
||||
slug=slug,
|
||||
agent_name=metadata.agent_name if metadata else "?",
|
||||
started_at=metadata.started_at if metadata else "",
|
||||
services=services_by_slug.get(slug, ()),
|
||||
label=metadata.label if metadata else "",
|
||||
color=metadata.color if metadata else "",
|
||||
))
|
||||
return out
|
||||
|
||||
|
||||
def _query_bundle_services() -> dict[str, tuple[str, ...]]:
|
||||
"""`{slug: ('egress', ...)}` from each running bundle container's
|
||||
`BOT_BOTTLE_SIDECAR_DAEMONS` env var.
|
||||
Smolmachines bundles all run the PRD-0024 image with the
|
||||
same daemon set declared via env, so one inspect per bundle
|
||||
gets us the picture without exec'ing into the container.
|
||||
|
||||
Returns an empty mapping when the docker backend isn't
|
||||
available — the bundle services field on each ActiveAgent
|
||||
just shows up empty, matching the docker backend's "starting"
|
||||
state."""
|
||||
# Late import: `has_backend` lives on the backend package's
|
||||
# __init__, which imports this module transitively. Pulling
|
||||
# the name in at call time sidesteps the cycle.
|
||||
from .. import has_backend
|
||||
if not has_backend("docker"):
|
||||
return {}
|
||||
ps = subprocess.run(
|
||||
["docker", "ps",
|
||||
"--filter", "name=" + _bundle.bundle_container_name(""),
|
||||
"--format", "{{.Names}}"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if ps.returncode != 0:
|
||||
return {}
|
||||
out: dict[str, tuple[str, ...]] = {}
|
||||
for line in (ps.stdout or "").splitlines():
|
||||
name = line.strip()
|
||||
if not name:
|
||||
continue
|
||||
slug = name.removeprefix(_bundle.bundle_container_name(""))
|
||||
if not slug:
|
||||
continue
|
||||
inspect = subprocess.run(
|
||||
["docker", "inspect", name, "--format", "{{json .Config.Env}}"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if inspect.returncode != 0:
|
||||
continue
|
||||
try:
|
||||
env_list = json.loads(inspect.stdout or "[]")
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
for entry in env_list:
|
||||
key, _, value = entry.partition("=")
|
||||
if key == "BOT_BOTTLE_SIDECAR_DAEMONS":
|
||||
out[slug] = tuple(sorted(
|
||||
d for d in value.split(",") if d
|
||||
))
|
||||
break
|
||||
return out
|
||||
@@ -1,145 +0,0 @@
|
||||
"""SmolmachinesFreezer — snapshot a smolmachines bottle.
|
||||
|
||||
`smolvm pack create --from-vm` requires the VM to be stopped, and smolvm
|
||||
removes VMs when stopped (same issue as Apple Container). Instead, exec
|
||||
into the running VM as root to write a gzip-compressed tar of the root
|
||||
filesystem to /var/tmp, then copy it to the host with `smolvm machine cp`,
|
||||
build a Docker image from the archive, convert it to a smolmachine artifact
|
||||
via the existing registry pipeline, and record the sidecar path. The VM
|
||||
stays running throughout."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
from .. import ActiveAgent
|
||||
from ..freeze import Freezer
|
||||
from ..docker import util as docker_mod
|
||||
from .local_registry import crane_push_tarball, ephemeral_registry
|
||||
from .smolvm import machine_cp, machine_exec, pack_create
|
||||
from ...bottle_state import bottle_state_dir
|
||||
from ...log import die, info
|
||||
|
||||
|
||||
# Temp file written inside the VM during commit. Lives in /var/tmp
|
||||
# (on-disk, unlike tmpfs /tmp) to survive for machine_cp.
|
||||
_VM_COMMIT_TAR = "/var/tmp/.bot-bottle-commit.tar.gz"
|
||||
|
||||
|
||||
class SmolmachinesFreezer(Freezer):
|
||||
"""Freezes a smolmachines bottle via exec-tar + Docker image + smolmachine pack.
|
||||
|
||||
The VM is NOT stopped. We exec into the running VM to write a compressed
|
||||
tar of the root filesystem to /var/tmp, copy it to the host with
|
||||
machine_cp, build a Docker image (Docker's ADD decompresses .tar.gz
|
||||
automatically), then run the same image→registry→pack_create pipeline
|
||||
that _ensure_smolmachine uses for fresh builds."""
|
||||
|
||||
backend_name = "smolmachines"
|
||||
|
||||
def _freeze(self, agent: ActiveAgent) -> str:
|
||||
machine = f"bot-bottle-{agent.slug}"
|
||||
image_ref = f"bot-bottle-committed-{agent.slug}:latest"
|
||||
output_dir = bottle_state_dir(agent.slug)
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
binary = output_dir / "committed-smolmachine"
|
||||
sidecar = output_dir / "committed-smolmachine.smolmachine"
|
||||
_snapshot_running_vm(machine, image_ref, binary)
|
||||
return str(sidecar)
|
||||
|
||||
def _export_hint(self, slug: str, image_ref: str) -> None:
|
||||
info(f"to export for migration: cp {image_ref} {slug}.smolmachine")
|
||||
|
||||
|
||||
def _snapshot_running_vm(machine: str, image_ref: str, binary: Path) -> None:
|
||||
"""Exec-tar the running VM, build a Docker image, and pack to a smolmachine.
|
||||
|
||||
binary: destination for the launcher (sibling .smolmachine is the artifact
|
||||
that machine_create --from consumes, same convention as pack_create).
|
||||
"""
|
||||
with tempfile.TemporaryDirectory(prefix="bot-bottle-vm-commit.") as tmp:
|
||||
tmp_path = Path(tmp)
|
||||
# Use .tar.gz — Docker ADD decompresses automatically and the
|
||||
# compressed archive fits in the VM's /var/tmp more easily.
|
||||
rootfs_tar_gz = tmp_path / "rootfs.tar.gz"
|
||||
dockerfile = tmp_path / "Dockerfile"
|
||||
|
||||
_exec_tar_to_file(machine, rootfs_tar_gz)
|
||||
|
||||
dockerfile.write_text(
|
||||
"FROM scratch\n"
|
||||
"ADD rootfs.tar.gz /\n"
|
||||
"USER node\n"
|
||||
"WORKDIR /home/node\n"
|
||||
)
|
||||
docker_mod.build_image(image_ref, str(tmp_path), dockerfile=str(dockerfile))
|
||||
|
||||
image_tarball = binary.parent / "committed.image.tar"
|
||||
docker_mod.save(image_ref, str(image_tarball))
|
||||
try:
|
||||
with ephemeral_registry() as handle:
|
||||
digest = docker_mod.image_id(image_ref).split(":", 1)[-1][:16]
|
||||
push_ref = f"{handle.push_endpoint}/bot-bottle-committed:{digest}"
|
||||
pack_ref = f"{handle.pull_endpoint}/bot-bottle-committed:{digest}"
|
||||
crane_push_tarball(handle, str(image_tarball), push_ref)
|
||||
pack_create(pack_ref, binary)
|
||||
finally:
|
||||
image_tarball.unlink(missing_ok=True)
|
||||
|
||||
|
||||
def _exec_tar_to_file(machine: str, dest: Path) -> None:
|
||||
"""Snapshot the running VM's root filesystem to dest (.tar.gz).
|
||||
|
||||
Writes a gzip-compressed tar to _VM_COMMIT_TAR inside the VM via
|
||||
machine_exec (same mechanism as provisioning), then copies it to the
|
||||
host with machine_cp. This avoids binary-stdout piping through the
|
||||
smolvm exec channel, which does not reliably handle large binary output.
|
||||
|
||||
A connectivity probe (machine_exec true) runs first so a concurrent-exec
|
||||
limitation (smolvm may reject a second exec while -i -t is active) is
|
||||
reported clearly rather than as a silent failure."""
|
||||
# Connectivity probe — if smolvm rejects concurrent exec while an
|
||||
# interactive session is running, fail clearly here.
|
||||
probe = machine_exec(machine, ["true"])
|
||||
if probe.returncode != 0:
|
||||
die(
|
||||
f"smolvm exec is not available for {machine!r} "
|
||||
f"(exit {probe.returncode}: {probe.stderr.strip() or probe.stdout.strip() or '<no output>'}). "
|
||||
f"If an interactive session is active, smolvm may not support concurrent exec."
|
||||
)
|
||||
|
||||
# Create the compressed tar inside the VM.
|
||||
# tar exits 1 when files change during archiving (normal for a live
|
||||
# filesystem); only treat exit > 1 as fatal.
|
||||
tar_result = machine_exec(
|
||||
machine,
|
||||
[
|
||||
"tar", "--create", "--gzip",
|
||||
"--exclude=./proc",
|
||||
"--exclude=./sys",
|
||||
"--exclude=./dev",
|
||||
"--exclude=./run",
|
||||
# /tmp and /var/tmp are ephemeral. Their stale contents
|
||||
# (e.g. /tmp/claude-<uid>) have uid remapped by smolvm's
|
||||
# pack process, causing Claude Code to refuse to use them
|
||||
# on resume. Exclude both; _init_vm recreates them with
|
||||
# mkdir -p + correct ownership on every boot.
|
||||
"--exclude=./tmp",
|
||||
"--exclude=./var/tmp",
|
||||
f"--file={_VM_COMMIT_TAR}",
|
||||
"--directory=/",
|
||||
".",
|
||||
],
|
||||
)
|
||||
if tar_result.returncode > 1:
|
||||
die(
|
||||
f"smolvm exec tar {machine!r} failed (exit {tar_result.returncode}): "
|
||||
f"{tar_result.stderr.strip() or tar_result.stdout.strip() or '<no output>'}"
|
||||
)
|
||||
|
||||
# Copy from VM to host, then clean up.
|
||||
try:
|
||||
machine_cp(f"{machine}:{_VM_COMMIT_TAR}", str(dest))
|
||||
finally:
|
||||
machine_exec(machine, ["rm", "-f", _VM_COMMIT_TAR])
|
||||
@@ -1,521 +0,0 @@
|
||||
"""End-to-end launch flow for the smolmachines backend
|
||||
(PRD 0023 chunks 2d + 4b).
|
||||
|
||||
Brings up the per-bottle docker bridge + sidecar bundle (with
|
||||
real daemons + their config files), creates + starts the smolvm
|
||||
guest pointed at the bundle's pinned IP via TSI's
|
||||
`--allow-cidr <bundle-ip>/32` allowlist, yields a
|
||||
`SmolmachinesBottle` handle, tears everything down on context
|
||||
exit.
|
||||
|
||||
The bundle's daemons consume the inner Plans the docker backend
|
||||
already produces: egress reads routes + CAs from the EgressPlan.
|
||||
Git-gate + supervise plumb through the same plans the docker
|
||||
backend uses, minus the docker-network fields that don't apply here."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import dataclasses
|
||||
import os
|
||||
import platform
|
||||
from contextlib import ExitStack, contextmanager
|
||||
from pathlib import Path
|
||||
from typing import Callable, Generator
|
||||
|
||||
from ...egress import (
|
||||
EGRESS_ROUTES_IN_CONTAINER,
|
||||
egress_agent_env_entries,
|
||||
egress_resolve_token_values,
|
||||
egress_sidecar_env_entries,
|
||||
)
|
||||
from ...supervise import DB_PATH_IN_CONTAINER, SUPERVISE_PORT
|
||||
from ...util import expand_tilde
|
||||
from ..docker import util as docker_mod
|
||||
from ..docker.egress import (
|
||||
EGRESS_CA_IN_CONTAINER,
|
||||
EGRESS_PORT as _EGRESS_PORT,
|
||||
egress_tls_init,
|
||||
)
|
||||
from ..docker.git_gate import (
|
||||
GIT_GATE_ACCESS_HOOK_IN_CONTAINER,
|
||||
GIT_GATE_CREDS_DIR_IN_CONTAINER,
|
||||
GIT_GATE_ENTRYPOINT_IN_CONTAINER,
|
||||
GIT_GATE_HOOK_IN_CONTAINER,
|
||||
)
|
||||
from ...git_gate import (
|
||||
provision_git_gate_dynamic_keys,
|
||||
revoke_git_gate_provisioned_keys,
|
||||
)
|
||||
from ...log import info, warn
|
||||
from ...bottle_state import (
|
||||
egress_state_dir,
|
||||
git_gate_state_dir,
|
||||
read_committed_image,
|
||||
)
|
||||
from . import loopback_alias as _loopback
|
||||
from . import sidecar_bundle as _bundle
|
||||
from . import smolvm as _smolvm
|
||||
from .bottle import SmolmachinesBottle
|
||||
from .bottle_plan import SmolmachinesBottlePlan
|
||||
from .local_registry import crane_push_tarball, ephemeral_registry
|
||||
|
||||
|
||||
# Repo root, used as the `docker build` context for the agent image.
|
||||
_REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent)
|
||||
|
||||
|
||||
# Per-host cache for `smolvm pack create` outputs. Keyed by the
|
||||
# docker image ID so a Dockerfile change automatically invalidates
|
||||
# the cache. `pack create` is idempotent on the smolvm side but
|
||||
# takes several seconds even on a no-op rebuild.
|
||||
_SMOLMACHINE_CACHE_DIR = Path.home() / ".cache" / "bot-bottle" / "smolmachines"
|
||||
|
||||
|
||||
# Container-internal listening ports for each bundle daemon. The
|
||||
# bundle publishes each one on a random host loopback port (see
|
||||
# `_bundle.start_bundle`), and `_bundle.bundle_host_port` looks
|
||||
# them up post-start.
|
||||
_GIT_HTTP_PORT = 9420
|
||||
_SUPERVISE_PORT = SUPERVISE_PORT
|
||||
|
||||
|
||||
@contextmanager
|
||||
def launch(
|
||||
plan: SmolmachinesBottlePlan,
|
||||
*,
|
||||
provision: Callable[[SmolmachinesBottlePlan, "SmolmachinesBottle"], str | None],
|
||||
) -> Generator[SmolmachinesBottle, None, None]:
|
||||
"""Build + run the bottle and yield a handle; tear everything
|
||||
down on exit. Errors during bringup unwind any partial state
|
||||
via the ExitStack."""
|
||||
stack = ExitStack()
|
||||
try:
|
||||
loopback_ip, network = _allocate_resources(plan, stack)
|
||||
plan = _mint_certs(plan)
|
||||
proxy_host = _proxy_host(plan, loopback_ip)
|
||||
plan = _start_bundle(plan, network, proxy_host, stack)
|
||||
plan = _discover_urls(plan, proxy_host)
|
||||
|
||||
agent_from_path = _agent_from_path(plan)
|
||||
|
||||
_launch_vm(plan, agent_from_path, proxy_host, stack)
|
||||
_init_vm(plan)
|
||||
|
||||
bottle = SmolmachinesBottle(
|
||||
plan.machine_name,
|
||||
prompt_path=None,
|
||||
guest_env=plan.guest_env,
|
||||
agent_command=plan.agent_command,
|
||||
agent_prompt_mode=plan.agent_prompt_mode,
|
||||
agent_provider_template=plan.agent_provider_template,
|
||||
terminal_title=f"{plan.spec.label} ({plan.spec.agent_name})" if plan.spec.label else plan.spec.agent_name,
|
||||
terminal_color=plan.spec.color,
|
||||
agent_workdir=plan.workspace_plan.workdir,
|
||||
)
|
||||
bottle.prompt_path = provision(plan, bottle)
|
||||
|
||||
yield bottle
|
||||
finally:
|
||||
_teardown_smolmachines(stack, plan)
|
||||
|
||||
|
||||
def _teardown_smolmachines(
|
||||
stack: ExitStack,
|
||||
plan: SmolmachinesBottlePlan,
|
||||
) -> None:
|
||||
"""Unwind the ExitStack, then revoke any provisioned deploy keys.
|
||||
|
||||
ExitStack errors are caught and logged (non-fatal) so that key
|
||||
revocation always runs. Revocation errors propagate — a stranded
|
||||
deploy key is a security concern the operator must address."""
|
||||
teardown_exc: BaseException | None = None
|
||||
try:
|
||||
stack.close()
|
||||
except BaseException as exc: # noqa: W0718 — teardown must not fail
|
||||
teardown_exc = exc
|
||||
warn(f"smolmachines teardown failed: {exc!r}")
|
||||
bottle = plan.manifest.bottle
|
||||
revoke_git_gate_provisioned_keys(bottle, git_gate_state_dir(plan.slug))
|
||||
if teardown_exc is not None:
|
||||
raise teardown_exc
|
||||
|
||||
|
||||
def _allocate_resources(
|
||||
plan: SmolmachinesBottlePlan,
|
||||
stack: ExitStack,
|
||||
) -> tuple[str, str]:
|
||||
"""Reserve a loopback alias and create the per-bottle docker bridge.
|
||||
|
||||
The per-bottle alias scopes TSI's allowlist to this bottle's
|
||||
published ports so the agent can't reach other bottles' or host
|
||||
services' ports on loopback. On macOS `ensure_pool` first
|
||||
sudo-aliases the pool on `lo0`; on Linux that's a no-op since
|
||||
all of 127.0.0.0/8 is already loopback, but the per-bottle
|
||||
allocation runs on both."""
|
||||
_loopback.ensure_pool()
|
||||
loopback_ip = _loopback.allocate(plan.slug)
|
||||
network = _bundle.bundle_network_name(plan.slug)
|
||||
_bundle.create_bundle_network(network, plan.bundle_subnet, plan.bundle_gateway)
|
||||
stack.callback(_bundle.remove_bundle_network, network)
|
||||
return loopback_ip, network
|
||||
|
||||
|
||||
def _mint_certs(plan: SmolmachinesBottlePlan) -> SmolmachinesBottlePlan:
|
||||
"""Mint the egress MITM CA and return the plan with CA paths filled."""
|
||||
egress_ca_host, egress_ca_cert_only = egress_tls_init(
|
||||
egress_state_dir(plan.slug),
|
||||
)
|
||||
egress_plan = dataclasses.replace(
|
||||
plan.egress_plan,
|
||||
mitmproxy_ca_host_path=egress_ca_host,
|
||||
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
|
||||
)
|
||||
return dataclasses.replace(plan, egress_plan=egress_plan)
|
||||
|
||||
|
||||
def _start_bundle(
|
||||
plan: SmolmachinesBottlePlan,
|
||||
network: str,
|
||||
proxy_host: str,
|
||||
stack: ExitStack,
|
||||
) -> SmolmachinesBottlePlan:
|
||||
"""Build the BundleLaunchSpec, resolve token env, start the
|
||||
sidecar bundle container, and register teardown."""
|
||||
plan = _provision_git_gate_keys(plan)
|
||||
bundle_spec = _bundle_launch_spec(plan, network, proxy_host)
|
||||
token_env = _resolve_token_env(plan, dict(os.environ))
|
||||
_bundle.ensure_bundle_image(bundle_spec.image)
|
||||
_bundle.start_bundle(bundle_spec, env={**os.environ, **token_env})
|
||||
stack.callback(_bundle.stop_bundle, plan.slug)
|
||||
return plan
|
||||
|
||||
|
||||
def _provision_git_gate_keys(
|
||||
plan: SmolmachinesBottlePlan,
|
||||
) -> SmolmachinesBottlePlan:
|
||||
if not plan.git_gate_plan.upstreams:
|
||||
return plan
|
||||
git_gate_plan = provision_git_gate_dynamic_keys(
|
||||
plan.manifest.bottle,
|
||||
plan.git_gate_plan,
|
||||
git_gate_state_dir(plan.slug),
|
||||
)
|
||||
return dataclasses.replace(plan, git_gate_plan=git_gate_plan)
|
||||
|
||||
|
||||
def _discover_urls(
|
||||
plan: SmolmachinesBottlePlan,
|
||||
proxy_host: str,
|
||||
) -> SmolmachinesBottlePlan:
|
||||
"""Discover host-side ports for published container ports and
|
||||
return the plan with URLs + guest_env stamped in.
|
||||
|
||||
`proxy_host` is the host IP that both TSI's allowlist and
|
||||
docker's port-forward bindings are keyed to. On macOS it is the
|
||||
per-bottle loopback alias; on Linux it is the per-bottle bridge
|
||||
gateway (see `_proxy_host`). The agent dials the published port
|
||||
on this IP for all bundle-hosted services.
|
||||
|
||||
NO_PROXY includes `proxy_host` so supervise + git-gate URLs
|
||||
bypass HTTPS_PROXY."""
|
||||
agent_facing_host_port = _bundle.bundle_host_port(
|
||||
plan.slug, _EGRESS_PORT, host_ip=proxy_host,
|
||||
)
|
||||
agent_proxy_url = f"http://{proxy_host}:{agent_facing_host_port}"
|
||||
|
||||
agent_git_gate_host = ""
|
||||
if plan.git_gate_plan.upstreams:
|
||||
git_gate_host_port = _bundle.bundle_host_port(
|
||||
plan.slug, _GIT_HTTP_PORT, host_ip=proxy_host,
|
||||
)
|
||||
agent_git_gate_host = f"{proxy_host}:{git_gate_host_port}"
|
||||
|
||||
agent_supervise_url = ""
|
||||
if plan.supervise_plan is not None:
|
||||
supervise_host_port = _bundle.bundle_host_port(
|
||||
plan.slug, _SUPERVISE_PORT, host_ip=proxy_host,
|
||||
)
|
||||
agent_supervise_url = f"http://{proxy_host}:{supervise_host_port}/"
|
||||
|
||||
existing_no_proxy = plan.guest_env.get("NO_PROXY", "localhost,127.0.0.1")
|
||||
no_proxy = f"{existing_no_proxy},{proxy_host}"
|
||||
guest_env = {
|
||||
**plan.guest_env,
|
||||
"HTTPS_PROXY": agent_proxy_url,
|
||||
"HTTP_PROXY": agent_proxy_url,
|
||||
"https_proxy": agent_proxy_url,
|
||||
"http_proxy": agent_proxy_url,
|
||||
"NO_PROXY": no_proxy,
|
||||
"no_proxy": no_proxy,
|
||||
}
|
||||
if agent_git_gate_host:
|
||||
guest_env["GIT_GATE_URL"] = f"http://{agent_git_gate_host}"
|
||||
if agent_supervise_url:
|
||||
guest_env["MCP_SUPERVISE_URL"] = agent_supervise_url
|
||||
for entry in egress_agent_env_entries(plan.egress_plan):
|
||||
name, value = entry.split("=", 1)
|
||||
guest_env[name] = value
|
||||
|
||||
return dataclasses.replace(
|
||||
plan,
|
||||
guest_env=guest_env,
|
||||
agent_proxy_url=agent_proxy_url,
|
||||
agent_git_gate_host=agent_git_gate_host,
|
||||
agent_supervise_url=agent_supervise_url,
|
||||
)
|
||||
|
||||
|
||||
def _launch_vm(
|
||||
plan: SmolmachinesBottlePlan,
|
||||
agent_from_path: Path,
|
||||
proxy_host: str,
|
||||
stack: ExitStack,
|
||||
) -> None:
|
||||
"""Create, patch, and start the smolvm VM; register teardown.
|
||||
|
||||
--allow-cidr is `proxy_host/32` — the per-bottle loopback alias
|
||||
on macOS or the bridge gateway on Linux (see `_proxy_host`). This
|
||||
ensures the guest can only reach bundle ports published on that IP,
|
||||
not the container IP directly. force_allowlist confirms the
|
||||
allowlist persisted (patching smolvm 0.8.0's silent-drop of
|
||||
--allow-cidr when combined with --from) and fails closed if it
|
||||
can't. Smolfile isn't usable here — smolvm 0.8.0 makes --from
|
||||
and --smolfile mutually exclusive."""
|
||||
tsi_cidr = f"{proxy_host}/32"
|
||||
_smolvm.machine_create(
|
||||
plan.machine_name,
|
||||
from_path=agent_from_path,
|
||||
allow_cidrs=[tsi_cidr],
|
||||
env=plan.guest_env,
|
||||
)
|
||||
stack.callback(_smolvm.machine_delete, plan.machine_name)
|
||||
# Confirm the booted VM's TSI allowlist will actually enforce the
|
||||
# /32 before start (smolvm 0.8.0 silently drops `--allow-cidr`
|
||||
# with `--from`, so the persisted state DB is patched if needed).
|
||||
# Fails closed if enforcement can't be confirmed.
|
||||
_loopback.force_allowlist(plan.machine_name, [tsi_cidr])
|
||||
_smolvm.machine_start(plan.machine_name)
|
||||
stack.callback(_smolvm.machine_stop, plan.machine_name)
|
||||
|
||||
|
||||
def _init_vm(plan: SmolmachinesBottlePlan) -> None:
|
||||
"""Repair filesystem ownership and wait for exec channel readiness.
|
||||
|
||||
Ownership repair: smolvm's pack process remaps files to the host
|
||||
invoker's uid (e.g. 501 on macOS, 1000 on Linux). The chowns use
|
||||
names not numbers so they're correct on either. /home/node must
|
||||
be node:node so
|
||||
Claude Code can write ~/.claude.json; /tmp + /var/tmp need root
|
||||
mode 1777 so non-root processes can create per-uid scratch dirs.
|
||||
All folded into one sh -c to avoid back-to-back exec calls
|
||||
immediately after machine_start (libkrun exec-channel race).
|
||||
|
||||
mkdir -p guards: when booting from a committed snapshot, /tmp and
|
||||
/var/tmp are excluded from the archive (they're ephemeral and their
|
||||
stale contents would have wrong uid after smolvm's uid remap). The
|
||||
directories must be created before chown/chmod can set permissions.
|
||||
|
||||
wait_exec_ready polls until the exec channel is ready for the
|
||||
subsequent provision calls, replacing the empirical sleep."""
|
||||
_smolvm.machine_exec(plan.machine_name, [
|
||||
"sh", "-c",
|
||||
"mkdir -p /tmp /var/tmp && "
|
||||
"chown -R node:node /home/node && "
|
||||
"chown root:root /tmp /var/tmp && "
|
||||
"chmod 1777 /tmp /var/tmp",
|
||||
])
|
||||
_smolvm.wait_exec_ready(plan.machine_name)
|
||||
|
||||
|
||||
def _proxy_host(plan: SmolmachinesBottlePlan, loopback_ip: str) -> str:
|
||||
"""Return the host IP for TSI's allowlist and docker port-forward bindings.
|
||||
|
||||
On macOS, the per-bottle loopback alias (e.g. ``127.0.0.16``) works
|
||||
because macOS's network stack lets TSI intercept 127.x.x.x connects
|
||||
from the guest before they reach the host's own loopback.
|
||||
|
||||
On Linux, the guest kernel's LOCAL routing table routes all
|
||||
``127.0.0.0/8`` to the guest's own loopback — those packets never
|
||||
reach eth0 and TSI never sees them. Using the per-bottle bridge
|
||||
gateway (e.g. ``192.168.N.1``) instead sidesteps the problem: it
|
||||
is not a loopback address, so the guest routes it via eth0 and TSI
|
||||
intercepts it normally. The TSI allowlist is ``gateway/32``, which
|
||||
is distinct from the container IP (``192.168.N.2``), so the agent
|
||||
still can't reach the egress daemon directly — TSI blocks any
|
||||
connection to the container IP that isn't via the published port."""
|
||||
if platform.system() == "Linux":
|
||||
return plan.bundle_gateway
|
||||
return loopback_ip
|
||||
|
||||
|
||||
def _bundle_launch_spec(
|
||||
plan: SmolmachinesBottlePlan, network: str, proxy_host: str,
|
||||
) -> _bundle.BundleLaunchSpec:
|
||||
"""Build a BundleLaunchSpec from the resolved inner Plans.
|
||||
|
||||
Daemons in the CSV:
|
||||
- egress is always present.
|
||||
- git-gate + git-http are conditional on plan.git_gate_plan.upstreams.
|
||||
- supervise is conditional on plan.supervise_plan.
|
||||
|
||||
Env + volumes are the union of the sidecar daemons' needs, with
|
||||
daemon-private values only (HTTPS_PROXY is scoped to the
|
||||
egress process by egress_entrypoint.sh — see PRD 0024's bundle
|
||||
bind-address PR)."""
|
||||
daemons: list[str] = ["egress"]
|
||||
env: list[str] = []
|
||||
volumes: list[tuple[str, str, bool]] = []
|
||||
|
||||
# --- egress -----------------------------------------------
|
||||
ep = plan.egress_plan
|
||||
volumes.append((str(ep.mitmproxy_ca_host_path), EGRESS_CA_IN_CONTAINER, True))
|
||||
if ep.routes:
|
||||
volumes.append((str(ep.routes_path.parent), str(Path(EGRESS_ROUTES_IN_CONTAINER).parent), True))
|
||||
env.extend(egress_sidecar_env_entries(ep))
|
||||
|
||||
# --- git-gate ---------------------------------------------
|
||||
gp = plan.git_gate_plan
|
||||
if gp.upstreams:
|
||||
daemons += ["git-gate", "git-http"]
|
||||
volumes += [
|
||||
(str(gp.entrypoint_script), GIT_GATE_ENTRYPOINT_IN_CONTAINER, True),
|
||||
(str(gp.hook_script), GIT_GATE_HOOK_IN_CONTAINER, True),
|
||||
(str(gp.access_hook_script), GIT_GATE_ACCESS_HOOK_IN_CONTAINER, True),
|
||||
]
|
||||
for u in gp.upstreams:
|
||||
keypath = expand_tilde(u.identity_file)
|
||||
volumes.append((
|
||||
keypath,
|
||||
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{u.name}-key",
|
||||
True,
|
||||
))
|
||||
if u.known_hosts_file:
|
||||
volumes.append((
|
||||
str(u.known_hosts_file),
|
||||
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{u.name}-known_hosts",
|
||||
True,
|
||||
))
|
||||
|
||||
# --- supervise --------------------------------------------
|
||||
sp = plan.supervise_plan
|
||||
if sp is not None:
|
||||
daemons.append("supervise")
|
||||
env += [
|
||||
f"SUPERVISE_BOTTLE_SLUG={plan.slug}",
|
||||
f"SUPERVISE_DB_PATH={DB_PATH_IN_CONTAINER}",
|
||||
f"SUPERVISE_PORT={SUPERVISE_PORT}",
|
||||
]
|
||||
volumes.append((str(sp.db_path), DB_PATH_IN_CONTAINER, False))
|
||||
|
||||
# Container ports the agent reaches from the smolvm guest —
|
||||
# published on `proxy_host` so the TSI allowlist and the docker
|
||||
# port-forward bindings point at the same IP. Egress is always
|
||||
# the agent's HTTP/HTTPS proxy.
|
||||
ports_to_publish: list[int] = [_EGRESS_PORT]
|
||||
if gp.upstreams:
|
||||
ports_to_publish.append(_GIT_HTTP_PORT)
|
||||
if sp is not None:
|
||||
ports_to_publish.append(_SUPERVISE_PORT)
|
||||
|
||||
return _bundle.BundleLaunchSpec(
|
||||
slug=plan.slug,
|
||||
network_name=network,
|
||||
subnet=plan.bundle_subnet,
|
||||
gateway=plan.bundle_gateway,
|
||||
bundle_ip=plan.bundle_ip,
|
||||
daemons_csv=",".join(daemons),
|
||||
environment=tuple(env),
|
||||
volumes=tuple(volumes),
|
||||
ports_to_publish=tuple(ports_to_publish),
|
||||
publish_host_ip=proxy_host,
|
||||
)
|
||||
|
||||
|
||||
def _resolve_token_env(
|
||||
plan: SmolmachinesBottlePlan, host_env: dict[str, str],
|
||||
) -> dict[str, str]:
|
||||
"""Resolve the egress token env-var values from the host's
|
||||
environ so they reach the bundle's process env via docker's
|
||||
`-e NAME` inheritance. Empty when no routes declare auth."""
|
||||
effective_env = {**host_env, **plan.agent_provision.provisioned_env}
|
||||
return egress_resolve_token_values(plan.egress_plan.token_env_map, effective_env)
|
||||
|
||||
|
||||
def _agent_from_path(plan: SmolmachinesBottlePlan) -> Path:
|
||||
"""Return the `.smolmachine` artifact used for `machine create --from`.
|
||||
|
||||
Prefer a committed VM artifact when one is recorded and still
|
||||
present. If the file was removed, fall back to the normal image
|
||||
build + pack cache path.
|
||||
"""
|
||||
committed = read_committed_image(plan.slug)
|
||||
if committed:
|
||||
committed_path = Path(committed)
|
||||
if committed_path.is_file():
|
||||
info(f"using committed smolmachine {str(committed_path)!r}")
|
||||
return committed_path
|
||||
|
||||
# Build the agent image and pack it into a `.smolmachine`
|
||||
# artifact (or hit the per-Dockerfile-digest cache). Runs here,
|
||||
# not in prepare, so the docker-build output doesn't garble the
|
||||
# dashboard's preflight modal.
|
||||
return _ensure_smolmachine(
|
||||
plan.agent_image,
|
||||
dockerfile=plan.agent_dockerfile_path,
|
||||
)
|
||||
|
||||
|
||||
def _ensure_smolmachine(image_ref: str, *, dockerfile: str = "") -> Path:
|
||||
"""Build the agent docker image and convert it into a
|
||||
`.smolmachine` artifact, caching the result under
|
||||
`~/.cache/bot-bottle/smolmachines/` keyed by the docker image
|
||||
ID (so a Dockerfile change automatically invalidates the cache).
|
||||
|
||||
Returns the `.smolmachine.smolmachine` sidecar path — that's
|
||||
the file `machine create --from` consumes (pack create produces
|
||||
a launcher binary at `.smolmachine` plus the sidecar alongside
|
||||
it; the sidecar is the actual artifact).
|
||||
|
||||
Conversion path: `docker build` (the existing layer cache
|
||||
makes no-change rebuilds cheap) → `docker save` to a tarball
|
||||
→ spin up an ephemeral registry on a private docker network →
|
||||
`crane push --insecure` from a one-shot container on the same
|
||||
network → `smolvm pack create --image localhost:<host port>/...`
|
||||
→ tear down the registry + network. The crane push detour
|
||||
sidesteps the Docker-Desktop daemon's HTTPS preference for
|
||||
non-loopback registries — see the `local_registry` module
|
||||
docstring for the gory details.
|
||||
|
||||
Each pack-create costs several seconds even on a hot cache,
|
||||
so we skip the whole pipeline when the cached sidecar is
|
||||
already on disk for this image ID."""
|
||||
_SMOLMACHINE_CACHE_DIR.mkdir(parents=True, exist_ok=True)
|
||||
docker_mod.build_image(image_ref, _REPO_DIR, dockerfile=dockerfile)
|
||||
# `sha256:abcd...` -> `abcd...` first 16 chars: short enough to
|
||||
# keep filenames manageable, long enough to make collisions
|
||||
# astronomically unlikely.
|
||||
digest = docker_mod.image_id(image_ref).split(":", 1)[-1][:16]
|
||||
binary = _SMOLMACHINE_CACHE_DIR / f"{digest}.smolmachine"
|
||||
sidecar = _SMOLMACHINE_CACHE_DIR / f"{digest}.smolmachine.smolmachine"
|
||||
if sidecar.is_file():
|
||||
return sidecar
|
||||
tarball = _SMOLMACHINE_CACHE_DIR / f"{digest}.image.tar"
|
||||
docker_mod.save(image_ref, str(tarball))
|
||||
# On Linux, `docker save -o` writes the tarball with owner-only
|
||||
# permissions (mode 600). The crane push container runs as UID
|
||||
# 65532 (distroless nonroot) and can't read it through a bind
|
||||
# mount unless world-read is set. The tarball is temporary and
|
||||
# lives in ~/.cache, so 644 is safe.
|
||||
tarball.chmod(0o644)
|
||||
try:
|
||||
with ephemeral_registry() as handle:
|
||||
push_ref = f"{handle.push_endpoint}/bot-bottle:{digest}"
|
||||
pack_ref = f"{handle.pull_endpoint}/bot-bottle:{digest}"
|
||||
crane_push_tarball(handle, str(tarball), push_ref)
|
||||
_smolvm.pack_create(pack_ref, binary)
|
||||
finally:
|
||||
# Tarball is ~500MB-1GB for the agent image; reclaim once
|
||||
# the smolmachine artifact exists. The artifact itself is
|
||||
# the long-lived cache entry.
|
||||
tarball.unlink(missing_ok=True)
|
||||
return sidecar
|
||||
@@ -1,236 +0,0 @@
|
||||
"""Ephemeral local OCI registry for the smolmachines agent-image
|
||||
conversion path (PRD 0023 chunk 4c).
|
||||
|
||||
`smolvm pack create --image <ref>` only accepts OCI registry refs
|
||||
— it can't read the local docker daemon's image cache, an OCI
|
||||
layout directory, or a `docker save` tarball. To convert the
|
||||
agent's Dockerfile-built image into a `.smolmachine` artifact we
|
||||
spin up a short-lived `registry:2.8.3` container alongside a
|
||||
`crane` helper container on a private docker network, push via
|
||||
`crane push --insecure <tarball> <registry-container>:5000/...`,
|
||||
and let smolvm pull from the registry's published host port. The
|
||||
network + both containers are torn down after the pack completes.
|
||||
|
||||
Why this two-container dance instead of plain `docker push`:
|
||||
- Docker Desktop's daemon runs in its own Linux VM, so its
|
||||
`localhost` is not the host's loopback. A registry bound to
|
||||
the host's 127.0.0.1 is unreachable from the daemon side.
|
||||
- `host.docker.internal` is reachable from the daemon but isn't
|
||||
in Docker's default insecure-registries CIDRs (only `::1/128`
|
||||
and `127.0.0.0/8` are), so `docker push` to it tries HTTPS,
|
||||
hits a plain-HTTP registry, and dies with
|
||||
`http: server gave HTTP response to HTTPS client`. Adding
|
||||
`host.docker.internal` to daemon.json works but is a one-time
|
||||
manual step the user has to do in Docker Desktop's UI.
|
||||
- Going through a docker network sidesteps the host-vs-daemon
|
||||
loopback mismatch (crane and registry containers see each
|
||||
other on the network) AND the HTTPS preference (crane has an
|
||||
`--insecure` flag that forces plain HTTP).
|
||||
|
||||
The registry is also published on a random host port so smolvm
|
||||
— a host process — can pull from `localhost:<port>` via Docker's
|
||||
port-forward. smolvm's bundled crane auto-falls-back to HTTP for
|
||||
localhost addresses, so no insecure-registries config is needed
|
||||
on that side either."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import socket
|
||||
import subprocess
|
||||
import time
|
||||
import uuid
|
||||
from contextlib import contextmanager
|
||||
from dataclasses import dataclass
|
||||
from typing import Generator
|
||||
|
||||
from ...log import die
|
||||
|
||||
|
||||
# registry:2.8.3, pinned by digest. Same env-override pattern as the
|
||||
# sidecar image pin in bot_bottle/backend/docker/sidecar_bundle.py.
|
||||
REGISTRY_IMAGE = os.environ.get(
|
||||
"BOT_BOTTLE_REGISTRY_IMAGE",
|
||||
"registry@sha256:a3d8aaa63ed8681a604f1dea0aa03f100d5895b6a58ace528858a7b332415373",
|
||||
)
|
||||
|
||||
|
||||
# gcr.io/go-containerregistry/crane:latest, pinned by digest. ~10MB,
|
||||
# stable upstream from Google; we only invoke `crane push --insecure`
|
||||
# against a localhost-equivalent registry, so the trust surface is
|
||||
# narrow.
|
||||
CRANE_IMAGE = os.environ.get(
|
||||
"BOT_BOTTLE_CRANE_IMAGE",
|
||||
(
|
||||
"gcr.io/go-containerregistry/crane@sha256:"
|
||||
"0ae17ecb34315aa7cbff28f6eddee3b7adae0b2f90101260d990804db1eb0084"
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
# Internal port the registry binds to inside its container — fixed
|
||||
# by the registry:2 image. The host-side mapping is random.
|
||||
_REGISTRY_CONTAINER_PORT = "5000"
|
||||
|
||||
|
||||
# How long to wait for the registry's HTTP layer to bind before
|
||||
# giving up. Two seconds is empirically enough; 10s leaves headroom
|
||||
# for slow CI runners without making the failure mode chatty.
|
||||
_READY_TIMEOUT_S = 10.0
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RegistryHandle:
|
||||
"""Everything callers need to push to + pull from the ephemeral
|
||||
registry.
|
||||
|
||||
`network` is the per-session docker network — a `crane push`
|
||||
container has to join it to reach the registry by name.
|
||||
`push_endpoint` is the `<host>:<port>` form to embed in image
|
||||
refs given to the crane push container (resolves via docker
|
||||
network DNS). `pull_endpoint` is the `<host>:<port>` form a
|
||||
host process (smolvm) uses; the registry's host port mapping
|
||||
backs this."""
|
||||
|
||||
network: str
|
||||
push_endpoint: str
|
||||
pull_endpoint: str
|
||||
|
||||
|
||||
@contextmanager
|
||||
def ephemeral_registry() -> Generator[RegistryHandle, None, None]:
|
||||
"""Bring up a per-session docker network + a `registry:2.8.3`
|
||||
container on it (published on a random host port), yield a
|
||||
`RegistryHandle`, force-remove both on exit.
|
||||
|
||||
The container is started with `--rm` so a clean exit cleans up
|
||||
on its own; the `finally` block force-removes on abnormal exit
|
||||
(the calling process crashes between yield and close)."""
|
||||
session_id = uuid.uuid4().hex[:12]
|
||||
network = f"bot-bottle-registry-net-{session_id}"
|
||||
registry_name = f"bot-bottle-registry-{session_id}"
|
||||
|
||||
subprocess.run(
|
||||
["docker", "network", "create", network],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
)
|
||||
try:
|
||||
subprocess.run(
|
||||
[
|
||||
"docker", "run", "-d", "--rm",
|
||||
"--name", registry_name,
|
||||
"--network", network,
|
||||
# `-p :5000` (no IP prefix) binds the container's
|
||||
# port 5000 on a random host port across all
|
||||
# interfaces. The host side reaches the registry
|
||||
# via this port — smolvm's `pack create` pulls from
|
||||
# `localhost:<port>` and the docker port-forward
|
||||
# routes there.
|
||||
"-p", _REGISTRY_CONTAINER_PORT,
|
||||
REGISTRY_IMAGE,
|
||||
],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
)
|
||||
try:
|
||||
port = _host_port(registry_name)
|
||||
_wait_ready(port)
|
||||
yield RegistryHandle(
|
||||
network=network,
|
||||
push_endpoint=f"{registry_name}:{_REGISTRY_CONTAINER_PORT}",
|
||||
pull_endpoint=f"localhost:{port}",
|
||||
)
|
||||
finally:
|
||||
subprocess.run(
|
||||
["docker", "rm", "-f", registry_name],
|
||||
check=False,
|
||||
capture_output=True,
|
||||
)
|
||||
finally:
|
||||
subprocess.run(
|
||||
["docker", "network", "rm", network],
|
||||
check=False,
|
||||
capture_output=True,
|
||||
)
|
||||
|
||||
|
||||
def crane_push_tarball(handle: RegistryHandle, tarball_path: str, ref: str) -> None:
|
||||
"""Run `crane push --insecure <tarball> <ref>` inside a one-shot
|
||||
container on the registry's docker network. `ref` should
|
||||
reference the registry by `handle.push_endpoint` so the crane
|
||||
container resolves it via docker network DNS.
|
||||
|
||||
Doesn't go through `docker push` to avoid the Docker-Desktop
|
||||
daemon's HTTPS preference for non-loopback hostnames — crane's
|
||||
`--insecure` flag forces plain HTTP, which is what the
|
||||
registry container speaks."""
|
||||
r = subprocess.run(
|
||||
[
|
||||
"docker", "run", "--rm",
|
||||
"--network", handle.network,
|
||||
"-v", f"{tarball_path}:/img.tar:ro",
|
||||
CRANE_IMAGE,
|
||||
"push", "--insecure", "/img.tar", ref,
|
||||
],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if r.returncode != 0:
|
||||
die(
|
||||
f"crane push of {tarball_path!r} to {ref!r} failed: "
|
||||
f"{(r.stderr or r.stdout or '').strip() or '<no output>'}"
|
||||
)
|
||||
|
||||
|
||||
def _host_port(name: str) -> int:
|
||||
"""Resolve the host-side port docker mapped to the registry's
|
||||
container port. `docker port <name> 5000/tcp` returns one or
|
||||
more `host:port` lines (one per address family) — we take the
|
||||
first."""
|
||||
r = subprocess.run(
|
||||
["docker", "port", name, f"{_REGISTRY_CONTAINER_PORT}/tcp"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if r.returncode != 0:
|
||||
die(
|
||||
f"docker port {name} {_REGISTRY_CONTAINER_PORT}/tcp failed: "
|
||||
f"{(r.stderr or '').strip() or '<no stderr>'}"
|
||||
)
|
||||
# `0.0.0.0:54321\n[::]:54321\n` — split on the last colon to
|
||||
# handle either IPv4 or IPv6 host syntax.
|
||||
line = (r.stdout or "").splitlines()[0].strip()
|
||||
_, _, port_str = line.rpartition(":")
|
||||
try:
|
||||
return int(port_str)
|
||||
except ValueError:
|
||||
die(f"unexpected `docker port` output: {line!r}")
|
||||
|
||||
|
||||
def _wait_ready(port: int) -> None:
|
||||
"""Block until the registry's HTTP layer accepts a TCP
|
||||
connection on `127.0.0.1:<port>`, or `_READY_TIMEOUT_S`
|
||||
elapses.
|
||||
|
||||
A successful TCP connect is sufficient — registry:2.8.3 binds
|
||||
after it's ready to serve `/v2/` requests, so the push that
|
||||
follows will land on a working server. We probe loopback
|
||||
specifically (not via the docker network) because this helper
|
||||
runs on the host."""
|
||||
deadline = time.monotonic() + _READY_TIMEOUT_S
|
||||
last_err: Exception | None = None
|
||||
while time.monotonic() < deadline:
|
||||
try:
|
||||
with socket.create_connection(("127.0.0.1", port), timeout=0.5):
|
||||
return
|
||||
except OSError as e:
|
||||
last_err = e
|
||||
time.sleep(0.1)
|
||||
die(
|
||||
f"local registry on 127.0.0.1:{port} did not accept "
|
||||
f"connections within {_READY_TIMEOUT_S:.0f}s "
|
||||
f"(last error: {last_err})"
|
||||
)
|
||||
@@ -1,314 +0,0 @@
|
||||
"""Per-bottle loopback alias allocation + TSI allowlist
|
||||
enforcement (PRD 0023, follow-up to PR #74).
|
||||
|
||||
After the pivot to host-loopback port-forwards, the smolmachines
|
||||
TSI allowlist was `127.0.0.1/32` — which meant the agent VM could
|
||||
reach **any** service bound to macOS's loopback, not just the
|
||||
bundle's published ports. Real downgrade from the docker
|
||||
backend's `--internal` network isolation.
|
||||
|
||||
This module narrows the allowlist by allocating each bottle a
|
||||
unique loopback alias (`127.0.0.16` .. `127.0.0.31`). The
|
||||
bundle's port-forwards bind to that alias, and the alias's /32
|
||||
is what TSI allows.
|
||||
|
||||
**Smolvm 0.8.0 quirk + workaround.** `smolvm machine create
|
||||
--from <smolmachine> --net --allow-cidr X/32` silently drops the
|
||||
flag — verified empirically that the agent process's allowlist
|
||||
ends up `null` in smolvm's persistent state DB (`~/Library/
|
||||
Application Support/smolvm/server/smolvm.db`, `vms` table,
|
||||
`data` BLOB), and the booted VM reaches all of `127.0.0.0/8`
|
||||
regardless of what we passed. Workaround: after machine_create,
|
||||
open the SQLite DB and patch the row's `allowed_cidrs` field
|
||||
directly. Smolvm reads the DB at machine_start, so the patched
|
||||
value takes effect on boot. Tested: enforcement is real — the
|
||||
guest's connect to a non-allowlisted IP fails with `Permission
|
||||
denied`. Other paths we tried (machine update, stop-edit-
|
||||
agent.config.json-restart, --smolfile, --image localhost:N/...)
|
||||
were dead ends.
|
||||
|
||||
macOS only configures `127.0.0.1` on `lo0` by default; the
|
||||
additional aliases require `sudo ifconfig lo0 alias`. We lazily
|
||||
sudo-add the missing pool on first use per boot — the aliases
|
||||
persist on `lo0` until reboot, so subsequent launches don't
|
||||
prompt.
|
||||
|
||||
On Linux the whole `127.0.0.0/8` is already routed to `lo`, so
|
||||
docker can publish a bundle's ports directly on `127.0.0.<N>`
|
||||
with no `ifconfig`/sudo step. `ensure_pool` is therefore a no-op
|
||||
on Linux, but per-bottle alias *allocation* and the TSI allowlist
|
||||
DB patch run on both platforms — the isolation property is
|
||||
identical, it's just cheaper to set up on Linux. The state-DB
|
||||
path differs per platform (see `_smolvm_db_path`).
|
||||
|
||||
Allocation is coordinated by inspecting running bundle
|
||||
containers' published host IPs — each bottle's bundle owns the
|
||||
alias appearing in its port bindings. The lowest-numbered free
|
||||
alias gets handed to a new bottle."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import fcntl
|
||||
import json
|
||||
import os
|
||||
import platform
|
||||
import re
|
||||
import sqlite3
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
from typing import Iterable
|
||||
|
||||
from ...log import die, info
|
||||
|
||||
|
||||
def _smolvm_db_path() -> Path:
|
||||
"""smolvm's persistent VM state — a SQLite DB whose `vms` table
|
||||
holds one JSON BLOB per machine. macOS stores it under
|
||||
`Application Support`; Linux follows the XDG base-dir spec
|
||||
(`$XDG_DATA_HOME`, default `~/.local/share`).
|
||||
|
||||
NOTE: the Linux location is inferred from smolvm's documented
|
||||
`~/.local/share` install layout and must be confirmed against a
|
||||
real Linux smolvm install. If it's wrong, `force_allowlist`'s
|
||||
fail-closed check turns it into a clear launch-time error rather
|
||||
than a silent escape."""
|
||||
if platform.system() == "Darwin":
|
||||
return (
|
||||
Path.home()
|
||||
/ "Library"
|
||||
/ "Application Support"
|
||||
/ "smolvm"
|
||||
/ "server"
|
||||
/ "smolvm.db"
|
||||
)
|
||||
xdg_data = os.environ.get("XDG_DATA_HOME")
|
||||
base = Path(xdg_data) if xdg_data else Path.home() / ".local" / "share"
|
||||
return base / "smolvm" / "server" / "smolvm.db"
|
||||
|
||||
|
||||
# Resolved once at import: the host platform doesn't change within a
|
||||
# process. Tests patch this attribute directly.
|
||||
_SMOLVM_DB_PATH = _smolvm_db_path()
|
||||
|
||||
|
||||
# Sixteen aliases by default. Tunable for hosts that want more
|
||||
# concurrent bottles (each bottle reserves one alias for its
|
||||
# bundle bringup). The range is chosen to avoid the reserved
|
||||
# 127.0.0.1/2/3 ports (1 is the default, 2 is sometimes used by
|
||||
# CUPS, 3 by other macOS services) and stay well clear of
|
||||
# 127.0.0.53 (systemd-resolved) and 127.0.0.54 (libvirt).
|
||||
_POOL_START = 16
|
||||
_POOL_END = 31 # inclusive
|
||||
|
||||
|
||||
# File lock that serialises concurrent allocate() calls so two
|
||||
# simultaneous launches can't read the same docker state and claim
|
||||
# the same alias. Narrowed to the allocate() call itself; docker run
|
||||
# runs after the lock is released. Once the container is running it
|
||||
# appears in docker state and future allocate() calls will see it.
|
||||
_ALLOC_LOCK_PATH = Path.home() / ".cache" / "bot-bottle" / "smolmachines.lock"
|
||||
|
||||
|
||||
# Loopback aliases pool: 127.0.0.<start>..127.0.0.<end>.
|
||||
def _pool_addresses() -> list[str]:
|
||||
return [f"127.0.0.{i}" for i in range(_POOL_START, _POOL_END + 1)]
|
||||
|
||||
|
||||
def _is_macos() -> bool:
|
||||
return platform.system() == "Darwin"
|
||||
|
||||
|
||||
def ensure_pool() -> None:
|
||||
"""Make sure each address in the pool is up on `lo0`. Lazily
|
||||
runs `sudo ifconfig lo0 alias <ip>/32 up` for missing entries
|
||||
(sudo prompts once, then the aliases persist on lo0 until
|
||||
reboot). No-op on non-macOS hosts."""
|
||||
if not _is_macos():
|
||||
return
|
||||
missing = [ip for ip in _pool_addresses() if not _alias_present(ip)]
|
||||
if not missing:
|
||||
return
|
||||
info(
|
||||
f"smolmachines needs {len(missing)} loopback alias(es) on lo0 "
|
||||
f"({', '.join(missing[:3])}{', ...' if len(missing) > 3 else ''}) "
|
||||
f"to scope per-bottle TSI allowlists. sudo will prompt once; "
|
||||
f"aliases persist until reboot."
|
||||
)
|
||||
for ip in missing:
|
||||
result = subprocess.run(
|
||||
["sudo", "-p", "bot-bottle (loopback alias): ",
|
||||
"ifconfig", "lo0", "alias", f"{ip}/32", "up"],
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
die(
|
||||
f"sudo ifconfig lo0 alias {ip} failed (exit "
|
||||
f"{result.returncode}). Re-run with sudo available, "
|
||||
f"or add manually: sudo ifconfig lo0 alias {ip}/32 up"
|
||||
)
|
||||
|
||||
|
||||
def force_allowlist(machine_name: str, allowed_cidrs: list[str]) -> None:
|
||||
"""Ensure the machine's persisted TSI allowlist equals
|
||||
`allowed_cidrs`, failing **closed** if that can't be confirmed.
|
||||
|
||||
Runs on both macOS and Linux. It exists because smolvm 0.8.0
|
||||
silently drops `--allow-cidr` when combined with `--from`, so
|
||||
the allowlist has to be written into smolvm's persistent state
|
||||
DB before `machine start`. Rather than assume the flag was
|
||||
dropped, we read the persisted row and only patch when it
|
||||
doesn't already match — so a newer smolvm that honors the flag
|
||||
is left untouched.
|
||||
|
||||
Must run AFTER `smolvm machine create` (the row has to exist)
|
||||
and BEFORE `smolvm machine start` (smolvm reads the row on
|
||||
start; in-flight VMs don't pick up changes).
|
||||
|
||||
Fail-closed: if the state DB is missing, the row is missing, or
|
||||
the allowlist still doesn't match after patching, we `die()`
|
||||
rather than boot a VM whose egress confinement we can't verify
|
||||
— an unconfirmed allowlist is a sandbox-escape risk (the agent
|
||||
VM could reach all of host loopback)."""
|
||||
want = list(allowed_cidrs)
|
||||
if not _SMOLVM_DB_PATH.is_file():
|
||||
die(
|
||||
f"smolvm state DB not found at {_SMOLVM_DB_PATH}; cannot "
|
||||
f"confirm the TSI allowlist is enforced. Refusing to launch "
|
||||
f"(fail-closed). Check `smolvm --version` and the DB "
|
||||
f"location for your platform."
|
||||
)
|
||||
con = sqlite3.connect(str(_SMOLVM_DB_PATH))
|
||||
try:
|
||||
cfg = _read_machine_cfg(con, machine_name)
|
||||
if cfg.get("allowed_cidrs") != want:
|
||||
cfg["allowed_cidrs"] = want
|
||||
# Write as BLOB (the column type smolvm uses) — passing a
|
||||
# plain str makes sqlite store it as Text and smolvm then
|
||||
# fails to read it.
|
||||
con.execute(
|
||||
"UPDATE vms SET data = ? WHERE name = ?",
|
||||
(sqlite3.Binary(json.dumps(cfg).encode()), machine_name),
|
||||
)
|
||||
con.commit()
|
||||
cfg = _read_machine_cfg(con, machine_name)
|
||||
if cfg.get("allowed_cidrs") != want:
|
||||
die(
|
||||
f"could not enforce TSI allowlist {want!r} for machine "
|
||||
f"{machine_name!r} (persisted value is "
|
||||
f"{cfg.get('allowed_cidrs')!r}). Refusing to launch "
|
||||
f"(fail-closed)."
|
||||
)
|
||||
finally:
|
||||
con.close()
|
||||
|
||||
|
||||
def _read_machine_cfg(con: sqlite3.Connection, machine_name: str) -> dict[str, object]:
|
||||
"""Read + JSON-decode a machine's `data` BLOB from the smolvm
|
||||
state DB. Dies (fail-closed) if the row is missing — the caller
|
||||
can't confirm enforcement without it."""
|
||||
row = con.execute(
|
||||
"SELECT data FROM vms WHERE name = ?", (machine_name,),
|
||||
).fetchone()
|
||||
if row is None:
|
||||
die(
|
||||
f"smolvm DB has no row for machine {machine_name!r} — "
|
||||
f"machine_create must run before force_allowlist."
|
||||
)
|
||||
return json.loads(row[0])
|
||||
|
||||
|
||||
def allocate(_slug: str) -> str:
|
||||
"""Pick the lowest-numbered alias from the pool not already
|
||||
in use by a running smolmachines bundle. Bails when the pool
|
||||
is exhausted — the caller should report the limit to the
|
||||
operator. `_slug` is logged for traceability; not otherwise
|
||||
used (no on-disk reservation, allocation is purely
|
||||
docker-state-driven).
|
||||
|
||||
Runs on both platforms: the allocation logic (docker-state
|
||||
inspection + the file lock) is platform-independent. macOS
|
||||
needs `ensure_pool` to have aliased the addresses on `lo0`
|
||||
first; on Linux all of `127.0.0.0/8` is already loopback, so
|
||||
docker can publish on the chosen `127.0.0.<N>` with no setup.
|
||||
Per-bottle scoping (so the agent can't reach other bottles' or
|
||||
host services' loopback ports) therefore holds on both.
|
||||
|
||||
An exclusive file lock serialises concurrent calls so two
|
||||
simultaneous launches don't read the same docker state and
|
||||
claim the same alias."""
|
||||
_ALLOC_LOCK_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(_ALLOC_LOCK_PATH, "w", encoding="utf-8") as lf:
|
||||
fcntl.flock(lf, fcntl.LOCK_EX)
|
||||
return _allocate_locked()
|
||||
|
||||
|
||||
def _allocate_locked() -> str:
|
||||
in_use = _aliases_in_use()
|
||||
for ip in _pool_addresses():
|
||||
if ip not in in_use:
|
||||
return ip
|
||||
die(
|
||||
f"smolmachines loopback alias pool exhausted "
|
||||
f"({_POOL_END - _POOL_START + 1} aliases, all in use). "
|
||||
f"Stop a running bottle (`smolvm machine ls --json`) or "
|
||||
f"raise _POOL_END in loopback_alias.py."
|
||||
)
|
||||
|
||||
|
||||
def _alias_present(ip: str) -> bool:
|
||||
"""True iff `ifconfig lo0` shows `<ip>` as an inet address.
|
||||
Exact-match — `127.0.0.1` shouldn't match `127.0.0.16`."""
|
||||
result = subprocess.run(
|
||||
["/sbin/ifconfig", "lo0"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return False
|
||||
pattern = re.compile(rf"\binet {re.escape(ip)}\b")
|
||||
return bool(pattern.search(result.stdout or ""))
|
||||
|
||||
|
||||
def _aliases_in_use() -> set[str]:
|
||||
"""Aliases already bound by another smolmachines bundle's
|
||||
published-port mappings. We inspect every container whose
|
||||
name matches the smolmachines bundle prefix and pull the
|
||||
`HostIp` out of its port bindings."""
|
||||
result = subprocess.run(
|
||||
["docker", "ps", "--format", "{{.Names}}",
|
||||
"--filter", "name=bot-bottle-sidecars-"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return set()
|
||||
names = [n.strip() for n in (result.stdout or "").splitlines() if n.strip()]
|
||||
in_use: set[str] = set()
|
||||
for name in names:
|
||||
in_use.update(_host_ips_for_container(name))
|
||||
return in_use
|
||||
|
||||
|
||||
def _host_ips_for_container(name: str) -> Iterable[str]:
|
||||
"""Yield the `HostIp` values across all port bindings on
|
||||
container `name`. A bundle binds three or four ports and
|
||||
they all share the same HostIp, so callers can take any."""
|
||||
result = subprocess.run(
|
||||
["docker", "inspect", name,
|
||||
"--format", "{{json .HostConfig.PortBindings}}"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return ()
|
||||
try:
|
||||
bindings = json.loads(result.stdout or "{}")
|
||||
except json.JSONDecodeError:
|
||||
return ()
|
||||
seen: set[str] = set()
|
||||
for _port, mappings in (bindings or {}).items():
|
||||
for m in mappings or []:
|
||||
host_ip = m.get("HostIp") or ""
|
||||
if host_ip:
|
||||
seen.add(host_ip)
|
||||
return seen
|
||||
|
||||
|
||||
__all__ = ["allocate", "ensure_pool", "force_allowlist"]
|
||||
@@ -1,12 +0,0 @@
|
||||
"""Backend-infrastructure provisioners for the smolmachines backend.
|
||||
|
||||
Per PRD 0050 the per-provider provisioning steps (prompt, skills,
|
||||
declarative provision-plan apply, supervise MCP registration) live on
|
||||
the `AgentProvider` plugin under `bot_bottle/contrib/`. CA and git
|
||||
provisioning also moved to the AgentProvider ABC (with Debian/node
|
||||
defaults); user plugins override them for non-standard images.
|
||||
|
||||
No modules remain in this subpackage. Workspace copying now runs
|
||||
through `BottleBackend.provision_workspace` against the running
|
||||
bottle for every backend.
|
||||
"""
|
||||
@@ -1,151 +0,0 @@
|
||||
"""Host-side SIGWINCH → in-VM PTY resize bridge (issue #82).
|
||||
|
||||
smolvm 0.8.0 `machine exec -t` allocates an in-VM PTY but never
|
||||
forwards the host terminal's window size (TIOCSWINSZ) to it. The
|
||||
PTY's initial size is `0 0`, and any host-side resize during the
|
||||
session goes unnoticed — the in-VM claude TUI keeps rendering for
|
||||
whatever (typically tiny) box it last saw, ignoring the operator's
|
||||
tmux pane resize. `docker exec -it` does this forwarding
|
||||
automatically; smolvm doesn't.
|
||||
|
||||
This module wraps `smolvm machine exec` with a thin parent
|
||||
process that:
|
||||
|
||||
1. Spawns the original argv as a child (it gets the inherited
|
||||
TTY, so claude's stdin/stdout/stderr work unchanged).
|
||||
2. On startup + every host SIGWINCH, reads the host terminal
|
||||
size via TIOCGWINSZ on stdin (or stderr if stdin isn't a
|
||||
TTY — tmux respawn-pane gives us a TTY on stdout/stderr)
|
||||
and pushes it into the VM with a side-channel
|
||||
`smolvm machine exec -- sh -c 'for f in /dev/pts/*; do
|
||||
stty -F $f cols X rows Y; done'`. The kernel delivers
|
||||
SIGWINCH to the foreground process group on the slave end
|
||||
automatically, so claude picks up the new size without
|
||||
extra signalling.
|
||||
3. Waits on the child and exits with its returncode.
|
||||
|
||||
The dashboard's tmux pane respawn calls `bottle.agent_argv`
|
||||
which now prepends `[sys.executable, -m, ..., <machine>, --, ...]`
|
||||
to the smolvm argv. Foreground handoff (curses endwin →
|
||||
subprocess.run) goes through the same path so behavior is
|
||||
identical.
|
||||
|
||||
Removable once smolvm grows native SIGWINCH forwarding (upstream
|
||||
follow-up tracked separately)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import fcntl
|
||||
import signal
|
||||
import struct
|
||||
import subprocess
|
||||
import sys
|
||||
import termios
|
||||
import threading
|
||||
from types import FrameType
|
||||
|
||||
|
||||
# How long to wait after the main exec starts before pushing the
|
||||
# initial size. Concurrent `smolvm machine exec` invocations race
|
||||
# libkrun's per-exec OCI config write during the main exec's
|
||||
# bringup window; the side-channel firing immediately corrupts
|
||||
# `config.json` and the main exec dies with SIGKILL (rc=137) or
|
||||
# libkrun's "parse error: trailing garbage" depending on
|
||||
# scheduling. Two seconds is well past the bringup window on a
|
||||
# warm VM, well under the operator's "this is unresponsive"
|
||||
# threshold, and short enough that claude's initial render
|
||||
# almost always fires after the size has been set.
|
||||
_STARTUP_SYNC_DELAY_SEC = 2.0
|
||||
|
||||
|
||||
def _read_winsize() -> tuple[int, int] | None:
|
||||
"""Return `(rows, cols)` from whichever of stdin / stdout /
|
||||
stderr is a TTY, or None if none are. Different invocation
|
||||
surfaces give us different TTYs:
|
||||
|
||||
- foreground handoff (curses endwin → subprocess.run): all
|
||||
three are the operator's terminal.
|
||||
- tmux respawn-pane: tmux sets all three to the pane's PTY.
|
||||
- non-TTY (someone piped stdin in tests): none are; the
|
||||
sync just no-ops, which is the right behavior."""
|
||||
for stream in (sys.stdin, sys.stdout, sys.stderr):
|
||||
try:
|
||||
fd = stream.fileno()
|
||||
data = fcntl.ioctl(fd, termios.TIOCGWINSZ, b"\x00" * 8)
|
||||
except OSError:
|
||||
continue
|
||||
rows, cols, _, _ = struct.unpack("hhhh", data)
|
||||
if rows > 0 and cols > 0:
|
||||
return rows, cols
|
||||
return None
|
||||
|
||||
|
||||
def _push_size(machine: str, rows: int, cols: int) -> None:
|
||||
"""Side-channel `smolvm machine exec` that sets the size of
|
||||
every PTY in the VM. The shell `for` loop covers the case of
|
||||
multiple concurrent interactive sessions (rare but cheap to
|
||||
handle); `stty -F` returns silently on PTYs that don't apply.
|
||||
|
||||
Best-effort: swallow failures. A failed resize doesn't break
|
||||
the session — it just leaves the in-VM PTY at its old size.
|
||||
|
||||
`stdin=DEVNULL` is load-bearing: under tmux, inheriting the
|
||||
pane PTY here means two concurrent smolvm processes (this one
|
||||
and the agent session the wrapper is shepherding) share the
|
||||
PTY's foreground-process-group / input plumbing, and smolvm
|
||||
bails with an internal config-parse error or SIGKILL within
|
||||
~100ms of the side-channel firing. Outside tmux the same
|
||||
pattern survived, presumably because iTerm's PTY plumbing is
|
||||
more forgiving than tmux's, but the DEVNULL is the right
|
||||
default either way — the side-channel never needs stdin."""
|
||||
subprocess.run(
|
||||
["smolvm", "machine", "exec", "--name", machine, "--",
|
||||
"sh", "-c",
|
||||
f"for f in /dev/pts/*; do "
|
||||
f"stty -F \"$f\" cols {cols} rows {rows} 2>/dev/null; "
|
||||
f"done"],
|
||||
stdin=subprocess.DEVNULL,
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
)
|
||||
|
||||
|
||||
def main(argv: list[str]) -> int:
|
||||
"""Entry point. `argv` shape: `<machine> -- <smolvm-argv...>`.
|
||||
|
||||
We don't use argparse — the `--` separator is the contract and
|
||||
everything past it is forwarded verbatim. Keeps the wrapper
|
||||
transparent for callers building argv programmatically."""
|
||||
if len(argv) < 3 or argv[1] != "--":
|
||||
sys.stderr.write(
|
||||
"usage: python -m bot_bottle.backend.smolmachines.pty_resize "
|
||||
"<machine> -- <smolvm-argv...>\n"
|
||||
)
|
||||
return 2
|
||||
machine = argv[0]
|
||||
inner = argv[2:]
|
||||
|
||||
def sync(_signum: int | None = None, _frame: FrameType | None = None) -> None:
|
||||
size = _read_winsize()
|
||||
if size is None:
|
||||
return
|
||||
_push_size(machine, *size)
|
||||
|
||||
signal.signal(signal.SIGWINCH, sync) # type: ignore[arg-type]
|
||||
|
||||
proc = subprocess.Popen(inner)
|
||||
# Initial sync is deferred — see _STARTUP_SYNC_DELAY_SEC.
|
||||
# daemon=True so the timer doesn't block exit when the child
|
||||
# finishes before the delay elapses.
|
||||
timer = threading.Timer(_STARTUP_SYNC_DELAY_SEC, sync)
|
||||
timer.daemon = True
|
||||
timer.start()
|
||||
while True:
|
||||
try:
|
||||
return proc.wait()
|
||||
except KeyboardInterrupt:
|
||||
proc.send_signal(signal.SIGINT)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv[1:]))
|
||||
@@ -1,83 +0,0 @@
|
||||
"""smolmachines `_resolve_plan` (PRD 0023 chunks 2d + 4c).
|
||||
|
||||
Resolves the per-bottle docker subnet + bundle IP and assembles
|
||||
the guest env. The agent's docker image build → smolmachine
|
||||
pack pipeline runs in `launch.launch`, not here, so the
|
||||
dashboard's preflight modal isn't garbled by docker-build output
|
||||
before the operator has confirmed.
|
||||
|
||||
No VM bringup — that's `launch.launch`'s job."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from .. import BottleSpec
|
||||
from ...manifest import Manifest
|
||||
from ...env import ResolvedEnv
|
||||
from ...agent_provider import AgentProvisionPlan
|
||||
from ...egress import EgressPlan
|
||||
from ...supervise import SupervisePlan
|
||||
from ...git_gate import GitGatePlan
|
||||
from .bottle_plan import SmolmachinesBottlePlan
|
||||
from .util import smolmachines_bundle_subnet, smolmachines_preflight
|
||||
|
||||
def preflight() -> None:
|
||||
smolmachines_preflight()
|
||||
|
||||
|
||||
def build_guest_env(resolved_env: ResolvedEnv) -> dict[str, str]:
|
||||
# Agent's env: resolve through resolve_env() so ?prompt entries
|
||||
# are prompted and ${HOST_VAR} entries are interpolated — matching
|
||||
# the Docker backend's contract. Forwarded (secret/interpolated)
|
||||
# values still reach the guest as -e K=V smolvm flags because
|
||||
# smolvm 0.8.0 has no env-file or stdin injection path; this is
|
||||
# the known argv-exposure gap documented in PRD 0038.
|
||||
# HTTPS_PROXY / GIT_GATE_URL / MCP_SUPERVISE_URL are populated
|
||||
# in launch.py after bundle bringup.
|
||||
return {
|
||||
**resolved_env.literals,
|
||||
**resolved_env.forwarded,
|
||||
"NO_PROXY": "localhost,127.0.0.1",
|
||||
"NODE_EXTRA_CA_CERTS": "/etc/ssl/certs/ca-certificates.crt",
|
||||
"SSL_CERT_FILE": "/etc/ssl/certs/ca-certificates.crt",
|
||||
"REQUESTS_CA_BUNDLE": "/etc/ssl/certs/ca-certificates.crt",
|
||||
}
|
||||
|
||||
|
||||
def resolve_plan(
|
||||
spec: BottleSpec,
|
||||
manifest: Manifest,
|
||||
slug: str,
|
||||
resolved_env: ResolvedEnv,
|
||||
agent_provision_plan: AgentProvisionPlan,
|
||||
egress_plan: EgressPlan,
|
||||
supervise_plan: SupervisePlan | None,
|
||||
git_gate_plan: GitGatePlan,
|
||||
stage_dir: Path,
|
||||
) -> SmolmachinesBottlePlan:
|
||||
"""Materialize the smolmachines plan. The bundle's docker
|
||||
subnet + pinned IP are derived from the slug; the agent's
|
||||
`.smolmachine` artifact is built (or cache-hit) here so
|
||||
launch's `machine create --from` boots without a registry
|
||||
pull. Per-bottle guest env + the TSI allow_cidrs land on the
|
||||
plan for launch to pass straight through to
|
||||
`machine create` flags."""
|
||||
|
||||
# ==== smolmachines specific setup ====
|
||||
subnet, gateway, bundle_ip = smolmachines_bundle_subnet(slug)
|
||||
|
||||
return SmolmachinesBottlePlan(
|
||||
spec=spec,
|
||||
manifest=manifest,
|
||||
stage_dir=stage_dir,
|
||||
slug=slug,
|
||||
bundle_subnet=subnet,
|
||||
bundle_gateway=gateway,
|
||||
bundle_ip=bundle_ip,
|
||||
guest_env=agent_provision_plan.guest_env,
|
||||
git_gate_plan=git_gate_plan,
|
||||
egress_plan=egress_plan,
|
||||
supervise_plan=supervise_plan,
|
||||
agent_provision=agent_provision_plan,
|
||||
)
|
||||
@@ -1,242 +0,0 @@
|
||||
"""Per-bottle sidecar bundle bringup for the smolmachines backend
|
||||
(PRD 0023).
|
||||
|
||||
Two docker resources per bottle live here:
|
||||
|
||||
- **A dedicated bridge network**, subnet derived from the slug.
|
||||
The bundle container gets a pinned IP at `<subnet>.2` so the
|
||||
smolvm guest's TSI allowlist (`<bundle-ip>/32`) has a stable
|
||||
target. Without pinning, we'd have to inspect the container's
|
||||
assigned IP after start and feed it back into the Smolfile
|
||||
— a race we can sidestep with `--ip`.
|
||||
|
||||
- **The bundle container itself**, running the PRD 0024 bundle
|
||||
image (`bot-bottle-sidecars:latest` by default). Same
|
||||
image, same daemons, same daemon-private env / bind-mounts
|
||||
as the docker backend.
|
||||
|
||||
This module ships the lifecycle primitives only — create
|
||||
network, start bundle, stop bundle, remove network — wrapped
|
||||
around `subprocess.run(["docker", ...])`. Wiring them into the
|
||||
launch flow + populating the `BundleLaunchSpec` from the inner
|
||||
Plans (EgressPlan, …) lands in chunk 2d."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Sequence
|
||||
|
||||
from ...log import die, warn
|
||||
from ..docker import util as docker_mod
|
||||
from ..docker.sidecar_bundle import (
|
||||
SIDECAR_BUNDLE_DOCKERFILE,
|
||||
SIDECAR_BUNDLE_IMAGE,
|
||||
)
|
||||
|
||||
|
||||
_REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent)
|
||||
|
||||
|
||||
def bundle_network_name(slug: str) -> str:
|
||||
"""`bot-bottle-bundle-<slug>` — distinct from the docker
|
||||
backend's `bot-bottle-net-<slug>` so a smolmachines bottle
|
||||
and a docker bottle for the same agent don't collide on
|
||||
network name."""
|
||||
return f"bot-bottle-bundle-{slug}"
|
||||
|
||||
|
||||
def bundle_container_name(slug: str) -> str:
|
||||
"""`bot-bottle-sidecars-<slug>` — same name shape the docker
|
||||
backend uses for the bundle (PRD 0024 chunk 5). The dashboard's
|
||||
prefix-based discovery covers both backends with one filter."""
|
||||
return f"bot-bottle-sidecars-{slug}"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class BundleLaunchSpec:
|
||||
"""Everything `start_bundle` needs to bring up one bundle
|
||||
container. Populated by chunk-2d's launch flow from the inner
|
||||
Plans the prepare step already produces."""
|
||||
|
||||
slug: str
|
||||
network_name: str
|
||||
subnet: str
|
||||
gateway: str
|
||||
bundle_ip: str
|
||||
image: str = SIDECAR_BUNDLE_IMAGE
|
||||
# Daemon subset CSV for BOT_BOTTLE_SIDECAR_DAEMONS. The
|
||||
# supervisor inside the bundle reads it to skip
|
||||
# bottle-irrelevant daemons (e.g. supervise=False bottles).
|
||||
daemons_csv: str = "egress"
|
||||
# Plain "KEY=VALUE" strings + "KEY" bare names (the bare-name
|
||||
# form inherits the value from the docker-run subprocess env,
|
||||
# matching the docker backend's compose-up secret-forwarding
|
||||
# pattern).
|
||||
environment: Sequence[str] = field(default_factory=tuple)
|
||||
# (host_path, container_path, read_only) bind mounts.
|
||||
volumes: Sequence[tuple[str, str, bool]] = field(default_factory=tuple)
|
||||
# Container ports to publish on `publish_host_ip`, random
|
||||
# host-side port per entry. The smolvm guest's TSI talks via
|
||||
# macOS networking, so docker container IPs (192.168.x.x in
|
||||
# the daemon's bridge) aren't directly reachable from the
|
||||
# guest — host-loopback port-forwards are. Egress's port
|
||||
# is bundle-internal and never published.
|
||||
ports_to_publish: Sequence[int] = field(default_factory=tuple)
|
||||
# Loopback IP to bind published ports against. Per-bottle
|
||||
# loopback aliases (`127.0.0.16` etc., added via sudo
|
||||
# ifconfig lo0 alias) narrow the TSI allowlist so a bottle
|
||||
# can't reach other bottles' (or other host services') ports
|
||||
# via 127.0.0.1.
|
||||
publish_host_ip: str = "127.0.0.1"
|
||||
|
||||
|
||||
def ensure_bundle_image(image: str = SIDECAR_BUNDLE_IMAGE) -> None:
|
||||
"""Build the sidecar bundle image before `docker run`.
|
||||
|
||||
The Docker backend gets this for free from compose's `build:`
|
||||
stanza. smolmachines starts the bundle with plain `docker run`,
|
||||
so without an explicit build a first launch tries to pull the
|
||||
local-only `bot-bottle-sidecars:latest` tag from a registry.
|
||||
"""
|
||||
docker_mod.build_image(
|
||||
image,
|
||||
_REPO_DIR,
|
||||
dockerfile=SIDECAR_BUNDLE_DOCKERFILE,
|
||||
)
|
||||
|
||||
|
||||
def create_bundle_network(network_name: str, subnet: str, gateway: str) -> None:
|
||||
"""`docker network create` with an explicit subnet + gateway
|
||||
so the bundle's `--ip` lands on the address the Smolfile's
|
||||
TSI allowlist points at. Idempotent on the caller's side —
|
||||
`start_bundle` catches the "network exists" error and treats
|
||||
it as success (chunk-2d teardown is paired with each create).
|
||||
"""
|
||||
result = subprocess.run(
|
||||
["docker", "network", "create",
|
||||
"--subnet", subnet, "--gateway", gateway,
|
||||
network_name],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
# Already-exists is fine on a resume path; everything else
|
||||
# is fatal — the bundle won't have an addressable network.
|
||||
if "already exists" in (result.stderr or "").lower():
|
||||
return
|
||||
die(
|
||||
f"docker network create {network_name} failed: "
|
||||
f"{(result.stderr or '').strip()}"
|
||||
)
|
||||
|
||||
|
||||
def remove_bundle_network(network_name: str) -> None:
|
||||
"""Idempotent: a missing network returns success."""
|
||||
result = subprocess.run(
|
||||
["docker", "network", "rm", network_name],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if result.returncode == 0:
|
||||
return
|
||||
if "no such network" in (result.stderr or "").lower():
|
||||
return
|
||||
# Network with attached containers is the common non-fatal
|
||||
# case during a partial teardown — warn but don't die.
|
||||
warn(
|
||||
f"docker network rm {network_name} failed: "
|
||||
f"{(result.stderr or '').strip()}"
|
||||
)
|
||||
|
||||
|
||||
def start_bundle(spec: BundleLaunchSpec, *,
|
||||
env: dict[str, str] | None = None) -> None:
|
||||
"""Bring the bundle container up on the per-bottle bridge with
|
||||
the pinned IP. Argv is built deterministically from `spec`;
|
||||
`env` is the host subprocess env (forwarded values for any
|
||||
bare-name entries in `spec.environment`)."""
|
||||
container = bundle_container_name(spec.slug)
|
||||
argv = [
|
||||
"docker", "run",
|
||||
"--name", container,
|
||||
"--detach",
|
||||
"--rm",
|
||||
"--network", spec.network_name,
|
||||
"--ip", spec.bundle_ip,
|
||||
"-e", f"BOT_BOTTLE_SIDECAR_DAEMONS={spec.daemons_csv}",
|
||||
]
|
||||
for entry in spec.environment:
|
||||
argv += ["-e", entry]
|
||||
for host_path, container_path, read_only in spec.volumes:
|
||||
suffix = ":ro" if read_only else ""
|
||||
argv += ["-v", f"{host_path}:{container_path}{suffix}"]
|
||||
# Loopback-only host port-forwards — the smolvm guest's TSI
|
||||
# uses macOS networking, and macOS loopback is the only host
|
||||
# surface that round-trips into Docker Desktop's daemon VM.
|
||||
# Binds to the per-bottle alias so TSI's IP-only allowlist
|
||||
# narrows reachability to this bottle's bundle only.
|
||||
for port in spec.ports_to_publish:
|
||||
argv += ["-p", f"{spec.publish_host_ip}::{port}"]
|
||||
argv.append(spec.image)
|
||||
result = subprocess.run(
|
||||
argv, capture_output=True, text=True,
|
||||
env=dict(env) if env is not None else None, check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
die(
|
||||
f"docker run for bundle {container} failed: "
|
||||
f"{(result.stderr or '').strip()}"
|
||||
)
|
||||
|
||||
|
||||
def bundle_host_port(
|
||||
slug: str, container_port: int, *, host_ip: str = "127.0.0.1",
|
||||
) -> int:
|
||||
"""`docker port <bundle> <container_port>/tcp` → the random
|
||||
host-side port docker assigned for the binding on `host_ip`.
|
||||
Called after `start_bundle` on each container port listed in
|
||||
`BundleLaunchSpec.ports_to_publish` so the launch step can
|
||||
build the agent's HTTPS_PROXY / GIT_GATE / SUPERVISE URLs in
|
||||
`<host_ip>:<host port>` form."""
|
||||
container = bundle_container_name(slug)
|
||||
result = subprocess.run(
|
||||
["docker", "port", container, f"{container_port}/tcp"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
die(
|
||||
f"docker port {container} {container_port}/tcp failed: "
|
||||
f"{(result.stderr or '').strip() or '<no stderr>'}"
|
||||
)
|
||||
# Each line looks like `127.0.0.16:54321` — one per address
|
||||
# family / host IP. Match on the expected host_ip prefix so
|
||||
# bottles bound to per-bottle aliases pick the right line.
|
||||
for raw in (result.stdout or "").splitlines():
|
||||
line = raw.strip()
|
||||
if line.startswith(f"{host_ip}:"):
|
||||
_, _, port_str = line.rpartition(":")
|
||||
try:
|
||||
return int(port_str)
|
||||
except ValueError:
|
||||
die(f"unexpected `docker port` output: {line!r}")
|
||||
die(
|
||||
f"no port mapping on {host_ip} for {container} "
|
||||
f"{container_port}/tcp; got: {(result.stdout or '').strip()!r}"
|
||||
)
|
||||
|
||||
|
||||
def stop_bundle(slug: str) -> None:
|
||||
"""Idempotent: a missing container returns success."""
|
||||
container = bundle_container_name(slug)
|
||||
result = subprocess.run(
|
||||
["docker", "rm", "-f", container],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if result.returncode == 0:
|
||||
return
|
||||
if "no such container" in (result.stderr or "").lower():
|
||||
return
|
||||
warn(
|
||||
f"docker rm -f {container} failed: "
|
||||
f"{(result.stderr or '').strip()}"
|
||||
)
|
||||
@@ -1,268 +0,0 @@
|
||||
"""Thin subprocess wrapper around the `smolvm` CLI (PRD 0023).
|
||||
|
||||
One thin Python function per smolvm subcommand the launch flow
|
||||
needs. Two design choices worth flagging:
|
||||
|
||||
- **No daemon, no SDK.** smolvm 0.8.0 ships a `smolvm serve`
|
||||
HTTP API as the long-term-clean integration target. The
|
||||
project's stdlib-first ethos + the lower-overhead CLI calls
|
||||
push v1 to shell out via `subprocess.run`. If a future
|
||||
smolvm release makes `serve` mandatory (or significantly
|
||||
faster), revisit.
|
||||
|
||||
- **Two return shapes.** `SmolvmRunResult` (returncode + stdout
|
||||
+ stderr captured) is returned by `machine_exec` because the
|
||||
caller cares about the in-VM command's exit status, and by
|
||||
test helpers that introspect output. The other calls
|
||||
(`machine_start`, `machine_stop`, `pack_create`, etc.) raise
|
||||
`SmolvmError` on non-zero exit — failure to start a VM is
|
||||
fatal to the launch flow, not something callers want to
|
||||
branch on.
|
||||
|
||||
The wrapper is unit-tested with `subprocess.run` mocked; the
|
||||
integration smoke test (chunk 2d) exercises against a real
|
||||
smolvm binary."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import shutil
|
||||
import subprocess
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Mapping, Sequence
|
||||
|
||||
|
||||
|
||||
_SMOLVM = "smolvm"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SmolvmRunResult:
|
||||
"""Captured result of an in-VM command. Mirrors the structure
|
||||
`Bottle.exec` returns so callers can hand it straight through."""
|
||||
returncode: int
|
||||
stdout: str
|
||||
stderr: str
|
||||
|
||||
|
||||
class SmolvmError(RuntimeError):
|
||||
"""Raised when a smolvm subprocess returns non-zero on a path
|
||||
where the caller has no useful branch to take (start failed,
|
||||
pack failed, etc.). Carries the captured stderr for the
|
||||
operator-facing log line."""
|
||||
|
||||
def __init__(self, argv: Sequence[str], result: subprocess.CompletedProcess[str]):
|
||||
self.argv = list(argv)
|
||||
self.returncode = result.returncode
|
||||
self.stdout = result.stdout
|
||||
self.stderr = result.stderr
|
||||
cmd = " ".join(self.argv)
|
||||
super().__init__(
|
||||
f"{cmd!r} failed (exit {result.returncode}): "
|
||||
f"{(result.stderr or '').strip() or '<no stderr>'}"
|
||||
)
|
||||
|
||||
|
||||
def _smolvm(*args: str, env: Mapping[str, str] | None = None,
|
||||
check: bool = True) -> subprocess.CompletedProcess[str]:
|
||||
"""One subprocess call into the smolvm CLI. `check=True`
|
||||
raises SmolvmError on non-zero; `check=False` returns the
|
||||
CompletedProcess for the caller to inspect."""
|
||||
argv = [_SMOLVM, *args]
|
||||
result = subprocess.run(
|
||||
argv,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
env=dict(env) if env is not None else None,
|
||||
check=False,
|
||||
)
|
||||
if check and result.returncode != 0:
|
||||
raise SmolvmError(argv, result)
|
||||
return result
|
||||
|
||||
|
||||
# --- Pack ----------------------------------------------------------------
|
||||
|
||||
|
||||
def pack_create(image: str, output: Path) -> None:
|
||||
"""`smolvm pack create --image <image> -o <output>`. Converts
|
||||
an OCI image into a self-contained `.smolmachine` artifact
|
||||
smolvm can boot via `machine create --from`. Idempotent on the
|
||||
smolvm side — re-running with the same image+output rebuilds
|
||||
from layer cache."""
|
||||
_smolvm("pack", "create", "--image", image, "-o", str(output))
|
||||
|
||||
|
||||
def pack_create_from_vm(name: str, output: Path) -> None:
|
||||
"""`smolvm pack create --from-vm <name> -o <output>`.
|
||||
|
||||
Snapshots an existing persistent VM into a pack artifact. As
|
||||
with `pack_create`, smolvm writes a launcher at `output` and the
|
||||
bootable sidecar at `output.smolmachine`.
|
||||
"""
|
||||
_smolvm("pack", "create", "--from-vm", name, "-o", str(output))
|
||||
|
||||
|
||||
# --- Machine lifecycle ---------------------------------------------------
|
||||
|
||||
|
||||
def machine_create(
|
||||
name: str,
|
||||
*,
|
||||
image: str | None = None,
|
||||
from_path: Path | None = None,
|
||||
allow_cidrs: Sequence[str] = (),
|
||||
env: Mapping[str, str] | None = None,
|
||||
) -> None:
|
||||
"""`smolvm machine create --name NAME [--image IMG | --from PATH]
|
||||
[--allow-cidr CIDR ...] [-e K=V ...]`. NAME is passed as
|
||||
`--name` (smolvm 1.4.7+; earlier versions took it positionally).
|
||||
|
||||
`image` (registry ref like `alpine:latest`) and `from_path`
|
||||
(a `.smolmachine` artifact) are mutually exclusive — one or
|
||||
the other tells smolvm what to boot. The wrapper doesn't
|
||||
enforce exclusivity; smolvm errors clearly enough.
|
||||
|
||||
`allow_cidrs` and `env` are passed as CLI flags instead of a
|
||||
Smolfile because `--from` and `--smolfile` are themselves
|
||||
mutually exclusive in smolvm 0.8.0 — and we want `--from`'s
|
||||
no-pull-at-start property. The flag form gives the same
|
||||
result without the Smolfile complication.
|
||||
|
||||
`--net` is sent explicitly when `allow_cidrs` is non-empty.
|
||||
`--allow-cidr` implies `--net` per the CLI help, but sending
|
||||
`--net` explicitly is harmless and ensures the guest has
|
||||
network access even if that implication changes across versions."""
|
||||
args: list[str] = ["machine", "create", "--name", name]
|
||||
if image is not None:
|
||||
args += ["--image", image]
|
||||
if from_path is not None:
|
||||
args += ["--from", str(from_path)]
|
||||
if allow_cidrs:
|
||||
args.append("--net")
|
||||
for cidr in allow_cidrs:
|
||||
args += ["--allow-cidr", cidr]
|
||||
if env:
|
||||
for k, v in env.items():
|
||||
args += ["-e", f"{k}={v}"]
|
||||
_smolvm(*args)
|
||||
|
||||
|
||||
def machine_is_running(name: str) -> bool:
|
||||
"""Return True if the named VM is in the 'running' state."""
|
||||
result = _smolvm("machine", "ls", "--json", check=False)
|
||||
if result.returncode != 0:
|
||||
return False
|
||||
try:
|
||||
machines = json.loads(result.stdout or "[]")
|
||||
except ValueError:
|
||||
return False
|
||||
return any(
|
||||
isinstance(m, dict) and m.get("name") == name and m.get("state") == "running"
|
||||
for m in machines
|
||||
)
|
||||
|
||||
|
||||
def machine_start(name: str) -> None:
|
||||
"""`smolvm machine start --name NAME`."""
|
||||
_smolvm("machine", "start", "--name", name)
|
||||
|
||||
|
||||
def machine_stop(name: str) -> None:
|
||||
"""`smolvm machine stop --name NAME`. Idempotent against
|
||||
already-stopped machines: smolvm prints a notice and exits 0
|
||||
in that case, so no special handling here."""
|
||||
_smolvm("machine", "stop", "--name", name)
|
||||
|
||||
|
||||
def machine_delete(name: str) -> None:
|
||||
"""`smolvm machine delete --name NAME -f`. `-f` skips the
|
||||
interactive confirmation — required for non-interactive teardown."""
|
||||
_smolvm("machine", "delete", "--name", name, "-f")
|
||||
|
||||
|
||||
def machine_exec(
|
||||
name: str,
|
||||
argv: Sequence[str],
|
||||
*,
|
||||
env: Mapping[str, str] | None = None,
|
||||
workdir: str | None = None,
|
||||
timeout: str | None = None,
|
||||
) -> SmolvmRunResult:
|
||||
"""`smolvm machine exec --name NAME [-w DIR] [--timeout DUR]
|
||||
[-e K=V ...] -- ARGV...`. Returns the captured result rather
|
||||
than raising — callers (including `Bottle.exec`) care about
|
||||
the in-VM command's exit code, not just whether smolvm ran.
|
||||
|
||||
`env` here is in-VM env vars (`-e K=V`), not the host
|
||||
subprocess env — smolvm's own argv carries them through the
|
||||
VMM."""
|
||||
flags: list[str] = ["machine", "exec", "--name", name]
|
||||
if workdir is not None:
|
||||
flags += ["-w", workdir]
|
||||
if timeout is not None:
|
||||
flags += ["--timeout", timeout]
|
||||
if env:
|
||||
for k, v in env.items():
|
||||
flags += ["-e", f"{k}={v}"]
|
||||
# `--` separator before the command. smolvm's CLI requires it
|
||||
# so its own flag parser doesn't grab argv items that look
|
||||
# like flags.
|
||||
flags.append("--")
|
||||
flags += list(argv)
|
||||
result = _smolvm(*flags, check=False)
|
||||
return SmolvmRunResult(
|
||||
returncode=result.returncode,
|
||||
stdout=result.stdout or "",
|
||||
stderr=result.stderr or "",
|
||||
)
|
||||
|
||||
|
||||
def wait_exec_ready(name: str, *, timeout: float = 5.0) -> None:
|
||||
"""Poll `machine exec true` until exit 0 or `timeout` elapses.
|
||||
|
||||
Replaces `time.sleep(1.5)` after `machine_start`: libkrun's exec
|
||||
channel needs a brief warm-up before back-to-back exec calls are
|
||||
safe. Polling exits as soon as the channel is ready and fails
|
||||
loudly if the VM never responds."""
|
||||
deadline = time.monotonic() + timeout
|
||||
delay = 0.1
|
||||
while time.monotonic() < deadline:
|
||||
r = machine_exec(name, ["true"])
|
||||
if r.returncode == 0:
|
||||
return
|
||||
remaining = deadline - time.monotonic()
|
||||
if remaining <= 0:
|
||||
break
|
||||
time.sleep(min(delay, remaining))
|
||||
delay = min(delay * 2, 0.5)
|
||||
argv = ["smolvm", "machine", "exec", "--name", name, "--", "true"]
|
||||
raise SmolvmError(
|
||||
argv,
|
||||
subprocess.CompletedProcess(
|
||||
args=argv, returncode=-1, stdout="",
|
||||
stderr=f"exec channel not ready after {timeout:.0f}s — VM may have failed to boot.",
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def machine_cp(src: str, dst: str) -> None:
|
||||
"""`smolvm machine cp SRC DST`. Path syntax: `machine:path` to
|
||||
reference a path inside the VM, bare path for the host. Both
|
||||
SRC and DST are positional; either side can be machine: or
|
||||
bare. Empty path is a no-op (returns immediately without
|
||||
invoking smolvm)."""
|
||||
if not src or not dst:
|
||||
return
|
||||
_smolvm("machine", "cp", src, dst)
|
||||
|
||||
|
||||
# --- Discovery -----------------------------------------------------------
|
||||
|
||||
|
||||
def is_available() -> bool:
|
||||
"""True iff `smolvm` is on PATH. Used by the integration test
|
||||
suite's skip-guards."""
|
||||
return shutil.which(_SMOLVM) is not None
|
||||
@@ -1,82 +0,0 @@
|
||||
"""Slug / preflight / subnet helpers for the smolmachines backend
|
||||
(PRD 0023). Kept in its own module so the renderers can be
|
||||
unit-tested without importing the docker subprocess paths."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import os
|
||||
import platform
|
||||
import shutil
|
||||
|
||||
from ...log import die
|
||||
|
||||
# libkrun's Linux backend drives the guest through KVM, so the host
|
||||
# must expose `/dev/kvm` and the invoking user must be able to open
|
||||
# it. macOS uses Hypervisor.framework and needs no device node.
|
||||
_KVM_DEVICE = "/dev/kvm"
|
||||
|
||||
|
||||
def smolmachines_preflight() -> None:
|
||||
"""Ensure the host can run the smolmachines backend before the
|
||||
launch flow starts. Called from `_resolve_plan`; surfaces a
|
||||
clear, actionable error instead of a cryptic `smolvm` failure
|
||||
deep in launch.
|
||||
|
||||
Checks `smolvm` is on PATH (both platforms) and, on Linux,
|
||||
that `/dev/kvm` exists and is accessible. `gvproxy` is no
|
||||
longer required — see the PRD's design pivot section."""
|
||||
if shutil.which("smolvm") is None:
|
||||
die(
|
||||
"BOT_BOTTLE_BACKEND=smolmachines requires `smolvm` on "
|
||||
"PATH. Install with: "
|
||||
"curl -sSL https://smolmachines.com/install.sh | sh. "
|
||||
"To use the legacy Docker backend instead, set "
|
||||
"BOT_BOTTLE_BACKEND=docker or pass --backend=docker."
|
||||
)
|
||||
if platform.system() == "Linux":
|
||||
_preflight_kvm()
|
||||
|
||||
|
||||
def _preflight_kvm() -> None:
|
||||
"""Linux-only: libkrun needs `/dev/kvm`. Distinguish 'KVM not
|
||||
enabled' from 'no permission' so the operator knows which to
|
||||
fix."""
|
||||
if not os.path.exists(_KVM_DEVICE):
|
||||
die(
|
||||
f"BOT_BOTTLE_BACKEND=smolmachines needs {_KVM_DEVICE} on "
|
||||
"Linux but it is missing. Enable KVM: load the kvm-intel "
|
||||
"or kvm-amd kernel module (and confirm virtualization is "
|
||||
"enabled in BIOS/firmware). To use the legacy Docker "
|
||||
"backend instead, set BOT_BOTTLE_BACKEND=docker."
|
||||
)
|
||||
if not os.access(_KVM_DEVICE, os.R_OK | os.W_OK):
|
||||
die(
|
||||
f"{_KVM_DEVICE} exists but is not readable/writable by the "
|
||||
"current user. Add your user to the `kvm` group "
|
||||
"(`sudo usermod -aG kvm \"$USER\"`) and re-login, or run "
|
||||
"with access to the device."
|
||||
)
|
||||
|
||||
|
||||
def smolmachines_bundle_subnet(slug: str) -> tuple[str, str, str]:
|
||||
"""Derive a per-bottle docker subnet + gateway IP + bundle IP
|
||||
from the slug.
|
||||
|
||||
Returns `(subnet_cidr, gateway_ip, bundle_ip)`. The third
|
||||
octet comes from SHA-256 of the slug mod 254 (skipping 17 to
|
||||
avoid the docker-default bridge), so parallel bottles get
|
||||
distinct /24s and `resume` reuses the same /24. The bundle
|
||||
container always lands at `.2`; gateway is `.1`; the smolvm
|
||||
Smolfile's `allow_cidrs` is `<bundle_ip>/32`."""
|
||||
digest = hashlib.sha256(slug.encode("utf-8")).digest()
|
||||
octet = (digest[0] % 254) + 1
|
||||
# Skip the docker-default bridge to dodge the most common
|
||||
# collision (operators with `docker0` at 172.17.x.x or a
|
||||
# 192.168.17.x VPN client).
|
||||
if octet == 17:
|
||||
octet = 18
|
||||
subnet = f"192.168.{octet}.0/24"
|
||||
gateway = f"192.168.{octet}.1"
|
||||
bundle_ip = f"192.168.{octet}.2"
|
||||
return subnet, gateway, bundle_ip
|
||||
@@ -105,9 +105,9 @@ class BottleMetadata:
|
||||
# written before chunk 3 (resume / inspect should fall back to
|
||||
# deriving from identity in that case).
|
||||
compose_project: str = ""
|
||||
# PRD 0040: backend name ("docker" or "smolmachines"). Empty string
|
||||
# for state dirs written before PRD 0040; callers default to "docker"
|
||||
# for backward compatibility.
|
||||
# PRD 0040: backend name ("docker", "firecracker", "macos-container").
|
||||
# Empty string for state dirs written before PRD 0040; callers default
|
||||
# to "docker" for backward compatibility.
|
||||
backend: str = ""
|
||||
label: str = ""
|
||||
color: str = ""
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
"""Main CLI dispatcher.
|
||||
|
||||
Commands: cleanup, commit, edit, info, init, list, resume, start, supervise
|
||||
Commands: backend, cleanup, commit, edit, info, init, list, resume, start, supervise
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -13,6 +13,7 @@ from ..manifest import ManifestError
|
||||
from ..store_manager import StoreManager
|
||||
from ._common import PROG
|
||||
from . import list as _list_mod
|
||||
from .backend import cmd_backend
|
||||
from .cleanup import cmd_cleanup
|
||||
from .commit import cmd_commit
|
||||
from .edit import cmd_edit
|
||||
@@ -25,6 +26,7 @@ from .supervise import cmd_supervise
|
||||
cmd_list = _list_mod.cmd_list
|
||||
|
||||
COMMANDS = {
|
||||
"backend": cmd_backend,
|
||||
"cleanup": cmd_cleanup,
|
||||
"commit": cmd_commit,
|
||||
"edit": cmd_edit,
|
||||
@@ -40,6 +42,7 @@ COMMANDS = {
|
||||
def usage() -> None:
|
||||
sys.stderr.write(f"usage: {PROG} <command> [args...]\n\n")
|
||||
sys.stderr.write("Commands:\n")
|
||||
sys.stderr.write(" backend set up or check a backend's host prerequisites (setup|status)\n")
|
||||
sys.stderr.write(" cleanup stop and remove all active bot-bottle containers\n")
|
||||
sys.stderr.write(" commit snapshot a running bottle's container state to a Docker image\n")
|
||||
sys.stderr.write(" edit open an agent in vim for editing\n")
|
||||
|
||||
@@ -0,0 +1,41 @@
|
||||
"""`backend` CLI command — generic host setup/status across backends.
|
||||
|
||||
`./cli.py backend setup [--backend=NAME]` provisions (or points at how
|
||||
to provision) the chosen backend's one-time host prerequisites.
|
||||
`./cli.py backend status [--backend=NAME]` reports readiness.
|
||||
|
||||
Both dispatch to the backend's `setup()` / `status()` classmethods, so
|
||||
there are no backend-specific commands — swapping backends is just a
|
||||
different `--backend` (or `$BOT_BOTTLE_BACKEND`, or the host default).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
|
||||
from ..backend import get_bottle_backend, known_backend_names
|
||||
from ._common import PROG
|
||||
|
||||
|
||||
def cmd_backend(args: list[str]) -> int:
|
||||
parser = argparse.ArgumentParser(
|
||||
prog=f"{PROG} backend",
|
||||
description="Set up or check a backend's host prerequisites.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"action",
|
||||
choices=("setup", "status"),
|
||||
help="setup: provision/print host prerequisites; status: report readiness",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--backend",
|
||||
choices=known_backend_names(),
|
||||
default=None,
|
||||
help="backend to target (default: $BOT_BOTTLE_BACKEND or the host default)",
|
||||
)
|
||||
ns = parser.parse_args(args)
|
||||
|
||||
backend = get_bottle_backend(ns.backend)
|
||||
if ns.action == "setup":
|
||||
return backend.setup()
|
||||
return backend.status()
|
||||
@@ -1,14 +1,14 @@
|
||||
"""cleanup: stop and remove all orphaned bot-bottle resources.
|
||||
|
||||
Walks every registered backend (docker + smolmachines) so a single
|
||||
`./cli.py cleanup` reaps both backends' leftovers — orphaned
|
||||
smolvm machines won't survive a docker-only cleanup pass (issue
|
||||
addressed alongside #77).
|
||||
Walks every registered backend (docker, firecracker, macos-container)
|
||||
so a single `./cli.py cleanup` reaps every backend's leftovers — a
|
||||
firecracker bottle's sidecars won't survive a docker-only cleanup pass
|
||||
(issue addressed alongside #77).
|
||||
|
||||
Each backend's `prepare_cleanup` enumerates its own resources;
|
||||
docker's `_list_orphan_state_dirs` consults
|
||||
`enumerate_active_agents()` for the union of live identities so
|
||||
state dirs of running smolmachines bottles aren't reaped. State
|
||||
state dirs of running non-docker bottles aren't reaped. State
|
||||
dirs are shared layout, so docker is the single owner of that
|
||||
bucket.
|
||||
|
||||
|
||||
@@ -2,10 +2,10 @@
|
||||
|
||||
Docker bottles are committed to a local Docker image. Macos-container
|
||||
bottles are exported and rebuilt as a local Apple Container image.
|
||||
Smolmachines bottles are packed from the running VM into a
|
||||
`.smolmachine` artifact. The resulting reference is stored in
|
||||
per-bottle state so the next `./cli.py resume <slug>` boots from the
|
||||
snapshot instead of rebuilding from the Dockerfile.
|
||||
Firecracker bottles stream the guest rootfs out over SSH and rebuild a
|
||||
local Docker image. The resulting reference is stored in per-bottle
|
||||
state so the next `./cli.py resume <slug>` boots from the snapshot
|
||||
instead of rebuilding from the Dockerfile.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -45,8 +45,9 @@ def cmd_list(argv: list[str]) -> int:
|
||||
print(name)
|
||||
return 0
|
||||
|
||||
# `active` enumerates every backend (docker + smolmachines)
|
||||
# so smolmachines bottles aren't hidden behind the env var.
|
||||
# `active` enumerates every backend (docker, firecracker,
|
||||
# macos-container) so non-docker bottles aren't hidden behind
|
||||
# the env var.
|
||||
active = enumerate_active_agents()
|
||||
if not active:
|
||||
print("no active bot-bottle bottles", file=sys.stderr)
|
||||
|
||||
@@ -28,9 +28,6 @@ from ..backend.docker.egress_apply import (
|
||||
from ..backend.macos_container.egress_apply import (
|
||||
applicator as _macos_applicator,
|
||||
)
|
||||
from ..backend.smolmachines.egress_apply import (
|
||||
applicator as _smolmachines_applicator,
|
||||
)
|
||||
from ..log import Die, error, info
|
||||
|
||||
from ..supervise import (
|
||||
@@ -79,8 +76,6 @@ def apply_routes_change(slug: str, content: str) -> tuple[str, str]:
|
||||
backend = meta.backend if meta is not None else ""
|
||||
if backend == "macos-container":
|
||||
return _macos_applicator.apply_routes_change(slug, content)
|
||||
if backend == "smolmachines":
|
||||
return _smolmachines_applicator.apply_routes_change(slug, content)
|
||||
return _docker_applicator.apply_routes_change(slug, content)
|
||||
|
||||
|
||||
|
||||
@@ -37,8 +37,8 @@ if [ -n "$EGRESS_UPSTREAM_PROXY" ]; then
|
||||
fi
|
||||
|
||||
# Bind address. Docker backend wants `0.0.0.0` (agent dials egress
|
||||
# directly via the docker network alias). Smolmachines backend
|
||||
# uses EGRESS_LISTEN_HOST when a non-default binding is needed.
|
||||
# directly via the docker network alias). A VM backend uses
|
||||
# EGRESS_LISTEN_HOST when a non-default binding is needed.
|
||||
LISTEN_HOST_FLAG=""
|
||||
if [ -n "$EGRESS_LISTEN_HOST" ]; then
|
||||
LISTEN_HOST_FLAG="--listen-host $EGRESS_LISTEN_HOST"
|
||||
|
||||
@@ -76,14 +76,14 @@ def git_gate_render_gitconfig(
|
||||
entries: tuple[ManifestGitEntry, ...], gate_host: str, *, scheme: str = "git",
|
||||
) -> str:
|
||||
"""Render the agent's ~/.gitconfig content for git-gate
|
||||
`insteadOf` rewrites. Pure host-side, no docker / smolvm;
|
||||
`insteadOf` rewrites. Pure host-side, no docker / VM;
|
||||
exposed for tests + reuse across backends.
|
||||
|
||||
`gate_host` is the part of the URL between `<scheme>://` and the
|
||||
repo path — backends differ here:
|
||||
- docker: `git-gate` (the short network alias)
|
||||
- smolmachines: `<bundle_ip>:<port>` (no DNS in the
|
||||
TSI-allowlisted guest)
|
||||
- firecracker: `<bundle_ip>:<port>` (no DNS on the
|
||||
point-to-point TAP link)
|
||||
|
||||
Empty `entries` returns an empty string so callers can no-op
|
||||
cleanly without conditional formatting at the call site."""
|
||||
|
||||
@@ -1,8 +1,9 @@
|
||||
"""Tiny smart-HTTP wrapper for git-gate repos.
|
||||
|
||||
Used by the smolmachines backend where `git://` push traffic over the
|
||||
host-published Docker port can hang before receive-pack reaches hooks.
|
||||
The wrapper serves the same `/git/*.git` bare repos through
|
||||
Used where `git://` push traffic over a host-published Docker port can
|
||||
hang before receive-pack reaches hooks (e.g. the firecracker backend,
|
||||
where the guest reaches the sidecar over the point-to-point TAP). The
|
||||
wrapper serves the same `/git/*.git` bare repos through
|
||||
`git http-backend`, so pre-receive and upstream forwarding remain the
|
||||
git-gate enforcement point.
|
||||
"""
|
||||
|
||||
@@ -85,7 +85,7 @@ SUPERVISE_PORT = 9100
|
||||
# tool. The hostname + port match egress's docker network
|
||||
# listen port (see backend.docker.egress.EGRESS_PORT). The supervise
|
||||
# daemon runs inside the sidecar bundle alongside egress, so loopback
|
||||
# is the stable address across docker, smolmachines, and Apple
|
||||
# is the stable address across docker, firecracker, and Apple
|
||||
# Container backends.
|
||||
EGRESS_FORWARD_PROXY = "http://127.0.0.1:9099"
|
||||
EGRESS_INTROSPECT_URL = "http://_egress.local/allowlist"
|
||||
|
||||
@@ -1,14 +1,20 @@
|
||||
# Landscape: containerized Claude Code agent tools
|
||||
# Landscape: containerized AI coding agent tools
|
||||
|
||||
Research into whether bot-bottle is redundant with existing projects, and
|
||||
whether it's worth publishing.
|
||||
|
||||
## Summary
|
||||
|
||||
The "Claude Code in Docker" space is active but not saturated. bot-bottle
|
||||
occupies a distinct position: no surveyed project combines all five of its
|
||||
defining features. Publishing is likely worthwhile, with the main risk being
|
||||
claudebox expanding to absorb the same niche.
|
||||
The "AI coding agents in isolated sandboxes" space is active but not saturated.
|
||||
bot-bottle occupies a distinct position: no surveyed project combines all five
|
||||
of its defining features. Publishing is likely worthwhile, with the main risk
|
||||
being claudebox expanding to absorb the same niche.
|
||||
|
||||
**Updated 2026-07-09:** bot-bottle now supports three isolation backends
|
||||
(Docker, Apple `container`, smolmachines/libkrun microVMs) and three built-in
|
||||
agent providers (Claude Code, OpenAI Codex, Pi) with an open plugin system for
|
||||
arbitrary providers. This meaningfully strengthens the differentiation against
|
||||
all surveyed competitors.
|
||||
|
||||
## Closest competitor: claudebox
|
||||
|
||||
@@ -43,28 +49,78 @@ manifest merge.
|
||||
Still marked early-development.
|
||||
- **E2B, Northflank, Cloudflare Sandbox SDK** — cloud-hosted SaaS sandbox
|
||||
runtimes; fundamentally different architecture.
|
||||
- **superhq.ai / SuperHQ** (v0.4.4, April 2026) — macOS desktop app (Rust/GPUI)
|
||||
that runs Claude Code, Codex, and Pi inside microVMs via Apple's
|
||||
Virtualization.framework (their own shuru-sdk / libkrun). Auth gateway
|
||||
injects API keys on the wire so the sandbox never sees them; tmpfs overlay
|
||||
stages agent writes for diff-and-accept review; mobile remote access via
|
||||
remote.superhq.ai. Early alpha, free on launch, Apple Silicon only.
|
||||
|
||||
Overlap: both projects cover agent isolation, credential proxying, and
|
||||
multi-provider support (Claude Code / Codex / Pi). Differences: SuperHQ is a
|
||||
GUI desktop app with no manifest layer; bot-bottle is a CLI fleet manager with
|
||||
named agents, skills injection, per-agent system prompts, and cross-platform
|
||||
backends (Docker, Apple `container`, smolmachines). SuperHQ's microVM
|
||||
isolation story is now partially matched by bot-bottle's `macos_container` and
|
||||
smolmachines backends. Worth watching — it targets the same security-minded
|
||||
power-user audience and moves fast.
|
||||
|
||||
**Known gap in SuperHQ (user-requested, as of 2026-07-09):** A named user
|
||||
(Brian Cheong, Founder, Dunialabs.io) explicitly called out the absence of
|
||||
per-run audit logging: tool calls and network egress. Bot-bottle covers both:
|
||||
network egress is logged by pipelock/mitmproxy, and per-run op-log/audit state
|
||||
is persisted to SQLite.
|
||||
|
||||
## What no found project does
|
||||
|
||||
None combine:
|
||||
1. Named-agent JSON manifest with per-agent env resolution (prompt / host-forward / literal)
|
||||
2. Claude Code skills directory injection
|
||||
1. Named-agent manifest with per-agent env resolution (prompt / host-forward / literal), supporting multiple providers (Claude Code, Codex, Pi, arbitrary plugins)
|
||||
2. Skills directory injection
|
||||
3. Per-agent system prompts
|
||||
4. SSH-agent key forwarding without copying private keys into the container
|
||||
5. Home + project manifest merge
|
||||
6. Pluggable isolation backends: Docker (Linux/macOS), Apple `container` (macOS microVMs), smolmachines/libkrun microVMs
|
||||
7. Per-run audit log: network egress via pipelock/mitmproxy + op-log persisted to SQLite
|
||||
|
||||
**In-flight directions (not yet shipped):**
|
||||
|
||||
- **Forge-native dispatch (issue #317):** Gitea webhook → orchestrator spins up a bottle
|
||||
with the issue body as prompt → agent works → bottle freezes awaiting review comment →
|
||||
rehydrates on comment → tears down on PR close. The issue-to-PR lifecycle concept is not
|
||||
novel (Devin, Copilot Workspace, SWE-agent all do this as cloud services); what's
|
||||
distinct is doing it self-hosted, manifest-driven, inside bot-bottle's isolation
|
||||
primitives.
|
||||
- **Paid web control plane (issue #327):** Browser-based multi-host agent launch and
|
||||
monitoring; account-scoped bottle and agent definitions; secret custody (encrypted at
|
||||
rest, injected into the sidecar at launch, never exposed to the agent or returned by any
|
||||
read API). Monetization model: OSS runtime free, control plane paid — a standard split
|
||||
(HashiCorp, Grafana) applied to a self-hosted agent sandbox. The principled secret
|
||||
custody model (agent never sees real credentials, even via printenv) is more rigorous
|
||||
than most surveyed tools but not unprecedented.
|
||||
|
||||
## Publishing verdict
|
||||
|
||||
Worth publishing. Differentiators that matter to the target audience (power
|
||||
users running parallel Claude Code sessions with distinct personas/tooling):
|
||||
users running parallel AI coding agent sessions with distinct personas/tooling):
|
||||
|
||||
- The Python-stdlib-first, low-dependency design — competitors are npm-based or
|
||||
Kubernetes-native.
|
||||
- The Python-stdlib-first, low-dependency design — competitors are npm-based,
|
||||
Rust/GUI, or Kubernetes-native.
|
||||
- Named agents with distinct skills and system prompts, not just language profiles.
|
||||
- Multi-backend isolation: Docker, Apple `container` microVMs, and
|
||||
smolmachines/libkrun — single manifest works across all three.
|
||||
- Multi-provider: Claude Code, Codex, Pi, plus an open plugin system for
|
||||
arbitrary providers.
|
||||
- SSH forwarding without key copying.
|
||||
- Per-run audit log (tool calls + network egress) — an explicitly requested gap
|
||||
in SuperHQ as of 2026-07-09.
|
||||
- Forge-native dispatch and a paid control plane (in flight) bring bot-bottle
|
||||
into the same product category as cloud services like Devin and Copilot
|
||||
Workspace — but self-hosted, with stronger isolation guarantees and a
|
||||
manifest-driven fleet model those services don't have.
|
||||
|
||||
Main risk: claudebox adds manifest/agent config. The space is moving fast
|
||||
enough that publishing sooner is better if establishing prior art matters.
|
||||
Main risk: claudebox adds manifest/agent config; SuperHQ is moving fast on the
|
||||
GUI / microVM side. The space is moving fast enough that publishing sooner is
|
||||
better if establishing prior art matters.
|
||||
|
||||
Discovery will be slow without active promotion; an Anthropic Discord post or
|
||||
HN "Show HN" would do most of the work.
|
||||
@@ -73,4 +129,4 @@ HN "Show HN" would do most of the work.
|
||||
|
||||
- GitHub search cannot surface private or very new repos comprehensively.
|
||||
- Counts (stars, forks) were not confirmed for every project.
|
||||
- Research conducted 2026-05-07; the space moves fast.
|
||||
- Initial research conducted 2026-05-07; SuperHQ entry added 2026-07-09; the space moves fast.
|
||||
|
||||
@@ -13,6 +13,6 @@ agent_provider:
|
||||
|
||||
Common Claude provider boundary. Drop this file into
|
||||
`~/.bot-bottle/bottles/claude.md`, then extend it from task-specific
|
||||
bottles. The default smolmachines backend keeps DNS resolution under
|
||||
the VM-layer egress policy; use `BOT_BOTTLE_BACKEND=docker` only for
|
||||
legacy Docker-backed runs.
|
||||
bottles. On a KVM Linux host the default Firecracker backend confines
|
||||
the guest behind a fail-closed nftables boundary; use
|
||||
`BOT_BOTTLE_BACKEND=docker` only for legacy Docker-backed runs.
|
||||
|
||||
@@ -11,4 +11,5 @@ The `dev` bottle — backs a generic development workflow.
|
||||
Inherits the Claude provider boundary from `claude`. Drop this file
|
||||
into `~/.bot-bottle/bottles/dev.md` and any agent referencing
|
||||
`bottle: dev` will launch against this infrastructure. By default,
|
||||
bot-bottle runs this bottle on the smolmachines backend.
|
||||
bot-bottle runs this bottle on the host's default backend (Firecracker
|
||||
on KVM Linux, Apple Container on macOS).
|
||||
|
||||
@@ -0,0 +1,18 @@
|
||||
{
|
||||
description = "bot-bottle — sandboxed runtime for AI coding agents";
|
||||
|
||||
outputs = { self, ... }: {
|
||||
# Declarative host setup for the Firecracker backend's network pool.
|
||||
# Consume from a flake-based NixOS config:
|
||||
#
|
||||
# inputs.bot-bottle.url = "git+ssh://<your-bot-bottle-remote>"; # or path:/…
|
||||
# # then, in your host module:
|
||||
# imports = [ inputs.bot-bottle.nixosModules.firecracker-netpool ];
|
||||
# services.bot-bottle-firecracker = { enable = true; owner = "you"; };
|
||||
#
|
||||
# The module is plain (no nixpkgs pin), so channel users can import
|
||||
# ./nix/firecracker-netpool.nix directly without the flake.
|
||||
nixosModules.firecracker-netpool = import ./nix/firecracker-netpool.nix;
|
||||
nixosModules.default = self.nixosModules.firecracker-netpool;
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,152 @@
|
||||
# bot-bottle Firecracker network pool — declarative NixOS module.
|
||||
#
|
||||
# The one-time privileged setup the Firecracker backend needs: a pool of
|
||||
# user-owned point-to-point TAP devices plus a fail-closed nftables table
|
||||
# that confines every microVM to its own sidecar. Enabling this module is
|
||||
# the NixOS equivalent of `sudo ./scripts/firecracker-netpool.sh up` — but
|
||||
# declarative, so the taps/table survive `nixos-rebuild` and are only
|
||||
# reconfigured when this config changes (running VMs aren't dropped).
|
||||
#
|
||||
# The option defaults MUST match the backend constants in
|
||||
# bot_bottle/backend/firecracker/netpool.py (pool size, IP base, iface
|
||||
# prefix, table name). The backend reads the same values at launch from
|
||||
# the BOT_BOTTLE_FC_* env vars; if you override an option here, override
|
||||
# the matching env var for the CLI too (or set writeEnvFile = true below
|
||||
# to have this module emit them). Keep the two sides in lockstep.
|
||||
{ config, lib, ... }:
|
||||
|
||||
let
|
||||
cfg = config.services.bot-bottle-firecracker;
|
||||
|
||||
# --- IPv4 <-> int (full 32-bit carry math) -------------------------
|
||||
toOctets = s: map lib.toInt (lib.splitString "." s);
|
||||
ipToInt = s:
|
||||
let o = toOctets s; in
|
||||
(lib.elemAt o 0) * 16777216
|
||||
+ (lib.elemAt o 1) * 65536
|
||||
+ (lib.elemAt o 2) * 256
|
||||
+ (lib.elemAt o 3);
|
||||
intToIp = n:
|
||||
let
|
||||
b0 = n / 16777216; r0 = n - b0 * 16777216;
|
||||
b1 = r0 / 65536; r1 = r0 - b1 * 65536;
|
||||
b2 = r1 / 256;
|
||||
b3 = r1 - b2 * 256;
|
||||
in "${toString b0}.${toString b1}.${toString b2}.${toString b3}";
|
||||
|
||||
baseInt = ipToInt cfg.ipBase;
|
||||
|
||||
# Slot i: host = base + 2i (the VM's gateway), on its own /31.
|
||||
slots = lib.genList (i: {
|
||||
iface = "${cfg.ifacePrefix}${toString i}";
|
||||
hostIp = intToIp (baseInt + 2 * i);
|
||||
}) cfg.poolSize;
|
||||
|
||||
mkAttr = f: lib.listToAttrs (map (s: lib.nameValuePair "10-${s.iface}" (f s)) slots);
|
||||
in
|
||||
{
|
||||
options.services.bot-bottle-firecracker = {
|
||||
enable = lib.mkEnableOption "the bot-bottle Firecracker network pool (TAP devices + isolation nftables table)";
|
||||
|
||||
poolSize = lib.mkOption {
|
||||
type = lib.types.ints.positive;
|
||||
default = 8;
|
||||
description = "Number of pool slots (concurrent bottles). Must match BOT_BOTTLE_FC_POOL_SIZE.";
|
||||
};
|
||||
|
||||
ipBase = lib.mkOption {
|
||||
type = lib.types.str;
|
||||
default = "10.243.0.0";
|
||||
description = ''
|
||||
Base IPv4 of the /31 pool; slot i uses host = base + 2i. Must
|
||||
be /31-aligned (even final address) and must match
|
||||
BOT_BOTTLE_FC_IP_BASE. Default is an obscure RFC-1918 /16 that
|
||||
dodges docker/libvirt/k8s/LAN and, deliberately, Tailscale's
|
||||
100.64.0.0/10 CGNAT range.
|
||||
'';
|
||||
};
|
||||
|
||||
ifacePrefix = lib.mkOption {
|
||||
type = lib.types.str;
|
||||
default = "bbfc";
|
||||
description = "TAP interface name prefix. Must match BOT_BOTTLE_FC_IFACE_PREFIX.";
|
||||
};
|
||||
|
||||
owner = lib.mkOption {
|
||||
type = lib.types.str;
|
||||
example = "alice";
|
||||
description = "User that owns the TAP devices, so `./cli.py start` opens them without root.";
|
||||
};
|
||||
|
||||
tableName = lib.mkOption {
|
||||
type = lib.types.str;
|
||||
default = "bot_bottle_fc";
|
||||
description = "nftables table name for the isolation boundary. Must match netpool.NFT_TABLE.";
|
||||
};
|
||||
|
||||
writeEnvFile = lib.mkOption {
|
||||
type = lib.types.bool;
|
||||
default = false;
|
||||
description = ''
|
||||
When true, write /etc/bot-bottle/firecracker.env with the
|
||||
matching BOT_BOTTLE_FC_* values, so the host pool and the CLI
|
||||
launcher can't drift. Source it before running `./cli.py`.
|
||||
'';
|
||||
};
|
||||
};
|
||||
|
||||
config = lib.mkIf cfg.enable {
|
||||
assertions = [
|
||||
{
|
||||
assertion = lib.mod baseInt 2 == 0;
|
||||
message = "services.bot-bottle-firecracker.ipBase must be /31-aligned (even final octet); got ${cfg.ipBase}.";
|
||||
}
|
||||
];
|
||||
|
||||
# VM->sidecar traffic is DNAT'd and forwarded, so forwarding must be on.
|
||||
boot.kernel.sysctl."net.ipv4.ip_forward" = 1;
|
||||
|
||||
systemd.network.enable = true;
|
||||
systemd.network.netdevs = mkAttr (s: {
|
||||
netdevConfig = { Name = s.iface; Kind = "tap"; };
|
||||
tapConfig = { User = cfg.owner; };
|
||||
});
|
||||
systemd.network.networks = mkAttr (s: {
|
||||
matchConfig.Name = s.iface;
|
||||
address = [ "${s.hostIp}/31" ];
|
||||
networkConfig.ConfigureWithoutCarrier = true;
|
||||
});
|
||||
|
||||
# Independent, fail-closed table — coexists with Docker/ufw/firewalld
|
||||
# rather than replacing the ruleset. A VM (traffic from a
|
||||
# ${cfg.ifacePrefix}* TAP) may reach only its own sidecar (DNAT'd from
|
||||
# the host TAP IP); everything else is dropped.
|
||||
networking.nftables.enable = true;
|
||||
networking.nftables.tables.${cfg.tableName} = {
|
||||
family = "inet";
|
||||
content = ''
|
||||
chain forward {
|
||||
type filter hook forward priority -10; policy accept;
|
||||
iifname != "${cfg.ifacePrefix}*" return
|
||||
ct state established,related accept
|
||||
ct status dnat accept
|
||||
drop
|
||||
}
|
||||
chain input {
|
||||
type filter hook input priority -10; policy accept;
|
||||
iifname != "${cfg.ifacePrefix}*" return
|
||||
ct state established,related accept
|
||||
drop
|
||||
}
|
||||
'';
|
||||
};
|
||||
|
||||
environment.etc."bot-bottle/firecracker.env" = lib.mkIf cfg.writeEnvFile {
|
||||
text = ''
|
||||
BOT_BOTTLE_FC_POOL_SIZE=${toString cfg.poolSize}
|
||||
BOT_BOTTLE_FC_IP_BASE=${cfg.ipBase}
|
||||
BOT_BOTTLE_FC_IFACE_PREFIX=${cfg.ifacePrefix}
|
||||
'';
|
||||
};
|
||||
};
|
||||
}
|
||||
Executable
+162
@@ -0,0 +1,162 @@
|
||||
#!/usr/bin/env bash
|
||||
# One-time privileged network setup for the Firecracker backend.
|
||||
#
|
||||
# Creates a pool of point-to-point TAP devices (owned by the invoking
|
||||
# user so the backend can open them without root at launch) and a
|
||||
# dedicated nftables table that isolates every VM: a bottle VM can
|
||||
# reach only its own sidecar (published on the host-side TAP IP) and
|
||||
# nothing else on the host or network.
|
||||
#
|
||||
# Why a pool + one-time setup: creating a TAP and assigning it an IP
|
||||
# needs CAP_NET_ADMIN. Pre-creating user-owned, pre-addressed TAPs
|
||||
# means `./cli.py start` never needs root. The nft table is static
|
||||
# (keyed on the `bbfc*` interface wildcard), so it covers every slot
|
||||
# without per-launch changes.
|
||||
#
|
||||
# Design notes:
|
||||
# * No shared bridge — each slot is an isolated /31 host<->guest link,
|
||||
# so there are no bridge name/subnet collisions with docker0,
|
||||
# virbr0 (libvirt), cni0 (k8s) or br-* (docker networks).
|
||||
# * Own nftables table `bot_bottle_fc` — independent of the iptables
|
||||
# filter/nat tables Docker/ufw/firewalld use, so nothing is stomped.
|
||||
# * Default IP block is an obscure RFC-1918 /16 (10.243.0.0/16),
|
||||
# chosen to dodge the usual occupants (docker 172.17-31, libvirt
|
||||
# 192.168.122, k8s 10.42/10.244, home LANs). NOT 100.64.0.0/10 —
|
||||
# that's RFC-6598 CGNAT, which Tailscale hands node addresses from.
|
||||
#
|
||||
# NixOS: imperative rules here do NOT survive nixos-rebuild. Use the
|
||||
# declarative module in nix/firecracker-netpool.nix instead (exposed as
|
||||
# the flake output nixosModules.firecracker-netpool).
|
||||
#
|
||||
# Usage:
|
||||
# sudo ./scripts/firecracker-netpool.sh up
|
||||
# sudo ./scripts/firecracker-netpool.sh down
|
||||
# ./scripts/firecracker-netpool.sh status
|
||||
#
|
||||
# Env overrides (must match the backend's util.py constants):
|
||||
# BOT_BOTTLE_FC_POOL_SIZE number of slots (default 8)
|
||||
# BOT_BOTTLE_FC_IP_BASE base IPv4 of the /31 pool (default 10.243.0.0)
|
||||
# BOT_BOTTLE_FC_IFACE_PREFIX TAP name prefix (default bbfc)
|
||||
# BOT_BOTTLE_FC_OWNER owning user (default $SUDO_USER or $USER)
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
POOL_SIZE="${BOT_BOTTLE_FC_POOL_SIZE:-8}"
|
||||
IP_BASE="${BOT_BOTTLE_FC_IP_BASE:-10.243.0.0}"
|
||||
PREFIX="${BOT_BOTTLE_FC_IFACE_PREFIX:-bbfc}"
|
||||
OWNER="${BOT_BOTTLE_FC_OWNER:-${SUDO_USER:-$USER}}"
|
||||
TABLE="bot_bottle_fc"
|
||||
|
||||
# Sidecar ports (must match the backend). egress=9099, supervise=9100,
|
||||
# git-http=9420. Reached by the VM at its host-side TAP IP.
|
||||
SIDECAR_PORTS="9099,9100,9420"
|
||||
|
||||
# --- IP math ---------------------------------------------------------
|
||||
# Slot i occupies the /31 {base+2i, base+2i+1}: host = base+2i (the
|
||||
# gateway the VM routes through), guest = base+2i+1 (the VM's address).
|
||||
_ip_to_int() {
|
||||
local IFS=. ; read -r a b c d <<<"$1" ; echo $(( (a<<24) + (b<<16) + (c<<8) + d ))
|
||||
}
|
||||
_int_to_ip() {
|
||||
local n=$1 ; echo "$(( (n>>24)&255 )).$(( (n>>16)&255 )).$(( (n>>8)&255 )).$(( n&255 ))"
|
||||
}
|
||||
host_ip() { _int_to_ip $(( $(_ip_to_int "$IP_BASE") + 2*$1 )); }
|
||||
guest_ip() { _int_to_ip $(( $(_ip_to_int "$IP_BASE") + 2*$1 + 1 )); }
|
||||
iface() { echo "${PREFIX}$1"; }
|
||||
|
||||
require_root() {
|
||||
if [ "$(id -u)" -ne 0 ]; then
|
||||
echo "error: '$1' needs root (run under sudo)" >&2
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
cmd_up() {
|
||||
require_root up
|
||||
echo "firecracker net pool: $POOL_SIZE slots, base $IP_BASE, owner $OWNER"
|
||||
|
||||
# VM->sidecar traffic is DNAT'd to the sidecar container and
|
||||
# forwarded, so forwarding must be enabled (Docker also sets this).
|
||||
sysctl -qw net.ipv4.ip_forward=1
|
||||
|
||||
for i in $(seq 0 $((POOL_SIZE-1))); do
|
||||
local dev host
|
||||
dev="$(iface "$i")" ; host="$(host_ip "$i")"
|
||||
if ip link show "$dev" >/dev/null 2>&1; then
|
||||
ip link set "$dev" down 2>/dev/null || true
|
||||
ip tuntap del dev "$dev" mode tap 2>/dev/null || true
|
||||
fi
|
||||
ip tuntap add dev "$dev" mode tap user "$OWNER"
|
||||
ip addr add "$host/31" dev "$dev"
|
||||
ip link set "$dev" up
|
||||
echo " $dev host=$host guest=$(guest_ip "$i") owner=$OWNER"
|
||||
done
|
||||
|
||||
_install_nft
|
||||
echo "nftables table inet $TABLE installed (fail-closed boundary)"
|
||||
echo "done."
|
||||
}
|
||||
|
||||
_install_nft() {
|
||||
# Own table: dropping only matches our bbfc* interfaces, so no other
|
||||
# tool's traffic is affected. Priority -10 runs before Docker's
|
||||
# filter hooks (priority 0); a drop here is terminal for the packet.
|
||||
#
|
||||
# forward: VM egress is DNAT'd to the sidecar (established via
|
||||
# `ct status dnat`); return traffic via `ct state established`.
|
||||
# Anything else from a VM is dropped -> no route to the internet
|
||||
# or the rest of the host except through the sidecar proxy.
|
||||
# input: a VM never needs host-local delivery (its sidecar is
|
||||
# reached via DNAT->forward), so drop all direct input from VMs
|
||||
# -> host services bound on 0.0.0.0 are unreachable from the VM.
|
||||
nft -f - <<EOF
|
||||
table inet $TABLE {
|
||||
chain forward {
|
||||
type filter hook forward priority -10; policy accept;
|
||||
iifname != "${PREFIX}*" return
|
||||
ct state established,related accept
|
||||
ct status dnat accept
|
||||
drop
|
||||
}
|
||||
chain input {
|
||||
type filter hook input priority -10; policy accept;
|
||||
iifname != "${PREFIX}*" return
|
||||
ct state established,related accept
|
||||
drop
|
||||
}
|
||||
}
|
||||
EOF
|
||||
}
|
||||
|
||||
cmd_down() {
|
||||
require_root down
|
||||
nft delete table inet "$TABLE" 2>/dev/null || true
|
||||
for i in $(seq 0 $((POOL_SIZE-1))); do
|
||||
local dev ; dev="$(iface "$i")"
|
||||
if ip link show "$dev" >/dev/null 2>&1; then
|
||||
ip link set "$dev" down 2>/dev/null || true
|
||||
ip tuntap del dev "$dev" mode tap 2>/dev/null || true
|
||||
echo " removed $dev"
|
||||
fi
|
||||
done
|
||||
echo "done."
|
||||
}
|
||||
|
||||
cmd_status() {
|
||||
echo "table inet $TABLE:"
|
||||
nft list table inet "$TABLE" 2>/dev/null || echo " (absent)"
|
||||
echo "taps:"
|
||||
for i in $(seq 0 $((POOL_SIZE-1))); do
|
||||
local dev ; dev="$(iface "$i")"
|
||||
if ip -brief addr show "$dev" >/dev/null 2>&1; then
|
||||
ip -brief addr show "$dev" | sed 's/^/ /'
|
||||
fi
|
||||
done
|
||||
}
|
||||
|
||||
case "${1:-}" in
|
||||
up) cmd_up ;;
|
||||
down) cmd_down ;;
|
||||
status) cmd_status ;;
|
||||
*) echo "usage: $0 {up|down|status}" >&2 ; exit 2 ;;
|
||||
esac
|
||||
@@ -11,7 +11,7 @@ asserts each one is blocked:
|
||||
5. Secret exfil via README link pushed through git-gate
|
||||
|
||||
The suite is backend-agnostic — it goes through `get_bottle_backend()`
|
||||
so smolmachines can be tested by setting `BOT_BOTTLE_BACKEND=smolmachines`.
|
||||
so another backend can be tested by setting `BOT_BOTTLE_BACKEND`.
|
||||
When unset, this integration test pins Docker explicitly to preserve
|
||||
the Docker-backed CI path.
|
||||
|
||||
@@ -24,7 +24,6 @@ from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
@@ -70,27 +69,16 @@ class TestSandboxEscape(unittest.TestCase):
|
||||
_launch_cm = None # backend.launch context manager
|
||||
_bottle = None
|
||||
_identity: str = ""
|
||||
_backend_name: str = "docker"
|
||||
|
||||
@classmethod
|
||||
def setUpClass(cls) -> None:
|
||||
# Per-backend prerequisites. Docker is always required (both
|
||||
# backends use it — docker for the agent + sidecars, smolmachines
|
||||
# for the sidecar bundle); the class-level @skip_unless_docker
|
||||
# already covers that. Smolmachines additionally needs smolvm on
|
||||
# PATH and is macOS-only in v1 (libkrun/TSI). Skip cleanly when
|
||||
# those are missing rather than die-ing inside backend.prepare.
|
||||
backend_name = os.environ.get("BOT_BOTTLE_BACKEND", "docker")
|
||||
if backend_name == "smolmachines":
|
||||
if sys.platform not in ("darwin", "linux"):
|
||||
raise unittest.SkipTest(
|
||||
f"BOT_BOTTLE_BACKEND=smolmachines is not supported "
|
||||
f"on {sys.platform} (macOS and Linux only)"
|
||||
)
|
||||
if shutil.which("smolvm") is None:
|
||||
raise unittest.SkipTest(
|
||||
"BOT_BOTTLE_BACKEND=smolmachines requires `smolvm` "
|
||||
"on PATH: curl -sSL https://smolmachines.com/install.sh | sh"
|
||||
)
|
||||
# Docker is always required (the agent + sidecars run under it,
|
||||
# and VM backends still use it for the sidecar bundle); the
|
||||
# class-level @skip_unless_docker already covers that. Pin
|
||||
# Docker when BOT_BOTTLE_BACKEND is unset to preserve the
|
||||
# Docker-backed CI path.
|
||||
cls._backend_name = os.environ.get("BOT_BOTTLE_BACKEND", "docker")
|
||||
|
||||
# Throwaway static key for the git-gate fixture. It need not
|
||||
# be a real SSH key: test 5 reaches gitleaks before any SSH
|
||||
@@ -149,7 +137,7 @@ class TestSandboxEscape(unittest.TestCase):
|
||||
|
||||
cls._stage_dir = Path(tempfile.mkdtemp(prefix="sandbox-escape-stage."))
|
||||
try:
|
||||
backend = get_bottle_backend(backend_name)
|
||||
backend = get_bottle_backend(cls._backend_name)
|
||||
plan = backend.prepare(spec, stage_dir=cls._stage_dir)
|
||||
cls._identity = plan.slug
|
||||
|
||||
@@ -421,7 +409,7 @@ class TestSandboxEscape(unittest.TestCase):
|
||||
# ~/.gitconfig insteadOf rewrite (set up by provision_git)
|
||||
# redirects to the gate. This makes the test backend-
|
||||
# agnostic: docker resolves the gate via the short `git-gate`
|
||||
# alias, smolmachines via `<bundle_ip>:9418` — both
|
||||
# alias, a VM backend via `<bundle_ip>:9418` — both
|
||||
# transparent to the test through insteadOf.
|
||||
upstream_url = "ssh://git@unreachable.invalid:22/throwaway.git"
|
||||
|
||||
|
||||
@@ -1,111 +0,0 @@
|
||||
"""Integration: PRD 0023 chunk 2c — bundle bringup on a per-bottle
|
||||
docker bridge with the pinned IP.
|
||||
|
||||
End-to-end against the real docker daemon. Brings up just the
|
||||
sidecar bundle on its own bridge, confirms the container lands at
|
||||
the pinned IP, then tears down. Skipped under act_runner (docker
|
||||
socket mount topology breaks bridge visibility) and when the
|
||||
bundle image isn't available.
|
||||
|
||||
Full launch flow (smolvm + bundle + provisioning + the
|
||||
localhost-reach / egress-port-bypass probes) lives in chunk 2d."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
import time
|
||||
import unittest
|
||||
|
||||
from bot_bottle.backend.smolmachines.sidecar_bundle import (
|
||||
BundleLaunchSpec,
|
||||
bundle_container_name,
|
||||
bundle_network_name,
|
||||
create_bundle_network,
|
||||
remove_bundle_network,
|
||||
start_bundle,
|
||||
stop_bundle,
|
||||
)
|
||||
from tests._docker import skip_unless_docker
|
||||
|
||||
|
||||
@skip_unless_docker()
|
||||
@unittest.skipIf(
|
||||
os.environ.get("GITEA_ACTIONS") == "true",
|
||||
"skipped under act_runner: docker socket mount topology breaks "
|
||||
"in-process visibility of networks created on the host daemon",
|
||||
)
|
||||
class TestBundleBringup(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.slug = f"cb-test-bundle-{os.getpid()}-{int(time.time())}"
|
||||
self.network = bundle_network_name(self.slug)
|
||||
self.container = bundle_container_name(self.slug)
|
||||
|
||||
def tearDown(self):
|
||||
stop_bundle(self.slug)
|
||||
remove_bundle_network(self.network)
|
||||
|
||||
def _bundle_image_built(self) -> bool:
|
||||
"""The bundle image (`bot-bottle-sidecars:latest`) is
|
||||
built lazily by the docker backend's compose. If a
|
||||
smolmachines-only operator hasn't run the docker backend
|
||||
first, the image won't exist locally. Skip rather than
|
||||
fail."""
|
||||
r = subprocess.run(
|
||||
["docker", "image", "inspect", "bot-bottle-sidecars:latest"],
|
||||
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
)
|
||||
return r.returncode == 0
|
||||
|
||||
def test_create_network_then_start_bundle_pins_ip(self):
|
||||
if not self._bundle_image_built():
|
||||
self.skipTest(
|
||||
"bot-bottle-sidecars:latest not built; run a docker "
|
||||
"bottle first or `docker build -f Dockerfile.sidecars .`"
|
||||
)
|
||||
|
||||
# Pick a subnet unlikely to collide on the host. Last
|
||||
# octet of the slug hash isn't deterministic across runs;
|
||||
# we hardcode a high octet (.211) that the docker default
|
||||
# bridges almost never use.
|
||||
subnet = "192.168.211.0/24"
|
||||
gateway = "192.168.211.1"
|
||||
bundle_ip = "192.168.211.2"
|
||||
|
||||
create_bundle_network(self.network, subnet, gateway)
|
||||
|
||||
spec = BundleLaunchSpec(
|
||||
slug=self.slug,
|
||||
network_name=self.network,
|
||||
subnet=subnet,
|
||||
gateway=gateway,
|
||||
bundle_ip=bundle_ip,
|
||||
# Empty daemons_csv → init exits "no daemons selected"
|
||||
# immediately. We just need the container to land on
|
||||
# the network at the right IP before it exits.
|
||||
daemons_csv="", # empty → init exits "no daemons selected"
|
||||
)
|
||||
start_bundle(spec)
|
||||
|
||||
# Inspect the container's IP on the per-bottle network.
|
||||
r = subprocess.run(
|
||||
["docker", "inspect",
|
||||
"--format",
|
||||
"{{(index .NetworkSettings.Networks \"" + self.network + "\").IPAddress}}",
|
||||
self.container],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
# Container may have exited (no daemons selected → exit 0).
|
||||
# The inspect still works on exited containers as long as
|
||||
# `--rm` hasn't fired yet, which is a race. Even if it has,
|
||||
# the launch succeeded — the container existed, on the
|
||||
# right network, at the right IP. We don't fail here on
|
||||
# missing inspect.
|
||||
if r.returncode == 0 and r.stdout.strip():
|
||||
self.assertEqual(bundle_ip, r.stdout.strip(),
|
||||
f"bundle landed at wrong IP: {r.stdout!r}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,223 +0,0 @@
|
||||
"""Integration: PRD 0023 chunk 2d — end-to-end launch + exec
|
||||
round trip + the acceptance probes.
|
||||
|
||||
The smoke confirms the launch flow (per-bottle docker bridge →
|
||||
sidecar bundle with host-loopback published ports → smolvm guest
|
||||
with TSI allowlist → exec) plumbs together end to end. The probes confirm the
|
||||
security properties the design pivot was about:
|
||||
|
||||
- **localhost-reach probe** — guest tries to dial a service
|
||||
bound on the host's `127.0.0.1`. TSI's per-bottle loopback
|
||||
alias allowlist must refuse the connect.
|
||||
|
||||
- **egress proxy probe** — guest reaches the egress proxy through
|
||||
the injected `HTTPS_PROXY`/`HTTP_PROXY` URL on the per-bottle
|
||||
loopback alias, while direct egress with proxy vars unset fails.
|
||||
|
||||
- **egress-port-bypass probe** — guest tries to dial
|
||||
`<bundle-ip>:9099` (egress's port). TSI permits the IP but
|
||||
the bundle's egress daemon binds `127.0.0.1` inside its
|
||||
container, so the connect refuses at the socket level. The
|
||||
bind-address mitigation is what closes TSI's port-granularity
|
||||
gap.
|
||||
|
||||
Gated on macOS/Linux + smolvm + docker + not GITEA_ACTIONS — the
|
||||
runner can't host libkrun-backed VMs."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import platform
|
||||
import shutil
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
from bot_bottle.backend import BottleSpec, get_bottle_backend
|
||||
from bot_bottle.backend.smolmachines.smolvm import is_available as _smolvm_available
|
||||
from bot_bottle.manifest import ManifestIndex
|
||||
from tests._docker import skip_unless_docker
|
||||
|
||||
|
||||
_AGENT_PROMPT = "You are demo. Be brief."
|
||||
|
||||
|
||||
def _minimal_manifest() -> ManifestIndex:
|
||||
return ManifestIndex.from_json_obj({
|
||||
"bottles": {
|
||||
"dev": {
|
||||
"egress": {
|
||||
"routes": [
|
||||
{"host": "example.com"},
|
||||
],
|
||||
},
|
||||
},
|
||||
},
|
||||
"agents": {
|
||||
"demo": {
|
||||
"skills": [],
|
||||
"prompt": _AGENT_PROMPT,
|
||||
"bottle": "dev",
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
|
||||
@skip_unless_docker()
|
||||
@unittest.skipUnless(
|
||||
platform.system() in ("Darwin", "Linux"),
|
||||
"smolvm requires macOS or Linux",
|
||||
)
|
||||
@unittest.skipUnless(
|
||||
_smolvm_available(),
|
||||
"smolvm not on PATH; install via "
|
||||
"curl -sSL https://smolmachines.com/install.sh | sh",
|
||||
)
|
||||
@unittest.skipIf(
|
||||
os.environ.get("GITEA_ACTIONS") == "true",
|
||||
"skipped under act_runner: cannot host libkrun-backed VMs",
|
||||
)
|
||||
class TestSmolmachinesLaunch(unittest.TestCase):
|
||||
"""The full smoke + the two acceptance probes share one
|
||||
bottle bringup to amortize the ~10s cold-start cost across
|
||||
three assertions."""
|
||||
|
||||
@classmethod
|
||||
def setUpClass(cls) -> None:
|
||||
cls.stage = Path(tempfile.mkdtemp(prefix="cb-smol-launch."))
|
||||
os.environ["BOT_BOTTLE_BACKEND"] = "smolmachines"
|
||||
backend = get_bottle_backend()
|
||||
spec = BottleSpec(
|
||||
manifest=_minimal_manifest(),
|
||||
agent_name="demo",
|
||||
copy_cwd=False,
|
||||
user_cwd=str(cls.stage),
|
||||
)
|
||||
cls.plan = backend.prepare(spec, stage_dir=cls.stage)
|
||||
cls._launch = backend.launch(cls.plan)
|
||||
cls.bottle = cls._launch.__enter__()
|
||||
|
||||
@classmethod
|
||||
def tearDownClass(cls) -> None:
|
||||
try:
|
||||
cls._launch.__exit__(None, None, None)
|
||||
finally:
|
||||
shutil.rmtree(cls.stage, ignore_errors=True)
|
||||
os.environ.pop("BOT_BOTTLE_BACKEND", None)
|
||||
|
||||
def test_smoke_exec_echo(self):
|
||||
# The plumbing-verifies-end-to-end smoke: a shell command
|
||||
# round-trips through smolvm machine exec.
|
||||
r = self.bottle.exec("echo hello-from-vm")
|
||||
self.assertEqual(0, r.returncode, msg=r.stderr)
|
||||
self.assertIn("hello-from-vm", r.stdout)
|
||||
|
||||
def test_localhost_reach_probe(self):
|
||||
# Agent dials a 127.0.0.1 service on the host. TSI's
|
||||
# allowlist contains only <bundle-ip>/32, so this must
|
||||
# refuse. We use a port unlikely to be bound on the host
|
||||
# (high-numbered) so we're confirming TSI refusal, not
|
||||
# just "no service listening."
|
||||
r = self.bottle.exec(
|
||||
"curl -s --show-error --max-time 3 http://127.0.0.1:9 2>&1 || true"
|
||||
)
|
||||
# `curl` to a denied destination produces a connect error.
|
||||
# The exact phrasing varies by curl version; we assert
|
||||
# the response is NOT the body of any real service.
|
||||
self.assertNotIn("hello-from-vm", r.stdout)
|
||||
self.assertTrue(
|
||||
"refused" in r.stdout.lower()
|
||||
or "timed out" in r.stdout.lower()
|
||||
or "unreachable" in r.stdout.lower()
|
||||
or "failed" in r.stdout.lower(),
|
||||
f"expected a connect-refusal message; got: {r.stdout!r}",
|
||||
)
|
||||
|
||||
def test_egress_proxy_reachable_through_tsi_loopback_alias(self):
|
||||
r = self.bottle.exec(
|
||||
"printf '%s\n' \"$HTTPS_PROXY\" \"$HTTP_PROXY\""
|
||||
)
|
||||
self.assertEqual(0, r.returncode, msg=r.stderr)
|
||||
proxies = [line.strip() for line in r.stdout.splitlines()]
|
||||
self.assertEqual(2, len(proxies), proxies)
|
||||
self.assertEqual(proxies[0], proxies[1], proxies)
|
||||
# macOS: proxy binds to the per-bottle loopback alias (127.x.x.x) so
|
||||
# TSI can intercept guest connections to it. Linux: the guest kernel
|
||||
# routes 127.0.0.0/8 to its own loopback (TSI never sees those), so
|
||||
# the proxy instead binds to the per-bottle bridge gateway (192.168.x.1)
|
||||
# which routes via eth0 and is intercepted by TSI normally.
|
||||
self.assertRegex(
|
||||
proxies[0], r"^http://\d+\.\d+\.\d+\.\d+:\d+$",
|
||||
"expected proxy URL to be an http://IP:port address",
|
||||
)
|
||||
|
||||
r = self.bottle.exec(
|
||||
"curl -fsS --max-time 20 https://example.com >/dev/null && echo OK"
|
||||
)
|
||||
self.assertEqual(0, r.returncode, msg=r.stderr + r.stdout)
|
||||
self.assertIn("OK", r.stdout)
|
||||
|
||||
def test_direct_egress_bypass_without_proxy_fails(self):
|
||||
r = self.bottle.exec(
|
||||
"env -u HTTPS_PROXY -u HTTP_PROXY -u https_proxy -u http_proxy "
|
||||
"curl -s --show-error --max-time 5 https://example.com 2>&1 || true"
|
||||
)
|
||||
self.assertTrue(
|
||||
"refused" in r.stdout.lower()
|
||||
or "timed out" in r.stdout.lower()
|
||||
or "unreachable" in r.stdout.lower()
|
||||
or "failed" in r.stdout.lower()
|
||||
or "could not resolve" in r.stdout.lower()
|
||||
or "connection reset" in r.stdout.lower(),
|
||||
f"expected direct egress to fail; got: {r.stdout!r}",
|
||||
)
|
||||
|
||||
def test_non_allowlisted_host_fails_through_proxy(self):
|
||||
r = self.bottle.exec(
|
||||
"curl -s --show-error --max-time 10 https://iana.org 2>&1 || true"
|
||||
)
|
||||
self.assertTrue(
|
||||
"403" in r.stdout
|
||||
or "502" in r.stdout
|
||||
or "blocked" in r.stdout.lower()
|
||||
or "not allowed" in r.stdout.lower()
|
||||
or "not in the bottle's egress.routes allowlist" in r.stdout.lower()
|
||||
or "forbidden" in r.stdout.lower()
|
||||
or "failed" in r.stdout.lower(),
|
||||
f"expected non-allowlisted proxy request to fail; got: {r.stdout!r}",
|
||||
)
|
||||
|
||||
def test_prompt_file_lands_in_guest(self):
|
||||
# provision_prompt copies the host-side prompt.txt into the
|
||||
# guest at /home/node/.bot-bottle-prompt.txt. The content
|
||||
# must match what the manifest declared so claude-code's
|
||||
# --append-system-prompt-file reads the right text.
|
||||
r = self.bottle.exec("cat /home/node/.bot-bottle-prompt.txt")
|
||||
self.assertEqual(0, r.returncode, msg=r.stderr)
|
||||
self.assertEqual(_AGENT_PROMPT, r.stdout.rstrip("\n"))
|
||||
|
||||
def test_egress_port_bypass_probe(self):
|
||||
# Agent dials <bundle-ip>:9099 (egress's port). TSI
|
||||
# permits the IP, but egress will bind 127.0.0.1:9099
|
||||
# inside the bundle in chunk 3, so the connect refuses
|
||||
# at the socket level. NOTE: in chunk 2d the bundle's
|
||||
# daemons aren't running (daemons_csv=""), so nothing
|
||||
# is listening on :9099 anyway — this test asserts the
|
||||
# connect fails, which is the property chunk 3 will
|
||||
# preserve once egress is actually running.
|
||||
r = self.bottle.exec(
|
||||
"env -u HTTPS_PROXY -u HTTP_PROXY -u https_proxy -u http_proxy "
|
||||
f"curl -s --show-error --max-time 3 http://{self.plan.bundle_ip}:9099 "
|
||||
"2>&1 || true"
|
||||
)
|
||||
self.assertTrue(
|
||||
"refused" in r.stdout.lower()
|
||||
or "timed out" in r.stdout.lower()
|
||||
or "unreachable" in r.stdout.lower()
|
||||
or "failed" in r.stdout.lower(),
|
||||
f"expected egress port refusal; got: {r.stdout!r}",
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,64 +0,0 @@
|
||||
"""Integration: PRD 0023 chunk 2b — smolvm subprocess wrapper
|
||||
exercised against the real binary.
|
||||
|
||||
The full machine-lifecycle round trip (create → start → exec →
|
||||
delete) is gated behind macOS/Linux platform check and lives
|
||||
in chunk 2d's smoke. This file just verifies `is_available()`
|
||||
correctly reports presence and `_smolvm()` can run a no-op
|
||||
subcommand without errors — enough to flag wrapper drift if
|
||||
smolvm's flag parser changes shape across versions."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import platform
|
||||
import subprocess
|
||||
import unittest
|
||||
|
||||
from bot_bottle.backend.smolmachines.smolvm import is_available
|
||||
|
||||
|
||||
@unittest.skipIf(
|
||||
os.environ.get("GITEA_ACTIONS") == "true",
|
||||
"skipped under act_runner: smolvm not installed on the runner",
|
||||
)
|
||||
@unittest.skipUnless(
|
||||
platform.system() in ("Darwin", "Linux"),
|
||||
"smolvm requires macOS or Linux",
|
||||
)
|
||||
@unittest.skipUnless(
|
||||
is_available(),
|
||||
"smolvm not on PATH; install via "
|
||||
"curl -sSL https://smolmachines.com/install.sh | sh",
|
||||
)
|
||||
class TestSmolvmSmoke(unittest.TestCase):
|
||||
def test_smolvm_help_responds(self):
|
||||
# `smolvm --help` exits 0 (per `smolvm machine --help`
|
||||
# convention) — verifies the binary launches and the
|
||||
# top-level parser is intact.
|
||||
r = subprocess.run(
|
||||
["smolvm", "--help"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
# Either exit-code 0 (clean) or 1 (some CLIs return 1
|
||||
# from --help by convention; smolvm 0.8.0 does this). The
|
||||
# point is the binary runs and emits help text.
|
||||
self.assertIn("smolvm", r.stdout)
|
||||
self.assertIn("machine", r.stdout)
|
||||
|
||||
def test_machine_ls_empty_returns_json_array(self):
|
||||
# `machine ls --json` is the contract chunk 4's
|
||||
# list_active wires to. Lock in that the JSON shape is
|
||||
# parseable now so chunk 4 doesn't surprise us.
|
||||
import json
|
||||
r = subprocess.run(
|
||||
["smolvm", "machine", "ls", "--json"],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
self.assertEqual(0, r.returncode, r.stderr)
|
||||
parsed = json.loads(r.stdout)
|
||||
self.assertIsInstance(parsed, list)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -12,7 +12,7 @@ from bot_bottle.backend import ActiveAgent
|
||||
from bot_bottle.backend.freeze import get_freezer
|
||||
from bot_bottle.backend.docker.freezer import DockerFreezer
|
||||
from bot_bottle.backend.macos_container.freezer import MacosContainerFreezer
|
||||
from bot_bottle.backend.smolmachines.freezer import SmolmachinesFreezer
|
||||
from bot_bottle.backend.firecracker.freezer import FirecrackerFreezer
|
||||
|
||||
|
||||
class _FakeHomeMixin:
|
||||
@@ -51,8 +51,8 @@ class TestGetFreezer(unittest.TestCase):
|
||||
def test_macos_container(self):
|
||||
self.assertIsInstance(get_freezer("macos-container"), MacosContainerFreezer)
|
||||
|
||||
def test_smolmachines(self):
|
||||
self.assertIsInstance(get_freezer("smolmachines"), SmolmachinesFreezer)
|
||||
def test_firecracker(self):
|
||||
self.assertIsInstance(get_freezer("firecracker"), FirecrackerFreezer)
|
||||
|
||||
def test_unknown_backend_dies(self):
|
||||
with patch("bot_bottle.backend.freeze.die", side_effect=SystemExit("die")):
|
||||
@@ -176,39 +176,59 @@ class TestMacosContainerFreezer(_FakeHomeMixin, unittest.TestCase):
|
||||
self.assertTrue(bottle_state.is_preserved(slug))
|
||||
|
||||
|
||||
class TestSmolmachinesFreezer(_FakeHomeMixin, unittest.TestCase):
|
||||
class TestFirecrackerFreezer(_FakeHomeMixin, unittest.TestCase):
|
||||
def setUp(self):
|
||||
self._setup_fake_home()
|
||||
# The freezer resolves the running VM's SSH key + config from
|
||||
# the per-bottle run dir under the firecracker cache; point that
|
||||
# at a temp dir so we can stage a fake live bottle.
|
||||
self._cache = tempfile.TemporaryDirectory(prefix="fc-freezer-cache.")
|
||||
self._cache_patch = patch(
|
||||
"bot_bottle.backend.firecracker.freezer.util.cache_dir",
|
||||
return_value=Path(self._cache.name),
|
||||
)
|
||||
self._cache_patch.start()
|
||||
|
||||
def tearDown(self):
|
||||
self._cache_patch.stop()
|
||||
self._cache.cleanup()
|
||||
self._teardown_fake_home()
|
||||
|
||||
def _write_meta(self, slug: str) -> None:
|
||||
bottle_state.write_metadata(bottle_state.BottleMetadata(
|
||||
identity=slug, agent_name="dev", cwd="", copy_cwd=False,
|
||||
started_at="t", backend="smolmachines",
|
||||
started_at="t", backend="firecracker",
|
||||
))
|
||||
|
||||
def _stage_run_dir(self, slug: str, guest_ip: str = "100.64.0.1") -> None:
|
||||
run_dir = Path(self._cache.name) / "run" / slug
|
||||
run_dir.mkdir(parents=True)
|
||||
(run_dir / "bottle_id_ed25519").write_text("KEY")
|
||||
(run_dir / "config.json").write_text(
|
||||
'{"boot-source": {"boot_args": '
|
||||
f'"console=ttyS0 ip={guest_ip}::100.64.0.0:255.255.255.254::eth0:off"}}}}'
|
||||
)
|
||||
|
||||
def test_snapshots_running_vm_without_stopping(self):
|
||||
"""Commit should exec-tar the running VM, not stop it."""
|
||||
"""Commit should tar the running guest rootfs over SSH, not stop it."""
|
||||
slug = "dev-abc12"
|
||||
self._write_meta(slug)
|
||||
freezer = SmolmachinesFreezer()
|
||||
agent = _make_agent(slug, "smolmachines")
|
||||
self._stage_run_dir(slug)
|
||||
freezer = FirecrackerFreezer()
|
||||
agent = _make_agent(slug, "firecracker")
|
||||
|
||||
with patch("bot_bottle.backend.smolmachines.freezer._snapshot_running_vm") as mock_snap, \
|
||||
with patch("bot_bottle.backend.firecracker.freezer._commit_via_ssh") as mock_commit, \
|
||||
patch("bot_bottle.backend.freeze.info"), \
|
||||
patch("bot_bottle.backend.smolmachines.freezer.info"):
|
||||
patch("bot_bottle.backend.firecracker.freezer.info"):
|
||||
freezer.commit(agent)
|
||||
|
||||
expected_binary = bottle_state.bottle_state_dir(slug) / "committed-smolmachine"
|
||||
mock_snap.assert_called_once_with(
|
||||
f"bot-bottle-{slug}",
|
||||
f"bot-bottle-committed-{slug}:latest",
|
||||
expected_binary,
|
||||
)
|
||||
expected_sidecar = str(expected_binary.with_suffix(".smolmachine"))
|
||||
self.assertEqual(expected_sidecar, bottle_state.read_committed_image(slug))
|
||||
image_tag = f"bot-bottle-committed-{slug}:latest"
|
||||
self.assertEqual(1, mock_commit.call_count)
|
||||
# (private_key, guest_ip, image_tag) — guest_ip parsed from config.
|
||||
args = mock_commit.call_args.args
|
||||
self.assertEqual("100.64.0.1", args[1])
|
||||
self.assertEqual(image_tag, args[2])
|
||||
self.assertEqual(image_tag, bottle_state.read_committed_image(slug))
|
||||
self.assertTrue(bottle_state.is_preserved(slug))
|
||||
|
||||
|
||||
|
||||
@@ -1,18 +1,18 @@
|
||||
"""Cross-backend parity tests (PRD 0042).
|
||||
|
||||
Verifies that Docker and smolmachines bottles expose the same
|
||||
Verifies that Docker and firecracker bottles expose the same
|
||||
observable contracts for env injection, agent argv, and exec. Tests
|
||||
use mock subprocess layers so no live VM or Docker daemon is needed.
|
||||
|
||||
The scenarios here document what must hold across both backends. As
|
||||
PRDs 0038–0040 land these tests provide regression coverage for the
|
||||
contracts they establish.
|
||||
The scenarios here document what must hold across both backends and
|
||||
provide regression coverage for those contracts.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from typing import Callable
|
||||
from unittest.mock import patch
|
||||
|
||||
@@ -31,10 +31,12 @@ def _docker_bottle(guest_env: dict[str, str]) -> "object":
|
||||
)
|
||||
|
||||
|
||||
def _smolmachines_bottle(guest_env: dict[str, str]) -> "object":
|
||||
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
|
||||
return SmolmachinesBottle(
|
||||
def _firecracker_bottle(guest_env: dict[str, str]) -> "object":
|
||||
from bot_bottle.backend.firecracker.bottle import FirecrackerBottle
|
||||
return FirecrackerBottle(
|
||||
"bot-bottle-test",
|
||||
private_key=Path("/tmp/key"),
|
||||
guest_ip="100.64.0.1",
|
||||
guest_env=guest_env,
|
||||
agent_command="claude",
|
||||
)
|
||||
@@ -43,7 +45,7 @@ def _smolmachines_bottle(guest_env: dict[str, str]) -> "object":
|
||||
# One entry per backend: (label, factory).
|
||||
_BACKENDS: list[tuple[str, Callable[[dict[str, str]], object]]] = [
|
||||
("docker", _docker_bottle),
|
||||
("smolmachines", _smolmachines_bottle),
|
||||
("firecracker", _firecracker_bottle),
|
||||
]
|
||||
|
||||
|
||||
@@ -91,19 +93,16 @@ class TestAgentArgvParity(unittest.TestCase):
|
||||
)
|
||||
|
||||
|
||||
class TestSmolmachinesEnvInArgv(unittest.TestCase):
|
||||
"""smolmachines bottle includes guest_env values in exec argv."""
|
||||
class TestFirecrackerEnvInArgv(unittest.TestCase):
|
||||
"""firecracker bottle includes guest_env values in the agent argv."""
|
||||
|
||||
def test_guest_env_in_exec_argv(self):
|
||||
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
|
||||
bottle = SmolmachinesBottle(
|
||||
"bot-bottle-test",
|
||||
guest_env={"TOKEN": "abc123", "PROXY": "http://proxy:8888"},
|
||||
def test_guest_env_in_agent_argv(self):
|
||||
bottle = _firecracker_bottle(
|
||||
{"TOKEN": "abc123", "PROXY": "http://proxy:8888"},
|
||||
)
|
||||
argv = bottle.agent_argv([], tty=False)
|
||||
joined = " ".join(argv)
|
||||
self.assertIn("TOKEN=abc123", joined)
|
||||
self.assertIn("PROXY=http://proxy:8888", joined)
|
||||
argv = bottle.agent_argv([], tty=False) # type: ignore[union-attr]
|
||||
self.assertIn("TOKEN=abc123", argv)
|
||||
self.assertIn("PROXY=http://proxy:8888", argv)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -129,17 +128,16 @@ class TestExecUserSwitching(unittest.TestCase):
|
||||
self.assertIn("node", call_args,
|
||||
"docker exec should use 'node' user by default")
|
||||
|
||||
def test_smolmachines_exec_uses_node_user_by_default(self):
|
||||
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
|
||||
bottle = SmolmachinesBottle("bot-bottle-test", guest_env={})
|
||||
with patch("bot_bottle.backend.smolmachines.bottle.subprocess.run") as run:
|
||||
def test_firecracker_exec_uses_node_user_by_default(self):
|
||||
bottle = _firecracker_bottle({})
|
||||
with patch("bot_bottle.backend.firecracker.bottle.subprocess.run") as run:
|
||||
run.return_value = subprocess.CompletedProcess(
|
||||
[], 0, stdout="", stderr="",
|
||||
)
|
||||
bottle.exec("echo hi")
|
||||
call_args = run.call_args[0][0]
|
||||
self.assertIn("node", call_args,
|
||||
"smolvm exec should use 'node' user by default")
|
||||
bottle.exec("echo hi") # type: ignore[union-attr]
|
||||
call_args = " ".join(run.call_args[0][0])
|
||||
self.assertIn("runuser -u node", call_args,
|
||||
"firecracker exec should use 'node' user by default")
|
||||
|
||||
def test_docker_exec_respects_root_user(self):
|
||||
from bot_bottle.backend.docker.bottle import DockerBottle
|
||||
@@ -156,16 +154,15 @@ class TestExecUserSwitching(unittest.TestCase):
|
||||
call_args = run.call_args[0][0]
|
||||
self.assertIn("root", call_args)
|
||||
|
||||
def test_smolmachines_exec_respects_root_user(self):
|
||||
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
|
||||
bottle = SmolmachinesBottle("bot-bottle-test", guest_env={})
|
||||
with patch("bot_bottle.backend.smolmachines.bottle.subprocess.run") as run:
|
||||
def test_firecracker_exec_respects_root_user(self):
|
||||
bottle = _firecracker_bottle({})
|
||||
with patch("bot_bottle.backend.firecracker.bottle.subprocess.run") as run:
|
||||
run.return_value = subprocess.CompletedProcess(
|
||||
[], 0, stdout="", stderr="",
|
||||
)
|
||||
bottle.exec("id", user="root")
|
||||
call_args = run.call_args[0][0]
|
||||
self.assertIn("root", call_args)
|
||||
bottle.exec("id", user="root") # type: ignore[union-attr]
|
||||
call_args = " ".join(run.call_args[0][0])
|
||||
self.assertIn("runuser -u root", call_args)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -196,13 +193,12 @@ class TestExecResultParity(unittest.TestCase):
|
||||
self.assertIsInstance(result.stdout, str)
|
||||
self.assertIsInstance(result.stderr, str)
|
||||
|
||||
def test_smolmachines_exec_result_shape(self):
|
||||
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
|
||||
def test_firecracker_exec_result_shape(self):
|
||||
from bot_bottle.backend import ExecResult
|
||||
bottle = SmolmachinesBottle("bot-bottle-test", guest_env={})
|
||||
with patch("bot_bottle.backend.smolmachines.bottle.subprocess.run",
|
||||
bottle = _firecracker_bottle({})
|
||||
with patch("bot_bottle.backend.firecracker.bottle.subprocess.run",
|
||||
side_effect=self._stub_run):
|
||||
result = bottle.exec("echo hi")
|
||||
result = bottle.exec("echo hi") # type: ignore[union-attr]
|
||||
self.assertIsInstance(result, ExecResult)
|
||||
self.assertEqual(0, result.returncode)
|
||||
self.assertIsInstance(result.stdout, str)
|
||||
@@ -229,11 +225,10 @@ class TestCloseParity(unittest.TestCase):
|
||||
# DockerBottle.close calls teardown — once per call is fine;
|
||||
# what matters is it doesn't raise.
|
||||
|
||||
def test_smolmachines_close_is_noop(self):
|
||||
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
|
||||
bottle = SmolmachinesBottle("bot-bottle-test", guest_env={})
|
||||
bottle.close()
|
||||
bottle.close()
|
||||
def test_firecracker_close_is_noop(self):
|
||||
bottle = _firecracker_bottle({})
|
||||
bottle.close() # type: ignore[union-attr]
|
||||
bottle.close() # type: ignore[union-attr]
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -17,7 +17,7 @@ from bot_bottle import supervise
|
||||
from bot_bottle.backend import BottleSpec
|
||||
from bot_bottle.backend.docker import DockerBottleBackend
|
||||
from bot_bottle.backend.resolve_common import mint_slug
|
||||
from bot_bottle.backend.smolmachines import SmolmachinesBottleBackend
|
||||
from bot_bottle.backend.firecracker import FirecrackerBottleBackend
|
||||
from bot_bottle.manifest import ManifestIndex
|
||||
|
||||
|
||||
@@ -90,30 +90,28 @@ class TestDockerPrepare(_FakeStateMixin, unittest.TestCase):
|
||||
self.assertNotIn("FORWARDED_ENV", plan.agent_provision.guest_env)
|
||||
|
||||
|
||||
class TestSmolmachinesPrepare(_FakeStateMixin, unittest.TestCase):
|
||||
class TestFirecrackerPrepare(_FakeStateMixin, unittest.TestCase):
|
||||
def test_records_backend_and_builds_guest_env(self) -> None:
|
||||
backend = SmolmachinesBottleBackend()
|
||||
spec = _spec(Path(self.tmp.name), identity="demo-smol")
|
||||
backend = FirecrackerBottleBackend()
|
||||
spec = _spec(Path(self.tmp.name), identity="demo-fc")
|
||||
|
||||
with (
|
||||
patch.dict("os.environ", {"HOST_SECRET_ENV": "secret-value"}),
|
||||
patch(
|
||||
"bot_bottle.backend.smolmachines.resolve_plan.smolmachines_preflight",
|
||||
"bot_bottle.backend.firecracker.resolve_plan.util.require_firecracker",
|
||||
) as preflight,
|
||||
):
|
||||
plan = backend.prepare(spec, Path(self.tmp.name) / "stage")
|
||||
|
||||
preflight.assert_called_once_with()
|
||||
metadata = bottle_state.read_metadata("demo-smol")
|
||||
metadata = bottle_state.read_metadata("demo-fc")
|
||||
self.assertIsNotNone(metadata)
|
||||
assert metadata is not None
|
||||
self.assertEqual("smolmachines", metadata.backend)
|
||||
self.assertEqual("literal-value", plan.guest_env["LITERAL_ENV"])
|
||||
self.assertEqual("secret-value", plan.guest_env["FORWARDED_ENV"])
|
||||
self.assertEqual("firecracker", metadata.backend)
|
||||
self.assertEqual(
|
||||
"/etc/ssl/certs/ca-certificates.crt",
|
||||
plan.guest_env["SSL_CERT_FILE"],
|
||||
"literal-value", plan.agent_provision.guest_env["LITERAL_ENV"],
|
||||
)
|
||||
self.assertEqual({"FORWARDED_ENV": "secret-value"}, plan.forwarded_env)
|
||||
|
||||
|
||||
class TestMintSlug(unittest.TestCase):
|
||||
|
||||
@@ -23,14 +23,14 @@ from bot_bottle.backend import (
|
||||
|
||||
class TestGetBottleBackend(unittest.TestCase):
|
||||
def test_explicit_name_wins_over_env(self):
|
||||
with patch.dict(os.environ, {"BOT_BOTTLE_BACKEND": "smolmachines"}):
|
||||
with patch.dict(os.environ, {"BOT_BOTTLE_BACKEND": "firecracker"}):
|
||||
b = get_bottle_backend("docker")
|
||||
self.assertEqual("docker", b.name)
|
||||
|
||||
def test_env_var_fallback(self):
|
||||
with patch.dict(os.environ, {"BOT_BOTTLE_BACKEND": "smolmachines"}):
|
||||
with patch.dict(os.environ, {"BOT_BOTTLE_BACKEND": "firecracker"}):
|
||||
b = get_bottle_backend()
|
||||
self.assertEqual("smolmachines", b.name)
|
||||
self.assertEqual("firecracker", b.name)
|
||||
|
||||
def test_default_macos_container_when_available(self):
|
||||
class _FakeBackend:
|
||||
@@ -42,12 +42,12 @@ class TestGetBottleBackend(unittest.TestCase):
|
||||
with patch.dict(os.environ, {}, clear=True), \
|
||||
patch.object(backend_mod, "_BACKENDS", {
|
||||
"macos-container": _FakeBackend(),
|
||||
"smolmachines": _FakeBackend(),
|
||||
"docker": _FakeBackend(),
|
||||
}):
|
||||
b = get_bottle_backend()
|
||||
self.assertEqual("macos-container", b.name)
|
||||
|
||||
def test_default_smolmachines_when_macos_container_unavailable(self):
|
||||
def test_default_docker_when_no_macos_and_host_not_kvm(self):
|
||||
class _FakeBackend:
|
||||
def __init__(self, name: str, available: bool) -> None:
|
||||
self.name = name
|
||||
@@ -56,13 +56,40 @@ class TestGetBottleBackend(unittest.TestCase):
|
||||
def is_available(self) -> bool:
|
||||
return self._available
|
||||
|
||||
# No macOS container and the host can't run firecracker (no
|
||||
# KVM / not Linux) → docker is the last resort.
|
||||
with patch.dict(os.environ, {}, clear=True), \
|
||||
patch.object(backend_mod.FirecrackerBottleBackend,
|
||||
"is_host_capable", classmethod(lambda cls: False)), \
|
||||
patch.object(backend_mod, "_BACKENDS", {
|
||||
"macos-container": _FakeBackend("macos-container", False),
|
||||
"smolmachines": _FakeBackend("smolmachines", False),
|
||||
"docker": _FakeBackend("docker", True),
|
||||
}):
|
||||
b = get_bottle_backend()
|
||||
self.assertEqual("smolmachines", b.name)
|
||||
self.assertEqual("docker", b.name)
|
||||
|
||||
def test_default_firecracker_on_kvm_host_even_when_binary_missing(self):
|
||||
class _FakeBackend:
|
||||
def __init__(self, name: str, available: bool) -> None:
|
||||
self.name = name
|
||||
self._available = available
|
||||
|
||||
def is_available(self) -> bool:
|
||||
return self._available
|
||||
|
||||
# A KVM-capable Linux host defaults to firecracker even when the
|
||||
# binary isn't installed (is_available False) — start then prints
|
||||
# the install pointer instead of falling back to docker.
|
||||
with patch.dict(os.environ, {}, clear=True), \
|
||||
patch.object(backend_mod.FirecrackerBottleBackend,
|
||||
"is_host_capable", classmethod(lambda cls: True)), \
|
||||
patch.object(backend_mod, "_BACKENDS", {
|
||||
"macos-container": _FakeBackend("macos-container", False),
|
||||
"firecracker": _FakeBackend("firecracker", False),
|
||||
"docker": _FakeBackend("docker", True),
|
||||
}):
|
||||
b = get_bottle_backend()
|
||||
self.assertEqual("firecracker", b.name)
|
||||
|
||||
def test_unknown_dies(self):
|
||||
with patch.object(backend_mod, "die", side_effect=SystemExit("die")):
|
||||
@@ -73,7 +100,7 @@ class TestGetBottleBackend(unittest.TestCase):
|
||||
class TestKnownBackendNames(unittest.TestCase):
|
||||
def test_returns_backends_sorted(self):
|
||||
self.assertEqual(
|
||||
("docker", "macos-container", "smolmachines"),
|
||||
("docker", "firecracker", "macos-container"),
|
||||
known_backend_names(),
|
||||
)
|
||||
|
||||
@@ -81,8 +108,8 @@ class TestKnownBackendNames(unittest.TestCase):
|
||||
class TestEnumerateActiveAgents(unittest.TestCase):
|
||||
"""Combines each backend's `enumerate_active`. Each backend's
|
||||
implementation has its own tests (`test_docker_enumerate_active`,
|
||||
`test_smolmachines_*`); this just asserts the aggregator stitches
|
||||
them together."""
|
||||
`test_firecracker_backend`); this just asserts the aggregator
|
||||
stitches them together."""
|
||||
|
||||
def test_concatenates_per_backend(self):
|
||||
a = ActiveAgent(
|
||||
@@ -90,7 +117,7 @@ class TestEnumerateActiveAgents(unittest.TestCase):
|
||||
started_at="", services=("egress",),
|
||||
)
|
||||
b = ActiveAgent(
|
||||
backend_name="smolmachines", slug="b-2", agent_name="research",
|
||||
backend_name="firecracker", slug="b-2", agent_name="research",
|
||||
started_at="", services=(),
|
||||
)
|
||||
|
||||
@@ -107,7 +134,7 @@ class TestEnumerateActiveAgents(unittest.TestCase):
|
||||
|
||||
with patch.object(
|
||||
backend_mod, "_BACKENDS",
|
||||
{"docker": _FakeBackend([a]), "smolmachines": _FakeBackend([b])},
|
||||
{"docker": _FakeBackend([a]), "firecracker": _FakeBackend([b])},
|
||||
):
|
||||
self.assertEqual([a, b], enumerate_active_agents())
|
||||
|
||||
@@ -121,11 +148,11 @@ class TestEnumerateActiveAgents(unittest.TestCase):
|
||||
started_at="2026-06-02T11:00:00Z", services=(),
|
||||
)
|
||||
missing_metadata = ActiveAgent(
|
||||
backend_name="smolmachines", slug="missing-metadata",
|
||||
backend_name="firecracker", slug="missing-metadata",
|
||||
agent_name="?", started_at="", services=(),
|
||||
)
|
||||
tie_a = ActiveAgent(
|
||||
backend_name="smolmachines", slug="a-slug", agent_name="research",
|
||||
backend_name="firecracker", slug="a-slug", agent_name="research",
|
||||
started_at="2026-06-02T11:00:00Z", services=(),
|
||||
)
|
||||
|
||||
@@ -143,7 +170,7 @@ class TestEnumerateActiveAgents(unittest.TestCase):
|
||||
backend_mod, "_BACKENDS",
|
||||
{
|
||||
"docker": _FakeBackend([newer, tie_b]),
|
||||
"smolmachines": _FakeBackend([missing_metadata, tie_a]),
|
||||
"firecracker": _FakeBackend([missing_metadata, tie_a]),
|
||||
},
|
||||
):
|
||||
self.assertEqual(
|
||||
@@ -161,21 +188,21 @@ class TestEnumerateActiveAgents(unittest.TestCase):
|
||||
|
||||
with patch.object(
|
||||
backend_mod, "_BACKENDS",
|
||||
{"docker": _FakeBackend(), "smolmachines": _FakeBackend()},
|
||||
{"docker": _FakeBackend(), "firecracker": _FakeBackend()},
|
||||
):
|
||||
self.assertEqual([], enumerate_active_agents())
|
||||
|
||||
def test_skips_unavailable_backends(self):
|
||||
# If a backend's runtime isn't installed (smolvm missing on
|
||||
# a docker-only host, or docker missing on a smolmachines-
|
||||
# only host), the cross-backend enumerator skips it rather
|
||||
# than dying — `has_backend` gates the iteration.
|
||||
# If a backend's runtime isn't installed (docker missing on a
|
||||
# firecracker host, or KVM missing on a docker-only host), the
|
||||
# cross-backend enumerator skips it rather than dying —
|
||||
# `has_backend` gates the iteration.
|
||||
present = ActiveAgent(
|
||||
backend_name="docker", slug="a-1", agent_name="impl",
|
||||
started_at="", services=(),
|
||||
)
|
||||
hidden = ActiveAgent(
|
||||
backend_name="smolmachines", slug="x", agent_name="x",
|
||||
backend_name="firecracker", slug="x", agent_name="x",
|
||||
started_at="", services=(),
|
||||
)
|
||||
|
||||
@@ -194,7 +221,7 @@ class TestEnumerateActiveAgents(unittest.TestCase):
|
||||
backend_mod, "_BACKENDS",
|
||||
{
|
||||
"docker": _FakeBackend([present], available=True),
|
||||
"smolmachines": _FakeBackend([hidden], available=False),
|
||||
"firecracker": _FakeBackend([hidden], available=False),
|
||||
},
|
||||
):
|
||||
self.assertEqual([present], enumerate_active_agents())
|
||||
|
||||
@@ -34,7 +34,7 @@ class TestPalettePrintf(unittest.TestCase):
|
||||
|
||||
|
||||
class TestExecShellScript(unittest.TestCase):
|
||||
_ARGV = ["smolvm", "machine", "exec", "--name", "x", "--", "claude"]
|
||||
_ARGV = ["ssh", "-t", "100.64.0.1", "--", "claude"]
|
||||
|
||||
def test_no_decoration_returns_none(self):
|
||||
self.assertIsNone(exec_shell_script(self._ARGV))
|
||||
@@ -58,7 +58,7 @@ class TestExecShellScript(unittest.TestCase):
|
||||
self.assertIn("\\033]111", script) # background reset
|
||||
# No exec-replace when palette is active (shell must survive for reset)
|
||||
parts = script.split("; ")
|
||||
agent_part = next(p for p in parts if "smolvm" in p)
|
||||
agent_part = next(p for p in parts if "ssh" in p)
|
||||
self.assertFalse(agent_part.startswith("exec "))
|
||||
|
||||
def test_title_and_color_both_appear(self):
|
||||
|
||||
@@ -16,7 +16,7 @@ from bot_bottle import bottle_state
|
||||
from bot_bottle import supervise
|
||||
from bot_bottle.backend import Bottle, BottleSpec, ExecResult
|
||||
from bot_bottle.backend.docker import DockerBottleBackend
|
||||
from bot_bottle.backend.smolmachines import SmolmachinesBottleBackend
|
||||
from bot_bottle.backend.firecracker import FirecrackerBottleBackend
|
||||
from bot_bottle.manifest import ManifestIndex
|
||||
|
||||
|
||||
@@ -114,13 +114,13 @@ class TestRuntimeWorkspaceProvisioning(_FakeStateMixin, unittest.TestCase):
|
||||
bottle.exec.assert_not_called()
|
||||
bottle.cp_in.assert_not_called()
|
||||
|
||||
def test_smolmachines_uses_same_running_bottle_method(self) -> None:
|
||||
backend = SmolmachinesBottleBackend()
|
||||
def test_firecracker_uses_same_running_bottle_method(self) -> None:
|
||||
backend = FirecrackerBottleBackend()
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.resolve_plan.smolmachines_preflight",
|
||||
"bot_bottle.backend.firecracker.resolve_plan.util.require_firecracker",
|
||||
):
|
||||
plan = backend.prepare(
|
||||
_spec(self.tmp, identity="demo-smol-work"),
|
||||
_spec(self.tmp, identity="demo-fc-work"),
|
||||
self.tmp / "stage",
|
||||
)
|
||||
|
||||
@@ -128,10 +128,10 @@ class TestRuntimeWorkspaceProvisioning(_FakeStateMixin, unittest.TestCase):
|
||||
backend.provision_workspace(plan, bottle)
|
||||
|
||||
bottle.cp_in.assert_called_once_with(str(self.tmp), "/home/node/workspace")
|
||||
metadata = bottle_state.read_metadata("demo-smol-work")
|
||||
metadata = bottle_state.read_metadata("demo-fc-work")
|
||||
self.assertIsNotNone(metadata)
|
||||
assert metadata is not None
|
||||
self.assertEqual("smolmachines", metadata.backend)
|
||||
self.assertEqual("firecracker", metadata.backend)
|
||||
|
||||
|
||||
class TestWorkspaceTrustPath(_FakeStateMixin, unittest.TestCase):
|
||||
|
||||
@@ -241,7 +241,7 @@ class TestBottleMetadataBackend(_FakeHomeMixin, unittest.TestCase):
|
||||
assert loaded is not None
|
||||
self.assertEqual("docker", loaded.backend)
|
||||
|
||||
def test_backend_field_roundtrips_smolmachines(self):
|
||||
def test_backend_field_roundtrips_firecracker(self):
|
||||
meta = BottleMetadata(
|
||||
identity="dev-b2",
|
||||
agent_name="dev",
|
||||
@@ -249,13 +249,13 @@ class TestBottleMetadataBackend(_FakeHomeMixin, unittest.TestCase):
|
||||
copy_cwd=False,
|
||||
started_at="2026-06-02T00:00:00+00:00",
|
||||
compose_project="",
|
||||
backend="smolmachines",
|
||||
backend="firecracker",
|
||||
)
|
||||
write_metadata(meta)
|
||||
loaded = read_metadata("dev-b2")
|
||||
self.assertIsNotNone(loaded)
|
||||
assert loaded is not None
|
||||
self.assertEqual("smolmachines", loaded.backend)
|
||||
self.assertEqual("firecracker", loaded.backend)
|
||||
|
||||
def test_missing_backend_field_defaults_to_empty(self):
|
||||
# Old state dirs written before PRD 0040 have no backend key.
|
||||
|
||||
@@ -0,0 +1,79 @@
|
||||
"""Unit: the generic `backend` CLI command.
|
||||
|
||||
`./cli.py backend {setup,status} [--backend=NAME]` resolves a backend
|
||||
and dispatches to its `setup()` / `status()` classmethods — no
|
||||
backend-specific commands. Also covers the docker backend's own
|
||||
setup/status logic (the firecracker path is exercised by
|
||||
test_firecracker_backend via netpool)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
import unittest
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from bot_bottle.cli import backend as cmd
|
||||
from bot_bottle.backend.docker import setup as docker_setup
|
||||
|
||||
|
||||
class TestCmdBackendDispatch(unittest.TestCase):
|
||||
def _fake_backend(self):
|
||||
b = MagicMock()
|
||||
b.setup.return_value = 0
|
||||
b.status.return_value = 3
|
||||
return b
|
||||
|
||||
def test_setup_dispatches_to_backend(self):
|
||||
b = self._fake_backend()
|
||||
with patch.object(cmd, "get_bottle_backend", return_value=b) as gbb:
|
||||
rc = cmd.cmd_backend(["setup", "--backend", "firecracker"])
|
||||
gbb.assert_called_once_with("firecracker")
|
||||
b.setup.assert_called_once_with()
|
||||
b.status.assert_not_called()
|
||||
self.assertEqual(0, rc)
|
||||
|
||||
def test_status_dispatches_and_returns_backend_code(self):
|
||||
b = self._fake_backend()
|
||||
with patch.object(cmd, "get_bottle_backend", return_value=b):
|
||||
rc = cmd.cmd_backend(["status", "--backend", "docker"])
|
||||
b.status.assert_called_once_with()
|
||||
self.assertEqual(3, rc)
|
||||
|
||||
def test_no_backend_flag_uses_host_default(self):
|
||||
b = self._fake_backend()
|
||||
with patch.object(cmd, "get_bottle_backend", return_value=b) as gbb:
|
||||
cmd.cmd_backend(["status"])
|
||||
gbb.assert_called_once_with(None)
|
||||
|
||||
def test_unknown_action_exits(self):
|
||||
with self.assertRaises(SystemExit):
|
||||
cmd.cmd_backend(["bogus"])
|
||||
|
||||
def test_unknown_backend_rejected_by_argparse(self):
|
||||
with self.assertRaises(SystemExit):
|
||||
cmd.cmd_backend(["status", "--backend", "nope"])
|
||||
|
||||
|
||||
class TestDockerSetupStatus(unittest.TestCase):
|
||||
def _ok(self):
|
||||
return subprocess.CompletedProcess([], 0, stdout="", stderr="")
|
||||
|
||||
def test_setup_ok_when_docker_and_daemon_present(self):
|
||||
with patch.object(docker_setup.shutil, "which", return_value="/usr/bin/docker"), \
|
||||
patch.object(docker_setup, "_daemon_reachable", return_value=True), \
|
||||
patch.object(docker_setup._util, "runsc_available", return_value=True):
|
||||
self.assertEqual(0, docker_setup.setup())
|
||||
|
||||
def test_setup_fails_when_docker_missing(self):
|
||||
with patch.object(docker_setup.shutil, "which", return_value=None):
|
||||
self.assertEqual(1, docker_setup.setup())
|
||||
|
||||
def test_status_fails_when_daemon_unreachable(self):
|
||||
with patch.object(docker_setup.shutil, "which", return_value="/usr/bin/docker"), \
|
||||
patch.object(docker_setup, "_daemon_reachable", return_value=False), \
|
||||
patch.object(docker_setup._util, "runsc_available", return_value=False):
|
||||
self.assertEqual(1, docker_setup.status())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -23,12 +23,12 @@ def _make_backend(empty: bool = True):
|
||||
class TestCmdCleanup(unittest.TestCase):
|
||||
def test_iterates_every_backend(self):
|
||||
docker, docker_plan = _make_backend(empty=False)
|
||||
smol, smol_plan = _make_backend(empty=False)
|
||||
backends_by_name = {"docker": docker, "smolmachines": smol}
|
||||
fc, fc_plan = _make_backend(empty=False)
|
||||
backends_by_name = {"docker": docker, "firecracker": fc}
|
||||
|
||||
with patch.object(
|
||||
cmd, "known_backend_names",
|
||||
return_value=("docker", "smolmachines"),
|
||||
return_value=("docker", "firecracker"),
|
||||
), patch.object(
|
||||
cmd, "get_bottle_backend",
|
||||
side_effect=lambda name: backends_by_name[name], # type: ignore
|
||||
@@ -38,18 +38,18 @@ class TestCmdCleanup(unittest.TestCase):
|
||||
self.assertEqual(0, cmd.cmd_cleanup([]))
|
||||
|
||||
docker.prepare_cleanup.assert_called_once()
|
||||
smol.prepare_cleanup.assert_called_once()
|
||||
fc.prepare_cleanup.assert_called_once()
|
||||
docker.cleanup.assert_called_once_with(docker_plan)
|
||||
smol.cleanup.assert_called_once_with(smol_plan)
|
||||
fc.cleanup.assert_called_once_with(fc_plan)
|
||||
|
||||
def test_short_circuits_when_all_empty(self):
|
||||
docker, _ = _make_backend(empty=True)
|
||||
smol, _ = _make_backend(empty=True)
|
||||
backends_by_name = {"docker": docker, "smolmachines": smol}
|
||||
fc, _ = _make_backend(empty=True)
|
||||
backends_by_name = {"docker": docker, "firecracker": fc}
|
||||
|
||||
with patch.object(
|
||||
cmd, "known_backend_names",
|
||||
return_value=("docker", "smolmachines"),
|
||||
return_value=("docker", "firecracker"),
|
||||
), patch.object(
|
||||
cmd, "get_bottle_backend",
|
||||
side_effect=lambda name: backends_by_name[name], # type: ignore
|
||||
@@ -59,16 +59,16 @@ class TestCmdCleanup(unittest.TestCase):
|
||||
self.assertEqual(0, cmd.cmd_cleanup([]))
|
||||
prompt.assert_not_called()
|
||||
docker.cleanup.assert_not_called()
|
||||
smol.cleanup.assert_not_called()
|
||||
fc.cleanup.assert_not_called()
|
||||
|
||||
def test_abort_at_prompt_runs_nothing(self):
|
||||
docker, _ = _make_backend(empty=False)
|
||||
smol, _ = _make_backend(empty=True)
|
||||
backends_by_name = {"docker": docker, "smolmachines": smol}
|
||||
fc, _ = _make_backend(empty=True)
|
||||
backends_by_name = {"docker": docker, "firecracker": fc}
|
||||
|
||||
with patch.object(
|
||||
cmd, "known_backend_names",
|
||||
return_value=("docker", "smolmachines"),
|
||||
return_value=("docker", "firecracker"),
|
||||
), patch.object(
|
||||
cmd, "get_bottle_backend",
|
||||
side_effect=lambda name: backends_by_name[name], # type: ignore
|
||||
@@ -77,18 +77,18 @@ class TestCmdCleanup(unittest.TestCase):
|
||||
):
|
||||
self.assertEqual(0, cmd.cmd_cleanup([]))
|
||||
docker.cleanup.assert_not_called()
|
||||
smol.cleanup.assert_not_called()
|
||||
fc.cleanup.assert_not_called()
|
||||
|
||||
def test_skips_empty_plans_when_others_have_work(self):
|
||||
# docker has work, smolmachines doesn't — only docker.cleanup
|
||||
# docker has work, firecracker doesn't — only docker.cleanup
|
||||
# is called.
|
||||
docker, docker_plan = _make_backend(empty=False)
|
||||
smol, _ = _make_backend(empty=True)
|
||||
backends_by_name = {"docker": docker, "smolmachines": smol}
|
||||
fc, _ = _make_backend(empty=True)
|
||||
backends_by_name = {"docker": docker, "firecracker": fc}
|
||||
|
||||
with patch.object(
|
||||
cmd, "known_backend_names",
|
||||
return_value=("docker", "smolmachines"),
|
||||
return_value=("docker", "firecracker"),
|
||||
), patch.object(
|
||||
cmd, "get_bottle_backend",
|
||||
side_effect=lambda name: backends_by_name[name], # type: ignore
|
||||
@@ -97,7 +97,7 @@ class TestCmdCleanup(unittest.TestCase):
|
||||
):
|
||||
cmd.cmd_cleanup([])
|
||||
docker.cleanup.assert_called_once_with(docker_plan)
|
||||
smol.cleanup.assert_not_called()
|
||||
fc.cleanup.assert_not_called()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -83,9 +83,9 @@ class TestCmdCommitSlugArg(_FakeHomeMixin, unittest.TestCase):
|
||||
mock_gf.assert_called_once_with("macos-container")
|
||||
mock_freezer.commit_slug.assert_called_once_with(slug)
|
||||
|
||||
def test_commits_smolmachines_bottle(self):
|
||||
def test_commits_firecracker_bottle(self):
|
||||
slug = "dev-abc12"
|
||||
self._write_meta(slug, "smolmachines")
|
||||
self._write_meta(slug, "firecracker")
|
||||
|
||||
with patch("bot_bottle.cli.commit.get_freezer") as mock_gf:
|
||||
mock_freezer = MagicMock()
|
||||
@@ -93,7 +93,7 @@ class TestCmdCommitSlugArg(_FakeHomeMixin, unittest.TestCase):
|
||||
rc = cmd_commit([slug])
|
||||
|
||||
self.assertEqual(0, rc)
|
||||
mock_gf.assert_called_once_with("smolmachines")
|
||||
mock_gf.assert_called_once_with("firecracker")
|
||||
|
||||
def test_returns_zero_on_commit_cancelled(self):
|
||||
slug = "dev-abc12"
|
||||
|
||||
@@ -35,8 +35,8 @@ class TestStartBackendFlag(unittest.TestCase):
|
||||
return parser
|
||||
|
||||
def test_flag_recognized(self):
|
||||
args = self._build_parser().parse_args(["--backend=smolmachines", "researcher"])
|
||||
self.assertEqual("smolmachines", args.backend)
|
||||
args = self._build_parser().parse_args(["--backend=firecracker", "researcher"])
|
||||
self.assertEqual("firecracker", args.backend)
|
||||
self.assertEqual("researcher", args.name)
|
||||
|
||||
def test_flag_default_none_means_env_or_default_backend(self):
|
||||
@@ -53,7 +53,7 @@ class TestStartBackendFlag(unittest.TestCase):
|
||||
# `--backend` ultimately threads to) prefers the explicit
|
||||
# name over BOT_BOTTLE_BACKEND.
|
||||
from bot_bottle.backend import get_bottle_backend
|
||||
with patch.dict(os.environ, {"BOT_BOTTLE_BACKEND": "smolmachines"}):
|
||||
with patch.dict(os.environ, {"BOT_BOTTLE_BACKEND": "firecracker"}):
|
||||
self.assertEqual("docker", get_bottle_backend("docker").name)
|
||||
|
||||
|
||||
|
||||
@@ -99,25 +99,25 @@ class TestOrphanStateDirs(_FakeHomeMixin, unittest.TestCase):
|
||||
|
||||
def test_protected_identity_skips_dir(self):
|
||||
# `protected_identities` carries slugs that are live in
|
||||
# any backend (smolmachines included). docker's orphan
|
||||
# detection respects them so a running smolmachines
|
||||
# any backend (firecracker included). docker's orphan
|
||||
# detection respects them so a running firecracker
|
||||
# bottle's state dir isn't reaped while the VM is up.
|
||||
bottle_state.write_per_bottle_dockerfile("smol-hhh", "FROM x\n")
|
||||
bottle_state.write_per_bottle_dockerfile("fc-hhh", "FROM x\n")
|
||||
self.assertEqual(
|
||||
[],
|
||||
_list_orphan_state_dirs(set(), {"smol-hhh"}),
|
||||
_list_orphan_state_dirs(set(), {"fc-hhh"}),
|
||||
)
|
||||
|
||||
def test_protected_overrides_no_live_project(self):
|
||||
# A smolmachines bottle has no docker compose project but
|
||||
# A firecracker bottle has no docker compose project but
|
||||
# IS in the protected set; the absence of a project
|
||||
# shouldn't cause a reap.
|
||||
bottle_state.write_per_bottle_dockerfile("smol-iii", "FROM x\n")
|
||||
bottle_state.write_per_bottle_dockerfile("fc-iii", "FROM x\n")
|
||||
self.assertEqual(
|
||||
[],
|
||||
_list_orphan_state_dirs(
|
||||
{"bot-bottle-something-else"}, # different project up
|
||||
{"smol-iii"}, # but smol-iii is live
|
||||
{"fc-iii"}, # but fc-iii is live
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""Unit: image_id / tag / push helpers in
|
||||
"""Unit: commit_container helper in
|
||||
bot_bottle.backend.docker.util (PRD 0023 chunk 4c additions).
|
||||
|
||||
Tests mock `subprocess.run` and assert on argv shape + parsing.
|
||||
@@ -26,47 +26,6 @@ def _fail(stderr: str = "boom") -> subprocess.CompletedProcess: # type: ignore
|
||||
)
|
||||
|
||||
|
||||
class TestImageId(unittest.TestCase):
|
||||
def test_strips_trailing_newline(self):
|
||||
# docker image inspect --format ... emits a trailing newline.
|
||||
with patch.object(
|
||||
docker_mod.subprocess, "run",
|
||||
return_value=_ok(stdout="sha256:abcdef\n"),
|
||||
) as run:
|
||||
self.assertEqual(
|
||||
"sha256:abcdef", docker_mod.image_id("bot-bottle-claude:latest")
|
||||
)
|
||||
argv = run.call_args.args[0]
|
||||
self.assertEqual(
|
||||
["docker", "image", "inspect", "--format", "{{.Id}}", "bot-bottle-claude:latest"],
|
||||
argv,
|
||||
)
|
||||
|
||||
def test_dies_on_inspect_failure(self):
|
||||
with patch.object(
|
||||
docker_mod.subprocess, "run", return_value=_fail("No such image"),
|
||||
), patch.object(
|
||||
docker_mod, "die", side_effect=SystemExit("die"),
|
||||
) as die:
|
||||
with self.assertRaises(SystemExit):
|
||||
docker_mod.image_id("missing:tag")
|
||||
die.assert_called_once()
|
||||
self.assertIn("missing:tag", die.call_args.args[0])
|
||||
|
||||
|
||||
class TestSave(unittest.TestCase):
|
||||
def test_save_runs_docker_save(self):
|
||||
with patch.object(
|
||||
docker_mod.subprocess, "run", return_value=_ok(),
|
||||
) as run:
|
||||
docker_mod.save("bot-bottle-claude:latest", "/tmp/img.tar")
|
||||
argv = run.call_args.args[0]
|
||||
self.assertEqual(
|
||||
["docker", "save", "bot-bottle-claude:latest", "-o", "/tmp/img.tar"],
|
||||
argv,
|
||||
)
|
||||
|
||||
|
||||
class TestCommitContainer(unittest.TestCase):
|
||||
def test_runs_docker_commit(self):
|
||||
with patch.object(
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
"""Unit: egress_entrypoint.sh argv construction (PRD 0023 chunk 3).
|
||||
|
||||
The egress entrypoint is a small POSIX-sh script that builds
|
||||
the mitmdump argv from env vars. The smolmachines backend
|
||||
the mitmdump argv from env vars. A VM backend (e.g. firecracker)
|
||||
controls egress's bind address via EGRESS_LISTEN_HOST; the
|
||||
docker backend leaves it unset and gets mitmdump's default
|
||||
(all interfaces).
|
||||
@@ -67,7 +67,7 @@ class TestEgressEntrypointArgv(unittest.TestCase):
|
||||
)
|
||||
|
||||
def test_listen_host_127_0_0_1_emits_flag(self):
|
||||
# smolmachines backend sets EGRESS_LISTEN_HOST=127.0.0.1
|
||||
# A VM backend sets EGRESS_LISTEN_HOST=127.0.0.1
|
||||
# to scope egress to localhost inside the bundle.
|
||||
argv = _run_entrypoint({"EGRESS_LISTEN_HOST": "127.0.0.1"})
|
||||
self.assertIn("--listen-host\n127.0.0.1", argv)
|
||||
@@ -79,7 +79,7 @@ class TestEgressEntrypointArgv(unittest.TestCase):
|
||||
self.assertNotIn("--listen-host", argv)
|
||||
|
||||
def test_upstream_mode_combined_with_listen_host(self):
|
||||
# smolmachines mode also sets EGRESS_UPSTREAM_PROXY so
|
||||
# VM-backend mode also sets EGRESS_UPSTREAM_PROXY so
|
||||
# both flags should compose correctly.
|
||||
argv = _run_entrypoint({
|
||||
"EGRESS_UPSTREAM_PROXY": "http://192.168.50.2:8888",
|
||||
|
||||
@@ -0,0 +1,238 @@
|
||||
"""Unit: Firecracker backend helpers.
|
||||
|
||||
Covers the pieces that run without KVM/root: pool IP math + config
|
||||
renderers + allocation, the SSH-backed bottle's argv construction, and
|
||||
the VM boot-arg assembly.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import os
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from typing import Any, cast
|
||||
from unittest.mock import patch
|
||||
|
||||
from bot_bottle.backend.firecracker import firecracker_vm, netpool
|
||||
from bot_bottle.backend.firecracker.bottle import FirecrackerBottle
|
||||
|
||||
|
||||
def _bottle(**kw: Any) -> FirecrackerBottle:
|
||||
defaults = dict(
|
||||
name="bot-bottle-dev-abc",
|
||||
private_key=Path("/tmp/key"),
|
||||
guest_ip="100.64.0.1",
|
||||
)
|
||||
defaults.update(kw)
|
||||
return FirecrackerBottle(**defaults) # type: ignore[arg-type]
|
||||
|
||||
|
||||
class TestNetpoolSlots(unittest.TestCase):
|
||||
def test_slot_ip_math_31_pairs(self):
|
||||
with patch.dict(os.environ, {"BOT_BOTTLE_FC_IP_BASE": "100.64.0.0"}):
|
||||
s0, s1 = netpool.slot(0), netpool.slot(1)
|
||||
self.assertEqual(("bbfc0", "100.64.0.0", "100.64.0.1"),
|
||||
(s0.iface, s0.host_ip, s0.guest_ip))
|
||||
self.assertEqual(("bbfc1", "100.64.0.2", "100.64.0.3"),
|
||||
(s1.iface, s1.host_ip, s1.guest_ip))
|
||||
|
||||
def test_guest_cidr_is_31(self):
|
||||
with patch.dict(os.environ, {"BOT_BOTTLE_FC_IP_BASE": "100.64.0.0"}):
|
||||
self.assertEqual("100.64.0.1/31", netpool.slot(0).guest_cidr)
|
||||
|
||||
def test_default_base_avoids_cgnat_and_common_private_ranges(self):
|
||||
# The default base must NOT sit in 100.64.0.0/10 (RFC-6598 CGNAT,
|
||||
# which Tailscale hands node addresses from); it's an obscure
|
||||
# RFC-1918 block instead.
|
||||
with patch.dict(os.environ, {}, clear=True):
|
||||
self.assertEqual("10.243.0.0", netpool.ip_base())
|
||||
|
||||
def test_pool_size_env_override(self):
|
||||
with patch.dict(os.environ, {"BOT_BOTTLE_FC_POOL_SIZE": "4"}):
|
||||
self.assertEqual(4, netpool.pool_size())
|
||||
self.assertEqual(4, len(netpool.all_slots()))
|
||||
|
||||
|
||||
class TestNetpoolRenderers(unittest.TestCase):
|
||||
def test_nixos_module_has_table_and_taps(self):
|
||||
with patch.dict(os.environ, {"BOT_BOTTLE_FC_POOL_SIZE": "2"}):
|
||||
out = netpool.render_nixos_module()
|
||||
self.assertIn('networking.nftables.tables."bot_bottle_fc"', out)
|
||||
self.assertIn('Name = "bbfc0"', out)
|
||||
self.assertIn('Name = "bbfc1"', out)
|
||||
self.assertIn("net.ipv4.ip_forward", out)
|
||||
# fail-closed isolation rules present
|
||||
self.assertIn('iifname != "bbfc*" return', out)
|
||||
self.assertIn("ct status dnat accept", out)
|
||||
|
||||
def test_shell_setup_reflects_overrides(self):
|
||||
with patch.dict(os.environ, {"BOT_BOTTLE_FC_POOL_SIZE": "3"}):
|
||||
out = netpool.render_shell_setup()
|
||||
self.assertIn("BOT_BOTTLE_FC_POOL_SIZE=3", out)
|
||||
self.assertIn("firecracker-netpool.sh up", out)
|
||||
|
||||
|
||||
class TestNetpoolOverlap(unittest.TestCase):
|
||||
"""`overlapping_routes()` flags a pool base that collides with an
|
||||
existing host route (Tailscale CGNAT peer, docker/libvirt bridge,
|
||||
LAN) and ignores our own bbfc TAPs + the default route."""
|
||||
|
||||
def _routes(self, entries: list[dict]) -> object:
|
||||
import json
|
||||
import subprocess
|
||||
|
||||
def fake_run(argv, **kwargs):
|
||||
return subprocess.CompletedProcess(
|
||||
argv, 0, stdout=json.dumps(entries), stderr="",
|
||||
)
|
||||
return patch.object(netpool.subprocess, "run", side_effect=fake_run)
|
||||
|
||||
def test_flags_route_overlapping_pool(self):
|
||||
# base 100.64.0.0 + a Tailscale peer /32 inside the pool window.
|
||||
with patch.dict(os.environ, {"BOT_BOTTLE_FC_IP_BASE": "100.64.0.0",
|
||||
"BOT_BOTTLE_FC_POOL_SIZE": "8"}), \
|
||||
self._routes([
|
||||
{"dst": "default", "dev": "enp4s0"},
|
||||
{"dst": "100.64.0.5", "dev": "tailscale0"},
|
||||
]):
|
||||
conflicts = netpool.overlapping_routes()
|
||||
self.assertEqual(1, len(conflicts))
|
||||
self.assertEqual("tailscale0", conflicts[0].dev)
|
||||
|
||||
def test_ignores_own_taps_and_default(self):
|
||||
with patch.dict(os.environ, {"BOT_BOTTLE_FC_IP_BASE": "10.243.0.0",
|
||||
"BOT_BOTTLE_FC_POOL_SIZE": "8"}), \
|
||||
self._routes([
|
||||
{"dst": "default", "dev": "enp4s0"},
|
||||
{"dst": "10.243.0.0/31", "dev": "bbfc0"},
|
||||
{"dst": "192.168.1.0/24", "dev": "enp4s0"},
|
||||
]):
|
||||
self.assertEqual([], netpool.overlapping_routes())
|
||||
|
||||
def test_empty_when_ip_missing(self):
|
||||
with self._routes([]) as run:
|
||||
run.side_effect = FileNotFoundError()
|
||||
self.assertEqual([], netpool.overlapping_routes())
|
||||
|
||||
|
||||
class TestNetpoolAllocation(unittest.TestCase):
|
||||
def test_allocate_is_mutually_exclusive(self):
|
||||
with patch.dict(os.environ, {"BOT_BOTTLE_FC_POOL_SIZE": "1"}):
|
||||
# First allocation grabs the only slot; the lock is held
|
||||
# until the handle is closed, so a second call must fail
|
||||
# over (and here, exhaust the pool).
|
||||
slot, lock = netpool.allocate("first")
|
||||
self.addCleanup(lock.close)
|
||||
self.assertEqual("bbfc0", slot.iface)
|
||||
with patch.object(netpool, "die",
|
||||
side_effect=SystemExit("exhausted")):
|
||||
with self.assertRaises(SystemExit):
|
||||
netpool.allocate("second")
|
||||
|
||||
def test_allocate_releases_on_close(self):
|
||||
with patch.dict(os.environ, {"BOT_BOTTLE_FC_POOL_SIZE": "1"}):
|
||||
slot, lock = netpool.allocate("first")
|
||||
lock.close()
|
||||
# Slot is free again after close.
|
||||
slot2, lock2 = netpool.allocate("second")
|
||||
self.addCleanup(lock2.close)
|
||||
self.assertEqual(slot.iface, slot2.iface)
|
||||
|
||||
|
||||
class TestBottleAgentArgv(unittest.TestCase):
|
||||
def test_interactive_uses_ssh_tty_and_runuser(self):
|
||||
argv = _bottle().agent_argv([], tty=True)
|
||||
self.assertEqual("ssh", argv[0])
|
||||
self.assertIn("-t", argv)
|
||||
self.assertIn("100.64.0.1", " ".join(argv))
|
||||
idx = argv.index("--")
|
||||
self.assertEqual(["runuser", "-u", "node", "--"], argv[idx + 1:idx + 5])
|
||||
self.assertIn("claude", argv)
|
||||
|
||||
def test_non_interactive_has_no_tty(self):
|
||||
argv = _bottle().agent_argv([], tty=False)
|
||||
self.assertEqual("ssh", argv[0])
|
||||
self.assertNotIn("-t", argv)
|
||||
|
||||
def test_appends_extra_args_after_command(self):
|
||||
argv = _bottle().agent_argv(
|
||||
["--dangerously-skip-permissions", "--continue"], tty=False,
|
||||
)
|
||||
idx = argv.index("claude")
|
||||
self.assertEqual(
|
||||
["claude", "--dangerously-skip-permissions", "--continue"],
|
||||
argv[idx:],
|
||||
)
|
||||
|
||||
def test_prompt_file_flag_injected(self):
|
||||
argv = _bottle(
|
||||
prompt_path_in_guest="/home/node/.bot-bottle-prompt.txt",
|
||||
).agent_argv(["--continue"], tty=False)
|
||||
idx = argv.index("claude")
|
||||
self.assertEqual(
|
||||
["claude", "--continue",
|
||||
"--append-system-prompt-file", "/home/node/.bot-bottle-prompt.txt"],
|
||||
argv[idx:],
|
||||
)
|
||||
|
||||
def test_workdir_wraps_command(self):
|
||||
argv = _bottle(agent_workdir="/home/node/workspace").agent_argv([], tty=False)
|
||||
self.assertIn("sh", argv)
|
||||
joined = " ".join(argv)
|
||||
self.assertIn("cd /home/node/workspace", joined)
|
||||
|
||||
def test_guest_env_injected(self):
|
||||
argv = _bottle(guest_env={"HTTPS_PROXY": "http://100.64.0.0:9099"}).agent_argv(
|
||||
[], tty=False,
|
||||
)
|
||||
self.assertIn("HTTPS_PROXY=http://100.64.0.0:9099", argv)
|
||||
self.assertIn("HOME=/home/node", argv)
|
||||
self.assertIn("USER=node", argv)
|
||||
|
||||
|
||||
class TestSetupScriptConsistency(unittest.TestCase):
|
||||
"""The shell setup script duplicates the pool defaults; keep them in
|
||||
lockstep with the Python constants so the two setup paths agree."""
|
||||
|
||||
def _script(self) -> str:
|
||||
root = Path(__file__).resolve().parent.parent.parent
|
||||
return (root / "scripts" / "firecracker-netpool.sh").read_text()
|
||||
|
||||
def test_defaults_match_python(self):
|
||||
script = self._script()
|
||||
with patch.dict(os.environ, {}, clear=True):
|
||||
self.assertIn(f'POOL_SIZE:-{netpool.pool_size()}', script)
|
||||
self.assertIn(f'BOT_BOTTLE_FC_IP_BASE:-{netpool.ip_base()}', script)
|
||||
self.assertIn(f'IFACE_PREFIX:-{netpool.IFACE_PREFIX}', script)
|
||||
|
||||
def test_table_name_and_ports_match(self):
|
||||
script = self._script()
|
||||
self.assertIn(f'TABLE="{netpool.NFT_TABLE}"', script)
|
||||
for port in netpool.SIDECAR_PORTS:
|
||||
self.assertIn(str(port), script)
|
||||
|
||||
|
||||
class TestBootArgs(unittest.TestCase):
|
||||
def test_boot_args_carry_ip_and_pubkey(self):
|
||||
args = firecracker_vm._boot_args(
|
||||
guest_ip="100.64.0.1", host_ip="100.64.0.0", pubkey="ssh-ed25519 AAAA x",
|
||||
)
|
||||
self.assertIn("ip=100.64.0.1::100.64.0.0:255.255.255.254::eth0:off", args)
|
||||
self.assertIn("init=/bb-init", args)
|
||||
b64 = base64.b64encode(b"ssh-ed25519 AAAA x").decode()
|
||||
self.assertIn(f"bb_pubkey={b64}", args)
|
||||
|
||||
def test_config_has_rootfs_and_tap(self):
|
||||
cfg = cast(Any, firecracker_vm._config(
|
||||
rootfs=Path("/run/rootfs.ext4"), tap="bbfc0",
|
||||
guest_ip="100.64.0.1", host_ip="100.64.0.0", pubkey="k",
|
||||
vcpus=2, mem_mib=2048, guest_mac="06:00:AC:10:00:02",
|
||||
))
|
||||
self.assertEqual("/run/rootfs.ext4", cfg["drives"][0]["path_on_host"])
|
||||
self.assertFalse(cfg["drives"][0]["is_read_only"])
|
||||
self.assertEqual("bbfc0", cfg["network-interfaces"][0]["host_dev_name"])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,4 +1,4 @@
|
||||
"""Unit: BottlePlan.print parity across Docker and smolmachines (PRD 0044).
|
||||
"""Unit: BottlePlan.print parity across Docker and firecracker (PRD 0044).
|
||||
|
||||
Both backends inherit a single concrete print() from BottlePlan. These
|
||||
tests verify that identical git_gate_plan and egress_plan inputs produce
|
||||
@@ -16,7 +16,7 @@ from pathlib import Path
|
||||
from bot_bottle.agent_provider import AgentProvisionPlan
|
||||
from bot_bottle.backend import BottleSpec
|
||||
from bot_bottle.backend.docker.bottle_plan import DockerBottlePlan
|
||||
from bot_bottle.backend.smolmachines.bottle_plan import SmolmachinesBottlePlan
|
||||
from bot_bottle.backend.firecracker.bottle_plan import FirecrackerBottlePlan
|
||||
from bot_bottle.egress import EgressPlan, EgressRoute
|
||||
from bot_bottle.git_gate import GitGatePlan, GitGateUpstream
|
||||
from bot_bottle.manifest import Manifest, ManifestIndex
|
||||
@@ -107,9 +107,9 @@ def _docker_plan(spec: BottleSpec, manifest: Manifest, tmp: str) -> DockerBottle
|
||||
)
|
||||
|
||||
|
||||
def _smolmachines_plan(spec: BottleSpec, manifest: Manifest, tmp: str) -> SmolmachinesBottlePlan:
|
||||
def _firecracker_plan(spec: BottleSpec, manifest: Manifest, tmp: str) -> FirecrackerBottlePlan:
|
||||
stage = Path(tmp)
|
||||
return SmolmachinesBottlePlan(
|
||||
return FirecrackerBottlePlan(
|
||||
spec=spec,
|
||||
manifest=manifest,
|
||||
stage_dir=stage,
|
||||
@@ -118,14 +118,11 @@ def _smolmachines_plan(spec: BottleSpec, manifest: Manifest, tmp: str) -> Smolma
|
||||
supervise_plan=None,
|
||||
agent_provision=_agent_provision(tmp),
|
||||
slug="test-00001",
|
||||
bundle_subnet="10.99.0.0/24",
|
||||
bundle_gateway="10.99.0.1",
|
||||
bundle_ip="10.99.0.2",
|
||||
guest_env={"HTTPS_PROXY": "http://127.0.0.1:9999"},
|
||||
forwarded_env={},
|
||||
)
|
||||
|
||||
|
||||
def _capture_print(plan: DockerBottlePlan | SmolmachinesBottlePlan) -> list[str]:
|
||||
def _capture_print(plan: DockerBottlePlan | FirecrackerBottlePlan) -> list[str]:
|
||||
buf = io.StringIO()
|
||||
orig = sys.stderr
|
||||
sys.stderr = buf
|
||||
@@ -144,7 +141,7 @@ class TestGitGatePrintParity(unittest.TestCase):
|
||||
manifest = _INDEX.load_for_agent("demo")
|
||||
spec = _spec(_INDEX, self._tmp)
|
||||
self._docker_lines = _capture_print(_docker_plan(spec, manifest, self._tmp))
|
||||
self._smol_lines = _capture_print(_smolmachines_plan(spec, manifest, self._tmp))
|
||||
self._fc_lines = _capture_print(_firecracker_plan(spec, manifest, self._tmp))
|
||||
|
||||
def _git_gate_lines(self, lines: list[str]) -> list[str]:
|
||||
return [ln for ln in lines if "git gate" in ln]
|
||||
@@ -154,15 +151,15 @@ class TestGitGatePrintParity(unittest.TestCase):
|
||||
self.assertEqual(1, len(git_lines))
|
||||
self.assertIn("myrepo → gitea.example.com:30009", git_lines[0])
|
||||
|
||||
def test_smolmachines_renders_name_arrow_host_port(self) -> None:
|
||||
git_lines = self._git_gate_lines(self._smol_lines)
|
||||
def test_firecracker_renders_name_arrow_host_port(self) -> None:
|
||||
git_lines = self._git_gate_lines(self._fc_lines)
|
||||
self.assertEqual(1, len(git_lines))
|
||||
self.assertIn("myrepo → gitea.example.com:30009", git_lines[0])
|
||||
|
||||
def test_git_gate_lines_match_across_backends(self) -> None:
|
||||
self.assertEqual(
|
||||
self._git_gate_lines(self._docker_lines),
|
||||
self._git_gate_lines(self._smol_lines),
|
||||
self._git_gate_lines(self._fc_lines),
|
||||
)
|
||||
|
||||
|
||||
@@ -174,7 +171,7 @@ class TestEgressPrintParity(unittest.TestCase):
|
||||
manifest = _INDEX.load_for_agent("demo")
|
||||
spec = _spec(_INDEX, self._tmp)
|
||||
self._docker_lines = _capture_print(_docker_plan(spec, manifest, self._tmp))
|
||||
self._smol_lines = _capture_print(_smolmachines_plan(spec, manifest, self._tmp))
|
||||
self._fc_lines = _capture_print(_firecracker_plan(spec, manifest, self._tmp))
|
||||
|
||||
def _egress_section(self, lines: list[str]) -> list[str]:
|
||||
"""Return lines from the egress label through the last route entry.
|
||||
@@ -210,8 +207,8 @@ class TestEgressPrintParity(unittest.TestCase):
|
||||
combined = "\n".join(self._egress_section(self._docker_lines))
|
||||
self.assertIn("api.example.com [auth:bearer]", combined)
|
||||
|
||||
def test_smolmachines_includes_auth_annotation(self) -> None:
|
||||
combined = "\n".join(self._egress_section(self._smol_lines))
|
||||
def test_firecracker_includes_auth_annotation(self) -> None:
|
||||
combined = "\n".join(self._egress_section(self._fc_lines))
|
||||
self.assertIn("api.example.com [auth:bearer]", combined)
|
||||
|
||||
def test_unauthenticated_route_has_no_annotation(self) -> None:
|
||||
@@ -222,7 +219,7 @@ class TestEgressPrintParity(unittest.TestCase):
|
||||
def test_egress_lines_match_across_backends(self) -> None:
|
||||
self.assertEqual(
|
||||
self._egress_section(self._docker_lines),
|
||||
self._egress_section(self._smol_lines),
|
||||
self._egress_section(self._fc_lines),
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -53,15 +53,16 @@ class TestGitGateGitconfigRender(unittest.TestCase):
|
||||
self.assertNotIn("pushInsteadOf", out)
|
||||
|
||||
def test_gate_host_can_be_ip_port_form(self):
|
||||
# The smolmachines backend's TSI-allowlisted guest has no
|
||||
# DNS, so it dials git-gate via `<bundle_ip>:<port>`.
|
||||
# A VM backend's guest may have no DNS (e.g. firecracker over
|
||||
# the point-to-point TAP), so it dials git-gate via
|
||||
# `<bundle_ip>:<port>`.
|
||||
bottle = fixture_with_git().bottles["dev"]
|
||||
out = git_gate_render_gitconfig(bottle.git, "192.168.20.2:9418")
|
||||
self.assertIn(
|
||||
'[url "git://192.168.20.2:9418/bot-bottle.git"]', out,
|
||||
)
|
||||
|
||||
def test_scheme_can_be_http_for_smolmachines(self):
|
||||
def test_scheme_can_be_http_for_vm_backend(self):
|
||||
bottle = fixture_with_git().bottles["dev"]
|
||||
out = git_gate_render_gitconfig(
|
||||
bottle.git, "127.0.0.16:57001", scheme="http",
|
||||
|
||||
@@ -1,195 +0,0 @@
|
||||
"""Unit: SmolmachinesBottle's `agent_argv` builder.
|
||||
|
||||
The dashboard's tmux pane-respawn path calls `bottle.agent_argv`
|
||||
directly (it spawns claude inside a tmux pane rather than as a
|
||||
child of the current process), so the argv shape is the
|
||||
non-trivial part. `exec_agent` is a thin wrapper around the same
|
||||
builder + `subprocess.run`; we lock the shape here.
|
||||
|
||||
The TTY-mode argv is wrapped in the pty_resize helper (issue #82
|
||||
workaround); we assert both the wrapper presence and the wrapped
|
||||
smolvm argv shape. Non-TTY mode skips the wrapper.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
from bot_bottle.backend.smolmachines import pty_resize as _pty_resize
|
||||
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
|
||||
|
||||
|
||||
def _bottle(prompt_path: str | None = None, **env: str) -> SmolmachinesBottle:
|
||||
return SmolmachinesBottle(
|
||||
"bot-bottle-dev-abc",
|
||||
prompt_path=prompt_path,
|
||||
guest_env=env,
|
||||
)
|
||||
|
||||
|
||||
def _pi_bottle(prompt_path: str | None = None) -> SmolmachinesBottle:
|
||||
return SmolmachinesBottle(
|
||||
"bot-bottle-dev-abc",
|
||||
prompt_path=prompt_path,
|
||||
agent_command="pi",
|
||||
agent_prompt_mode="append_system_prompt",
|
||||
)
|
||||
|
||||
|
||||
def _workspace_bottle() -> SmolmachinesBottle:
|
||||
return SmolmachinesBottle(
|
||||
"bot-bottle-dev-abc",
|
||||
prompt_path=None,
|
||||
agent_workdir="/home/node/workspace",
|
||||
)
|
||||
|
||||
|
||||
def _unwrap(argv: list[str]) -> list[str]:
|
||||
"""Strip the pty_resize wrapper from the front of a TTY-mode
|
||||
argv, return the inner smolvm argv. Mirrors what the kernel
|
||||
sees inside the wrapper's `subprocess.Popen`."""
|
||||
idx = argv.index("--")
|
||||
return argv[idx + 1:]
|
||||
|
||||
|
||||
class TestClaudeArgvWrapped(unittest.TestCase):
|
||||
"""TTY-mode argv: pty_resize wrapper + inner smolvm exec."""
|
||||
|
||||
def test_pty_resize_wrapper_prefix(self):
|
||||
argv = _bottle().agent_argv([])
|
||||
# Absolute script path (not `-m <dotted>`) so the tmux
|
||||
# pane's cwd doesn't matter — see the `_PTY_RESIZE_SCRIPT`
|
||||
# docstring in bottle.py.
|
||||
self.assertEqual(
|
||||
[
|
||||
sys.executable, _pty_resize.__file__,
|
||||
"bot-bottle-dev-abc", "--",
|
||||
],
|
||||
argv[:4],
|
||||
)
|
||||
|
||||
def test_minimal_inner_argv_no_prompt(self):
|
||||
argv = _unwrap(_bottle().agent_argv([]))
|
||||
self.assertEqual(
|
||||
[
|
||||
"smolvm", "machine", "exec", "--name",
|
||||
"bot-bottle-dev-abc",
|
||||
"-i", "-t",
|
||||
"--",
|
||||
"runuser", "-u", "node", "--",
|
||||
"env", "HOME=/home/node", "USER=node",
|
||||
(
|
||||
"PATH=/home/node/.local/bin:"
|
||||
"/home/node/.codex/packages/standalone/current/bin:"
|
||||
"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
|
||||
),
|
||||
"claude",
|
||||
],
|
||||
argv,
|
||||
)
|
||||
|
||||
def test_appends_passed_args_after_claude(self):
|
||||
argv = _unwrap(_bottle().agent_argv(
|
||||
["--dangerously-skip-permissions", "--continue"],
|
||||
))
|
||||
self.assertEqual(
|
||||
["claude", "--dangerously-skip-permissions", "--continue"],
|
||||
argv[argv.index("claude"):],
|
||||
)
|
||||
|
||||
def test_appends_prompt_file_flag_when_set(self):
|
||||
argv = _unwrap(
|
||||
_bottle("/home/node/.bot-bottle-prompt.txt").agent_argv(
|
||||
["--dangerously-skip-permissions"],
|
||||
)
|
||||
)
|
||||
self.assertEqual(
|
||||
[
|
||||
"claude",
|
||||
"--append-system-prompt-file",
|
||||
"/home/node/.bot-bottle-prompt.txt",
|
||||
"--dangerously-skip-permissions",
|
||||
],
|
||||
argv[argv.index("claude"):],
|
||||
)
|
||||
|
||||
def test_no_prompt_flag_when_none(self):
|
||||
argv = _bottle(None).agent_argv(["--continue"])
|
||||
self.assertNotIn("--append-system-prompt-file", argv)
|
||||
|
||||
def test_empty_prompt_string_is_treated_as_no_prompt(self):
|
||||
argv = _bottle("").agent_argv(["--continue"])
|
||||
self.assertNotIn("--append-system-prompt-file", argv)
|
||||
|
||||
def test_guest_env_forwarded_as_e_flags(self):
|
||||
argv = _unwrap(_bottle(
|
||||
None,
|
||||
HTTPS_PROXY="http://127.0.0.1:1234",
|
||||
NO_PROXY="localhost",
|
||||
).agent_argv([]))
|
||||
self.assertIn("env", argv)
|
||||
self.assertIn("HTTPS_PROXY=http://127.0.0.1:1234", argv)
|
||||
self.assertIn("NO_PROXY=localhost", argv)
|
||||
|
||||
def test_guest_env_path_overrides_default_path(self):
|
||||
argv = _unwrap(_bottle(None, PATH="/custom/bin").agent_argv([]))
|
||||
self.assertIn("PATH=/custom/bin", argv)
|
||||
self.assertFalse(any(
|
||||
item.startswith("PATH=/home/node/.local/bin")
|
||||
for item in argv
|
||||
))
|
||||
|
||||
def test_runuser_switch_precedes_claude(self):
|
||||
# The dashboard's `_build_resume_argv_with_fallback` finds
|
||||
# the `claude` token to split exec-framing from the claude
|
||||
# tail. `runuser -u node --` must sit on the prefix side so
|
||||
# the shell wrap inherits the UID switch.
|
||||
argv = _bottle().agent_argv([])
|
||||
runuser_idx = argv.index("runuser")
|
||||
self.assertEqual(
|
||||
["runuser", "-u", "node", "--", "env"],
|
||||
argv[runuser_idx:runuser_idx + 5],
|
||||
)
|
||||
|
||||
def test_pi_provider_appends_system_prompt_without_print_mode(self):
|
||||
argv = _unwrap(
|
||||
_pi_bottle("/home/node/.bot-bottle-prompt.txt").agent_argv([])
|
||||
)
|
||||
self.assertEqual(
|
||||
["pi", "--append-system-prompt", "/home/node/.bot-bottle-prompt.txt"],
|
||||
argv[argv.index("pi"):],
|
||||
)
|
||||
self.assertNotIn("-p", argv)
|
||||
|
||||
def test_workspace_workdir_wraps_agent_command(self):
|
||||
argv = _unwrap(_workspace_bottle().agent_argv([]))
|
||||
agent_idx = argv.index("claude")
|
||||
self.assertEqual(
|
||||
[
|
||||
"sh", "-lc",
|
||||
"cd /home/node/workspace && exec \"$@\"",
|
||||
"bot-bottle-agent",
|
||||
"claude",
|
||||
],
|
||||
argv[agent_idx - 4:agent_idx + 1],
|
||||
)
|
||||
|
||||
|
||||
class TestClaudeArgvNoTTY(unittest.TestCase):
|
||||
"""`tty=False` paths skip the pty_resize wrapper — there's no
|
||||
PTY whose SIGWINCH we'd need to bridge."""
|
||||
|
||||
def test_no_wrapper_when_tty_false(self):
|
||||
argv = _bottle().agent_argv([], tty=False)
|
||||
self.assertEqual("smolvm", argv[0])
|
||||
self.assertFalse(any("pty_resize" in a for a in argv))
|
||||
|
||||
def test_tty_false_drops_it_flags(self):
|
||||
argv = _bottle().agent_argv([], tty=False)
|
||||
self.assertNotIn("-i", argv)
|
||||
self.assertNotIn("-t", argv)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,150 +0,0 @@
|
||||
"""Unit: smolmachines backend cleanup (`cleanup.py` +
|
||||
`bottle_cleanup_plan.py`).
|
||||
|
||||
Tests mock `subprocess.run` + `has_backend` so they execute
|
||||
without docker / smolvm on PATH. Each cleanup step verifies argv
|
||||
shape; teardown verifies order (machines → bundles → networks)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
from bot_bottle import backend as backend_mod
|
||||
from bot_bottle.backend.smolmachines import cleanup
|
||||
from bot_bottle.backend.smolmachines.bottle_cleanup_plan import (
|
||||
SmolmachinesBottleCleanupPlan,
|
||||
)
|
||||
|
||||
|
||||
def _ok(stdout: str = "", stderr: str = "") -> subprocess.CompletedProcess: # type: ignore
|
||||
return subprocess.CompletedProcess(
|
||||
args=[], returncode=0, stdout=stdout, stderr=stderr,
|
||||
)
|
||||
|
||||
|
||||
class TestPrepareCleanup(unittest.TestCase):
|
||||
def test_empty_when_nothing_running(self):
|
||||
with patch.object(cleanup, "_smolvm") as smolvm, \
|
||||
patch.object(cleanup.subprocess, "run") as run, \
|
||||
patch.object(backend_mod, "has_backend", return_value=True):
|
||||
smolvm.is_available.return_value = True
|
||||
run.return_value = _ok(stdout="[]")
|
||||
plan = cleanup.prepare_cleanup()
|
||||
self.assertTrue(plan.empty)
|
||||
|
||||
def test_lists_machines_bundles_networks(self):
|
||||
def fake_run(argv, *a, **kw): # type: ignore
|
||||
if argv[:3] == ["smolvm", "machine", "ls"]:
|
||||
return _ok(stdout=(
|
||||
'[{"name":"bot-bottle-a-1","state":"running"},'
|
||||
' {"name":"bot-bottle-b-2","state":"created"},'
|
||||
' {"name":"unrelated","state":"running"}]'
|
||||
))
|
||||
if argv[:2] == ["docker", "ps"]:
|
||||
return _ok(stdout=(
|
||||
"bot-bottle-sidecars-a-1\n"
|
||||
"bot-bottle-sidecars-b-2\n"
|
||||
))
|
||||
if argv[:3] == ["docker", "network", "ls"]:
|
||||
return _ok(stdout=(
|
||||
"bot-bottle-bundle-a-1\n"
|
||||
"bot-bottle-bundle-b-2\n"
|
||||
))
|
||||
return _ok()
|
||||
|
||||
with patch.object(cleanup, "_smolvm") as smolvm, \
|
||||
patch.object(cleanup.subprocess, "run", side_effect=fake_run), \
|
||||
patch.object(backend_mod, "has_backend", return_value=True):
|
||||
smolvm.is_available.return_value = True
|
||||
plan = cleanup.prepare_cleanup()
|
||||
|
||||
# `unrelated` filtered out (no bot-bottle- prefix).
|
||||
self.assertEqual(
|
||||
("bot-bottle-a-1", "bot-bottle-b-2"),
|
||||
plan.machines,
|
||||
)
|
||||
self.assertEqual(
|
||||
("bot-bottle-sidecars-a-1", "bot-bottle-sidecars-b-2"),
|
||||
plan.bundles,
|
||||
)
|
||||
self.assertEqual(
|
||||
("bot-bottle-bundle-a-1", "bot-bottle-bundle-b-2"),
|
||||
plan.networks,
|
||||
)
|
||||
|
||||
def test_no_smolvm_means_no_machines(self):
|
||||
with patch.object(cleanup, "_smolvm") as smolvm, \
|
||||
patch.object(cleanup.subprocess, "run", return_value=_ok()), \
|
||||
patch.object(backend_mod, "has_backend", return_value=True):
|
||||
smolvm.is_available.return_value = False
|
||||
plan = cleanup.prepare_cleanup()
|
||||
self.assertEqual((), plan.machines)
|
||||
|
||||
|
||||
class TestCleanup(unittest.TestCase):
|
||||
def test_machines_stopped_then_deleted_then_bundles_then_networks(self):
|
||||
plan = SmolmachinesBottleCleanupPlan(
|
||||
machines=("bot-bottle-a-1",),
|
||||
bundles=("bot-bottle-sidecars-a-1",),
|
||||
networks=("bot-bottle-bundle-a-1",),
|
||||
)
|
||||
calls: list[list[str]] = []
|
||||
|
||||
def fake_run(argv, *a, **kw): # type: ignore
|
||||
calls.append(list(argv[:4]))
|
||||
return _ok()
|
||||
|
||||
with patch.object(cleanup.subprocess, "run", side_effect=fake_run):
|
||||
cleanup.cleanup(plan)
|
||||
|
||||
# Stop precedes delete precedes bundle rm precedes network rm.
|
||||
self.assertEqual(
|
||||
["smolvm", "machine", "stop", "--name"], calls[0],
|
||||
)
|
||||
self.assertEqual(
|
||||
["smolvm", "machine", "delete", "-f"], calls[1],
|
||||
)
|
||||
self.assertEqual(
|
||||
["docker", "rm", "-f", "bot-bottle-sidecars-a-1"], calls[2],
|
||||
)
|
||||
self.assertEqual(
|
||||
["docker", "network", "rm", "bot-bottle-bundle-a-1"], calls[3],
|
||||
)
|
||||
|
||||
def test_failures_are_warnings_not_fatal(self):
|
||||
# smolvm machine delete -f returning non-zero should warn
|
||||
# but continue with bundles + networks. The cleanup is
|
||||
# idempotent on success and tries to remove every resource.
|
||||
plan = SmolmachinesBottleCleanupPlan(
|
||||
machines=("bot-bottle-a-1",),
|
||||
bundles=("bot-bottle-sidecars-a-1",),
|
||||
networks=(),
|
||||
)
|
||||
results = iter([
|
||||
_ok(), # stop succeeds
|
||||
subprocess.CompletedProcess(
|
||||
args=[], returncode=1, stdout="", stderr="boom"
|
||||
), # delete fails
|
||||
_ok(), # bundle rm succeeds
|
||||
])
|
||||
|
||||
def fake_run(argv, *a, **kw): # type: ignore
|
||||
return next(results)
|
||||
|
||||
with patch.object(cleanup.subprocess, "run", side_effect=fake_run), \
|
||||
patch.object(cleanup, "warn") as warn:
|
||||
cleanup.cleanup(plan)
|
||||
# warn called once for the delete failure.
|
||||
warn.assert_called_once()
|
||||
|
||||
def test_empty_plan_is_noop(self):
|
||||
plan = SmolmachinesBottleCleanupPlan()
|
||||
with patch.object(cleanup.subprocess, "run") as run:
|
||||
cleanup.cleanup(plan)
|
||||
run.assert_not_called()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,192 +0,0 @@
|
||||
"""Unit: smolmachines `_ensure_smolmachine` agent-image pipeline
|
||||
(PRD 0023 chunk 4c).
|
||||
|
||||
Asserts that the cache-hit path returns without touching the
|
||||
registry / pack pipeline, and that the cache-miss path runs
|
||||
build → tag → push → pack in order against a registry port the
|
||||
helper yields.
|
||||
|
||||
The pipeline lives in `launch.py` (moved from `prepare.py` so the
|
||||
docker build doesn't run before the dashboard's preflight modal;
|
||||
the curses-endwin / tmux pane-routing handoff happens around
|
||||
`launch`)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
from typing import Any, cast
|
||||
from unittest.mock import patch
|
||||
|
||||
from bot_bottle.backend.smolmachines import launch as _launch_mod
|
||||
|
||||
|
||||
class TestEnsureSmolmachine(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self._tmp = tempfile.TemporaryDirectory(prefix="cb-cache.")
|
||||
self._cache_patch = patch.object(
|
||||
_launch_mod, "_SMOLMACHINE_CACHE_DIR", Path(self._tmp.name),
|
||||
)
|
||||
self._cache_patch.start()
|
||||
|
||||
def tearDown(self):
|
||||
self._cache_patch.stop()
|
||||
self._tmp.cleanup()
|
||||
|
||||
def test_cache_hit_skips_registry_and_pack(self):
|
||||
# Pre-populate the cache for image id `sha256:abcdef0123456789...`.
|
||||
digest = "abcdef0123456789"
|
||||
sidecar = Path(self._tmp.name) / f"{digest}.smolmachine.smolmachine"
|
||||
sidecar.write_text("")
|
||||
|
||||
with patch.object(
|
||||
_launch_mod.docker_mod, "build_image",
|
||||
) as build, patch.object(
|
||||
_launch_mod.docker_mod, "image_id",
|
||||
return_value=f"sha256:{digest}fffffffffffffffff",
|
||||
), patch.object(
|
||||
_launch_mod.docker_mod, "save",
|
||||
) as save, patch.object(
|
||||
_launch_mod, "ephemeral_registry",
|
||||
) as registry, patch.object(
|
||||
_launch_mod, "crane_push_tarball",
|
||||
) as push, patch.object(
|
||||
_launch_mod._smolvm, "pack_create",
|
||||
) as pack:
|
||||
result = _launch_mod._ensure_smolmachine("bot-bottle-claude:latest")
|
||||
|
||||
self.assertEqual(sidecar, result)
|
||||
# build still runs (Dockerfile edits land without manual rmi).
|
||||
build.assert_called_once()
|
||||
# No save (500MB tarball), no registry, no push, no pack on
|
||||
# cache hit.
|
||||
save.assert_not_called()
|
||||
registry.assert_not_called()
|
||||
push.assert_not_called()
|
||||
pack.assert_not_called()
|
||||
|
||||
def test_cache_miss_runs_build_save_push_pack_in_order(self):
|
||||
digest = "0123456789abcdef"
|
||||
|
||||
# ephemeral_registry yields a RegistryHandle with the
|
||||
# docker network + a push endpoint (container DNS) and
|
||||
# pull endpoint (host port-forward).
|
||||
from bot_bottle.backend.smolmachines.local_registry import (
|
||||
RegistryHandle,
|
||||
)
|
||||
|
||||
class _Reg:
|
||||
def __enter__(self_inner): # type: ignore
|
||||
return RegistryHandle(
|
||||
network="cb-net-xyz",
|
||||
push_endpoint="cb-registry-xyz:5000",
|
||||
pull_endpoint="localhost:54321",
|
||||
)
|
||||
def __exit__(self_inner, *exc): # type: ignore
|
||||
return False
|
||||
|
||||
calls: list[str] = []
|
||||
|
||||
def record(name): # type: ignore
|
||||
def _f(*a, **kw): # type: ignore
|
||||
calls.append(name)
|
||||
return _f
|
||||
|
||||
def save_and_record(image_ref: str, path: str) -> None:
|
||||
Path(path).touch()
|
||||
calls.append("save")
|
||||
|
||||
with patch.object(
|
||||
_launch_mod.docker_mod, "build_image",
|
||||
side_effect=record("build"),
|
||||
), patch.object(
|
||||
_launch_mod.docker_mod, "image_id",
|
||||
return_value=f"sha256:{digest}fffffffffffffffff",
|
||||
), patch.object(
|
||||
_launch_mod.docker_mod, "save",
|
||||
side_effect=save_and_record,
|
||||
) as save, patch.object(
|
||||
_launch_mod, "ephemeral_registry",
|
||||
return_value=_Reg(),
|
||||
), patch.object(
|
||||
_launch_mod, "crane_push_tarball",
|
||||
side_effect=record("push"),
|
||||
) as push, patch.object(
|
||||
_launch_mod._smolvm, "pack_create",
|
||||
side_effect=record("pack"),
|
||||
) as pack:
|
||||
_launch_mod._ensure_smolmachine("bot-bottle-claude:latest")
|
||||
|
||||
# Build → save → push → pack in that order. No `docker
|
||||
# push` (the daemon's HTTPS-by-default path is what we're
|
||||
# sidestepping).
|
||||
self.assertEqual(["build", "save", "push", "pack"], calls)
|
||||
|
||||
# docker save targets a per-digest tarball alongside the
|
||||
# cached sidecar.
|
||||
save_args = save.call_args.args
|
||||
self.assertEqual("bot-bottle-claude:latest", save_args[0])
|
||||
self.assertTrue(save_args[1].endswith(f"{digest}.image.tar"))
|
||||
|
||||
# crane push runs against the push_endpoint (container DNS
|
||||
# on the registry network) with the digest as the tag.
|
||||
push_args = push.call_args.args
|
||||
self.assertEqual(
|
||||
f"cb-registry-xyz:5000/bot-bottle:{digest}", push_args[2],
|
||||
)
|
||||
|
||||
# pack_create reads from the pull_endpoint (host port-
|
||||
# forward, smolvm is on the host). Same repo+tag, just a
|
||||
# different routing hostname — the registry stores one blob.
|
||||
pack_args = pack.call_args.args
|
||||
self.assertEqual(
|
||||
f"localhost:54321/bot-bottle:{digest}", pack_args[0],
|
||||
)
|
||||
self.assertTrue(str(pack_args[1]).endswith(f"{digest}.smolmachine"))
|
||||
|
||||
|
||||
class TestAgentFromPath(unittest.TestCase):
|
||||
def _plan(self) -> Any:
|
||||
return cast(Any, SimpleNamespace(
|
||||
slug="dev-abc12",
|
||||
agent_image="bot-bottle-claude:latest",
|
||||
agent_dockerfile_path="/repo/Dockerfile",
|
||||
))
|
||||
|
||||
def test_uses_committed_artifact_when_present(self):
|
||||
with tempfile.TemporaryDirectory(prefix="committed-smolmachine.") as tmp:
|
||||
artifact = Path(tmp) / "committed-smolmachine.smolmachine"
|
||||
artifact.write_text("")
|
||||
with patch.object(
|
||||
_launch_mod, "read_committed_image", return_value=str(artifact),
|
||||
), patch.object(
|
||||
_launch_mod, "_ensure_smolmachine",
|
||||
) as ensure, patch.object(
|
||||
_launch_mod, "info",
|
||||
):
|
||||
result = _launch_mod._agent_from_path(self._plan())
|
||||
|
||||
self.assertEqual(artifact, result)
|
||||
ensure.assert_not_called()
|
||||
|
||||
def test_falls_back_when_committed_artifact_missing(self):
|
||||
packed = Path("/cache/agent.smolmachine")
|
||||
with patch.object(
|
||||
_launch_mod, "read_committed_image",
|
||||
return_value="/missing/committed.smolmachine",
|
||||
), patch.object(
|
||||
_launch_mod, "_ensure_smolmachine", return_value=packed,
|
||||
) as ensure:
|
||||
result = _launch_mod._agent_from_path(self._plan())
|
||||
|
||||
self.assertEqual(packed, result)
|
||||
ensure.assert_called_once_with(
|
||||
"bot-bottle-claude:latest",
|
||||
dockerfile="/repo/Dockerfile",
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,250 +0,0 @@
|
||||
"""Unit: ephemeral local-registry helper (PRD 0023 chunk 4c).
|
||||
|
||||
The helper brings up a `registry:2.8.3` container on a private
|
||||
docker network with a random host-side port, yields a
|
||||
`RegistryHandle`, and tears the container + network down on exit.
|
||||
Tests mock `subprocess.run` + `socket.create_connection` so they
|
||||
run without docker."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
from bot_bottle.backend.smolmachines import local_registry
|
||||
|
||||
|
||||
def _ok(stdout: str = "", stderr: str = "") -> subprocess.CompletedProcess: # type: ignore
|
||||
return subprocess.CompletedProcess(
|
||||
args=[], returncode=0, stdout=stdout, stderr=stderr,
|
||||
)
|
||||
|
||||
|
||||
def _fail(stderr: str = "boom") -> subprocess.CompletedProcess: # type: ignore
|
||||
return subprocess.CompletedProcess(
|
||||
args=[], returncode=1, stdout="", stderr=stderr,
|
||||
)
|
||||
|
||||
|
||||
# Run sequence per ephemeral_registry() call:
|
||||
# docker network create -> ok
|
||||
# docker run -d (registry) -> ok (container id)
|
||||
# docker port (host port) -> ok (mapping line)
|
||||
# docker rm -f (registry) -> ok (in finally)
|
||||
# docker network rm -> ok (in finally)
|
||||
def _stock_run_sequence(port_line: str = "0.0.0.0:54321\n"):
|
||||
return [
|
||||
_ok(), # docker network create
|
||||
_ok(stdout="<container-id>\n"), # docker run
|
||||
_ok(stdout=port_line), # docker port
|
||||
_ok(), # docker rm -f
|
||||
_ok(), # docker network rm
|
||||
]
|
||||
|
||||
|
||||
class TestEphemeralRegistry(unittest.TestCase):
|
||||
def test_yields_handle_with_network_and_endpoints(self):
|
||||
with patch.object(
|
||||
local_registry.subprocess, "run",
|
||||
side_effect=_stock_run_sequence(),
|
||||
) as run, patch.object(
|
||||
local_registry.socket, "create_connection",
|
||||
return_value=_FakeSocket(),
|
||||
):
|
||||
with local_registry.ephemeral_registry() as handle:
|
||||
# push_endpoint points at the registry container by
|
||||
# its docker-network name on its container port.
|
||||
self.assertTrue(
|
||||
handle.push_endpoint.startswith(
|
||||
"bot-bottle-registry-"
|
||||
)
|
||||
)
|
||||
self.assertTrue(handle.push_endpoint.endswith(":5000"))
|
||||
# pull_endpoint is the host-side mapping for smolvm.
|
||||
self.assertEqual("localhost:54321", handle.pull_endpoint)
|
||||
# network name is the per-session bridge crane joins.
|
||||
self.assertTrue(
|
||||
handle.network.startswith("bot-bottle-registry-net-")
|
||||
)
|
||||
# docker network create + docker run + docker port + rm -f + network rm
|
||||
self.assertEqual(5, run.call_count)
|
||||
|
||||
def test_registry_run_publishes_random_port_across_interfaces(self):
|
||||
with patch.object(
|
||||
local_registry.subprocess, "run",
|
||||
side_effect=_stock_run_sequence(),
|
||||
) as run, patch.object(
|
||||
local_registry.socket, "create_connection",
|
||||
return_value=_FakeSocket(),
|
||||
):
|
||||
with local_registry.ephemeral_registry():
|
||||
pass
|
||||
# second call is the docker run for the registry
|
||||
run_argv = run.call_args_list[1].args[0]
|
||||
self.assertEqual(["docker", "run"], run_argv[:2])
|
||||
self.assertIn("--rm", run_argv)
|
||||
# `-p 5000` (no IP prefix) — needed so the host-published
|
||||
# port is reachable from BOTH the host (for smolvm) and the
|
||||
# docker daemon (for the docker port command to find it).
|
||||
self.assertIn("5000", run_argv)
|
||||
# And the registry is attached to the same per-session
|
||||
# network the crane push container joins.
|
||||
self.assertIn("--network", run_argv)
|
||||
|
||||
def test_force_removes_container_and_network_on_clean_exit(self):
|
||||
with patch.object(
|
||||
local_registry.subprocess, "run",
|
||||
side_effect=_stock_run_sequence(),
|
||||
) as run, patch.object(
|
||||
local_registry.socket, "create_connection",
|
||||
return_value=_FakeSocket(),
|
||||
):
|
||||
with local_registry.ephemeral_registry():
|
||||
pass
|
||||
|
||||
# Last two calls are `docker rm -f <container>` then
|
||||
# `docker network rm <network>`.
|
||||
argvs = [c.args[0] for c in run.call_args_list]
|
||||
self.assertEqual(["docker", "rm", "-f"], argvs[-2][:3])
|
||||
self.assertEqual(["docker", "network", "rm"], argvs[-1][:3])
|
||||
|
||||
def test_force_removes_on_exception_inside_with(self):
|
||||
with patch.object(
|
||||
local_registry.subprocess, "run",
|
||||
side_effect=_stock_run_sequence(),
|
||||
) as run, patch.object(
|
||||
local_registry.socket, "create_connection",
|
||||
return_value=_FakeSocket(),
|
||||
):
|
||||
with self.assertRaises(RuntimeError):
|
||||
with local_registry.ephemeral_registry():
|
||||
raise RuntimeError("inside with")
|
||||
|
||||
# Both teardowns still ran.
|
||||
argvs = [c.args[0] for c in run.call_args_list]
|
||||
self.assertEqual(["docker", "rm", "-f"], argvs[-2][:3])
|
||||
self.assertEqual(["docker", "network", "rm"], argvs[-1][:3])
|
||||
|
||||
def test_wait_ready_times_out(self):
|
||||
with patch.object(local_registry, "_READY_TIMEOUT_S", 0.1), patch.object(
|
||||
local_registry.subprocess, "run",
|
||||
side_effect=_stock_run_sequence(),
|
||||
) as run, patch.object(
|
||||
local_registry.socket, "create_connection",
|
||||
side_effect=OSError("conn refused"),
|
||||
), patch.object(
|
||||
local_registry, "die",
|
||||
side_effect=SystemExit("die called"),
|
||||
) as die:
|
||||
with self.assertRaises(SystemExit):
|
||||
with local_registry.ephemeral_registry():
|
||||
self.fail("yield reached despite unreachable registry")
|
||||
die.assert_called_once()
|
||||
# Teardown still ran via the finally blocks.
|
||||
argvs = [c.args[0] for c in run.call_args_list]
|
||||
self.assertEqual(["docker", "rm", "-f"], argvs[-2][:3])
|
||||
self.assertEqual(["docker", "network", "rm"], argvs[-1][:3])
|
||||
|
||||
def test_unique_session_ids_per_call(self):
|
||||
sessions: list[tuple[str, str]] = []
|
||||
|
||||
def capture(argv, *a, **kw): # type: ignore
|
||||
if argv[:3] == ["docker", "network", "create"]:
|
||||
return _ok()
|
||||
if argv[:2] == ["docker", "run"]:
|
||||
# `--name <registry-name>` and `--network <net-name>`
|
||||
# both encode the session id.
|
||||
name = argv[argv.index("--name") + 1]
|
||||
network = argv[argv.index("--network") + 1]
|
||||
sessions.append((name, network))
|
||||
return _ok(stdout="cid\n")
|
||||
if argv[:2] == ["docker", "port"]:
|
||||
return _ok(stdout="0.0.0.0:1\n")
|
||||
return _ok()
|
||||
|
||||
with patch.object(
|
||||
local_registry.subprocess, "run", side_effect=capture,
|
||||
), patch.object(
|
||||
local_registry.socket, "create_connection",
|
||||
return_value=_FakeSocket(),
|
||||
):
|
||||
with local_registry.ephemeral_registry():
|
||||
pass
|
||||
with local_registry.ephemeral_registry():
|
||||
pass
|
||||
|
||||
self.assertEqual(2, len(sessions))
|
||||
self.assertNotEqual(sessions[0], sessions[1])
|
||||
|
||||
|
||||
class TestCranePushTarball(unittest.TestCase):
|
||||
def test_runs_crane_container_on_registry_network_with_insecure_flag(self):
|
||||
handle = local_registry.RegistryHandle(
|
||||
network="cb-registry-net-x",
|
||||
push_endpoint="cb-registry-x:5000",
|
||||
pull_endpoint="localhost:54321",
|
||||
)
|
||||
with patch.object(
|
||||
local_registry.subprocess, "run", return_value=_ok(),
|
||||
) as run:
|
||||
local_registry.crane_push_tarball(
|
||||
handle, "/tmp/img.tar", "cb-registry-x:5000/cb:abc",
|
||||
)
|
||||
|
||||
argv = run.call_args.args[0]
|
||||
# Joined to the same docker network so it can reach the
|
||||
# registry by container name (no host port-forward needed
|
||||
# for the push leg).
|
||||
self.assertEqual("docker", argv[0])
|
||||
self.assertEqual("run", argv[1])
|
||||
self.assertIn("--rm", argv)
|
||||
self.assertIn("--network", argv)
|
||||
self.assertEqual(
|
||||
"cb-registry-net-x", argv[argv.index("--network") + 1],
|
||||
)
|
||||
# The tarball is mounted read-only at /img.tar.
|
||||
self.assertIn("-v", argv)
|
||||
self.assertIn("/tmp/img.tar:/img.tar:ro", argv)
|
||||
# And the crane command itself uses --insecure so plain
|
||||
# HTTP is allowed against the registry container.
|
||||
self.assertIn("push", argv)
|
||||
self.assertIn("--insecure", argv)
|
||||
self.assertIn("/img.tar", argv)
|
||||
self.assertIn("cb-registry-x:5000/cb:abc", argv)
|
||||
|
||||
def test_dies_when_crane_returns_non_zero(self):
|
||||
handle = local_registry.RegistryHandle(
|
||||
network="cb-net", push_endpoint="cb:5000", pull_endpoint="localhost:1",
|
||||
)
|
||||
with patch.object(
|
||||
local_registry.subprocess, "run",
|
||||
return_value=_fail("push failed"),
|
||||
), patch.object(
|
||||
local_registry, "die", side_effect=SystemExit("die"),
|
||||
) as die:
|
||||
with self.assertRaises(SystemExit):
|
||||
local_registry.crane_push_tarball(
|
||||
handle, "/tmp/img.tar", "cb:5000/cb:abc",
|
||||
)
|
||||
die.assert_called_once()
|
||||
# Error message names what was being pushed where.
|
||||
msg = die.call_args.args[0]
|
||||
self.assertIn("/tmp/img.tar", msg)
|
||||
self.assertIn("cb:5000/cb:abc", msg)
|
||||
|
||||
|
||||
class _FakeSocket:
|
||||
"""Minimal context-manager stand-in for the socket
|
||||
`create_connection` returns. The helper only uses `with` on it
|
||||
and discards the value, so we don't need any real network."""
|
||||
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, *exc): # type: ignore
|
||||
return False
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,430 +0,0 @@
|
||||
"""Unit: per-bottle loopback alias pool (follow-up to the
|
||||
Docker-Desktop fix in PR #74).
|
||||
|
||||
`ensure_pool` lazily sudo-adds missing aliases on macOS; no-ops
|
||||
on Linux. `allocate` picks the lowest-numbered unused alias by
|
||||
inspecting running bundle containers' port bindings."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import sqlite3
|
||||
import subprocess
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
from bot_bottle.backend.smolmachines import loopback_alias
|
||||
|
||||
|
||||
def _ok(stdout: str = "") -> subprocess.CompletedProcess: # type: ignore
|
||||
return subprocess.CompletedProcess(
|
||||
args=[], returncode=0, stdout=stdout, stderr="",
|
||||
)
|
||||
|
||||
|
||||
def _fail(stderr: str = "boom") -> subprocess.CompletedProcess: # type: ignore
|
||||
return subprocess.CompletedProcess(
|
||||
args=[], returncode=1, stdout="", stderr=stderr,
|
||||
)
|
||||
|
||||
|
||||
# `ifconfig lo0` on macOS with the default lo0 config: just
|
||||
# 127.0.0.1. We craft fixtures around this shape.
|
||||
_LO0_DEFAULT = (
|
||||
"lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384\n"
|
||||
"\tinet 127.0.0.1 netmask 0xff000000\n"
|
||||
"\tinet6 ::1 prefixlen 128\n"
|
||||
)
|
||||
|
||||
_LO0_PARTIAL = (
|
||||
_LO0_DEFAULT
|
||||
+ "\tinet 127.0.0.16 netmask 0xffffffff\n"
|
||||
+ "\tinet 127.0.0.17 netmask 0xffffffff\n"
|
||||
)
|
||||
|
||||
|
||||
def _lo0_full() -> str:
|
||||
"""All 16 pool addresses already aliased."""
|
||||
aliases = "".join(
|
||||
f"\tinet 127.0.0.{i} netmask 0xffffffff\n"
|
||||
for i in range(16, 32)
|
||||
)
|
||||
return _LO0_DEFAULT + aliases
|
||||
|
||||
|
||||
class TestEnsurePool(unittest.TestCase):
|
||||
def test_noop_on_linux(self):
|
||||
# `_is_macos` returns False on Linux; ensure_pool should
|
||||
# never shell out to sudo.
|
||||
with patch.object(loopback_alias, "_is_macos", return_value=False), \
|
||||
patch.object(loopback_alias.subprocess, "run") as run:
|
||||
loopback_alias.ensure_pool()
|
||||
run.assert_not_called()
|
||||
|
||||
def test_all_present_skips_sudo(self):
|
||||
with patch.object(loopback_alias, "_is_macos", return_value=True), \
|
||||
patch.object(
|
||||
loopback_alias.subprocess, "run",
|
||||
return_value=_ok(stdout=_lo0_full()),
|
||||
) as run:
|
||||
loopback_alias.ensure_pool()
|
||||
# Just the ifconfig probe per pool address; no sudo at all.
|
||||
for call in run.call_args_list:
|
||||
self.assertNotIn("sudo", call.args[0])
|
||||
|
||||
def test_missing_aliases_dispatch_sudo(self):
|
||||
# lo0 only has 16+17 already; sudo runs for 18..31 (14 missing).
|
||||
runs: list[list[str]] = []
|
||||
|
||||
def fake_run(argv, *a, **kw): # type: ignore
|
||||
runs.append(argv)
|
||||
if argv[:2] == ["/sbin/ifconfig", "lo0"]:
|
||||
return _ok(stdout=_LO0_PARTIAL)
|
||||
return _ok()
|
||||
|
||||
with patch.object(loopback_alias, "_is_macos", return_value=True), \
|
||||
patch.object(loopback_alias.subprocess, "run", side_effect=fake_run):
|
||||
loopback_alias.ensure_pool()
|
||||
|
||||
sudo_calls = [r for r in runs if r and r[0] == "sudo"]
|
||||
self.assertEqual(14, len(sudo_calls))
|
||||
sudo_ips = {call[call.index("alias") + 1].split("/")[0] for call in sudo_calls}
|
||||
self.assertEqual(
|
||||
{f"127.0.0.{i}" for i in range(18, 32)},
|
||||
sudo_ips,
|
||||
)
|
||||
|
||||
def test_sudo_failure_dies(self):
|
||||
def fake_run(argv, *a, **kw): # type: ignore
|
||||
if argv[:2] == ["/sbin/ifconfig", "lo0"]:
|
||||
return _ok(stdout=_LO0_DEFAULT)
|
||||
if argv[:1] == ["sudo"]:
|
||||
return _fail()
|
||||
return _ok()
|
||||
|
||||
with patch.object(loopback_alias, "_is_macos", return_value=True), \
|
||||
patch.object(loopback_alias.subprocess, "run", side_effect=fake_run), \
|
||||
patch.object(loopback_alias, "die", side_effect=SystemExit("die")):
|
||||
with self.assertRaises(SystemExit):
|
||||
loopback_alias.ensure_pool()
|
||||
|
||||
|
||||
class TestAllocate(unittest.TestCase):
|
||||
def test_per_bottle_alias_on_linux(self):
|
||||
# Linux gets the same per-bottle scoping as macOS (127/8 is
|
||||
# already loopback, so no ifconfig is needed). A fresh host
|
||||
# with no running bundles allocates the first pool entry.
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
lock_path = Path(tmp) / "smolmachines.lock"
|
||||
with patch.object(loopback_alias, "_is_macos", return_value=False), \
|
||||
patch.object(loopback_alias, "_ALLOC_LOCK_PATH", lock_path), \
|
||||
patch.object(loopback_alias, "_aliases_in_use", return_value=set()):
|
||||
self.assertEqual("127.0.0.16", loopback_alias.allocate("demo"))
|
||||
|
||||
def test_picks_lowest_unused_on_macos(self):
|
||||
# No bundles running -> first pool entry.
|
||||
with patch.object(loopback_alias, "_is_macos", return_value=True), \
|
||||
patch.object(loopback_alias, "_aliases_in_use", return_value=set()):
|
||||
self.assertEqual("127.0.0.16", loopback_alias.allocate("demo-1"))
|
||||
|
||||
def test_skips_in_use_aliases(self):
|
||||
with patch.object(loopback_alias, "_is_macos", return_value=True), \
|
||||
patch.object(
|
||||
loopback_alias, "_aliases_in_use",
|
||||
return_value={"127.0.0.16", "127.0.0.17", "127.0.0.19"},
|
||||
):
|
||||
# First unused = 127.0.0.18.
|
||||
self.assertEqual("127.0.0.18", loopback_alias.allocate("demo-3"))
|
||||
|
||||
def test_dies_when_pool_exhausted(self):
|
||||
all_in_use = {f"127.0.0.{i}" for i in range(16, 32)}
|
||||
with patch.object(loopback_alias, "_is_macos", return_value=True), \
|
||||
patch.object(
|
||||
loopback_alias, "_aliases_in_use",
|
||||
return_value=all_in_use,
|
||||
), patch.object(
|
||||
loopback_alias, "die", side_effect=SystemExit("die"),
|
||||
):
|
||||
with self.assertRaises(SystemExit):
|
||||
loopback_alias.allocate("demo-overflow")
|
||||
|
||||
|
||||
class TestAllocateLock(unittest.TestCase):
|
||||
"""allocate() on macOS acquires a file lock so concurrent calls
|
||||
serialise rather than racing on docker state."""
|
||||
|
||||
def test_acquires_exclusive_lock_on_macos(self):
|
||||
import fcntl as fcntl_mod
|
||||
flock_calls: list[int] = []
|
||||
|
||||
def record_flock(fd, op): # type: ignore
|
||||
flock_calls.append(op)
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
lock_path = Path(tmp) / "smolmachines.lock"
|
||||
with patch.object(loopback_alias, "_is_macos", return_value=True), \
|
||||
patch.object(loopback_alias, "_ALLOC_LOCK_PATH", lock_path), \
|
||||
patch.object(loopback_alias, "_aliases_in_use", return_value=set()), \
|
||||
patch.object(loopback_alias.fcntl, "flock",
|
||||
side_effect=record_flock):
|
||||
loopback_alias.allocate("demo")
|
||||
|
||||
self.assertIn(fcntl_mod.LOCK_EX, flock_calls)
|
||||
|
||||
def test_acquires_exclusive_lock_on_linux(self):
|
||||
# Linux allocates per-bottle too, so it must take the same
|
||||
# lock to serialise concurrent launches.
|
||||
import fcntl as fcntl_mod
|
||||
flock_calls: list[int] = []
|
||||
|
||||
def record_flock(fd, op): # type: ignore
|
||||
flock_calls.append(op)
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
lock_path = Path(tmp) / "smolmachines.lock"
|
||||
with patch.object(loopback_alias, "_is_macos", return_value=False), \
|
||||
patch.object(loopback_alias, "_ALLOC_LOCK_PATH", lock_path), \
|
||||
patch.object(loopback_alias, "_aliases_in_use", return_value=set()), \
|
||||
patch.object(loopback_alias.fcntl, "flock",
|
||||
side_effect=record_flock):
|
||||
loopback_alias.allocate("demo")
|
||||
|
||||
self.assertIn(fcntl_mod.LOCK_EX, flock_calls)
|
||||
|
||||
def test_sequential_allocations_with_shared_lock_are_serialised(self):
|
||||
# Two sequential calls share the same lock file. The second
|
||||
# call sees {127.0.0.16} in use (as if the first caller's
|
||||
# docker run completed between the two lock acquisitions) and
|
||||
# returns the next alias.
|
||||
in_use_seq = [set(), {"127.0.0.16"}]
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
lock_path = Path(tmp) / "smolmachines.lock"
|
||||
results: list[str] = []
|
||||
for _ in range(2):
|
||||
with patch.object(loopback_alias, "_is_macos", return_value=True), \
|
||||
patch.object(loopback_alias, "_ALLOC_LOCK_PATH", lock_path), \
|
||||
patch.object(loopback_alias, "_aliases_in_use",
|
||||
return_value=in_use_seq.pop(0)):
|
||||
results.append(loopback_alias.allocate("demo"))
|
||||
|
||||
self.assertEqual(["127.0.0.16", "127.0.0.17"], results)
|
||||
|
||||
|
||||
class TestAliasInUseDetection(unittest.TestCase):
|
||||
"""`_aliases_in_use` inspects every running bundle and pulls
|
||||
each container's port-binding `HostIp` out. The detection has
|
||||
to survive: no running bundles, multiple bundles, docker
|
||||
inspect failures."""
|
||||
|
||||
def test_no_bundles_returns_empty(self):
|
||||
with patch.object(
|
||||
loopback_alias.subprocess, "run",
|
||||
return_value=_ok(stdout=""),
|
||||
):
|
||||
self.assertEqual(set(), loopback_alias._aliases_in_use())
|
||||
|
||||
def test_walks_bundles_and_pulls_host_ips(self):
|
||||
# First call: docker ps -> two bundle names.
|
||||
# Then docker inspect each, returning a port-bindings JSON
|
||||
# blob with a HostIp on the per-bottle alias.
|
||||
ps_out = "bot-bottle-sidecars-a\nbot-bottle-sidecars-b\n"
|
||||
inspect_a = (
|
||||
'{"8888/tcp":[{"HostIp":"127.0.0.16","HostPort":"54000"}]}'
|
||||
)
|
||||
inspect_b = (
|
||||
'{"9099/tcp":[{"HostIp":"127.0.0.17","HostPort":"54001"}]}'
|
||||
)
|
||||
|
||||
seq = [
|
||||
_ok(stdout=ps_out),
|
||||
_ok(stdout=inspect_a),
|
||||
_ok(stdout=inspect_b),
|
||||
]
|
||||
with patch.object(
|
||||
loopback_alias.subprocess, "run", side_effect=seq,
|
||||
):
|
||||
self.assertEqual(
|
||||
{"127.0.0.16", "127.0.0.17"},
|
||||
loopback_alias._aliases_in_use(),
|
||||
)
|
||||
|
||||
def test_inspect_failures_are_skipped(self):
|
||||
ps_out = "bot-bottle-sidecars-c\n"
|
||||
with patch.object(
|
||||
loopback_alias.subprocess, "run",
|
||||
side_effect=[_ok(stdout=ps_out), _fail("inspect failed")],
|
||||
):
|
||||
self.assertEqual(set(), loopback_alias._aliases_in_use())
|
||||
|
||||
|
||||
class TestForceAllowlist(unittest.TestCase):
|
||||
"""Smolvm 0.8.0 silently drops `--allow-cidr` with `--from`, so
|
||||
`force_allowlist` opens the state DB directly and sets the row's
|
||||
`allowed_cidrs` field — on both macOS and Linux. It is
|
||||
fail-closed: it dies rather than launching a VM whose allowlist
|
||||
it can't confirm. Round-trip tests against a real SQLite DB to
|
||||
lock down the BLOB encoding."""
|
||||
|
||||
def setUp(self):
|
||||
self._tmp = tempfile.TemporaryDirectory(prefix="smolvm-db.")
|
||||
self.db = Path(self._tmp.name) / "smolvm.db"
|
||||
con = sqlite3.connect(str(self.db))
|
||||
con.execute(
|
||||
"CREATE TABLE vms (name TEXT PRIMARY KEY NOT NULL, data BLOB NOT NULL)"
|
||||
)
|
||||
# Mimic smolvm's row shape (the JSON keys that exist on
|
||||
# creation; allowed_cidrs is the field we patch).
|
||||
cfg = {
|
||||
"name": "demo-vm",
|
||||
"cpus": 4,
|
||||
"mem": 8192,
|
||||
"network": True,
|
||||
"allowed_cidrs": None,
|
||||
}
|
||||
con.execute(
|
||||
"INSERT INTO vms (name, data) VALUES (?, ?)",
|
||||
("demo-vm", sqlite3.Binary(json.dumps(cfg).encode())),
|
||||
)
|
||||
con.commit()
|
||||
con.close()
|
||||
|
||||
def tearDown(self):
|
||||
self._tmp.cleanup()
|
||||
|
||||
def test_patches_allowed_cidrs_on_row(self):
|
||||
with patch.object(loopback_alias, "_is_macos", return_value=True), \
|
||||
patch.object(loopback_alias, "_SMOLVM_DB_PATH", self.db):
|
||||
loopback_alias.force_allowlist("demo-vm", ["127.0.0.16/32"])
|
||||
|
||||
con = sqlite3.connect(str(self.db))
|
||||
row = con.execute(
|
||||
"SELECT typeof(data), data FROM vms WHERE name='demo-vm'",
|
||||
).fetchone()
|
||||
con.close()
|
||||
# Must round-trip as BLOB (the column type smolvm reads).
|
||||
self.assertEqual("blob", row[0])
|
||||
cfg = json.loads(row[1])
|
||||
self.assertEqual(["127.0.0.16/32"], cfg["allowed_cidrs"])
|
||||
# Other fields preserved verbatim.
|
||||
self.assertEqual(4, cfg["cpus"])
|
||||
self.assertTrue(cfg["network"])
|
||||
|
||||
def test_patches_on_linux_too(self):
|
||||
# force_allowlist no longer no-ops on Linux — the TSI
|
||||
# allowlist must be enforced there as well.
|
||||
with patch.object(loopback_alias, "_is_macos", return_value=False), \
|
||||
patch.object(loopback_alias, "_SMOLVM_DB_PATH", self.db):
|
||||
loopback_alias.force_allowlist("demo-vm", ["127.0.0.16/32"])
|
||||
con = sqlite3.connect(str(self.db))
|
||||
cfg = json.loads(con.execute(
|
||||
"SELECT data FROM vms WHERE name='demo-vm'",
|
||||
).fetchone()[0])
|
||||
con.close()
|
||||
self.assertEqual(["127.0.0.16/32"], cfg["allowed_cidrs"])
|
||||
|
||||
def test_skips_write_when_already_matching(self):
|
||||
# A newer smolvm that honors --allow-cidr at create leaves the
|
||||
# row already correct; force_allowlist must not rewrite it. We
|
||||
# detect a no-write by comparing the raw BLOB byte-for-byte
|
||||
# (a rewrite re-serialises the JSON, changing key order/bytes
|
||||
# is not guaranteed, but mtime/identity isn't observable — so
|
||||
# we assert the stored bytes are exactly what we pre-seeded).
|
||||
seeded = json.dumps({
|
||||
"name": "demo-vm", "cpus": 4, "mem": 8192,
|
||||
"network": True, "allowed_cidrs": ["127.0.0.16/32"],
|
||||
}).encode()
|
||||
con = sqlite3.connect(str(self.db))
|
||||
con.execute(
|
||||
"UPDATE vms SET data=? WHERE name='demo-vm'",
|
||||
(sqlite3.Binary(seeded),),
|
||||
)
|
||||
con.commit()
|
||||
con.close()
|
||||
|
||||
with patch.object(loopback_alias, "_is_macos", return_value=True), \
|
||||
patch.object(loopback_alias, "_SMOLVM_DB_PATH", self.db):
|
||||
loopback_alias.force_allowlist("demo-vm", ["127.0.0.16/32"])
|
||||
|
||||
con = sqlite3.connect(str(self.db))
|
||||
stored = con.execute(
|
||||
"SELECT data FROM vms WHERE name='demo-vm'").fetchone()[0]
|
||||
con.close()
|
||||
self.assertEqual(seeded, bytes(stored))
|
||||
|
||||
def test_dies_when_patch_does_not_take(self):
|
||||
# If the persisted allowlist still doesn't match after the
|
||||
# patch (e.g. wrong schema / smolvm stores it elsewhere),
|
||||
# force_allowlist must fail closed rather than boot the VM.
|
||||
original = loopback_alias._read_machine_cfg
|
||||
|
||||
def stale_cfg(con: sqlite3.Connection, name: str) -> dict[str, object]:
|
||||
# Always report the un-patched row so the post-write
|
||||
# verification never sees the requested cidrs.
|
||||
cfg = original(con, name)
|
||||
cfg["allowed_cidrs"] = None
|
||||
return cfg
|
||||
|
||||
with patch.object(loopback_alias, "_is_macos", return_value=True), \
|
||||
patch.object(loopback_alias, "_SMOLVM_DB_PATH", self.db), \
|
||||
patch.object(loopback_alias, "_read_machine_cfg", side_effect=stale_cfg), \
|
||||
patch.object(loopback_alias, "die", side_effect=SystemExit("die")):
|
||||
with self.assertRaises(SystemExit):
|
||||
loopback_alias.force_allowlist("demo-vm", ["127.0.0.16/32"])
|
||||
|
||||
def test_dies_on_missing_db(self):
|
||||
with patch.object(loopback_alias, "_is_macos", return_value=True), \
|
||||
patch.object(
|
||||
loopback_alias, "_SMOLVM_DB_PATH",
|
||||
Path("/nonexistent/smolvm.db"),
|
||||
), patch.object(
|
||||
loopback_alias, "die", side_effect=SystemExit("die"),
|
||||
):
|
||||
with self.assertRaises(SystemExit):
|
||||
loopback_alias.force_allowlist("demo-vm", ["127.0.0.16/32"])
|
||||
|
||||
def test_dies_on_missing_row(self):
|
||||
with patch.object(loopback_alias, "_is_macos", return_value=True), \
|
||||
patch.object(loopback_alias, "_SMOLVM_DB_PATH", self.db), \
|
||||
patch.object(
|
||||
loopback_alias, "die", side_effect=SystemExit("die"),
|
||||
):
|
||||
with self.assertRaises(SystemExit):
|
||||
loopback_alias.force_allowlist("not-in-db", ["127.0.0.16/32"])
|
||||
|
||||
|
||||
class TestSmolvmDbPath(unittest.TestCase):
|
||||
"""The smolvm state-DB path is platform-derived: Application
|
||||
Support on macOS, XDG data dir on Linux."""
|
||||
|
||||
def test_macos_path(self):
|
||||
with patch.object(loopback_alias.platform, "system", return_value="Darwin"):
|
||||
p = loopback_alias._smolvm_db_path()
|
||||
self.assertEqual(
|
||||
("Library", "Application Support", "smolvm", "server", "smolvm.db"),
|
||||
p.parts[-5:],
|
||||
)
|
||||
|
||||
def test_linux_default_xdg_path(self):
|
||||
env = {k: v for k, v in os.environ.items() if k != "XDG_DATA_HOME"}
|
||||
with patch.object(loopback_alias.platform, "system", return_value="Linux"), \
|
||||
patch.dict(loopback_alias.os.environ, env, clear=True):
|
||||
p = loopback_alias._smolvm_db_path()
|
||||
self.assertEqual(
|
||||
(".local", "share", "smolvm", "server", "smolvm.db"),
|
||||
p.parts[-5:],
|
||||
)
|
||||
|
||||
def test_linux_respects_xdg_data_home(self):
|
||||
with patch.object(loopback_alias.platform, "system", return_value="Linux"), \
|
||||
patch.dict(loopback_alias.os.environ,
|
||||
{"XDG_DATA_HOME": "/custom/data"}, clear=False):
|
||||
p = loopback_alias._smolvm_db_path()
|
||||
self.assertEqual(Path("/custom/data/smolvm/server/smolvm.db"), p)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,559 +0,0 @@
|
||||
"""Unit: smolmachines provisioning helpers (PRD 0023 chunks 4a + 4d).
|
||||
|
||||
Tests mock `bottle.exec` / `bottle.cp_in` and assert on the
|
||||
dispatched script shape. The real round-trip lives in the chunk-4
|
||||
integration smoke."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
import tempfile
|
||||
import unittest
|
||||
from dataclasses import replace
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from bot_bottle.agent_provider import (
|
||||
AgentProvider,
|
||||
AgentProviderRuntime,
|
||||
AgentProvisionCommand,
|
||||
AgentProvisionDir,
|
||||
AgentProvisionFile,
|
||||
AgentProvisionPlan,
|
||||
)
|
||||
from bot_bottle.backend import Bottle, BottleSpec, ExecResult
|
||||
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
|
||||
from bot_bottle.backend.smolmachines.bottle_plan import (
|
||||
SmolmachinesBottlePlan,
|
||||
)
|
||||
from bot_bottle.backend.smolmachines import launch as _launch
|
||||
from bot_bottle.backend.smolmachines.launch import _bundle_launch_spec
|
||||
from bot_bottle.backend.util import AGENT_CA_PATH
|
||||
from bot_bottle.egress import EgressPlan, EgressRoute
|
||||
from bot_bottle.git_gate import GitGatePlan, GitGateUpstream
|
||||
from bot_bottle.manifest import ManifestGitEntry, ManifestKeyConfig, ManifestIndex
|
||||
from bot_bottle.supervise import SupervisePlan
|
||||
|
||||
|
||||
class _Provider(AgentProvider):
|
||||
"""Minimal concrete subclass for testing the default provision_ca/provision_git."""
|
||||
@property
|
||||
def runtime(self) -> AgentProviderRuntime:
|
||||
return AgentProviderRuntime(
|
||||
template="test", command="test", image="",
|
||||
prompt_mode="append_file", bypass_args=(), resume_args=(),
|
||||
)
|
||||
def provision_plan(self, **kwargs): # type: ignore[override]
|
||||
raise NotImplementedError
|
||||
def provision_skills(self, plan, bottle): ... # type: ignore[override]
|
||||
def provision_prompt(self, plan, bottle): ... # type: ignore[override]
|
||||
def provision(self, plan, bottle): ... # type: ignore[override]
|
||||
def provision_supervise_mcp(self, plan, bottle, supervise_url): ... # type: ignore[override]
|
||||
def headless_prompt(self, prompt): return [] # type: ignore[override]
|
||||
|
||||
|
||||
_PROVIDER = _Provider()
|
||||
|
||||
|
||||
def _make_bottle(
|
||||
name: str = "bot-bottle-demo-abc12",
|
||||
exec_result: ExecResult | None = None,
|
||||
) -> MagicMock:
|
||||
bottle = MagicMock(spec=Bottle)
|
||||
bottle.name = name
|
||||
bottle.exec.return_value = (
|
||||
exec_result if exec_result is not None
|
||||
else ExecResult(returncode=0, stdout="", stderr="")
|
||||
)
|
||||
return bottle
|
||||
|
||||
|
||||
def _exec_users(bottle: MagicMock) -> list[str]: # type: ignore
|
||||
"""user= kwarg from each bottle.exec call, in order."""
|
||||
return [c.kwargs.get("user", "node") for c in bottle.exec.call_args_list]
|
||||
|
||||
|
||||
def _plan(
|
||||
*,
|
||||
agent_prompt: str = "",
|
||||
skills: list[str] | None = None,
|
||||
git: list[ManifestGitEntry] = (), # type: ignore
|
||||
git_user: dict | None = None, # type: ignore
|
||||
copy_cwd: bool = False,
|
||||
user_cwd: str = "/tmp/x",
|
||||
stage_dir: Path | None = None,
|
||||
egress_routes: tuple[EgressRoute, ...] = (),
|
||||
egress_ca_path: Path = Path(),
|
||||
canary: bool = False,
|
||||
supervise: bool = False,
|
||||
bundle_ip: str = "192.168.50.2",
|
||||
agent_git_gate_host: str = "127.0.0.1:55555",
|
||||
agent_supervise_url: str = "http://127.0.0.1:55556/",
|
||||
codex_auth_file: Path | None = None,
|
||||
agent_provider_template: str = "claude",
|
||||
guest_env: dict[str, str] | None = None,
|
||||
) -> SmolmachinesBottlePlan:
|
||||
bottle_json: dict = {} # type: ignore
|
||||
git_gate_json: dict = {} # type: ignore
|
||||
if git:
|
||||
git_gate_json["repos"] = {
|
||||
g.Name: {
|
||||
"url": g.Upstream,
|
||||
"key": {"provider": g.Key.provider or "static", "path": g.Key.path or g.IdentityFile},
|
||||
}
|
||||
for g in git
|
||||
}
|
||||
if git_user is not None:
|
||||
git_gate_json["user"] = git_user
|
||||
if git_gate_json:
|
||||
bottle_json["git-gate"] = git_gate_json
|
||||
if supervise:
|
||||
bottle_json["supervise"] = True
|
||||
index = ManifestIndex.from_json_obj({
|
||||
"bottles": {"dev": bottle_json},
|
||||
"agents": {
|
||||
"demo": {
|
||||
"skills": list(skills or []),
|
||||
"prompt": agent_prompt,
|
||||
"bottle": "dev",
|
||||
},
|
||||
},
|
||||
})
|
||||
manifest = index.load_for_agent("demo")
|
||||
spec = BottleSpec(
|
||||
manifest=index,
|
||||
agent_name="demo",
|
||||
copy_cwd=copy_cwd,
|
||||
user_cwd=user_cwd,
|
||||
)
|
||||
supervise_plan = None
|
||||
if supervise:
|
||||
supervise_plan = SupervisePlan(
|
||||
slug="demo-abc12",
|
||||
db_path=Path("/tmp/bot-bottle.db"),
|
||||
)
|
||||
return SmolmachinesBottlePlan(
|
||||
spec=spec,
|
||||
manifest=manifest,
|
||||
stage_dir=stage_dir or Path("/tmp/stage"),
|
||||
slug="demo-abc12",
|
||||
bundle_subnet="192.168.50.0/24",
|
||||
bundle_gateway="192.168.50.1",
|
||||
bundle_ip=bundle_ip,
|
||||
guest_env=dict(guest_env or {}),
|
||||
git_gate_plan=GitGatePlan(
|
||||
slug="demo-abc12",
|
||||
entrypoint_script=Path("/tmp/git-gate-entrypoint.sh"),
|
||||
hook_script=Path("/tmp/git-gate-hook"),
|
||||
access_hook_script=Path("/tmp/git-gate-access-hook"),
|
||||
upstreams=(),
|
||||
),
|
||||
egress_plan=EgressPlan(
|
||||
slug="demo-abc12",
|
||||
routes_path=Path("/tmp/routes.yaml"),
|
||||
routes=egress_routes,
|
||||
token_env_map={},
|
||||
mitmproxy_ca_cert_only_host_path=egress_ca_path,
|
||||
canary="fake-canary-value" if canary else "",
|
||||
canary_env="CANON_ALPHA_SECRET" if canary else "",
|
||||
),
|
||||
supervise_plan=supervise_plan,
|
||||
agent_git_gate_host=agent_git_gate_host,
|
||||
agent_supervise_url=agent_supervise_url,
|
||||
agent_provision=_agent_provision(
|
||||
agent_provider_template,
|
||||
codex_auth_file=codex_auth_file,
|
||||
guest_env=dict(guest_env or {}),
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def _agent_provision(
|
||||
template: str,
|
||||
*,
|
||||
codex_auth_file: Path | None = None,
|
||||
guest_env: dict[str, str] | None = None,
|
||||
) -> AgentProvisionPlan:
|
||||
if template != "codex":
|
||||
return AgentProvisionPlan(
|
||||
template=template,
|
||||
command=template,
|
||||
prompt_mode="append_file",
|
||||
image="bot-bottle-claude:latest",
|
||||
dockerfile="",
|
||||
guest_home="/home/node",
|
||||
instance_name="bot-bottle-demo-abc12",
|
||||
prompt_file=Path("/tmp/state/demo-abc12/agent/prompt.txt"),
|
||||
guest_env=dict(guest_env or {}),
|
||||
)
|
||||
auth_dir = (guest_env or {}).get("CODEX_HOME", "/home/node/.codex")
|
||||
files = [
|
||||
AgentProvisionFile(
|
||||
Path("/tmp/codex-config.toml"),
|
||||
f"{auth_dir}/config.toml",
|
||||
),
|
||||
]
|
||||
pre_copy: tuple[AgentProvisionCommand, ...] = ()
|
||||
verify: tuple[AgentProvisionCommand, ...] = ()
|
||||
if codex_auth_file is not None:
|
||||
files.append(AgentProvisionFile(codex_auth_file, f"{auth_dir}/auth.json"))
|
||||
pre_copy = (AgentProvisionCommand((
|
||||
"find", auth_dir,
|
||||
"-maxdepth", "1",
|
||||
"-type", "f",
|
||||
"(",
|
||||
"-name", "*.sqlite",
|
||||
"-o", "-name", "*.sqlite-*",
|
||||
"-o", "-name", "*.codex-repair-*.bak",
|
||||
")",
|
||||
"-delete",
|
||||
), "codex host credentials: could not reset runtime db files"),)
|
||||
verify = (AgentProvisionCommand((
|
||||
"runuser", "-u", "node", "--",
|
||||
"env",
|
||||
"HOME=/home/node",
|
||||
f"CODEX_HOME={auth_dir}",
|
||||
"codex", "login", "status",
|
||||
), "codex host credentials: dummy auth was copied into the guest"),)
|
||||
return AgentProvisionPlan(
|
||||
template="codex",
|
||||
command="codex",
|
||||
prompt_mode="read_prompt_file",
|
||||
image="bot-bottle-codex:latest",
|
||||
dockerfile="",
|
||||
guest_home="/home/node",
|
||||
instance_name="bot-bottle-demo-abc12",
|
||||
prompt_file=Path("/tmp/state/demo-abc12/agent/prompt.txt"),
|
||||
guest_env=dict(guest_env or {}),
|
||||
dirs=(AgentProvisionDir(auth_dir),),
|
||||
files=tuple(files),
|
||||
pre_copy=pre_copy,
|
||||
verify=verify,
|
||||
)
|
||||
|
||||
|
||||
def _write_self_signed_cert(path: Path) -> None:
|
||||
"""Drop a real self-signed PEM at `path` so provision_ca's
|
||||
fingerprint computation (PEM_cert_to_DER_cert + sha256) has
|
||||
actual bytes to chew on. Generated once per test via openssl."""
|
||||
subprocess.run(
|
||||
["openssl", "req", "-x509", "-newkey", "rsa:2048", "-nodes",
|
||||
"-keyout", "/dev/null",
|
||||
"-out", str(path),
|
||||
"-days", "1",
|
||||
"-subj", "/CN=test"],
|
||||
check=True, capture_output=True,
|
||||
)
|
||||
|
||||
|
||||
class TestProvisionCA(unittest.TestCase):
|
||||
"""provision_ca always uses the egress MITM CA and dispatches
|
||||
cp_in + exec in the right order."""
|
||||
|
||||
def setUp(self):
|
||||
self._tmp = tempfile.TemporaryDirectory(prefix="cb-prov-ca.") # pylint: disable=consider-using-with
|
||||
self.tmp = Path(self._tmp.name)
|
||||
self.egress_ca = self.tmp / "egress-ca.pem"
|
||||
_write_self_signed_cert(self.egress_ca)
|
||||
|
||||
def tearDown(self):
|
||||
self._tmp.cleanup()
|
||||
|
||||
# provision_ca dies hard if update-ca-certificates' exit
|
||||
# is non-zero; supply a stock success return so the bulk of
|
||||
# the tests below exercise the happy path.
|
||||
_UPDATE_OK = ExecResult(
|
||||
returncode=0,
|
||||
stdout="Updating certificates in /etc/ssl/certs...\n1 added, 0 removed; done.\n",
|
||||
stderr="",
|
||||
)
|
||||
|
||||
def test_egress_ca_always_installed(self):
|
||||
plan = _plan(egress_ca_path=self.egress_ca)
|
||||
bottle = _make_bottle(exec_result=self._UPDATE_OK)
|
||||
_PROVIDER.provision_ca(bottle, plan)
|
||||
bottle.cp_in.assert_called_once_with(
|
||||
str(self.egress_ca),
|
||||
AGENT_CA_PATH,
|
||||
)
|
||||
self.assertEqual(2, bottle.exec.call_count)
|
||||
script = bottle.exec.call_args_list[1].args[0]
|
||||
self.assertIn("chmod 644", script)
|
||||
self.assertIn("update-ca-certificates", script)
|
||||
self.assertEqual("root", bottle.exec.call_args.kwargs.get("user"))
|
||||
|
||||
def test_dies_when_egress_cert_missing(self):
|
||||
plan = _plan(egress_ca_path=self.tmp / "does-not-exist.pem")
|
||||
bottle = _make_bottle()
|
||||
with self.assertRaises(SystemExit):
|
||||
_PROVIDER.provision_ca(bottle, plan)
|
||||
|
||||
|
||||
class TestSmolmachinesBottleExec(unittest.TestCase):
|
||||
"""SmolmachinesBottle.exec retries once on SIGKILL (exit 137)."""
|
||||
|
||||
_SIGKILL = subprocess.CompletedProcess(
|
||||
args=[], returncode=137, stdout="", stderr="",
|
||||
)
|
||||
_SUCCESS = subprocess.CompletedProcess(
|
||||
args=[], returncode=0, stdout="done", stderr="",
|
||||
)
|
||||
|
||||
def test_retries_on_sigkill(self):
|
||||
bottle = SmolmachinesBottle("test-machine")
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.bottle.subprocess.run",
|
||||
side_effect=[self._SIGKILL, self._SUCCESS],
|
||||
) as mock_run, patch(
|
||||
"bot_bottle.backend.smolmachines.bottle.time.sleep"
|
||||
) as mock_sleep:
|
||||
result = bottle.exec("echo hi")
|
||||
|
||||
self.assertEqual(0, result.returncode)
|
||||
self.assertEqual(2, mock_run.call_count)
|
||||
mock_sleep.assert_called_once_with(1.0)
|
||||
|
||||
def test_no_retry_on_success(self):
|
||||
bottle = SmolmachinesBottle("test-machine")
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.bottle.subprocess.run",
|
||||
return_value=self._SUCCESS,
|
||||
) as mock_run:
|
||||
result = bottle.exec("echo hi")
|
||||
|
||||
self.assertEqual(0, result.returncode)
|
||||
self.assertEqual(1, mock_run.call_count)
|
||||
|
||||
def test_no_retry_on_other_error(self):
|
||||
fail = subprocess.CompletedProcess(args=[], returncode=1, stdout="", stderr="err")
|
||||
bottle = SmolmachinesBottle("test-machine")
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.bottle.subprocess.run",
|
||||
return_value=fail,
|
||||
) as mock_run:
|
||||
result = bottle.exec("bad-cmd")
|
||||
|
||||
self.assertEqual(1, result.returncode)
|
||||
self.assertEqual(1, mock_run.call_count)
|
||||
|
||||
|
||||
class TestProvisionGit(unittest.TestCase):
|
||||
"""provision_git writes gitconfig insteadOf rules when configured."""
|
||||
|
||||
def setUp(self):
|
||||
self._tmp = tempfile.TemporaryDirectory(prefix="cb-prov-git.") # pylint: disable=consider-using-with
|
||||
self.stage = Path(self._tmp.name)
|
||||
|
||||
def tearDown(self):
|
||||
self._tmp.cleanup()
|
||||
|
||||
def test_noop_when_no_cwd_and_no_git_entries(self):
|
||||
bottle = _make_bottle()
|
||||
_PROVIDER.provision_git(bottle, _plan(stage_dir=self.stage))
|
||||
bottle.cp_in.assert_not_called()
|
||||
bottle.exec.assert_not_called()
|
||||
|
||||
def test_writes_gitconfig_with_ip_port_form_for_smolmachines(self):
|
||||
# Smolmachines's TSI-allowlisted guest dials git-gate via
|
||||
# smart HTTP at `127.0.0.1:<host port>` — the bundle's
|
||||
# git HTTP port is published on host loopback at launch
|
||||
# time, and the plan carries the discovered host port.
|
||||
plan = _plan(
|
||||
git=[ManifestGitEntry(
|
||||
Name="bot-bottle",
|
||||
Upstream="ssh://git@host/repo.git",
|
||||
Key=ManifestKeyConfig(provider="static", path="~/.ssh/id_ed25519"),
|
||||
IdentityFile="~/.ssh/id_ed25519",
|
||||
)],
|
||||
stage_dir=self.stage,
|
||||
agent_git_gate_host="127.0.0.1:9418",
|
||||
)
|
||||
bottle = _make_bottle()
|
||||
_PROVIDER.provision_git(bottle, plan)
|
||||
# The staged gitconfig path is whatever NamedTemporaryFile
|
||||
# picked; we read its contents.
|
||||
cp_call = bottle.cp_in.call_args
|
||||
staged_path = Path(cp_call.args[0])
|
||||
self.assertEqual(self.stage, staged_path.parent)
|
||||
content = staged_path.read_text(encoding="utf-8")
|
||||
self.assertIn(
|
||||
'[url "http://127.0.0.1:9418/bot-bottle.git"]', content,
|
||||
)
|
||||
self.assertIn(
|
||||
"\tinsteadOf = ssh://git@host/repo.git", content,
|
||||
)
|
||||
|
||||
|
||||
class TestBundleLaunchSpec(unittest.TestCase):
|
||||
def test_git_gate_uses_http_daemon_for_smolmachines(self):
|
||||
plan = _plan()
|
||||
plan = replace(
|
||||
plan,
|
||||
git_gate_plan=replace(
|
||||
plan.git_gate_plan,
|
||||
upstreams=(GitGateUpstream(
|
||||
name="bot-bottle",
|
||||
upstream_url="ssh://git@host/repo.git",
|
||||
upstream_host="host",
|
||||
upstream_port="22",
|
||||
identity_file="/tmp/key",
|
||||
known_host_key="",
|
||||
),),
|
||||
),
|
||||
)
|
||||
|
||||
spec = _bundle_launch_spec(plan, "net", "127.0.0.16")
|
||||
|
||||
self.assertEqual(
|
||||
"egress,git-gate,git-http",
|
||||
spec.daemons_csv,
|
||||
)
|
||||
self.assertIn(9420, spec.ports_to_publish)
|
||||
self.assertNotIn(9418, spec.ports_to_publish)
|
||||
|
||||
def test_canary_env_registered_as_sensitive_in_bundle(self):
|
||||
plan = _plan(canary=True)
|
||||
|
||||
spec = _bundle_launch_spec(plan, "net", "127.0.0.16")
|
||||
|
||||
self.assertIn("CANON_ALPHA_SECRET=fake-canary-value", spec.environment)
|
||||
self.assertIn(
|
||||
"BOT_BOTTLE_SENSITIVE_PREFIXES=CANON_ALPHA_SECRET",
|
||||
spec.environment,
|
||||
)
|
||||
|
||||
def test_supervise_adds_daemon_volume_and_env(self):
|
||||
from bot_bottle.supervise import DB_PATH_IN_CONTAINER
|
||||
plan = _plan(supervise=True)
|
||||
spec = _bundle_launch_spec(plan, "net", "127.0.0.16")
|
||||
self.assertIn("supervise", spec.daemons_csv)
|
||||
self.assertIn(f"SUPERVISE_DB_PATH={DB_PATH_IN_CONTAINER}", spec.environment)
|
||||
self.assertIn(("/tmp/bot-bottle.db", DB_PATH_IN_CONTAINER, False), spec.volumes)
|
||||
|
||||
def test_canary_env_visible_to_smolvm_guest(self):
|
||||
plan = _plan(canary=True)
|
||||
with patch.object(
|
||||
_launch._bundle,
|
||||
"bundle_host_port",
|
||||
return_value="65000",
|
||||
):
|
||||
stamped = _launch._discover_urls(plan, "127.0.0.16")
|
||||
|
||||
self.assertEqual(
|
||||
"fake-canary-value",
|
||||
stamped.guest_env["CANON_ALPHA_SECRET"],
|
||||
)
|
||||
|
||||
|
||||
class TestProvisionGitUser(unittest.TestCase):
|
||||
"""`provision_git` runs `git config --global` inside the
|
||||
guest as the node user. SmolmachinesBottle.exec sets HOME and
|
||||
USER automatically for the requested user, so --global lands
|
||||
in /home/node/.gitconfig. No-op when the bottle didn't declare
|
||||
git_user (issue #86)."""
|
||||
|
||||
def _git_config_calls(self, bottle: MagicMock) -> list[tuple[str, str]]:
|
||||
"""Filter bottle.exec calls down to git-config invocations,
|
||||
return list of (script, user) tuples."""
|
||||
out = []
|
||||
for c in bottle.exec.call_args_list:
|
||||
script = c.args[0] if c.args else ""
|
||||
user = c.kwargs.get("user", "node")
|
||||
if "git config" in script:
|
||||
out.append((script, user))
|
||||
return out
|
||||
|
||||
def test_noop_when_no_git_user(self):
|
||||
bottle = _make_bottle()
|
||||
_PROVIDER.provision_git(bottle, _plan())
|
||||
self.assertEqual([], self._git_config_calls(bottle))
|
||||
|
||||
def test_sets_name_and_email_as_node(self):
|
||||
plan = _plan(git_user={
|
||||
"name": "Eric Bauerfeld",
|
||||
"email": "eric@dideric.is",
|
||||
})
|
||||
bottle = _make_bottle()
|
||||
_PROVIDER.provision_git(bottle, plan)
|
||||
calls = self._git_config_calls(bottle)
|
||||
self.assertEqual(2, len(calls))
|
||||
# Both run as node so SmolmachinesBottle.exec sets HOME=/home/node
|
||||
# automatically, ensuring --global writes to /home/node/.gitconfig.
|
||||
for script, user in calls:
|
||||
self.assertEqual("node", user)
|
||||
self.assertIn("git config --global", script)
|
||||
self.assertIn("user.name", calls[0][0])
|
||||
self.assertIn("Eric Bauerfeld", calls[0][0])
|
||||
self.assertIn("user.email", calls[1][0])
|
||||
self.assertIn("eric@dideric.is", calls[1][0])
|
||||
|
||||
def test_name_only(self):
|
||||
plan = _plan(git_user={"name": "Bot"})
|
||||
bottle = _make_bottle()
|
||||
_PROVIDER.provision_git(bottle, plan)
|
||||
calls = self._git_config_calls(bottle)
|
||||
self.assertEqual(1, len(calls))
|
||||
self.assertIn("user.name", calls[0][0])
|
||||
self.assertIn("Bot", calls[0][0])
|
||||
|
||||
def test_email_only(self):
|
||||
plan = _plan(git_user={"email": "bot@example.com"})
|
||||
bottle = _make_bottle()
|
||||
_PROVIDER.provision_git(bottle, plan)
|
||||
calls = self._git_config_calls(bottle)
|
||||
self.assertEqual(1, len(calls))
|
||||
self.assertIn("user.email", calls[0][0])
|
||||
self.assertIn("bot@example.com", calls[0][0])
|
||||
|
||||
|
||||
class TestProxyHost(unittest.TestCase):
|
||||
"""_proxy_host returns the bridge gateway on Linux and the loopback
|
||||
alias on other platforms."""
|
||||
|
||||
def test_linux_returns_bundle_gateway(self):
|
||||
plan = _plan()
|
||||
with patch("bot_bottle.backend.smolmachines.launch.platform.system",
|
||||
return_value="Linux"):
|
||||
result = _launch._proxy_host(plan, "127.0.0.16")
|
||||
self.assertEqual(plan.bundle_gateway, result)
|
||||
|
||||
def test_non_linux_returns_loopback(self):
|
||||
plan = _plan()
|
||||
with patch("bot_bottle.backend.smolmachines.launch.platform.system",
|
||||
return_value="Darwin"):
|
||||
result = _launch._proxy_host(plan, "127.0.0.16")
|
||||
self.assertEqual("127.0.0.16", result)
|
||||
|
||||
|
||||
class TestDiscoverUrls(unittest.TestCase):
|
||||
"""_discover_urls stamps git-gate host + supervise URL into the plan."""
|
||||
|
||||
def test_git_gate_host_set_when_upstreams_present(self):
|
||||
plan = _plan()
|
||||
plan = replace(
|
||||
plan,
|
||||
git_gate_plan=replace(
|
||||
plan.git_gate_plan,
|
||||
upstreams=(GitGateUpstream(
|
||||
name="bot-bottle",
|
||||
upstream_url="ssh://git@host/repo.git",
|
||||
upstream_host="host",
|
||||
upstream_port="22",
|
||||
identity_file="/tmp/key",
|
||||
known_host_key="",
|
||||
),),
|
||||
),
|
||||
)
|
||||
with patch.object(_launch._bundle, "bundle_host_port", return_value="9420"):
|
||||
stamped = _launch._discover_urls(plan, "127.0.0.16")
|
||||
self.assertEqual("127.0.0.16:9420", stamped.agent_git_gate_host)
|
||||
|
||||
def test_supervise_url_set_when_supervise_present(self):
|
||||
plan = _plan(supervise=True)
|
||||
with patch.object(_launch._bundle, "bundle_host_port", return_value="55556"):
|
||||
stamped = _launch._discover_urls(plan, "127.0.0.16")
|
||||
self.assertEqual("http://127.0.0.16:55556/", stamped.agent_supervise_url)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,164 +0,0 @@
|
||||
"""Unit: smolmachines pty_resize bridge (issue #82).
|
||||
|
||||
Locks down the parts of the wrapper we can test without spawning
|
||||
real children or signalling — argument parsing, the side-channel
|
||||
`smolvm machine exec` argv shape, and TTY-resolution fallback
|
||||
across stdin/stdout/stderr.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
import unittest
|
||||
import unittest.mock
|
||||
from unittest.mock import patch
|
||||
|
||||
from bot_bottle.backend.smolmachines import pty_resize
|
||||
|
||||
|
||||
class TestPushSize(unittest.TestCase):
|
||||
def test_emits_for_loop_over_all_pts_devices(self):
|
||||
# The shell `for f in /dev/pts/*` handles multiple
|
||||
# interactive sessions in the same VM (rare but cheap).
|
||||
# Per-PTY `stty -F ... 2>/dev/null` swallows EBADF when a
|
||||
# session has already exited.
|
||||
with patch.object(pty_resize.subprocess, "run") as run:
|
||||
pty_resize._push_size("bot-bottle-m", 50, 200)
|
||||
argv = run.call_args.args[0]
|
||||
self.assertEqual(
|
||||
["smolvm", "machine", "exec", "--name",
|
||||
"bot-bottle-m", "--", "sh", "-c"],
|
||||
argv[:8],
|
||||
)
|
||||
# cols / rows land in the order stty wants them.
|
||||
self.assertIn("cols 200", argv[8])
|
||||
self.assertIn("rows 50", argv[8])
|
||||
self.assertIn("for f in /dev/pts/*", argv[8])
|
||||
|
||||
def test_side_channel_uses_devnull_stdin(self):
|
||||
# Load-bearing regression: under tmux, inheriting the
|
||||
# pane PTY as the side-channel's stdin makes smolvm crash
|
||||
# within ~100ms (concurrent smolvm processes sharing the
|
||||
# PTY's FG-PG / input plumbing). DEVNULL stdin sidesteps
|
||||
# the interaction.
|
||||
with patch.object(pty_resize.subprocess, "run") as run:
|
||||
pty_resize._push_size("bot-bottle-m", 24, 80)
|
||||
self.assertEqual(
|
||||
pty_resize.subprocess.DEVNULL,
|
||||
run.call_args.kwargs.get("stdin"),
|
||||
)
|
||||
|
||||
def test_swallows_subprocess_failures(self):
|
||||
# `check=False` + DEVNULL streams: a side-channel failure
|
||||
# mustn't break the operator's session.
|
||||
with patch.object(
|
||||
pty_resize.subprocess, "run",
|
||||
side_effect=OSError("boom"),
|
||||
):
|
||||
with self.assertRaises(OSError):
|
||||
pty_resize._push_size("m", 24, 80)
|
||||
# The wrapper-level `sync()` is what swallows; `_push_size`
|
||||
# itself raises so the test above documents that. The
|
||||
# signal-handler-side `sync` in main wraps in try/except
|
||||
# via the `if size is None: return` guard for the
|
||||
# no-TTY case (no separate try needed because subprocess
|
||||
# already has check=False; only fcntl.ioctl raising would
|
||||
# surface, and _read_winsize handles that).
|
||||
|
||||
|
||||
class TestReadWinsize(unittest.TestCase):
|
||||
def test_returns_none_when_no_tty(self):
|
||||
# Patch ioctl to always OSError — simulates the case where
|
||||
# none of stdin/stdout/stderr is a TTY (e.g., tests, piped
|
||||
# automation).
|
||||
with patch.object(
|
||||
pty_resize.fcntl, "ioctl",
|
||||
side_effect=OSError("ENOTTY"),
|
||||
):
|
||||
self.assertIsNone(pty_resize._read_winsize())
|
||||
|
||||
def test_returns_first_tty_size(self):
|
||||
# First fd that responds with a non-zero size wins —
|
||||
# matches the "different surfaces give different TTYs"
|
||||
# invariant noted in the module docstring.
|
||||
import struct
|
||||
|
||||
calls: list[int] = []
|
||||
|
||||
def fake_ioctl(fd, req, buf): # type: ignore
|
||||
calls.append(fd)
|
||||
if fd == 0:
|
||||
raise OSError("stdin not a tty")
|
||||
return struct.pack("hhhh", 42, 137, 0, 0)
|
||||
|
||||
with patch.object(pty_resize.fcntl, "ioctl", side_effect=fake_ioctl):
|
||||
self.assertEqual((42, 137), pty_resize._read_winsize())
|
||||
|
||||
def test_skips_zero_sizes(self):
|
||||
# A TTY that reports `0 0` (the smolvm-allocated PTY's
|
||||
# initial state, ironically) shouldn't be used as the
|
||||
# source of truth — keep probing fallback fds.
|
||||
import struct
|
||||
|
||||
responses = iter([
|
||||
struct.pack("hhhh", 0, 0, 0, 0), # stdin: zero
|
||||
struct.pack("hhhh", 24, 80, 0, 0), # stdout: real
|
||||
])
|
||||
|
||||
def fake_ioctl(fd, req, buf): # type: ignore
|
||||
return next(responses)
|
||||
|
||||
with patch.object(pty_resize.fcntl, "ioctl", side_effect=fake_ioctl):
|
||||
self.assertEqual((24, 80), pty_resize._read_winsize())
|
||||
|
||||
|
||||
class TestMainArgvParsing(unittest.TestCase):
|
||||
def test_missing_separator_returns_error_exit_code(self):
|
||||
# No `--` between machine name and inner argv.
|
||||
with patch.object(pty_resize.sys, "stderr", new=io.StringIO()) as err:
|
||||
rc = pty_resize.main(["bot-bottle-m", "smolvm", "machine"])
|
||||
self.assertEqual(2, rc)
|
||||
self.assertIn("usage:", err.getvalue())
|
||||
|
||||
def test_too_few_args_returns_error_exit_code(self):
|
||||
with patch.object(pty_resize.sys, "stderr", new=io.StringIO()):
|
||||
self.assertEqual(2, pty_resize.main([]))
|
||||
self.assertEqual(2, pty_resize.main(["m"]))
|
||||
self.assertEqual(2, pty_resize.main(["m", "--"]))
|
||||
|
||||
|
||||
class TestStartupSyncDeferred(unittest.TestCase):
|
||||
"""Regression: the initial sync MUST be deferred (timer), not
|
||||
called synchronously between Popen + wait. Calling it
|
||||
immediately races libkrun's per-exec OCI config write during
|
||||
the main exec's bringup and crashes the child (rc=137 or
|
||||
'parse error: trailing garbage')."""
|
||||
|
||||
def test_main_schedules_timer_does_not_call_sync_synchronously(self):
|
||||
# Fake Popen + wait so main returns immediately. Patch
|
||||
# Timer to record args without spawning a real thread.
|
||||
# _push_size patched so any rogue synchronous call would
|
||||
# be observable.
|
||||
fake_proc = unittest.mock.MagicMock()
|
||||
fake_proc.wait.return_value = 0
|
||||
with patch.object(
|
||||
pty_resize.subprocess, "Popen", return_value=fake_proc,
|
||||
), patch.object(
|
||||
pty_resize.threading, "Timer",
|
||||
) as timer_cls, patch.object(
|
||||
pty_resize, "_push_size",
|
||||
) as push:
|
||||
rc = pty_resize.main(["machine-name", "--", "echo", "hi"])
|
||||
|
||||
self.assertEqual(0, rc)
|
||||
# Timer scheduled with the documented delay constant.
|
||||
timer_cls.assert_called_once()
|
||||
delay, callback = timer_cls.call_args.args # type: ignore
|
||||
self.assertEqual(pty_resize._STARTUP_SYNC_DELAY_SEC, delay)
|
||||
# _push_size never called synchronously — the only path to
|
||||
# it is via the (mocked) timer's callback firing.
|
||||
push.assert_not_called()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,226 +0,0 @@
|
||||
"""Unit: bundle bringup primitives for the smolmachines backend
|
||||
(PRD 0023 chunk 2c).
|
||||
|
||||
Tests mock `subprocess.run` and assert on the docker argv shape.
|
||||
The end-to-end integration smoke (real docker daemon, real
|
||||
bundle image) lands in chunk 2d."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
from bot_bottle.backend.smolmachines.sidecar_bundle import (
|
||||
BundleLaunchSpec,
|
||||
bundle_container_name,
|
||||
bundle_network_name,
|
||||
create_bundle_network,
|
||||
ensure_bundle_image,
|
||||
remove_bundle_network,
|
||||
start_bundle,
|
||||
stop_bundle,
|
||||
)
|
||||
|
||||
|
||||
def _ok(stdout: str = "", stderr: str = "") -> subprocess.CompletedProcess: # type: ignore
|
||||
return subprocess.CompletedProcess(
|
||||
args=[], returncode=0, stdout=stdout, stderr=stderr,
|
||||
)
|
||||
|
||||
|
||||
def _fail(stderr: str = "boom") -> subprocess.CompletedProcess: # type: ignore
|
||||
return subprocess.CompletedProcess(
|
||||
args=[], returncode=1, stdout="", stderr=stderr,
|
||||
)
|
||||
|
||||
|
||||
def _spec(**kwargs) -> BundleLaunchSpec: # type: ignore
|
||||
defaults = dict(
|
||||
slug="demo-abc12",
|
||||
network_name="bot-bottle-bundle-demo-abc12",
|
||||
subnet="192.168.50.0/24",
|
||||
gateway="192.168.50.1",
|
||||
bundle_ip="192.168.50.2",
|
||||
)
|
||||
defaults.update(kwargs)
|
||||
return BundleLaunchSpec(**defaults) # type: ignore
|
||||
|
||||
|
||||
class TestNamingHelpers(unittest.TestCase):
|
||||
def test_network_name_uses_bundle_prefix(self):
|
||||
# Distinct from the docker backend's
|
||||
# `bot-bottle-net-<slug>` so two backends running the
|
||||
# same agent slug don't collide.
|
||||
self.assertEqual(
|
||||
"bot-bottle-bundle-myagent-xyz",
|
||||
bundle_network_name("myagent-xyz"),
|
||||
)
|
||||
|
||||
def test_container_name_matches_docker_bundle_shape(self):
|
||||
# Same shape PRD 0024 chunk 5 set for the docker backend's
|
||||
# bundle container — dashboard prefix-discovery covers
|
||||
# both backends with one filter.
|
||||
self.assertEqual(
|
||||
"bot-bottle-sidecars-myagent-xyz",
|
||||
bundle_container_name("myagent-xyz"),
|
||||
)
|
||||
|
||||
|
||||
class TestNetworkLifecycle(unittest.TestCase):
|
||||
def _patch_run(self, **kwargs): # type: ignore
|
||||
return patch(
|
||||
"bot_bottle.backend.smolmachines.sidecar_bundle.subprocess.run",
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
def test_create_argv_explicit_subnet_and_gateway(self):
|
||||
with self._patch_run(return_value=_ok()) as m:
|
||||
create_bundle_network("nn", "192.168.50.0/24", "192.168.50.1")
|
||||
self.assertEqual(
|
||||
["docker", "network", "create",
|
||||
"--subnet", "192.168.50.0/24",
|
||||
"--gateway", "192.168.50.1",
|
||||
"nn"],
|
||||
m.call_args.args[0],
|
||||
)
|
||||
|
||||
def test_create_treats_existing_network_as_success(self):
|
||||
with self._patch_run(return_value=_fail("network nn already exists")):
|
||||
# No SystemExit.
|
||||
create_bundle_network("nn", "192.168.50.0/24", "192.168.50.1")
|
||||
|
||||
def test_create_other_failure_is_fatal(self):
|
||||
with self._patch_run(return_value=_fail("invalid subnet")):
|
||||
with self.assertRaises(SystemExit):
|
||||
create_bundle_network("nn", "bogus", "bogus")
|
||||
|
||||
def test_remove_missing_network_is_idempotent(self):
|
||||
# No SystemExit / no warn-and-continue noise; missing
|
||||
# network is the expected case during a partial teardown.
|
||||
with self._patch_run(return_value=_fail("Error: No such network: nn")):
|
||||
remove_bundle_network("nn")
|
||||
|
||||
def test_remove_clean_returns_success(self):
|
||||
with self._patch_run(return_value=_ok()):
|
||||
remove_bundle_network("nn")
|
||||
|
||||
|
||||
class TestStartBundle(unittest.TestCase):
|
||||
def _patch_run(self):
|
||||
return patch(
|
||||
"bot_bottle.backend.smolmachines.sidecar_bundle.subprocess.run",
|
||||
return_value=_ok(),
|
||||
)
|
||||
|
||||
def test_argv_pins_ip_on_network(self):
|
||||
with self._patch_run() as m:
|
||||
start_bundle(_spec())
|
||||
argv = m.call_args.args[0]
|
||||
# --network NETNAME --ip <bundle-ip> on the docker run.
|
||||
self.assertIn("--network", argv)
|
||||
self.assertIn("bot-bottle-bundle-demo-abc12", argv)
|
||||
self.assertIn("--ip", argv)
|
||||
self.assertIn("192.168.50.2", argv)
|
||||
# Detached and auto-removed.
|
||||
self.assertIn("--detach", argv)
|
||||
self.assertIn("--rm", argv)
|
||||
# Container name uses the per-slug bundle prefix.
|
||||
i = argv.index("--name")
|
||||
self.assertEqual("bot-bottle-sidecars-demo-abc12", argv[i + 1])
|
||||
# Image at the end.
|
||||
self.assertEqual("bot-bottle-sidecars:latest", argv[-1])
|
||||
|
||||
def test_daemons_env_passed_in(self):
|
||||
with self._patch_run() as m:
|
||||
start_bundle(_spec(daemons_csv="egress,supervise"))
|
||||
argv = m.call_args.args[0]
|
||||
self.assertIn("-e", argv)
|
||||
self.assertIn(
|
||||
"BOT_BOTTLE_SIDECAR_DAEMONS=egress,supervise",
|
||||
argv,
|
||||
)
|
||||
|
||||
def test_environment_entries_pass_through(self):
|
||||
with self._patch_run() as m:
|
||||
start_bundle(_spec(environment=(
|
||||
"SUPERVISE_BOTTLE_SLUG=demo-abc12",
|
||||
"EGRESS_TOKEN_0", # bare-name → host env inherit
|
||||
)))
|
||||
argv = m.call_args.args[0]
|
||||
self.assertIn("SUPERVISE_BOTTLE_SLUG=demo-abc12", argv)
|
||||
self.assertIn("EGRESS_TOKEN_0", argv)
|
||||
|
||||
def test_volumes_render_with_ro_flag(self):
|
||||
with self._patch_run() as m:
|
||||
start_bundle(_spec(volumes=(
|
||||
("/host/egress-ca.pem", "/home/mitmproxy/.mitmproxy/mitmproxy-ca.pem", True),
|
||||
("/host/queue", "/run/supervise/queue", False),
|
||||
)))
|
||||
argv = m.call_args.args[0]
|
||||
self.assertIn(
|
||||
"/host/egress-ca.pem:/home/mitmproxy/.mitmproxy/mitmproxy-ca.pem:ro",
|
||||
argv,
|
||||
)
|
||||
self.assertIn("/host/queue:/run/supervise/queue", argv)
|
||||
|
||||
def test_failure_dies(self):
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.sidecar_bundle.subprocess.run",
|
||||
return_value=_fail("invalid mount"),
|
||||
):
|
||||
with self.assertRaises(SystemExit):
|
||||
start_bundle(_spec())
|
||||
|
||||
def test_host_env_inherited_to_subprocess(self):
|
||||
# Bare-name entries in spec.environment rely on the docker
|
||||
# subprocess being run with the host env. Confirm `env=`
|
||||
# threads through.
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.sidecar_bundle.subprocess.run",
|
||||
return_value=_ok(),
|
||||
) as m:
|
||||
start_bundle(_spec(), env={"FOO": "bar"})
|
||||
self.assertEqual({"FOO": "bar"}, m.call_args.kwargs["env"])
|
||||
|
||||
|
||||
class TestEnsureBundleImage(unittest.TestCase):
|
||||
def test_builds_sidecar_dockerfile_before_plain_docker_run(self):
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.sidecar_bundle.docker_mod.build_image",
|
||||
) as build:
|
||||
ensure_bundle_image()
|
||||
|
||||
build.assert_called_once()
|
||||
args = build.call_args.args
|
||||
kwargs = build.call_args.kwargs
|
||||
self.assertEqual("bot-bottle-sidecars:latest", args[0])
|
||||
self.assertTrue((Path(args[1]) / "Dockerfile.sidecars").is_file())
|
||||
self.assertEqual("Dockerfile.sidecars", kwargs["dockerfile"])
|
||||
|
||||
|
||||
class TestStopBundle(unittest.TestCase):
|
||||
def _patch_run(self, **kwargs): # type: ignore
|
||||
return patch(
|
||||
"bot_bottle.backend.smolmachines.sidecar_bundle.subprocess.run",
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
def test_argv_force_removes(self):
|
||||
with self._patch_run(return_value=_ok()) as m:
|
||||
stop_bundle("demo-abc12")
|
||||
self.assertEqual(
|
||||
["docker", "rm", "-f", "bot-bottle-sidecars-demo-abc12"],
|
||||
m.call_args.args[0],
|
||||
)
|
||||
|
||||
def test_missing_container_is_idempotent(self):
|
||||
with self._patch_run(return_value=_fail(
|
||||
"Error: No such container: bot-bottle-sidecars-demo-abc12"
|
||||
)):
|
||||
stop_bundle("demo-abc12") # no raise
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,284 +0,0 @@
|
||||
"""Unit: smolvm subprocess wrapper (PRD 0023 chunk 2b).
|
||||
|
||||
The wrapper is one thin function per smolvm CLI subcommand. Tests
|
||||
mock `subprocess.run` and assert on the constructed argv +
|
||||
SmolvmError raising on non-zero. The actual smolvm binary's
|
||||
behavior is exercised in chunk 2d's integration smoke test."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
from bot_bottle.backend.smolmachines import smolvm as smolvm_mod
|
||||
from bot_bottle.backend.smolmachines.smolvm import (
|
||||
SmolvmError,
|
||||
SmolvmRunResult,
|
||||
is_available,
|
||||
machine_cp,
|
||||
machine_create,
|
||||
machine_delete,
|
||||
machine_exec,
|
||||
machine_start,
|
||||
machine_stop,
|
||||
pack_create,
|
||||
pack_create_from_vm,
|
||||
wait_exec_ready,
|
||||
)
|
||||
|
||||
|
||||
def _ok(stdout: str = "", stderr: str = "") -> subprocess.CompletedProcess: # type: ignore
|
||||
return subprocess.CompletedProcess(
|
||||
args=[], returncode=0, stdout=stdout, stderr=stderr,
|
||||
)
|
||||
|
||||
|
||||
def _fail(stderr: str = "boom") -> subprocess.CompletedProcess: # type: ignore
|
||||
return subprocess.CompletedProcess(
|
||||
args=[], returncode=1, stdout="", stderr=stderr,
|
||||
)
|
||||
|
||||
|
||||
class TestArgvShapes(unittest.TestCase):
|
||||
"""The CLI mixes `--name NAME` and positional-NAME styles
|
||||
across subcommands. The wrapper hides that inconsistency
|
||||
behind a uniform `name=` kwarg; lock down which form lands
|
||||
in each argv."""
|
||||
|
||||
def _patch_run(self):
|
||||
return patch(
|
||||
"bot_bottle.backend.smolmachines.smolvm.subprocess.run",
|
||||
return_value=_ok(),
|
||||
)
|
||||
|
||||
def test_pack_create_argv(self):
|
||||
with self._patch_run() as m:
|
||||
pack_create("bot-bottle-claude:latest", Path("/tmp/agent.smolmachine"))
|
||||
argv = m.call_args.args[0]
|
||||
self.assertEqual(
|
||||
["smolvm", "pack", "create",
|
||||
"--image", "bot-bottle-claude:latest",
|
||||
"-o", "/tmp/agent.smolmachine"],
|
||||
argv,
|
||||
)
|
||||
|
||||
def test_pack_create_from_vm_argv(self):
|
||||
with self._patch_run() as m:
|
||||
pack_create_from_vm("bot-bottle-dev-abc12", Path("/tmp/committed"))
|
||||
argv = m.call_args.args[0]
|
||||
self.assertEqual(
|
||||
["smolvm", "pack", "create",
|
||||
"--from-vm", "bot-bottle-dev-abc12",
|
||||
"-o", "/tmp/committed"],
|
||||
argv,
|
||||
)
|
||||
|
||||
def test_machine_create_minimal(self):
|
||||
with self._patch_run() as m:
|
||||
machine_create("agent-xyz")
|
||||
self.assertEqual(
|
||||
["smolvm", "machine", "create", "--name", "agent-xyz"],
|
||||
m.call_args.args[0],
|
||||
)
|
||||
|
||||
def test_machine_create_with_from_and_allow_cidr_and_env(self):
|
||||
with self._patch_run() as m:
|
||||
machine_create(
|
||||
"agent-xyz",
|
||||
from_path=Path("/stage/agent.smolmachine"),
|
||||
allow_cidrs=["192.168.50.2/32"],
|
||||
env={"HTTPS_PROXY": "http://192.168.50.2:8888"},
|
||||
)
|
||||
argv = m.call_args.args[0]
|
||||
# --from + --allow-cidr + -e are all flags, name is positional.
|
||||
self.assertEqual("smolvm", argv[0])
|
||||
self.assertIn("--from", argv)
|
||||
self.assertIn("/stage/agent.smolmachine", argv)
|
||||
# `--net` is explicit because smolvm 0.8.0's implied-net
|
||||
# from --allow-cidr doesn't fire when --from is set.
|
||||
self.assertIn("--net", argv)
|
||||
self.assertIn("--allow-cidr", argv)
|
||||
self.assertIn("192.168.50.2/32", argv)
|
||||
self.assertIn("-e", argv)
|
||||
self.assertIn("HTTPS_PROXY=http://192.168.50.2:8888", argv)
|
||||
self.assertIn("--name", argv)
|
||||
self.assertIn("agent-xyz", argv)
|
||||
|
||||
def test_machine_create_omits_net_when_no_allow_cidrs(self):
|
||||
with self._patch_run() as m:
|
||||
machine_create("agent-xyz", from_path=Path("/x.smolmachine"))
|
||||
self.assertNotIn("--net", m.call_args.args[0])
|
||||
|
||||
def test_machine_start_uses_dash_name(self):
|
||||
# `start` is the --name flag form, NOT positional.
|
||||
with self._patch_run() as m:
|
||||
machine_start("agent-xyz")
|
||||
self.assertEqual(
|
||||
["smolvm", "machine", "start", "--name", "agent-xyz"],
|
||||
m.call_args.args[0],
|
||||
)
|
||||
|
||||
def test_machine_stop_uses_dash_name(self):
|
||||
with self._patch_run() as m:
|
||||
machine_stop("agent-xyz")
|
||||
self.assertEqual(
|
||||
["smolvm", "machine", "stop", "--name", "agent-xyz"],
|
||||
m.call_args.args[0],
|
||||
)
|
||||
|
||||
def test_machine_delete_uses_name_flag_and_force(self):
|
||||
# delete uses --name flag; -f required so no interactive
|
||||
# confirmation blocks teardown.
|
||||
with self._patch_run() as m:
|
||||
machine_delete("agent-xyz")
|
||||
self.assertEqual(
|
||||
["smolvm", "machine", "delete", "--name", "agent-xyz", "-f"],
|
||||
m.call_args.args[0],
|
||||
)
|
||||
|
||||
def test_machine_exec_argv_with_separator(self):
|
||||
# `--` separator before the command — smolvm's flag parser
|
||||
# would otherwise grab argv items that look like flags.
|
||||
with self._patch_run() as m:
|
||||
machine_exec("agent-xyz", ["echo", "hello"])
|
||||
self.assertEqual(
|
||||
["smolvm", "machine", "exec", "--name", "agent-xyz",
|
||||
"--", "echo", "hello"],
|
||||
m.call_args.args[0],
|
||||
)
|
||||
|
||||
def test_machine_exec_env_workdir_timeout(self):
|
||||
with self._patch_run() as m:
|
||||
machine_exec(
|
||||
"agent-xyz",
|
||||
["ls"],
|
||||
env={"FOO": "bar", "BAZ": "qux"},
|
||||
workdir="/app",
|
||||
timeout="30s",
|
||||
)
|
||||
argv = m.call_args.args[0]
|
||||
self.assertIn("-w", argv); self.assertIn("/app", argv)
|
||||
self.assertIn("--timeout", argv); self.assertIn("30s", argv)
|
||||
# Both env vars present as -e K=V pairs.
|
||||
for pair in ("FOO=bar", "BAZ=qux"):
|
||||
self.assertIn(pair, argv)
|
||||
|
||||
def test_machine_cp_positional_argv(self):
|
||||
with self._patch_run() as m:
|
||||
machine_cp("/host/file", "agent-xyz:/dest/file")
|
||||
self.assertEqual(
|
||||
["smolvm", "machine", "cp",
|
||||
"/host/file", "agent-xyz:/dest/file"],
|
||||
m.call_args.args[0],
|
||||
)
|
||||
|
||||
def test_machine_cp_empty_is_noop(self):
|
||||
# Guard against an upstream caller passing an unset path —
|
||||
# cp with an empty string is meaningless and would just
|
||||
# confuse smolvm's error message.
|
||||
with self._patch_run() as m:
|
||||
machine_cp("", "agent-xyz:/dest")
|
||||
machine_cp("/host", "")
|
||||
self.assertEqual(0, m.call_count)
|
||||
|
||||
|
||||
class TestErrorPath(unittest.TestCase):
|
||||
"""`check=True` paths raise SmolvmError on non-zero; `exec` is
|
||||
the one path that returns a result regardless."""
|
||||
|
||||
def test_create_failure_raises(self):
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.smolvm.subprocess.run",
|
||||
return_value=_fail("no such image"),
|
||||
):
|
||||
with self.assertRaises(SmolvmError) as cm:
|
||||
machine_create("agent-xyz")
|
||||
self.assertEqual(1, cm.exception.returncode)
|
||||
self.assertIn("no such image", str(cm.exception))
|
||||
|
||||
def test_pack_create_failure_raises(self):
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.smolvm.subprocess.run",
|
||||
return_value=_fail("pack failed"),
|
||||
):
|
||||
with self.assertRaises(SmolvmError):
|
||||
pack_create("missing:tag", Path("/tmp/out"))
|
||||
|
||||
def test_pack_create_from_vm_failure_raises(self):
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.smolvm.subprocess.run",
|
||||
return_value=_fail("pack failed"),
|
||||
):
|
||||
with self.assertRaises(SmolvmError):
|
||||
pack_create_from_vm("bot-bottle-dev-abc12", Path("/tmp/out"))
|
||||
|
||||
def test_exec_failure_returns_result(self):
|
||||
# The in-VM command's exit code is what Bottle.exec sees;
|
||||
# `false` exiting non-zero is not a smolvm failure.
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.smolvm.subprocess.run",
|
||||
return_value=subprocess.CompletedProcess(
|
||||
args=[], returncode=42, stdout="", stderr="nope",
|
||||
),
|
||||
):
|
||||
r = machine_exec("agent-xyz", ["sh", "-c", "exit 42"])
|
||||
self.assertEqual(SmolvmRunResult(42, "", "nope"), r)
|
||||
|
||||
|
||||
class TestWaitExecReady(unittest.TestCase):
|
||||
"""wait_exec_ready polls machine_exec(name, ["true"]) until it
|
||||
returns 0, then exits. On timeout it calls die()."""
|
||||
|
||||
def test_returns_immediately_when_exec_succeeds_first_try(self):
|
||||
with patch.object(smolvm_mod, "machine_exec",
|
||||
return_value=SmolvmRunResult(0, "", "")) as m:
|
||||
wait_exec_ready("vm-x")
|
||||
m.assert_called_once_with("vm-x", ["true"])
|
||||
|
||||
def test_retries_on_nonzero_and_returns_on_success(self):
|
||||
results = [
|
||||
SmolvmRunResult(1, "", "not ready"),
|
||||
SmolvmRunResult(1, "", "not ready"),
|
||||
SmolvmRunResult(0, "", ""),
|
||||
]
|
||||
with patch.object(smolvm_mod, "machine_exec",
|
||||
side_effect=results) as m, \
|
||||
patch.object(smolvm_mod.time, "sleep"):
|
||||
wait_exec_ready("vm-x")
|
||||
self.assertEqual(3, m.call_count)
|
||||
|
||||
def test_raises_smolvm_error_on_timeout(self):
|
||||
# machine_exec always returns non-zero; monotonic advances past
|
||||
# the deadline after the first sleep so the loop exits.
|
||||
ticks = [0.0, 0.0, 10.0] # third call puts us past deadline
|
||||
with patch.object(smolvm_mod, "machine_exec",
|
||||
return_value=SmolvmRunResult(1, "", "")), \
|
||||
patch.object(smolvm_mod.time, "monotonic",
|
||||
side_effect=ticks), \
|
||||
patch.object(smolvm_mod.time, "sleep"):
|
||||
with self.assertRaises(SmolvmError) as cm:
|
||||
wait_exec_ready("vm-x", timeout=5.0)
|
||||
self.assertIn("vm-x", str(cm.exception))
|
||||
self.assertIn("not ready", str(cm.exception))
|
||||
|
||||
|
||||
class TestIsAvailable(unittest.TestCase):
|
||||
def test_true_when_on_path(self):
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.smolvm.shutil.which",
|
||||
return_value="/usr/local/bin/smolvm",
|
||||
):
|
||||
self.assertTrue(is_available())
|
||||
|
||||
def test_false_when_missing(self):
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.smolvm.shutil.which",
|
||||
return_value=None,
|
||||
):
|
||||
self.assertFalse(is_available())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,155 +0,0 @@
|
||||
"""Unit: smolmachines backend util helpers (PRD 0023)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
from bot_bottle.backend.smolmachines.util import (
|
||||
smolmachines_bundle_subnet,
|
||||
smolmachines_preflight,
|
||||
)
|
||||
|
||||
|
||||
class TestBundleSubnet(unittest.TestCase):
|
||||
def test_returns_subnet_gateway_and_bundle_ip(self):
|
||||
subnet, gateway, bundle_ip = smolmachines_bundle_subnet("demo-abc12")
|
||||
self.assertTrue(subnet.startswith("192.168."))
|
||||
self.assertTrue(subnet.endswith(".0/24"))
|
||||
# Gateway at .1, bundle at .2 — fixed convention.
|
||||
self.assertTrue(gateway.endswith(".1"))
|
||||
self.assertTrue(bundle_ip.endswith(".2"))
|
||||
# All three share the same third octet.
|
||||
third = subnet.split(".")[2]
|
||||
self.assertEqual(third, gateway.split(".")[2])
|
||||
self.assertEqual(third, bundle_ip.split(".")[2])
|
||||
|
||||
def test_stable_for_same_slug(self):
|
||||
# Recoverability: `cli.py resume` reuses the slug and
|
||||
# expects to find the same per-bottle subnet (a fresh
|
||||
# docker bridge would mean a different IP, and smolvm's
|
||||
# allow_cidrs would no longer match).
|
||||
a = smolmachines_bundle_subnet("demo-abc12")
|
||||
b = smolmachines_bundle_subnet("demo-abc12")
|
||||
self.assertEqual(a, b)
|
||||
|
||||
def test_different_slugs_likely_differ(self):
|
||||
# Not a guarantee — it's hash-mod-254, collisions exist —
|
||||
# but two arbitrary slugs shouldn't share a subnet in the
|
||||
# typical case.
|
||||
seen = {
|
||||
smolmachines_bundle_subnet(s)[0]
|
||||
for s in ("a", "b", "c", "d", "e", "alpha", "beta", "gamma")
|
||||
}
|
||||
self.assertGreater(len(seen), 1)
|
||||
|
||||
def test_skips_docker_default_octet(self):
|
||||
# docker's default bridge sits at 172.17.x.x; operators
|
||||
# often also see 192.168.17.x from VPN clients on macOS.
|
||||
# The util skips octet 17 → 18 so the smolmachines subnet
|
||||
# doesn't collide with that historical pain point.
|
||||
for slug in (f"slug-{i}" for i in range(500)):
|
||||
subnet, _, _ = smolmachines_bundle_subnet(slug)
|
||||
self.assertNotEqual("192.168.17.0/24", subnet,
|
||||
f"slug {slug!r} landed on the skipped octet")
|
||||
|
||||
|
||||
class TestPreflight(unittest.TestCase):
|
||||
def test_smolvm_present_returns_none(self):
|
||||
# Pin macOS so the Linux KVM gate doesn't fire on a CI runner
|
||||
# (ubuntu, no /dev/kvm) — this test isolates the PATH check.
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.util.shutil.which",
|
||||
return_value="/usr/local/bin/smolvm",
|
||||
), patch(
|
||||
"bot_bottle.backend.smolmachines.util.platform.system",
|
||||
return_value="Darwin",
|
||||
):
|
||||
self.assertIsNone(smolmachines_preflight())
|
||||
|
||||
def test_missing_smolvm_dies(self):
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.util.shutil.which",
|
||||
return_value=None,
|
||||
):
|
||||
with self.assertRaises(SystemExit) as cm:
|
||||
smolmachines_preflight()
|
||||
self.assertNotEqual(0, cm.exception.code)
|
||||
|
||||
def test_install_pointer_in_error(self):
|
||||
import io
|
||||
import sys
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.util.shutil.which",
|
||||
return_value=None,
|
||||
):
|
||||
captured = io.StringIO()
|
||||
with patch.object(sys, "stderr", captured):
|
||||
with self.assertRaises(SystemExit):
|
||||
smolmachines_preflight()
|
||||
msg = captured.getvalue()
|
||||
self.assertIn("smolvm", msg)
|
||||
self.assertIn("smolmachines.com/install.sh", msg)
|
||||
self.assertIn("BOT_BOTTLE_BACKEND=docker", msg)
|
||||
|
||||
|
||||
class TestKvmPreflight(unittest.TestCase):
|
||||
"""Linux-only KVM gate: smolvm needs /dev/kvm present and
|
||||
accessible. macOS skips this entirely (Hypervisor.framework)."""
|
||||
|
||||
def _run(self, *, system: str, exists: bool, access: bool) -> None:
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.util.shutil.which",
|
||||
return_value="/usr/bin/smolvm",
|
||||
), patch(
|
||||
"bot_bottle.backend.smolmachines.util.platform.system",
|
||||
return_value=system,
|
||||
), patch(
|
||||
"bot_bottle.backend.smolmachines.util.os.path.exists",
|
||||
return_value=exists,
|
||||
), patch(
|
||||
"bot_bottle.backend.smolmachines.util.os.access",
|
||||
return_value=access,
|
||||
):
|
||||
return smolmachines_preflight()
|
||||
|
||||
def test_macos_skips_kvm_check(self):
|
||||
# Even with /dev/kvm absent, macOS must not run the gate.
|
||||
self.assertIsNone(self._run(system="Darwin", exists=False, access=False))
|
||||
|
||||
def test_linux_ok_returns_none(self):
|
||||
self.assertIsNone(self._run(system="Linux", exists=True, access=True))
|
||||
|
||||
def test_linux_missing_device_dies(self):
|
||||
with self.assertRaises(SystemExit):
|
||||
self._run(system="Linux", exists=False, access=False)
|
||||
|
||||
def test_linux_no_access_dies(self):
|
||||
with self.assertRaises(SystemExit):
|
||||
self._run(system="Linux", exists=True, access=False)
|
||||
|
||||
def test_linux_missing_device_message(self):
|
||||
import io
|
||||
import sys
|
||||
captured = io.StringIO()
|
||||
with patch.object(sys, "stderr", captured):
|
||||
with self.assertRaises(SystemExit):
|
||||
self._run(system="Linux", exists=False, access=False)
|
||||
msg = captured.getvalue()
|
||||
self.assertIn("/dev/kvm", msg)
|
||||
self.assertIn("kvm-intel", msg)
|
||||
|
||||
def test_linux_no_access_message(self):
|
||||
import io
|
||||
import sys
|
||||
captured = io.StringIO()
|
||||
with patch.object(sys, "stderr", captured):
|
||||
with self.assertRaises(SystemExit):
|
||||
self._run(system="Linux", exists=True, access=False)
|
||||
msg = captured.getvalue()
|
||||
self.assertIn("kvm", msg)
|
||||
self.assertIn("group", msg)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in New Issue
Block a user