Compare commits
18 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| c3f55357cc | |||
| 36e62cd7e8 | |||
| 70e58f1333 | |||
| 3703b6f59f | |||
| afd943c4f9 | |||
| 8f31741c9f | |||
| 5bfe45bc1d | |||
| 7aca7d5289 | |||
| f6d4eb9e3c | |||
| c5b9f05467 | |||
| 09a2329f31 | |||
| 03c89dae81 | |||
| dd7232f368 | |||
| 76fdefded3 | |||
| 1085a2280d | |||
| c97f4ecb8d | |||
| 142974a4b8 | |||
| 041d9bbbf5 |
@@ -25,7 +25,7 @@
|
|||||||
- **Provider templates (Claude, Codex)** — `Dockerfile.claude` / `Dockerfile.codex`, or a bottle-supplied Dockerfile. Claude auth via long-lived OAuth token; Codex via opt-in host device-auth forwarding.
|
- **Provider templates (Claude, Codex)** — `Dockerfile.claude` / `Dockerfile.codex`, or a bottle-supplied Dockerfile. Claude auth via long-lived OAuth token; Codex via opt-in host device-auth forwarding.
|
||||||
- **gVisor auto-detect** — on Linux hosts where `runsc` is registered with Docker, every bottle launches under it for a userspace syscall barrier; no manifest config required.
|
- **gVisor auto-detect** — on Linux hosts where `runsc` is registered with Docker, every bottle launches under it for a userspace syscall barrier; no manifest config required.
|
||||||
- **Apple Container backend (macOS default when available)** — runs the agent and sidecar bundle with Apple's `container` CLI, using a host-only agent network plus a separate sidecar egress network.
|
- **Apple Container backend (macOS default when available)** — runs the agent and sidecar bundle with Apple's `container` CLI, using a host-only agent network plus a separate sidecar egress network.
|
||||||
- **Smolmachines backend** — runs the agent in a libkrun micro-VM while the sidecar bundle stays in Docker. TSI and smolmachines DNS filtering close the raw DNS exfiltration gap that exists in the legacy Docker backend.
|
- **Smolmachines backend** — runs the agent in a libkrun micro-VM while the sidecar bundle stays in Docker. TSI and smolmachines DNS filtering close the raw DNS exfiltration gap that exists in the legacy Docker backend. Runs on macOS (Hypervisor.framework) and Linux (KVM, `/dev/kvm`).
|
||||||
- **Legacy Docker backend** — still available for examples, CI, and hosts without Apple Container via `BOT_BOTTLE_BACKEND=docker` or `--backend=docker`.
|
- **Legacy Docker backend** — still available for examples, CI, and hosts without Apple Container via `BOT_BOTTLE_BACKEND=docker` or `--backend=docker`.
|
||||||
|
|
||||||
## Architecture
|
## Architecture
|
||||||
@@ -71,10 +71,26 @@ When the agent exits, `cli.py` tears down every sidecar and both networks; nothi
|
|||||||
|
|
||||||
## Quickstart
|
## Quickstart
|
||||||
|
|
||||||
On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The smolmachines backend requires Docker on the host for the sidecar bundle plus smolvm. The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
|
On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The smolmachines backend requires Docker on the host for the sidecar bundle plus `smolvm` (macOS or Linux). The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
|
||||||
|
|
||||||
Use `BOT_BOTTLE_BACKEND=docker ./cli.py start <agent>` on hosts where Apple Container is not installed and Docker is the desired backend.
|
Use `BOT_BOTTLE_BACKEND=docker ./cli.py start <agent>` on hosts where Apple Container is not installed and Docker is the desired backend.
|
||||||
|
|
||||||
|
### smolmachines on Linux
|
||||||
|
|
||||||
|
The smolmachines backend runs on Linux as well as macOS. On Linux, `smolvm`/libkrun use KVM, so the host needs:
|
||||||
|
|
||||||
|
- **`/dev/kvm`** present and accessible. Load `kvm-intel` or `kvm-amd` (and enable virtualization in BIOS/firmware). The invoking user must be in the `kvm` group: `sudo usermod -aG kvm "$USER"` then re-login. bot-bottle preflights this and reports exactly what's missing.
|
||||||
|
- **`smolvm`** on `PATH`: `curl -sSL https://smolmachines.com/install.sh | sh`.
|
||||||
|
- **Docker** for the sidecar bundle and image build, same as macOS.
|
||||||
|
|
||||||
|
Per-bottle isolation works the same as macOS without any `ifconfig`/sudo step — all of `127.0.0.0/8` is already loopback on Linux, so each bottle's sidecar bundle is published on its own `127.0.0.<N>` and TSI's allowlist is scoped to that `/32`.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
BOT_BOTTLE_BACKEND=smolmachines ./cli.py start <agent>
|
||||||
|
```
|
||||||
|
|
||||||
|
> **NixOS:** enable `virtualisation.docker`, ensure the KVM module is loaded (`boot.kernelModules = [ "kvm-intel" ];` or `kvm-amd`), and add your user to the `kvm` and `docker` groups. If you run bottles from a Gitea Actions runner, use a `host`-label runner so Docker, `smolvm`, and `/dev/kvm` are all reachable from the job. `smolvm` isn't in nixpkgs — install the release binary (pin the version) and put it on the runner's `PATH`.
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
./cli.py start <agent> # builds the image on first run, drops you into claude
|
./cli.py start <agent> # builds the image on first run, drops you into claude
|
||||||
```
|
```
|
||||||
|
|||||||
@@ -227,6 +227,10 @@ class AgentProvider(ABC):
|
|||||||
from .backend.util import AGENT_CA_PATH, log_ca_fingerprint, select_ca_cert
|
from .backend.util import AGENT_CA_PATH, log_ca_fingerprint, select_ca_cert
|
||||||
from .log import die
|
from .log import die
|
||||||
cert_host_path, label = select_ca_cert(plan.egress_plan)
|
cert_host_path, label = select_ca_cert(plan.egress_plan)
|
||||||
|
# Ensure the target directory exists. smolvm's pack step may not
|
||||||
|
# preserve the empty /usr/local/share/ca-certificates/ directory
|
||||||
|
# on Linux; mkdir -p is idempotent and safe for all backends.
|
||||||
|
bottle.exec("mkdir -p /usr/local/share/ca-certificates", user="root")
|
||||||
bottle.cp_in(str(cert_host_path), AGENT_CA_PATH)
|
bottle.cp_in(str(cert_host_path), AGENT_CA_PATH)
|
||||||
r = bottle.exec(
|
r = bottle.exec(
|
||||||
f"chmod 644 {AGENT_CA_PATH} && update-ca-certificates",
|
f"chmod 644 {AGENT_CA_PATH} && update-ca-certificates",
|
||||||
|
|||||||
@@ -15,10 +15,14 @@ except ImportError:
|
|||||||
from migrations import TableMigrations # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
|
from migrations import TableMigrations # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
|
||||||
|
|
||||||
|
|
||||||
# One entry per schema version: _MIGRATIONS.migrations[0] brings a fresh DB
|
class AuditStore(DbStore):
|
||||||
# to version 1, [1] to version 2, and so on. Add new migrations at the end;
|
"""SQLite-backed persistent store for supervise audit entries."""
|
||||||
# never edit existing ones.
|
|
||||||
_MIGRATIONS = TableMigrations("audit_store", [
|
def __init__(self, db_path: Path | None = None) -> None:
|
||||||
|
# One entry per schema version: migrations[0] brings a fresh DB to
|
||||||
|
# version 1, [1] to version 2, etc. Add new entries at the end; never
|
||||||
|
# edit existing ones.
|
||||||
|
migrations = TableMigrations("audit_store", [
|
||||||
# v1 — initial schema
|
# v1 — initial schema
|
||||||
"""
|
"""
|
||||||
CREATE TABLE IF NOT EXISTS supervise_audit_entries (
|
CREATE TABLE IF NOT EXISTS supervise_audit_entries (
|
||||||
@@ -33,13 +37,7 @@ _MIGRATIONS = TableMigrations("audit_store", [
|
|||||||
)
|
)
|
||||||
""",
|
""",
|
||||||
])
|
])
|
||||||
|
super().__init__(db_path or host_db_path(), migrations)
|
||||||
|
|
||||||
class AuditStore(DbStore):
|
|
||||||
"""SQLite-backed persistent store for supervise audit entries."""
|
|
||||||
|
|
||||||
def __init__(self, db_path: Path | None = None) -> None:
|
|
||||||
super().__init__(db_path or host_db_path(), _MIGRATIONS)
|
|
||||||
|
|
||||||
def write_audit_entry(self, entry: AuditEntry) -> Path:
|
def write_audit_entry(self, entry: AuditEntry) -> Path:
|
||||||
with self._connect() as conn:
|
with self._connect() as conn:
|
||||||
|
|||||||
@@ -405,7 +405,15 @@ def _sidecar_mounts(
|
|||||||
|
|
||||||
sp = plan.supervise_plan
|
sp = plan.supervise_plan
|
||||||
if sp is not None:
|
if sp is not None:
|
||||||
mounts.append((str(sp.db_path), DB_PATH_IN_CONTAINER, False))
|
# `container run --mount type=bind` only accepts directory
|
||||||
|
# sources (a file source fails with "is not a directory") —
|
||||||
|
# mount db_path's dedicated parent dir instead of the file
|
||||||
|
# itself, same as the CA/routes mounts above.
|
||||||
|
mounts.append((
|
||||||
|
str(sp.db_path.parent),
|
||||||
|
str(Path(DB_PATH_IN_CONTAINER).parent),
|
||||||
|
False,
|
||||||
|
))
|
||||||
|
|
||||||
return tuple(mounts)
|
return tuple(mounts)
|
||||||
|
|
||||||
|
|||||||
@@ -47,10 +47,24 @@ _HOME_FOR = {
|
|||||||
"root": "/root",
|
"root": "/root",
|
||||||
}
|
}
|
||||||
|
|
||||||
|
_DEFAULT_PATH_FOR = {
|
||||||
|
# Committed smolmachine snapshots are rebuilt from a rootfs tarball and
|
||||||
|
# lose Docker image ENV metadata. Restore the provider CLI path here so
|
||||||
|
# resumed Codex bottles can still find the per-user install.
|
||||||
|
"node": (
|
||||||
|
"/home/node/.local/bin:"
|
||||||
|
"/home/node/.codex/packages/standalone/current/bin:"
|
||||||
|
"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
|
||||||
|
),
|
||||||
|
"root": "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
def _env_assignments_for(user: str, env: Mapping[str, str]) -> list[str]:
|
def _env_assignments_for(user: str, env: Mapping[str, str]) -> list[str]:
|
||||||
home = _HOME_FOR.get(user, f"/home/{user}")
|
home = _HOME_FOR.get(user, f"/home/{user}")
|
||||||
out = [f"HOME={home}", f"USER={user}"]
|
out = [f"HOME={home}", f"USER={user}"]
|
||||||
|
if "PATH" not in env:
|
||||||
|
out.append(f"PATH={_DEFAULT_PATH_FOR.get(user, _DEFAULT_PATH_FOR['root'])}")
|
||||||
for k, v in env.items():
|
for k, v in env.items():
|
||||||
out.append(f"{k}={v}")
|
out.append(f"{k}={v}")
|
||||||
return out
|
return out
|
||||||
|
|||||||
@@ -17,6 +17,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import dataclasses
|
import dataclasses
|
||||||
import os
|
import os
|
||||||
|
import platform
|
||||||
from contextlib import ExitStack, contextmanager
|
from contextlib import ExitStack, contextmanager
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Callable, Generator
|
from typing import Callable, Generator
|
||||||
@@ -91,12 +92,13 @@ def launch(
|
|||||||
try:
|
try:
|
||||||
loopback_ip, network = _allocate_resources(plan, stack)
|
loopback_ip, network = _allocate_resources(plan, stack)
|
||||||
plan = _mint_certs(plan)
|
plan = _mint_certs(plan)
|
||||||
plan = _start_bundle(plan, network, loopback_ip, stack)
|
proxy_host = _proxy_host(plan, loopback_ip)
|
||||||
plan = _discover_urls(plan, loopback_ip)
|
plan = _start_bundle(plan, network, proxy_host, stack)
|
||||||
|
plan = _discover_urls(plan, proxy_host)
|
||||||
|
|
||||||
agent_from_path = _agent_from_path(plan)
|
agent_from_path = _agent_from_path(plan)
|
||||||
|
|
||||||
_launch_vm(plan, agent_from_path, loopback_ip, stack)
|
_launch_vm(plan, agent_from_path, proxy_host, stack)
|
||||||
_init_vm(plan)
|
_init_vm(plan)
|
||||||
|
|
||||||
bottle = SmolmachinesBottle(
|
bottle = SmolmachinesBottle(
|
||||||
@@ -144,10 +146,12 @@ def _allocate_resources(
|
|||||||
) -> tuple[str, str]:
|
) -> tuple[str, str]:
|
||||||
"""Reserve a loopback alias and create the per-bottle docker bridge.
|
"""Reserve a loopback alias and create the per-bottle docker bridge.
|
||||||
|
|
||||||
macOS only routes 127.0.0.1 by default; the per-bottle alias
|
The per-bottle alias scopes TSI's allowlist to this bottle's
|
||||||
scopes TSI's allowlist to this bottle's published ports so the
|
published ports so the agent can't reach other bottles' or host
|
||||||
agent can't reach other bottles' or host services' ports on
|
services' ports on loopback. On macOS `ensure_pool` first
|
||||||
loopback. No-op on Linux."""
|
sudo-aliases the pool on `lo0`; on Linux that's a no-op since
|
||||||
|
all of 127.0.0.0/8 is already loopback, but the per-bottle
|
||||||
|
allocation runs on both."""
|
||||||
_loopback.ensure_pool()
|
_loopback.ensure_pool()
|
||||||
loopback_ip = _loopback.allocate(plan.slug)
|
loopback_ip = _loopback.allocate(plan.slug)
|
||||||
network = _bundle.bundle_network_name(plan.slug)
|
network = _bundle.bundle_network_name(plan.slug)
|
||||||
@@ -172,13 +176,13 @@ def _mint_certs(plan: SmolmachinesBottlePlan) -> SmolmachinesBottlePlan:
|
|||||||
def _start_bundle(
|
def _start_bundle(
|
||||||
plan: SmolmachinesBottlePlan,
|
plan: SmolmachinesBottlePlan,
|
||||||
network: str,
|
network: str,
|
||||||
loopback_ip: str,
|
proxy_host: str,
|
||||||
stack: ExitStack,
|
stack: ExitStack,
|
||||||
) -> SmolmachinesBottlePlan:
|
) -> SmolmachinesBottlePlan:
|
||||||
"""Build the BundleLaunchSpec, resolve token env, start the
|
"""Build the BundleLaunchSpec, resolve token env, start the
|
||||||
sidecar bundle container, and register teardown."""
|
sidecar bundle container, and register teardown."""
|
||||||
plan = _provision_git_gate_keys(plan)
|
plan = _provision_git_gate_keys(plan)
|
||||||
bundle_spec = _bundle_launch_spec(plan, network, loopback_ip)
|
bundle_spec = _bundle_launch_spec(plan, network, proxy_host)
|
||||||
token_env = _resolve_token_env(plan, dict(os.environ))
|
token_env = _resolve_token_env(plan, dict(os.environ))
|
||||||
_bundle.ensure_bundle_image(bundle_spec.image)
|
_bundle.ensure_bundle_image(bundle_spec.image)
|
||||||
_bundle.start_bundle(bundle_spec, env={**os.environ, **token_env})
|
_bundle.start_bundle(bundle_spec, env={**os.environ, **token_env})
|
||||||
@@ -201,39 +205,40 @@ def _provision_git_gate_keys(
|
|||||||
|
|
||||||
def _discover_urls(
|
def _discover_urls(
|
||||||
plan: SmolmachinesBottlePlan,
|
plan: SmolmachinesBottlePlan,
|
||||||
loopback_ip: str,
|
proxy_host: str,
|
||||||
) -> SmolmachinesBottlePlan:
|
) -> SmolmachinesBottlePlan:
|
||||||
"""Discover host-side ports for published container ports and
|
"""Discover host-side ports for published container ports and
|
||||||
return the plan with URLs + guest_env stamped in.
|
return the plan with URLs + guest_env stamped in.
|
||||||
|
|
||||||
Docker container IPs (192.168.x.x in the daemon's bridge)
|
`proxy_host` is the host IP that both TSI's allowlist and
|
||||||
aren't reachable from the smolvm guest on macOS — TSI uses
|
docker's port-forward bindings are keyed to. On macOS it is the
|
||||||
macOS networking, and macOS sees the daemon's bridge via the
|
per-bottle loopback alias; on Linux it is the per-bottle bridge
|
||||||
published-port loopback forward only.
|
gateway (see `_proxy_host`). The agent dials the published port
|
||||||
|
on this IP for all bundle-hosted services.
|
||||||
|
|
||||||
NO_PROXY includes the per-bottle loopback alias so the
|
NO_PROXY includes `proxy_host` so supervise + git-gate URLs
|
||||||
supervise + git-gate URLs bypass HTTPS_PROXY."""
|
bypass HTTPS_PROXY."""
|
||||||
agent_facing_host_port = _bundle.bundle_host_port(
|
agent_facing_host_port = _bundle.bundle_host_port(
|
||||||
plan.slug, _EGRESS_PORT, host_ip=loopback_ip,
|
plan.slug, _EGRESS_PORT, host_ip=proxy_host,
|
||||||
)
|
)
|
||||||
agent_proxy_url = f"http://{loopback_ip}:{agent_facing_host_port}"
|
agent_proxy_url = f"http://{proxy_host}:{agent_facing_host_port}"
|
||||||
|
|
||||||
agent_git_gate_host = ""
|
agent_git_gate_host = ""
|
||||||
if plan.git_gate_plan.upstreams:
|
if plan.git_gate_plan.upstreams:
|
||||||
git_gate_host_port = _bundle.bundle_host_port(
|
git_gate_host_port = _bundle.bundle_host_port(
|
||||||
plan.slug, _GIT_HTTP_PORT, host_ip=loopback_ip,
|
plan.slug, _GIT_HTTP_PORT, host_ip=proxy_host,
|
||||||
)
|
)
|
||||||
agent_git_gate_host = f"{loopback_ip}:{git_gate_host_port}"
|
agent_git_gate_host = f"{proxy_host}:{git_gate_host_port}"
|
||||||
|
|
||||||
agent_supervise_url = ""
|
agent_supervise_url = ""
|
||||||
if plan.supervise_plan is not None:
|
if plan.supervise_plan is not None:
|
||||||
supervise_host_port = _bundle.bundle_host_port(
|
supervise_host_port = _bundle.bundle_host_port(
|
||||||
plan.slug, _SUPERVISE_PORT, host_ip=loopback_ip,
|
plan.slug, _SUPERVISE_PORT, host_ip=proxy_host,
|
||||||
)
|
)
|
||||||
agent_supervise_url = f"http://{loopback_ip}:{supervise_host_port}/"
|
agent_supervise_url = f"http://{proxy_host}:{supervise_host_port}/"
|
||||||
|
|
||||||
existing_no_proxy = plan.guest_env.get("NO_PROXY", "localhost,127.0.0.1")
|
existing_no_proxy = plan.guest_env.get("NO_PROXY", "localhost,127.0.0.1")
|
||||||
no_proxy = f"{existing_no_proxy},{loopback_ip}"
|
no_proxy = f"{existing_no_proxy},{proxy_host}"
|
||||||
guest_env = {
|
guest_env = {
|
||||||
**plan.guest_env,
|
**plan.guest_env,
|
||||||
"HTTPS_PROXY": agent_proxy_url,
|
"HTTPS_PROXY": agent_proxy_url,
|
||||||
@@ -263,27 +268,32 @@ def _discover_urls(
|
|||||||
def _launch_vm(
|
def _launch_vm(
|
||||||
plan: SmolmachinesBottlePlan,
|
plan: SmolmachinesBottlePlan,
|
||||||
agent_from_path: Path,
|
agent_from_path: Path,
|
||||||
loopback_ip: str,
|
proxy_host: str,
|
||||||
stack: ExitStack,
|
stack: ExitStack,
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Create, patch, and start the smolvm VM; register teardown.
|
"""Create, patch, and start the smolvm VM; register teardown.
|
||||||
|
|
||||||
--allow-cidr is the per-bottle loopback alias so the guest can
|
--allow-cidr is `proxy_host/32` — the per-bottle loopback alias
|
||||||
only reach this bottle's bundle ports. force_allowlist patches
|
on macOS or the bridge gateway on Linux (see `_proxy_host`). This
|
||||||
smolvm 0.8.0's silent-drop of --allow-cidr when combined with
|
ensures the guest can only reach bundle ports published on that IP,
|
||||||
--from. Smolfile isn't usable here — smolvm 0.8.0 makes --from
|
not the container IP directly. force_allowlist confirms the
|
||||||
|
allowlist persisted (patching smolvm 0.8.0's silent-drop of
|
||||||
|
--allow-cidr when combined with --from) and fails closed if it
|
||||||
|
can't. Smolfile isn't usable here — smolvm 0.8.0 makes --from
|
||||||
and --smolfile mutually exclusive."""
|
and --smolfile mutually exclusive."""
|
||||||
|
tsi_cidr = f"{proxy_host}/32"
|
||||||
_smolvm.machine_create(
|
_smolvm.machine_create(
|
||||||
plan.machine_name,
|
plan.machine_name,
|
||||||
from_path=agent_from_path,
|
from_path=agent_from_path,
|
||||||
allow_cidrs=[f"{loopback_ip}/32"],
|
allow_cidrs=[tsi_cidr],
|
||||||
env=plan.guest_env,
|
env=plan.guest_env,
|
||||||
)
|
)
|
||||||
stack.callback(_smolvm.machine_delete, plan.machine_name)
|
stack.callback(_smolvm.machine_delete, plan.machine_name)
|
||||||
# Workaround smolvm 0.8.0: `--allow-cidr` is silently dropped
|
# Confirm the booted VM's TSI allowlist will actually enforce the
|
||||||
# when combined with `--from`. Patch the persisted state DB
|
# /32 before start (smolvm 0.8.0 silently drops `--allow-cidr`
|
||||||
# before start so the booted VM's TSI actually enforces.
|
# with `--from`, so the persisted state DB is patched if needed).
|
||||||
_loopback.force_allowlist(plan.machine_name, [f"{loopback_ip}/32"])
|
# Fails closed if enforcement can't be confirmed.
|
||||||
|
_loopback.force_allowlist(plan.machine_name, [tsi_cidr])
|
||||||
_smolvm.machine_start(plan.machine_name)
|
_smolvm.machine_start(plan.machine_name)
|
||||||
stack.callback(_smolvm.machine_stop, plan.machine_name)
|
stack.callback(_smolvm.machine_stop, plan.machine_name)
|
||||||
|
|
||||||
@@ -292,7 +302,9 @@ def _init_vm(plan: SmolmachinesBottlePlan) -> None:
|
|||||||
"""Repair filesystem ownership and wait for exec channel readiness.
|
"""Repair filesystem ownership and wait for exec channel readiness.
|
||||||
|
|
||||||
Ownership repair: smolvm's pack process remaps files to the host
|
Ownership repair: smolvm's pack process remaps files to the host
|
||||||
invoker's uid (501 on macOS). /home/node must be node:node so
|
invoker's uid (e.g. 501 on macOS, 1000 on Linux). The chowns use
|
||||||
|
names not numbers so they're correct on either. /home/node must
|
||||||
|
be node:node so
|
||||||
Claude Code can write ~/.claude.json; /tmp + /var/tmp need root
|
Claude Code can write ~/.claude.json; /tmp + /var/tmp need root
|
||||||
mode 1777 so non-root processes can create per-uid scratch dirs.
|
mode 1777 so non-root processes can create per-uid scratch dirs.
|
||||||
All folded into one sh -c to avoid back-to-back exec calls
|
All folded into one sh -c to avoid back-to-back exec calls
|
||||||
@@ -315,8 +327,29 @@ def _init_vm(plan: SmolmachinesBottlePlan) -> None:
|
|||||||
_smolvm.wait_exec_ready(plan.machine_name)
|
_smolvm.wait_exec_ready(plan.machine_name)
|
||||||
|
|
||||||
|
|
||||||
|
def _proxy_host(plan: SmolmachinesBottlePlan, loopback_ip: str) -> str:
|
||||||
|
"""Return the host IP for TSI's allowlist and docker port-forward bindings.
|
||||||
|
|
||||||
|
On macOS, the per-bottle loopback alias (e.g. ``127.0.0.16``) works
|
||||||
|
because macOS's network stack lets TSI intercept 127.x.x.x connects
|
||||||
|
from the guest before they reach the host's own loopback.
|
||||||
|
|
||||||
|
On Linux, the guest kernel's LOCAL routing table routes all
|
||||||
|
``127.0.0.0/8`` to the guest's own loopback — those packets never
|
||||||
|
reach eth0 and TSI never sees them. Using the per-bottle bridge
|
||||||
|
gateway (e.g. ``192.168.N.1``) instead sidesteps the problem: it
|
||||||
|
is not a loopback address, so the guest routes it via eth0 and TSI
|
||||||
|
intercepts it normally. The TSI allowlist is ``gateway/32``, which
|
||||||
|
is distinct from the container IP (``192.168.N.2``), so the agent
|
||||||
|
still can't reach the egress daemon directly — TSI blocks any
|
||||||
|
connection to the container IP that isn't via the published port."""
|
||||||
|
if platform.system() == "Linux":
|
||||||
|
return plan.bundle_gateway
|
||||||
|
return loopback_ip
|
||||||
|
|
||||||
|
|
||||||
def _bundle_launch_spec(
|
def _bundle_launch_spec(
|
||||||
plan: SmolmachinesBottlePlan, network: str, loopback_ip: str,
|
plan: SmolmachinesBottlePlan, network: str, proxy_host: str,
|
||||||
) -> _bundle.BundleLaunchSpec:
|
) -> _bundle.BundleLaunchSpec:
|
||||||
"""Build a BundleLaunchSpec from the resolved inner Plans.
|
"""Build a BundleLaunchSpec from the resolved inner Plans.
|
||||||
|
|
||||||
@@ -375,8 +408,9 @@ def _bundle_launch_spec(
|
|||||||
volumes.append((str(sp.db_path), DB_PATH_IN_CONTAINER, False))
|
volumes.append((str(sp.db_path), DB_PATH_IN_CONTAINER, False))
|
||||||
|
|
||||||
# Container ports the agent reaches from the smolvm guest —
|
# Container ports the agent reaches from the smolvm guest —
|
||||||
# published on host loopback so the guest can dial via TSI +
|
# published on `proxy_host` so the TSI allowlist and the docker
|
||||||
# macOS networking. Egress is always the agent's HTTP/HTTPS proxy.
|
# port-forward bindings point at the same IP. Egress is always
|
||||||
|
# the agent's HTTP/HTTPS proxy.
|
||||||
ports_to_publish: list[int] = [_EGRESS_PORT]
|
ports_to_publish: list[int] = [_EGRESS_PORT]
|
||||||
if gp.upstreams:
|
if gp.upstreams:
|
||||||
ports_to_publish.append(_GIT_HTTP_PORT)
|
ports_to_publish.append(_GIT_HTTP_PORT)
|
||||||
@@ -393,7 +427,7 @@ def _bundle_launch_spec(
|
|||||||
environment=tuple(env),
|
environment=tuple(env),
|
||||||
volumes=tuple(volumes),
|
volumes=tuple(volumes),
|
||||||
ports_to_publish=tuple(ports_to_publish),
|
ports_to_publish=tuple(ports_to_publish),
|
||||||
publish_host_ip=loopback_ip,
|
publish_host_ip=proxy_host,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -467,6 +501,12 @@ def _ensure_smolmachine(image_ref: str, *, dockerfile: str = "") -> Path:
|
|||||||
return sidecar
|
return sidecar
|
||||||
tarball = _SMOLMACHINE_CACHE_DIR / f"{digest}.image.tar"
|
tarball = _SMOLMACHINE_CACHE_DIR / f"{digest}.image.tar"
|
||||||
docker_mod.save(image_ref, str(tarball))
|
docker_mod.save(image_ref, str(tarball))
|
||||||
|
# On Linux, `docker save -o` writes the tarball with owner-only
|
||||||
|
# permissions (mode 600). The crane push container runs as UID
|
||||||
|
# 65532 (distroless nonroot) and can't read it through a bind
|
||||||
|
# mount unless world-read is set. The tarball is temporary and
|
||||||
|
# lives in ~/.cache, so 644 is safe.
|
||||||
|
tarball.chmod(0o644)
|
||||||
try:
|
try:
|
||||||
with ephemeral_registry() as handle:
|
with ephemeral_registry() as handle:
|
||||||
push_ref = f"{handle.push_endpoint}/bot-bottle:{digest}"
|
push_ref = f"{handle.push_endpoint}/bot-bottle:{digest}"
|
||||||
|
|||||||
@@ -33,10 +33,13 @@ sudo-add the missing pool on first use per boot — the aliases
|
|||||||
persist on `lo0` until reboot, so subsequent launches don't
|
persist on `lo0` until reboot, so subsequent launches don't
|
||||||
prompt.
|
prompt.
|
||||||
|
|
||||||
Linux native daemons share the host's network namespace; the
|
On Linux the whole `127.0.0.0/8` is already routed to `lo`, so
|
||||||
whole `127.0.0.0/8` is reachable by default and aliases are
|
docker can publish a bundle's ports directly on `127.0.0.<N>`
|
||||||
unnecessary. The pool logic detects native-Linux and skips sudo
|
with no `ifconfig`/sudo step. `ensure_pool` is therefore a no-op
|
||||||
entirely; the DB patch is also gated on macOS.
|
on Linux, but per-bottle alias *allocation* and the TSI allowlist
|
||||||
|
DB patch run on both platforms — the isolation property is
|
||||||
|
identical, it's just cheaper to set up on Linux. The state-DB
|
||||||
|
path differs per platform (see `_smolvm_db_path`).
|
||||||
|
|
||||||
Allocation is coordinated by inspecting running bundle
|
Allocation is coordinated by inspecting running bundle
|
||||||
containers' published host IPs — each bottle's bundle owns the
|
containers' published host IPs — each bottle's bundle owns the
|
||||||
@@ -47,6 +50,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import fcntl
|
import fcntl
|
||||||
import json
|
import json
|
||||||
|
import os
|
||||||
import platform
|
import platform
|
||||||
import re
|
import re
|
||||||
import sqlite3
|
import sqlite3
|
||||||
@@ -57,13 +61,19 @@ from typing import Iterable
|
|||||||
from ...log import die, info
|
from ...log import die, info
|
||||||
|
|
||||||
|
|
||||||
# smolvm's persistent VM state on macOS — a SQLite DB whose `vms`
|
def _smolvm_db_path() -> Path:
|
||||||
# table holds one JSON BLOB per machine. The Linux path is
|
"""smolvm's persistent VM state — a SQLite DB whose `vms` table
|
||||||
# different, but smolmachines is macOS-only in v1 (PRD 0023) so
|
holds one JSON BLOB per machine. macOS stores it under
|
||||||
# we hard-code this. If the file moves under us we'll see a
|
`Application Support`; Linux follows the XDG base-dir spec
|
||||||
# clear FileNotFoundError; not worth defensive cross-platform
|
(`$XDG_DATA_HOME`, default `~/.local/share`).
|
||||||
# detection until the backend actually needs Linux.
|
|
||||||
_SMOLVM_DB_PATH = (
|
NOTE: the Linux location is inferred from smolvm's documented
|
||||||
|
`~/.local/share` install layout and must be confirmed against a
|
||||||
|
real Linux smolvm install. If it's wrong, `force_allowlist`'s
|
||||||
|
fail-closed check turns it into a clear launch-time error rather
|
||||||
|
than a silent escape."""
|
||||||
|
if platform.system() == "Darwin":
|
||||||
|
return (
|
||||||
Path.home()
|
Path.home()
|
||||||
/ "Library"
|
/ "Library"
|
||||||
/ "Application Support"
|
/ "Application Support"
|
||||||
@@ -71,6 +81,14 @@ _SMOLVM_DB_PATH = (
|
|||||||
/ "server"
|
/ "server"
|
||||||
/ "smolvm.db"
|
/ "smolvm.db"
|
||||||
)
|
)
|
||||||
|
xdg_data = os.environ.get("XDG_DATA_HOME")
|
||||||
|
base = Path(xdg_data) if xdg_data else Path.home() / ".local" / "share"
|
||||||
|
return base / "smolvm" / "server" / "smolvm.db"
|
||||||
|
|
||||||
|
|
||||||
|
# Resolved once at import: the host platform doesn't change within a
|
||||||
|
# process. Tests patch this attribute directly.
|
||||||
|
_SMOLVM_DB_PATH = _smolvm_db_path()
|
||||||
|
|
||||||
|
|
||||||
# Sixteen aliases by default. Tunable for hosts that want more
|
# Sixteen aliases by default. Tunable for hosts that want more
|
||||||
@@ -131,30 +149,64 @@ def ensure_pool() -> None:
|
|||||||
|
|
||||||
|
|
||||||
def force_allowlist(machine_name: str, allowed_cidrs: list[str]) -> None:
|
def force_allowlist(machine_name: str, allowed_cidrs: list[str]) -> None:
|
||||||
"""Patch smolvm's persistent VM-state DB to set the machine's
|
"""Ensure the machine's persisted TSI allowlist equals
|
||||||
`allowed_cidrs` to the given list. Workaround for smolvm
|
`allowed_cidrs`, failing **closed** if that can't be confirmed.
|
||||||
0.8.0's silent-drop of `--allow-cidr` when used with `--from`.
|
|
||||||
|
|
||||||
Must run AFTER `smolvm machine create` (the row has to
|
Runs on both macOS and Linux. It exists because smolvm 0.8.0
|
||||||
exist) and BEFORE `smolvm machine start` (smolvm reads the
|
silently drops `--allow-cidr` when combined with `--from`, so
|
||||||
row on start; in-flight VMs don't pick up changes). Once
|
the allowlist has to be written into smolvm's persistent state
|
||||||
smolvm honors the CLI flag upstream this whole function is
|
DB before `machine start`. Rather than assume the flag was
|
||||||
redundant — flag-respecting create + remove this call from
|
dropped, we read the persisted row and only patch when it
|
||||||
launch.
|
doesn't already match — so a newer smolvm that honors the flag
|
||||||
|
is left untouched.
|
||||||
|
|
||||||
No-op on non-macOS — the DB path differs and the Linux
|
Must run AFTER `smolvm machine create` (the row has to exist)
|
||||||
smolmachines code path isn't exercised in v1."""
|
and BEFORE `smolvm machine start` (smolvm reads the row on
|
||||||
if not _is_macos():
|
start; in-flight VMs don't pick up changes).
|
||||||
return
|
|
||||||
|
Fail-closed: if the state DB is missing, the row is missing, or
|
||||||
|
the allowlist still doesn't match after patching, we `die()`
|
||||||
|
rather than boot a VM whose egress confinement we can't verify
|
||||||
|
— an unconfirmed allowlist is a sandbox-escape risk (the agent
|
||||||
|
VM could reach all of host loopback)."""
|
||||||
|
want = list(allowed_cidrs)
|
||||||
if not _SMOLVM_DB_PATH.is_file():
|
if not _SMOLVM_DB_PATH.is_file():
|
||||||
die(
|
die(
|
||||||
f"smolvm state DB not found at {_SMOLVM_DB_PATH}. "
|
f"smolvm state DB not found at {_SMOLVM_DB_PATH}; cannot "
|
||||||
f"smolvm 0.8.0 expected? `smolvm --version` to check."
|
f"confirm the TSI allowlist is enforced. Refusing to launch "
|
||||||
|
f"(fail-closed). Check `smolvm --version` and the DB "
|
||||||
|
f"location for your platform."
|
||||||
)
|
)
|
||||||
con = sqlite3.connect(str(_SMOLVM_DB_PATH))
|
con = sqlite3.connect(str(_SMOLVM_DB_PATH))
|
||||||
try:
|
try:
|
||||||
cur = con.cursor()
|
cfg = _read_machine_cfg(con, machine_name)
|
||||||
row = cur.execute(
|
if cfg.get("allowed_cidrs") != want:
|
||||||
|
cfg["allowed_cidrs"] = want
|
||||||
|
# Write as BLOB (the column type smolvm uses) — passing a
|
||||||
|
# plain str makes sqlite store it as Text and smolvm then
|
||||||
|
# fails to read it.
|
||||||
|
con.execute(
|
||||||
|
"UPDATE vms SET data = ? WHERE name = ?",
|
||||||
|
(sqlite3.Binary(json.dumps(cfg).encode()), machine_name),
|
||||||
|
)
|
||||||
|
con.commit()
|
||||||
|
cfg = _read_machine_cfg(con, machine_name)
|
||||||
|
if cfg.get("allowed_cidrs") != want:
|
||||||
|
die(
|
||||||
|
f"could not enforce TSI allowlist {want!r} for machine "
|
||||||
|
f"{machine_name!r} (persisted value is "
|
||||||
|
f"{cfg.get('allowed_cidrs')!r}). Refusing to launch "
|
||||||
|
f"(fail-closed)."
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
con.close()
|
||||||
|
|
||||||
|
|
||||||
|
def _read_machine_cfg(con: sqlite3.Connection, machine_name: str) -> dict[str, object]:
|
||||||
|
"""Read + JSON-decode a machine's `data` BLOB from the smolvm
|
||||||
|
state DB. Dies (fail-closed) if the row is missing — the caller
|
||||||
|
can't confirm enforcement without it."""
|
||||||
|
row = con.execute(
|
||||||
"SELECT data FROM vms WHERE name = ?", (machine_name,),
|
"SELECT data FROM vms WHERE name = ?", (machine_name,),
|
||||||
).fetchone()
|
).fetchone()
|
||||||
if row is None:
|
if row is None:
|
||||||
@@ -162,18 +214,7 @@ def force_allowlist(machine_name: str, allowed_cidrs: list[str]) -> None:
|
|||||||
f"smolvm DB has no row for machine {machine_name!r} — "
|
f"smolvm DB has no row for machine {machine_name!r} — "
|
||||||
f"machine_create must run before force_allowlist."
|
f"machine_create must run before force_allowlist."
|
||||||
)
|
)
|
||||||
cfg = json.loads(row[0])
|
return json.loads(row[0])
|
||||||
cfg["allowed_cidrs"] = list(allowed_cidrs)
|
|
||||||
# Write as BLOB (the column type smolvm uses) — passing a
|
|
||||||
# plain str makes sqlite store it as Text and smolvm then
|
|
||||||
# fails to read it.
|
|
||||||
cur.execute(
|
|
||||||
"UPDATE vms SET data = ? WHERE name = ?",
|
|
||||||
(sqlite3.Binary(json.dumps(cfg).encode()), machine_name),
|
|
||||||
)
|
|
||||||
con.commit()
|
|
||||||
finally:
|
|
||||||
con.close()
|
|
||||||
|
|
||||||
|
|
||||||
def allocate(_slug: str) -> str:
|
def allocate(_slug: str) -> str:
|
||||||
@@ -184,16 +225,17 @@ def allocate(_slug: str) -> str:
|
|||||||
used (no on-disk reservation, allocation is purely
|
used (no on-disk reservation, allocation is purely
|
||||||
docker-state-driven).
|
docker-state-driven).
|
||||||
|
|
||||||
On non-macOS the whole `127.0.0.0/8` is loopback by default;
|
Runs on both platforms: the allocation logic (docker-state
|
||||||
`127.0.0.1` is fine to share and we skip the alias dance.
|
inspection + the file lock) is platform-independent. macOS
|
||||||
This still returns a deterministic address so launch.py's
|
needs `ensure_pool` to have aliased the addresses on `lo0`
|
||||||
callers don't have to branch on platform.
|
first; on Linux all of `127.0.0.0/8` is already loopback, so
|
||||||
|
docker can publish on the chosen `127.0.0.<N>` with no setup.
|
||||||
|
Per-bottle scoping (so the agent can't reach other bottles' or
|
||||||
|
host services' loopback ports) therefore holds on both.
|
||||||
|
|
||||||
An exclusive file lock serialises concurrent calls so two
|
An exclusive file lock serialises concurrent calls so two
|
||||||
simultaneous launches don't read the same docker state and
|
simultaneous launches don't read the same docker state and
|
||||||
claim the same alias."""
|
claim the same alias."""
|
||||||
if not _is_macos():
|
|
||||||
return "127.0.0.1"
|
|
||||||
_ALLOC_LOCK_PATH.parent.mkdir(parents=True, exist_ok=True)
|
_ALLOC_LOCK_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||||
with open(_ALLOC_LOCK_PATH, "w", encoding="utf-8") as lf:
|
with open(_ALLOC_LOCK_PATH, "w", encoding="utf-8") as lf:
|
||||||
fcntl.flock(lf, fcntl.LOCK_EX)
|
fcntl.flock(lf, fcntl.LOCK_EX)
|
||||||
|
|||||||
@@ -116,10 +116,9 @@ def machine_create(
|
|||||||
allow_cidrs: Sequence[str] = (),
|
allow_cidrs: Sequence[str] = (),
|
||||||
env: Mapping[str, str] | None = None,
|
env: Mapping[str, str] | None = None,
|
||||||
) -> None:
|
) -> None:
|
||||||
"""`smolvm machine create NAME [--image IMG | --from PATH]
|
"""`smolvm machine create --name NAME [--image IMG | --from PATH]
|
||||||
[--allow-cidr CIDR ...] [-e K=V ...]`. NAME is positional
|
[--allow-cidr CIDR ...] [-e K=V ...]`. NAME is passed as
|
||||||
(the CLI's exception to the `--name` pattern other
|
`--name` (smolvm 1.4.7+; earlier versions took it positionally).
|
||||||
subcommands use).
|
|
||||||
|
|
||||||
`image` (registry ref like `alpine:latest`) and `from_path`
|
`image` (registry ref like `alpine:latest`) and `from_path`
|
||||||
(a `.smolmachine` artifact) are mutually exclusive — one or
|
(a `.smolmachine` artifact) are mutually exclusive — one or
|
||||||
@@ -133,12 +132,10 @@ def machine_create(
|
|||||||
result without the Smolfile complication.
|
result without the Smolfile complication.
|
||||||
|
|
||||||
`--net` is sent explicitly when `allow_cidrs` is non-empty.
|
`--net` is sent explicitly when `allow_cidrs` is non-empty.
|
||||||
smolvm 0.8.0's docs say `--allow-cidr` implies `--net`, but
|
`--allow-cidr` implies `--net` per the CLI help, but sending
|
||||||
empirically the implication only fires when no `--from` is
|
`--net` explicitly is harmless and ensures the guest has
|
||||||
set — `--from PATH --allow-cidr X/32` silently produces a
|
network access even if that implication changes across versions."""
|
||||||
machine with `network: false` and no routes in the guest, so
|
args: list[str] = ["machine", "create", "--name", name]
|
||||||
the agent can't reach the bundle's pinned IP."""
|
|
||||||
args: list[str] = ["machine", "create"]
|
|
||||||
if image is not None:
|
if image is not None:
|
||||||
args += ["--image", image]
|
args += ["--image", image]
|
||||||
if from_path is not None:
|
if from_path is not None:
|
||||||
@@ -150,7 +147,6 @@ def machine_create(
|
|||||||
if env:
|
if env:
|
||||||
for k, v in env.items():
|
for k, v in env.items():
|
||||||
args += ["-e", f"{k}={v}"]
|
args += ["-e", f"{k}={v}"]
|
||||||
args.append(name)
|
|
||||||
_smolvm(*args)
|
_smolvm(*args)
|
||||||
|
|
||||||
|
|
||||||
@@ -182,10 +178,9 @@ def machine_stop(name: str) -> None:
|
|||||||
|
|
||||||
|
|
||||||
def machine_delete(name: str) -> None:
|
def machine_delete(name: str) -> None:
|
||||||
"""`smolvm machine delete -f NAME`. NAME is positional. `-f`
|
"""`smolvm machine delete --name NAME -f`. `-f` skips the
|
||||||
skips the interactive confirmation — required for
|
interactive confirmation — required for non-interactive teardown."""
|
||||||
non-interactive teardown."""
|
_smolvm("machine", "delete", "--name", name, "-f")
|
||||||
_smolvm("machine", "delete", "-f", name)
|
|
||||||
|
|
||||||
|
|
||||||
def machine_exec(
|
def machine_exec(
|
||||||
|
|||||||
@@ -5,19 +5,28 @@ unit-tested without importing the docker subprocess paths."""
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import hashlib
|
import hashlib
|
||||||
|
import os
|
||||||
|
import platform
|
||||||
import shutil
|
import shutil
|
||||||
|
|
||||||
from ...log import die
|
from ...log import die
|
||||||
|
|
||||||
|
# libkrun's Linux backend drives the guest through KVM, so the host
|
||||||
|
# must expose `/dev/kvm` and the invoking user must be able to open
|
||||||
|
# it. macOS uses Hypervisor.framework and needs no device node.
|
||||||
|
_KVM_DEVICE = "/dev/kvm"
|
||||||
|
|
||||||
|
|
||||||
def smolmachines_preflight() -> None:
|
def smolmachines_preflight() -> None:
|
||||||
"""Ensure `smolvm` is on PATH before the launch flow runs.
|
"""Ensure the host can run the smolmachines backend before the
|
||||||
Called from `_resolve_plan`; gives the operator a clear
|
launch flow starts. Called from `_resolve_plan`; surfaces a
|
||||||
install pointer rather than a cryptic FileNotFoundError
|
clear, actionable error instead of a cryptic `smolvm` failure
|
||||||
later. `gvproxy` is no longer required — see the PRD's design
|
deep in launch.
|
||||||
pivot section."""
|
|
||||||
if shutil.which("smolvm") is not None:
|
Checks `smolvm` is on PATH (both platforms) and, on Linux,
|
||||||
return
|
that `/dev/kvm` exists and is accessible. `gvproxy` is no
|
||||||
|
longer required — see the PRD's design pivot section."""
|
||||||
|
if shutil.which("smolvm") is None:
|
||||||
die(
|
die(
|
||||||
"BOT_BOTTLE_BACKEND=smolmachines requires `smolvm` on "
|
"BOT_BOTTLE_BACKEND=smolmachines requires `smolvm` on "
|
||||||
"PATH. Install with: "
|
"PATH. Install with: "
|
||||||
@@ -25,6 +34,29 @@ def smolmachines_preflight() -> None:
|
|||||||
"To use the legacy Docker backend instead, set "
|
"To use the legacy Docker backend instead, set "
|
||||||
"BOT_BOTTLE_BACKEND=docker or pass --backend=docker."
|
"BOT_BOTTLE_BACKEND=docker or pass --backend=docker."
|
||||||
)
|
)
|
||||||
|
if platform.system() == "Linux":
|
||||||
|
_preflight_kvm()
|
||||||
|
|
||||||
|
|
||||||
|
def _preflight_kvm() -> None:
|
||||||
|
"""Linux-only: libkrun needs `/dev/kvm`. Distinguish 'KVM not
|
||||||
|
enabled' from 'no permission' so the operator knows which to
|
||||||
|
fix."""
|
||||||
|
if not os.path.exists(_KVM_DEVICE):
|
||||||
|
die(
|
||||||
|
f"BOT_BOTTLE_BACKEND=smolmachines needs {_KVM_DEVICE} on "
|
||||||
|
"Linux but it is missing. Enable KVM: load the kvm-intel "
|
||||||
|
"or kvm-amd kernel module (and confirm virtualization is "
|
||||||
|
"enabled in BIOS/firmware). To use the legacy Docker "
|
||||||
|
"backend instead, set BOT_BOTTLE_BACKEND=docker."
|
||||||
|
)
|
||||||
|
if not os.access(_KVM_DEVICE, os.R_OK | os.W_OK):
|
||||||
|
die(
|
||||||
|
f"{_KVM_DEVICE} exists but is not readable/writable by the "
|
||||||
|
"current user. Add your user to the `kvm` group "
|
||||||
|
"(`sudo usermod -aG kvm \"$USER\"`) and re-login, or run "
|
||||||
|
"with access to the device."
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def smolmachines_bundle_subnet(slug: str) -> tuple[str, str, str]:
|
def smolmachines_bundle_subnet(slug: str) -> tuple[str, str, str]:
|
||||||
|
|||||||
@@ -7,6 +7,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import sys
|
import sys
|
||||||
|
|
||||||
|
from ..errors import MissingEnvVarError
|
||||||
from ..log import Die, die, error
|
from ..log import Die, die, error
|
||||||
from ..manifest import ManifestError
|
from ..manifest import ManifestError
|
||||||
from ..store_manager import StoreManager
|
from ..store_manager import StoreManager
|
||||||
@@ -76,7 +77,7 @@ def main(argv: list[str] | None = None) -> int:
|
|||||||
usage()
|
usage()
|
||||||
die(f"unknown command: {command}")
|
die(f"unknown command: {command}")
|
||||||
mgr = StoreManager.instance()
|
mgr = StoreManager.instance()
|
||||||
if not mgr.check_migrations():
|
if not mgr.is_migrated():
|
||||||
sys.stderr.write("bot-bottle: database schema is out of date\n")
|
sys.stderr.write("bot-bottle: database schema is out of date\n")
|
||||||
sys.stderr.write("Migrate now? [y/N] ")
|
sys.stderr.write("Migrate now? [y/N] ")
|
||||||
sys.stderr.flush()
|
sys.stderr.flush()
|
||||||
@@ -90,9 +91,10 @@ def main(argv: list[str] | None = None) -> int:
|
|||||||
mgr.migrate()
|
mgr.migrate()
|
||||||
try:
|
try:
|
||||||
return handler(rest) or 0
|
return handler(rest) or 0
|
||||||
|
except MissingEnvVarError as e:
|
||||||
|
error(str(e))
|
||||||
|
return 1
|
||||||
except ManifestError as e:
|
except ManifestError as e:
|
||||||
# Manifest/config problems surface as a catchable exception;
|
|
||||||
# print the reason and exit non-zero (same UX die() used to give).
|
|
||||||
error(str(e))
|
error(str(e))
|
||||||
return 1
|
return 1
|
||||||
except Die as e:
|
except Die as e:
|
||||||
|
|||||||
+11
-3
@@ -108,6 +108,13 @@ def cmd_start(argv: list[str]) -> int:
|
|||||||
|
|
||||||
agent_name: str | None = args.name
|
agent_name: str | None = args.name
|
||||||
if agent_name is None:
|
if agent_name is None:
|
||||||
|
if not manifest.all_agent_names:
|
||||||
|
print(
|
||||||
|
"bot-bottle: no agents defined. "
|
||||||
|
"Add an agent to ~/.bot-bottle/agents/ or ./bot-bottle/agents/ to get started.",
|
||||||
|
file=sys.stderr,
|
||||||
|
)
|
||||||
|
return 1
|
||||||
agent_name = tui.filter_select(
|
agent_name = tui.filter_select(
|
||||||
manifest.all_agent_names,
|
manifest.all_agent_names,
|
||||||
title="Select agent",
|
title="Select agent",
|
||||||
@@ -235,7 +242,7 @@ def prepare_with_preflight(
|
|||||||
spec: BottleSpec,
|
spec: BottleSpec,
|
||||||
*,
|
*,
|
||||||
stage_dir: Path,
|
stage_dir: Path,
|
||||||
render_preflight: Callable[[DockerBottlePlan], None],
|
render_preflight: Callable[[DockerBottlePlan, str], None],
|
||||||
prompt_yes: Callable[[], bool],
|
prompt_yes: Callable[[], bool],
|
||||||
dry_run: bool = False,
|
dry_run: bool = False,
|
||||||
backend_name: str | None = None,
|
backend_name: str | None = None,
|
||||||
@@ -256,7 +263,7 @@ def prepare_with_preflight(
|
|||||||
plan = backend.prepare(spec, stage_dir=stage_dir)
|
plan = backend.prepare(spec, stage_dir=stage_dir)
|
||||||
identity = _identity_from_plan(plan)
|
identity = _identity_from_plan(plan)
|
||||||
|
|
||||||
render_preflight(plan)
|
render_preflight(plan, backend.name)
|
||||||
|
|
||||||
if dry_run:
|
if dry_run:
|
||||||
info("dry-run requested; not starting container.")
|
info("dry-run requested; not starting container.")
|
||||||
@@ -383,8 +390,9 @@ def _text_prompt_yes() -> bool:
|
|||||||
|
|
||||||
|
|
||||||
def _text_render_preflight():
|
def _text_render_preflight():
|
||||||
def _render(plan: DockerBottlePlan) -> None:
|
def _render(plan: DockerBottlePlan, backend_name: str) -> None:
|
||||||
print(file=sys.stderr)
|
print(file=sys.stderr)
|
||||||
|
print(f"backend: {backend_name}", file=sys.stderr)
|
||||||
print(_manifest_to_yaml(plan.manifest), file=sys.stderr)
|
print(_manifest_to_yaml(plan.manifest), file=sys.stderr)
|
||||||
return _render
|
return _render
|
||||||
|
|
||||||
|
|||||||
@@ -21,7 +21,7 @@ FROM node:22-slim
|
|||||||
# to it) works against egress's bumped TLS without the agent needing
|
# to it) works against egress's bumped TLS without the agent needing
|
||||||
# local DNS.
|
# local DNS.
|
||||||
RUN apt-get update \
|
RUN apt-get update \
|
||||||
&& apt-get install -y --no-install-recommends git ca-certificates curl ripgrep \
|
&& apt-get install -y --no-install-recommends git ca-certificates curl ripgrep iproute2 dnsutils \
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
# App-specific deps. Python isn't required by claude-code itself
|
# App-specific deps. Python isn't required by claude-code itself
|
||||||
|
|||||||
@@ -34,6 +34,12 @@ if TYPE_CHECKING:
|
|||||||
|
|
||||||
|
|
||||||
_SUPERVISE_MCP_NAME = "supervise"
|
_SUPERVISE_MCP_NAME = "supervise"
|
||||||
|
_CODEX_CLI = "/home/node/.codex/packages/standalone/current/bin/codex"
|
||||||
|
_CODEX_CLI_PATH = (
|
||||||
|
"/home/node/.local/bin:"
|
||||||
|
"/home/node/.codex/packages/standalone/current/bin:"
|
||||||
|
"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def _skills_dir(guest_home: str) -> str:
|
def _skills_dir(guest_home: str) -> str:
|
||||||
@@ -50,7 +56,7 @@ def _prompt_path(guest_home: str) -> str:
|
|||||||
|
|
||||||
_RUNTIME = AgentProviderRuntime(
|
_RUNTIME = AgentProviderRuntime(
|
||||||
template="codex",
|
template="codex",
|
||||||
command="codex",
|
command=_CODEX_CLI,
|
||||||
image="bot-bottle-codex:latest",
|
image="bot-bottle-codex:latest",
|
||||||
prompt_mode="read_prompt_file",
|
prompt_mode="read_prompt_file",
|
||||||
bypass_args=("--dangerously-bypass-approvals-and-sandbox",),
|
bypass_args=("--dangerously-bypass-approvals-and-sandbox",),
|
||||||
@@ -145,7 +151,8 @@ class CodexAgentProvider(AgentProvider):
|
|||||||
"env",
|
"env",
|
||||||
f"HOME={guest_home}",
|
f"HOME={guest_home}",
|
||||||
f"CODEX_HOME={auth_dir}",
|
f"CODEX_HOME={auth_dir}",
|
||||||
"codex", "login", "status",
|
f"PATH={_CODEX_CLI_PATH}",
|
||||||
|
_CODEX_CLI, "login", "status",
|
||||||
), (
|
), (
|
||||||
"codex host credentials: dummy auth was copied into the "
|
"codex host credentials: dummy auth was copied into the "
|
||||||
"guest, but Codex did not accept it"
|
"guest, but Codex did not accept it"
|
||||||
@@ -267,7 +274,7 @@ class CodexAgentProvider(AgentProvider):
|
|||||||
return
|
return
|
||||||
info(f"registering supervise MCP server in agent codex config → {supervise_url}")
|
info(f"registering supervise MCP server in agent codex config → {supervise_url}")
|
||||||
r = bottle.exec(
|
r = bottle.exec(
|
||||||
f"codex mcp add {_SUPERVISE_MCP_NAME} --url "
|
f"{shlex.quote(_CODEX_CLI)} mcp add {_SUPERVISE_MCP_NAME} --url "
|
||||||
f"{shlex.quote(supervise_url)}",
|
f"{shlex.quote(supervise_url)}",
|
||||||
user="node",
|
user="node",
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -28,7 +28,7 @@ class DbStore:
|
|||||||
conn.row_factory = sqlite3.Row
|
conn.row_factory = sqlite3.Row
|
||||||
return conn
|
return conn
|
||||||
|
|
||||||
def check_migrations(self) -> bool:
|
def is_migrated(self) -> bool:
|
||||||
"""Return True if the DB is fully up-to-date, False if migration is needed."""
|
"""Return True if the DB is fully up-to-date, False if migration is needed."""
|
||||||
if not self.db_path.exists():
|
if not self.db_path.exists():
|
||||||
return False
|
return False
|
||||||
|
|||||||
@@ -23,6 +23,7 @@ from .egress_addon_core import (
|
|||||||
PathMatch as CorePathMatch,
|
PathMatch as CorePathMatch,
|
||||||
Route,
|
Route,
|
||||||
)
|
)
|
||||||
|
from .errors import MissingEnvVarError
|
||||||
from .log import die
|
from .log import die
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
@@ -354,16 +355,18 @@ def egress_resolve_token_values(
|
|||||||
for token_env, token_ref in token_env_map.items():
|
for token_env, token_ref in token_env_map.items():
|
||||||
value = host_env.get(token_ref)
|
value = host_env.get(token_ref)
|
||||||
if value is None:
|
if value is None:
|
||||||
die(
|
raise MissingEnvVarError(
|
||||||
|
token_ref,
|
||||||
f"egress: host env var '{token_ref}' is unset. Set it "
|
f"egress: host env var '{token_ref}' is unset. Set it "
|
||||||
f"before launching, or remove the corresponding auth block "
|
f"before launching, or remove the corresponding auth block "
|
||||||
f"from bottle.egress.routes."
|
f"from bottle.egress.routes.",
|
||||||
)
|
)
|
||||||
if not value:
|
if not value:
|
||||||
die(
|
raise MissingEnvVarError(
|
||||||
|
token_ref,
|
||||||
f"egress: host env var '{token_ref}' is empty. The "
|
f"egress: host env var '{token_ref}' is empty. The "
|
||||||
f"egress will not inject an empty token; set it to "
|
f"egress will not inject an empty token; set it to "
|
||||||
f"the real value or remove the route's auth block."
|
f"the real value or remove the route's auth block.",
|
||||||
)
|
)
|
||||||
out[token_env] = value
|
out[token_env] = value
|
||||||
return out
|
return out
|
||||||
|
|||||||
+4
-2
@@ -35,6 +35,7 @@ import re
|
|||||||
import sys
|
import sys
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
|
|
||||||
|
from .errors import MissingEnvVarError
|
||||||
from .log import die
|
from .log import die
|
||||||
from .manifest import Manifest
|
from .manifest import Manifest
|
||||||
|
|
||||||
@@ -136,9 +137,10 @@ def resolve_env(manifest: Manifest) -> ResolvedEnv:
|
|||||||
host_var = env_entry_interpolated_from(raw)
|
host_var = env_entry_interpolated_from(raw)
|
||||||
host_value = os.environ.get(host_var, "")
|
host_value = os.environ.get(host_var, "")
|
||||||
if not host_value:
|
if not host_value:
|
||||||
die(
|
raise MissingEnvVarError(
|
||||||
|
host_var,
|
||||||
f"env entry {name} is interpolated from ${host_var}, "
|
f"env entry {name} is interpolated from ${host_var}, "
|
||||||
f"but ${host_var} is unset or empty in the host environment."
|
f"but ${host_var} is unset or empty in the host environment.",
|
||||||
)
|
)
|
||||||
forwarded[name] = host_value
|
forwarded[name] = host_value
|
||||||
else: # literal
|
else: # literal
|
||||||
|
|||||||
@@ -0,0 +1,18 @@
|
|||||||
|
"""bot-bottle application-level exception hierarchy.
|
||||||
|
|
||||||
|
Exceptions here are caught by the CLI dispatcher (``cli/__init__.py``)
|
||||||
|
and rendered as ``bot-bottle: error: …`` lines — same UX as ``die()``,
|
||||||
|
but without coupling library code to stderr output."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
|
||||||
|
class MissingEnvVarError(Exception):
|
||||||
|
"""Raised when a required host environment variable is unset or empty.
|
||||||
|
|
||||||
|
``var_name`` is the exact name of the missing variable so callers
|
||||||
|
can display it without re-parsing the message string."""
|
||||||
|
|
||||||
|
def __init__(self, var_name: str, message: str) -> None:
|
||||||
|
self.var_name = var_name
|
||||||
|
super().__init__(message)
|
||||||
@@ -82,6 +82,7 @@ class GitGatePlan:
|
|||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
class GitGate(ABC):
|
class GitGate(ABC):
|
||||||
"""The per-agent git-gate. Encapsulates the host-side prepare
|
"""The per-agent git-gate. Encapsulates the host-side prepare
|
||||||
(upstream lift + entrypoint/hook render); the sidecar's
|
(upstream lift + entrypoint/hook render); the sidecar's
|
||||||
|
|||||||
@@ -13,6 +13,7 @@ import dataclasses
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import TYPE_CHECKING
|
from typing import TYPE_CHECKING
|
||||||
|
|
||||||
|
from .errors import MissingEnvVarError
|
||||||
from .log import info
|
from .log import info
|
||||||
from .manifest import ManifestBottle, ManifestGitEntry
|
from .manifest import ManifestBottle, ManifestGitEntry
|
||||||
from .git_gate_render import GitGateUpstream
|
from .git_gate_render import GitGateUpstream
|
||||||
@@ -34,9 +35,10 @@ def _provision_dynamic_key(
|
|||||||
pk = entry.Key
|
pk = entry.Key
|
||||||
token = os.environ.get(pk.forge_token_env)
|
token = os.environ.get(pk.forge_token_env)
|
||||||
if token is None:
|
if token is None:
|
||||||
raise RuntimeError(
|
raise MissingEnvVarError(
|
||||||
|
pk.forge_token_env,
|
||||||
f"git-gate.repos[{entry.Name!r}] key.forge_token_env"
|
f"git-gate.repos[{entry.Name!r}] key.forge_token_env"
|
||||||
f" = {pk.forge_token_env!r}: env var is not set"
|
f" = {pk.forge_token_env!r}: env var is not set",
|
||||||
)
|
)
|
||||||
api_url = pk.api_url or f"https://{entry.UpstreamHost}"
|
api_url = pk.api_url or f"https://{entry.UpstreamHost}"
|
||||||
provisioner = get_provisioner(pk.provider, token, api_url)
|
provisioner = get_provisioner(pk.provider, token, api_url)
|
||||||
@@ -78,10 +80,11 @@ def revoke_git_gate_provisioned_keys(bottle: ManifestBottle, stage_dir: Path) ->
|
|||||||
key_id = id_file.read_text().strip()
|
key_id = id_file.read_text().strip()
|
||||||
token = os.environ.get(pk.forge_token_env)
|
token = os.environ.get(pk.forge_token_env)
|
||||||
if token is None:
|
if token is None:
|
||||||
raise RuntimeError(
|
raise MissingEnvVarError(
|
||||||
|
pk.forge_token_env,
|
||||||
f"git-gate.repos[{entry.Name!r}] key.forge_token_env"
|
f"git-gate.repos[{entry.Name!r}] key.forge_token_env"
|
||||||
f" = {pk.forge_token_env!r}: env var is not set;"
|
f" = {pk.forge_token_env!r}: env var is not set;"
|
||||||
f" cannot revoke deploy key {key_id}"
|
f" cannot revoke deploy key {key_id}",
|
||||||
)
|
)
|
||||||
api_url = pk.api_url or f"https://{entry.UpstreamHost}"
|
api_url = pk.api_url or f"https://{entry.UpstreamHost}"
|
||||||
provisioner = get_provisioner(pk.provider, token, api_url)
|
provisioner = get_provisioner(pk.provider, token, api_url)
|
||||||
|
|||||||
@@ -228,7 +228,7 @@ supervise_gitleaks_allow() {
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
proposal_id=$(
|
proposal_id=$(
|
||||||
GITLEAKS_ALLOW_REF="$ref" python3 - "$report_file" <<'PY'
|
PYTHONPATH="/app${PYTHONPATH:+:$PYTHONPATH}" GITLEAKS_ALLOW_REF="$ref" python3 - "$report_file" <<'PY'
|
||||||
import datetime
|
import datetime
|
||||||
import hashlib
|
import hashlib
|
||||||
import json
|
import json
|
||||||
@@ -236,6 +236,9 @@ import os
|
|||||||
import sys
|
import sys
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
|
try:
|
||||||
|
import supervise as _sv
|
||||||
|
except ImportError:
|
||||||
from bot_bottle import supervise as _sv
|
from bot_bottle import supervise as _sv
|
||||||
|
|
||||||
report_path = Path(sys.argv[1])
|
report_path = Path(sys.argv[1])
|
||||||
@@ -314,9 +317,12 @@ PY
|
|||||||
echo "git-gate: approve with './cli.py supervise' to continue this push" >&2
|
echo "git-gate: approve with './cli.py supervise' to continue this push" >&2
|
||||||
waited=0
|
waited=0
|
||||||
while [ "$waited" -lt "$timeout" ]; do
|
while [ "$waited" -lt "$timeout" ]; do
|
||||||
status=$(python3 - "$slug" "$proposal_id" <<'PY'
|
status=$(PYTHONPATH="/app${PYTHONPATH:+:$PYTHONPATH}" python3 - "$slug" "$proposal_id" <<'PY'
|
||||||
import sys
|
import sys
|
||||||
|
|
||||||
|
try:
|
||||||
|
import supervise as _sv
|
||||||
|
except ImportError:
|
||||||
from bot_bottle import supervise as _sv
|
from bot_bottle import supervise as _sv
|
||||||
|
|
||||||
slug = sys.argv[1]
|
slug = sys.argv[1]
|
||||||
@@ -336,9 +342,12 @@ PY
|
|||||||
if [ -n "$status" ]; then
|
if [ -n "$status" ]; then
|
||||||
case "$status" in
|
case "$status" in
|
||||||
approved|modified)
|
approved|modified)
|
||||||
python3 - "$slug" "$proposal_id" <<'PY' || true
|
PYTHONPATH="/app${PYTHONPATH:+:$PYTHONPATH}" python3 - "$slug" "$proposal_id" <<'PY' || true
|
||||||
import sys
|
import sys
|
||||||
|
|
||||||
|
try:
|
||||||
|
import supervise as _sv
|
||||||
|
except ImportError:
|
||||||
from bot_bottle import supervise as _sv
|
from bot_bottle import supervise as _sv
|
||||||
|
|
||||||
_sv.archive_proposal(sys.argv[1], sys.argv[2])
|
_sv.archive_proposal(sys.argv[1], sys.argv[2])
|
||||||
|
|||||||
+18
-20
@@ -16,10 +16,23 @@ except ImportError:
|
|||||||
from migrations import TableMigrations # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
|
from migrations import TableMigrations # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
|
||||||
|
|
||||||
|
|
||||||
# One entry per schema version: _MIGRATIONS.migrations[0] brings a fresh DB
|
class QueueStore(DbStore):
|
||||||
# to version 1, [1] to version 2, and so on. Add new migrations at the end;
|
"""SQLite-backed persistent store for supervise proposals and responses."""
|
||||||
# never edit existing ones.
|
|
||||||
_MIGRATIONS = TableMigrations("queue_store", [
|
def __init__(self, queue_key: str, db_path: Path | None = None) -> None:
|
||||||
|
self.queue_key = queue_key
|
||||||
|
if db_path is not None:
|
||||||
|
resolved = db_path
|
||||||
|
else:
|
||||||
|
# In the sidecar container SUPERVISE_DB_PATH points at the
|
||||||
|
# bind-mounted host DB. On the host this env var is never set,
|
||||||
|
# so we always fall through to host_db_path().
|
||||||
|
env_path = os.environ.get("SUPERVISE_DB_PATH", "").strip()
|
||||||
|
resolved = Path(env_path) if env_path else host_db_path()
|
||||||
|
# One entry per schema version: migrations[0] brings a fresh DB to
|
||||||
|
# version 1, [1] to version 2, etc. Add new entries at the end; never
|
||||||
|
# edit existing ones.
|
||||||
|
migrations = TableMigrations("queue_store", [
|
||||||
# v1 — proposals table
|
# v1 — proposals table
|
||||||
"""
|
"""
|
||||||
CREATE TABLE IF NOT EXISTS supervise_proposals (
|
CREATE TABLE IF NOT EXISTS supervise_proposals (
|
||||||
@@ -48,22 +61,7 @@ _MIGRATIONS = TableMigrations("queue_store", [
|
|||||||
)
|
)
|
||||||
""",
|
""",
|
||||||
])
|
])
|
||||||
|
super().__init__(resolved, migrations)
|
||||||
|
|
||||||
class QueueStore(DbStore):
|
|
||||||
"""SQLite-backed persistent store for supervise proposals and responses."""
|
|
||||||
|
|
||||||
def __init__(self, queue_key: str, db_path: Path | None = None) -> None:
|
|
||||||
self.queue_key = queue_key
|
|
||||||
if db_path is not None:
|
|
||||||
resolved = db_path
|
|
||||||
else:
|
|
||||||
# In the sidecar container SUPERVISE_DB_PATH points at the
|
|
||||||
# bind-mounted host DB. On the host this env var is never set,
|
|
||||||
# so we always fall through to host_db_path().
|
|
||||||
env_path = os.environ.get("SUPERVISE_DB_PATH", "").strip()
|
|
||||||
resolved = Path(env_path) if env_path else host_db_path()
|
|
||||||
super().__init__(resolved, _MIGRATIONS)
|
|
||||||
|
|
||||||
def write_proposal(self, proposal: Proposal) -> Path:
|
def write_proposal(self, proposal: Proposal) -> Path:
|
||||||
with self._connect() as conn:
|
with self._connect() as conn:
|
||||||
|
|||||||
@@ -15,7 +15,7 @@ _instance: StoreManager | None = None
|
|||||||
|
|
||||||
|
|
||||||
class StoreManager:
|
class StoreManager:
|
||||||
"""Owns db_path and delegates migrate/check_migrations across all stores.
|
"""Owns db_path and delegates migrate/is_migrated across all stores.
|
||||||
|
|
||||||
Use instance() for normal access. Call reset(db_path) in tests to swap
|
Use instance() for normal access. Call reset(db_path) in tests to swap
|
||||||
the singleton to a temp path, then reset() with no args to restore the
|
the singleton to a temp path, then reset() with no args to restore the
|
||||||
@@ -46,10 +46,10 @@ class StoreManager:
|
|||||||
global _instance
|
global _instance
|
||||||
_instance = cls(db_path)
|
_instance = cls(db_path)
|
||||||
|
|
||||||
def check_migrations(self) -> bool:
|
def is_migrated(self) -> bool:
|
||||||
return (
|
return (
|
||||||
QueueStore("", self.db_path).check_migrations()
|
QueueStore("", self.db_path).is_migrated()
|
||||||
and AuditStore(self.db_path).check_migrations()
|
and AuditStore(self.db_path).is_migrated()
|
||||||
)
|
)
|
||||||
|
|
||||||
def migrate(self) -> None:
|
def migrate(self) -> None:
|
||||||
|
|||||||
@@ -110,7 +110,10 @@ def host_db_path() -> Path:
|
|||||||
# Calls bot_bottle_root() through this module's globals so that patches
|
# Calls bot_bottle_root() through this module's globals so that patches
|
||||||
# on supervise.bot_bottle_root propagate to callers going through
|
# on supervise.bot_bottle_root propagate to callers going through
|
||||||
# supervise.host_db_path().
|
# supervise.host_db_path().
|
||||||
return bot_bottle_root() / HOST_DB_FILENAME
|
#
|
||||||
|
# Kept in its own "db" subdirectory (see supervise_types.host_db_path
|
||||||
|
# for why) — must stay in sync with that copy.
|
||||||
|
return bot_bottle_root() / "db" / HOST_DB_FILENAME
|
||||||
|
|
||||||
|
|
||||||
def audit_log_path(component: str, slug: str) -> Path:
|
def audit_log_path(component: str, slug: str) -> Path:
|
||||||
|
|||||||
@@ -50,7 +50,14 @@ def bot_bottle_root() -> Path:
|
|||||||
def host_db_path() -> Path:
|
def host_db_path() -> Path:
|
||||||
# Look up bot_bottle_root through this module's live namespace so that
|
# Look up bot_bottle_root through this module's live namespace so that
|
||||||
# monkey-patches on supervise_types.bot_bottle_root take effect.
|
# monkey-patches on supervise_types.bot_bottle_root take effect.
|
||||||
return sys.modules[__name__].bot_bottle_root() / HOST_DB_FILENAME
|
#
|
||||||
|
# Lives in its own "db" subdirectory, not directly under
|
||||||
|
# bot_bottle_root(), so backends that can only bind-mount
|
||||||
|
# directories (macos_container's `container run --mount`, which
|
||||||
|
# rejects file sources with "is not a directory") can share this
|
||||||
|
# one file with a sidecar without also exposing bot_bottle_root()'s
|
||||||
|
# other contents (git-gate keys, per-bottle state, etc.).
|
||||||
|
return sys.modules[__name__].bot_bottle_root() / "db" / HOST_DB_FILENAME
|
||||||
|
|
||||||
|
|
||||||
def _require_str(raw: dict[str, object], key: str) -> str:
|
def _require_str(raw: dict[str, object], key: str) -> str:
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
# PRD prd-new: SQLite local storage
|
# PRD 0067: SQLite local storage
|
||||||
|
|
||||||
- **Status:** Active
|
- **Status:** Active
|
||||||
- **Author:** codex
|
- **Author:** codex
|
||||||
@@ -0,0 +1,227 @@
|
|||||||
|
# PRD prd-new: smolmachines backend on Linux
|
||||||
|
|
||||||
|
- **Status:** Draft
|
||||||
|
- **Author:** Claude
|
||||||
|
- **Created:** 2026-06-25
|
||||||
|
- **Issue:** #283
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
Make the `smolmachines` backend (PRD 0023) runnable on Linux, not
|
||||||
|
just macOS. `smolvm` already supports Linux via KVM (`/dev/kvm`);
|
||||||
|
the gap is entirely in bot-bottle's host-side glue, which hard-codes
|
||||||
|
macOS assumptions in three places:
|
||||||
|
|
||||||
|
1. **Preflight** only checks that `smolvm` is on `PATH` — it never
|
||||||
|
checks the Linux KVM prerequisite, so a misconfigured host fails
|
||||||
|
deep in the launch flow with an opaque `smolvm` error.
|
||||||
|
2. **The TSI allowlist enforcement** (`force_allowlist`) — the
|
||||||
|
security property that confines the agent VM to its sidecar
|
||||||
|
bundle's `/32` — **no-ops on Linux today, failing _open_**. The
|
||||||
|
smolvm state-DB path it patches is hard-coded to macOS's
|
||||||
|
`~/Library/Application Support/...`.
|
||||||
|
3. **Per-bottle loopback scoping** (`allocate`) returns the shared
|
||||||
|
`127.0.0.1` on Linux, which would let the agent VM reach every
|
||||||
|
service on host loopback — a downgrade from the per-bottle alias
|
||||||
|
isolation macOS gets.
|
||||||
|
|
||||||
|
This PRD closes all three so a bottle launched with
|
||||||
|
`BOT_BOTTLE_BACKEND=smolmachines` on Linux gets the same isolation
|
||||||
|
guarantee it gets on macOS, and documents the Linux/NixOS host
|
||||||
|
setup. The primary validation target is NixOS, but the changes are
|
||||||
|
distro-agnostic.
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
The smolmachines backend runs each bottle's agent inside a libkrun
|
||||||
|
microVM via `smolvm`, with egress confined by TSI's `--allow-cidr`
|
||||||
|
allowlist set to a single `/32` — the sidecar bundle's loopback
|
||||||
|
address. Everything else (host loopback, LAN, internet) is denied at
|
||||||
|
the VMM layer. That security property is the entire reason the
|
||||||
|
backend exists.
|
||||||
|
|
||||||
|
libkrun runs on Hypervisor.framework (macOS) **and** KVM (Linux), and
|
||||||
|
`smolvm` ships Linux x86_64 / aarch64 builds that require `/dev/kvm`.
|
||||||
|
So the microVM layer already works on Linux. What does not work is
|
||||||
|
bot-bottle's host integration, which PRD 0023 explicitly scoped to
|
||||||
|
macOS-only for v1. Three concrete blockers:
|
||||||
|
|
||||||
|
- **No KVM preflight.** On a Linux host without `/dev/kvm` (kernel
|
||||||
|
module not loaded) or without access to it (user not in the `kvm`
|
||||||
|
group), the failure surfaces as a cryptic `smolvm` non-zero exit
|
||||||
|
mid-launch instead of an actionable message.
|
||||||
|
|
||||||
|
- **TSI enforcement fails open on Linux.** `force_allowlist`
|
||||||
|
early-returns on non-macOS. It exists because `smolvm` 0.8.0
|
||||||
|
silently drops `--allow-cidr` when combined with `--from`, so the
|
||||||
|
allowlist has to be patched into smolvm's persisted state DB before
|
||||||
|
`machine start`. On Linux that patch never runs **and** the DB path
|
||||||
|
is the macOS path, so the booted VM's TSI allowlist is whatever
|
||||||
|
smolvm defaulted to — potentially all of `127.0.0.0/8`. That is the
|
||||||
|
exact sandbox-escape the backend is supposed to prevent.
|
||||||
|
|
||||||
|
- **No per-bottle loopback isolation on Linux.** `allocate` returns
|
||||||
|
`127.0.0.1` on Linux. Even with a correct allowlist, `127.0.0.1/32`
|
||||||
|
is shared by every service on host loopback, so the agent could
|
||||||
|
reach other bottles' published ports and host services. On macOS
|
||||||
|
this is solved with per-bottle `127.0.0.16..31` aliases added via
|
||||||
|
`sudo ifconfig lo0 alias`. On Linux the whole `127.0.0.0/8` is
|
||||||
|
already routed to `lo`, so docker can publish to `127.0.0.<N>`
|
||||||
|
with **no `ifconfig`/sudo step at all** — the isolation is actually
|
||||||
|
cheaper to achieve than on macOS.
|
||||||
|
|
||||||
|
## Goals / Success Criteria
|
||||||
|
|
||||||
|
- `BOT_BOTTLE_BACKEND=smolmachines ./cli.py start <agent>` launches,
|
||||||
|
runs, and tears down a bottle on a Linux host with `/dev/kvm`.
|
||||||
|
- The TSI allowlist is enforced on Linux: PRD 0022's
|
||||||
|
`tests/integration/test_sandbox_escape.py` passes against
|
||||||
|
`BOT_BOTTLE_BACKEND=smolmachines` on Linux (the acceptance gate).
|
||||||
|
- Each Linux bottle is scoped to its own `127.0.0.<N>/32`, matching
|
||||||
|
the macOS per-bottle isolation property.
|
||||||
|
- A clear, actionable preflight error when `/dev/kvm` is missing or
|
||||||
|
inaccessible, with remediation (load `kvm-intel`/`kvm-amd`, join the
|
||||||
|
`kvm` group).
|
||||||
|
- **Fail-closed:** if bot-bottle cannot positively confirm the TSI
|
||||||
|
allowlist was persisted for a machine (DB missing, row missing,
|
||||||
|
patch didn't take), it `die()`s before `machine start` rather than
|
||||||
|
booting a VM with an unverified allowlist.
|
||||||
|
- macOS behavior is unchanged.
|
||||||
|
- README documents Linux + NixOS host setup.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- Rootless / non-KVM fallbacks (e.g. software emulation). Linux
|
||||||
|
smolmachines requires `/dev/kvm`, full stop.
|
||||||
|
- Removing Docker as a host dependency — the sidecar bundle and
|
||||||
|
image-build pipeline still use Docker on Linux, same as macOS.
|
||||||
|
- Auto-installing `smolvm` or configuring KVM on the operator's
|
||||||
|
behalf. Preflight reports; the operator remediates.
|
||||||
|
- Nested-virtualization tuning for running the runner itself inside a
|
||||||
|
VM (documented as a caveat, not solved here).
|
||||||
|
|
||||||
|
## Design
|
||||||
|
|
||||||
|
### Platform detection
|
||||||
|
|
||||||
|
Reuse the existing `platform.system()` check already in
|
||||||
|
`loopback_alias.py` (`_is_macos()`). "Linux" is "not macOS" for every
|
||||||
|
branch below; no new third-platform path.
|
||||||
|
|
||||||
|
### Preflight: KVM gate (`util.smolmachines_preflight`)
|
||||||
|
|
||||||
|
After the existing `smolvm`-on-`PATH` check, add a Linux-only gate:
|
||||||
|
|
||||||
|
- `/dev/kvm` must exist → else `die()` with "enable KVM
|
||||||
|
(`kvm-intel`/`kvm-amd` kernel module)".
|
||||||
|
- `/dev/kvm` must be readable + writable by the current user
|
||||||
|
(`os.access(..., R_OK | W_OK)`) → else `die()` with "add your user
|
||||||
|
to the `kvm` group (and re-login)".
|
||||||
|
|
||||||
|
macOS is unaffected (Hypervisor.framework needs no device node).
|
||||||
|
|
||||||
|
### smolvm state-DB path (platform-aware)
|
||||||
|
|
||||||
|
`loopback_alias._SMOLVM_DB_PATH` becomes platform-derived:
|
||||||
|
|
||||||
|
- macOS: `~/Library/Application Support/smolvm/server/smolvm.db`
|
||||||
|
(unchanged).
|
||||||
|
- Linux: `$XDG_DATA_HOME/smolvm/server/smolvm.db`, defaulting to
|
||||||
|
`~/.local/share/smolvm/server/smolvm.db`.
|
||||||
|
|
||||||
|
> **Verification note:** the Linux DB location is inferred from
|
||||||
|
> smolvm's documented `~/.local/share` install layout and the XDG
|
||||||
|
> base-dir spec. It must be confirmed on a real Linux smolvm install;
|
||||||
|
> if smolvm uses a different path or schema, the fail-closed check
|
||||||
|
> below turns that into a clear `die()` at launch rather than a silent
|
||||||
|
> escape.
|
||||||
|
|
||||||
|
### TSI enforcement: cross-platform + fail-closed (`force_allowlist`)
|
||||||
|
|
||||||
|
Rework `force_allowlist(machine_name, allowed_cidrs)` to run on
|
||||||
|
**both** platforms and to fail closed:
|
||||||
|
|
||||||
|
1. Resolve the state DB; if the file is missing, `die()` (cannot
|
||||||
|
confirm enforcement → refuse to launch).
|
||||||
|
2. Read the machine's persisted row; if the row is missing, `die()`.
|
||||||
|
3. If the row's `allowed_cidrs` already equals the requested list
|
||||||
|
(e.g. a newer `smolvm` that honors `--allow-cidr` at create), do
|
||||||
|
nothing — no write.
|
||||||
|
4. Otherwise patch `allowed_cidrs` (the existing BLOB-encoded write)
|
||||||
|
and re-read.
|
||||||
|
5. If, after the patch, `allowed_cidrs` still does not equal the
|
||||||
|
requested list, `die()`.
|
||||||
|
|
||||||
|
This is robust across smolvm versions: it works whether `--allow-cidr`
|
||||||
|
is silently dropped (0.8.0) or honored (newer), and it never boots a
|
||||||
|
VM whose persisted allowlist it could not confirm. It is a strict
|
||||||
|
improvement on macOS too (today's code writes unconditionally and
|
||||||
|
never verifies).
|
||||||
|
|
||||||
|
> The persisted-row check confirms our write took, not that smolvm's
|
||||||
|
> runtime TSI enforces it. The runtime guarantee is covered by the
|
||||||
|
> sandbox-escape acceptance test; the persisted check is the cheap
|
||||||
|
> fail-closed guard at launch.
|
||||||
|
|
||||||
|
### Per-bottle loopback scoping on Linux (`allocate`)
|
||||||
|
|
||||||
|
`allocate` runs the same docker-state-driven allocation on Linux as on
|
||||||
|
macOS (`_allocate_locked`, the file lock, and `_aliases_in_use` via
|
||||||
|
`docker inspect` are all already cross-platform). The only macOS-only
|
||||||
|
step, `ensure_pool` (the `sudo ifconfig lo0 alias` dance), stays
|
||||||
|
macOS-only: on Linux `127.0.0.0/8` is already loopback, so docker can
|
||||||
|
publish bundle ports directly on `127.0.0.<N>` with no setup.
|
||||||
|
|
||||||
|
Net effect: Linux bottles get per-bottle `127.0.0.16..31/32` scoping
|
||||||
|
identical to macOS, without sudo.
|
||||||
|
|
||||||
|
### Launch flow
|
||||||
|
|
||||||
|
`launch.py` needs no structural change — `_allocate_resources` already
|
||||||
|
calls `ensure_pool()` (now a Linux no-op) then `allocate()` (now
|
||||||
|
per-bottle on Linux), and `_launch_vm` already calls
|
||||||
|
`force_allowlist()` (now active on Linux). Only the macOS-specific
|
||||||
|
docstrings are updated to describe the cross-platform behavior.
|
||||||
|
|
||||||
|
## Implementation chunks
|
||||||
|
|
||||||
|
1. **Preflight KVM gate** — `util.smolmachines_preflight` +
|
||||||
|
unit tests for the missing-device and no-access branches.
|
||||||
|
2. **Platform-aware DB path + fail-closed `force_allowlist`** —
|
||||||
|
`loopback_alias.py`; update/extend `TestForceAllowlist`.
|
||||||
|
3. **Cross-platform `allocate`** — drop the Linux early-return; update
|
||||||
|
`TestAllocate` / `TestAllocateLock` for the new Linux behavior.
|
||||||
|
4. **Docstring + comment cleanup** in `launch.py` and module headers.
|
||||||
|
5. **Docs** — README requirements + a Linux/NixOS host-setup section.
|
||||||
|
|
||||||
|
## Testing Strategy
|
||||||
|
|
||||||
|
- **Unit (CI, any OS):** the suite mocks `platform.system()` /
|
||||||
|
`subprocess` and patches `_SMOLVM_DB_PATH`, so the new Linux
|
||||||
|
branches are testable on the macOS/Linux CI runner without `smolvm`
|
||||||
|
or KVM. Covers: KVM preflight branches, fail-closed `force_allowlist`
|
||||||
|
(DB missing, row missing, patch-doesn't-take), per-bottle Linux
|
||||||
|
allocation + locking, platform-derived DB path.
|
||||||
|
- **Integration (Linux host with KVM — the acceptance gate):**
|
||||||
|
`tests/integration/test_sandbox_escape.py` against
|
||||||
|
`BOT_BOTTLE_BACKEND=smolmachines`. This cannot run on the macOS dev
|
||||||
|
box and must be executed on NixOS before merge.
|
||||||
|
|
||||||
|
## Open questions / verification pending
|
||||||
|
|
||||||
|
- **Confirm the Linux smolvm state-DB path and schema** on a real
|
||||||
|
install (the `~/.local/share/...` inference above).
|
||||||
|
- **Confirm whether the current smolvm Linux build still drops
|
||||||
|
`--allow-cidr` with `--from`** (the 0.8.0 bug). The fail-closed
|
||||||
|
design handles either answer, but knowing lets us drop the DB patch
|
||||||
|
if upstream fixed it.
|
||||||
|
- **Confirm docker publishing to `127.0.0.<N>` on Linux** behaves as
|
||||||
|
expected end-to-end with TSI (high confidence; standard loopback
|
||||||
|
behavior, but unverified on the target host).
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- PRD 0023 — smolmachines bottle backend (macOS v1).
|
||||||
|
- PRD 0022 — `test_sandbox_escape.py` acceptance gate.
|
||||||
|
- PRD 0024 — sidecar bundle image.
|
||||||
|
- smolvm: https://github.com/smol-machines/smolvm
|
||||||
@@ -81,10 +81,10 @@ class TestSandboxEscape(unittest.TestCase):
|
|||||||
# those are missing rather than die-ing inside backend.prepare.
|
# those are missing rather than die-ing inside backend.prepare.
|
||||||
backend_name = os.environ.get("BOT_BOTTLE_BACKEND", "docker")
|
backend_name = os.environ.get("BOT_BOTTLE_BACKEND", "docker")
|
||||||
if backend_name == "smolmachines":
|
if backend_name == "smolmachines":
|
||||||
if sys.platform != "darwin":
|
if sys.platform not in ("darwin", "linux"):
|
||||||
raise unittest.SkipTest(
|
raise unittest.SkipTest(
|
||||||
"BOT_BOTTLE_BACKEND=smolmachines is macOS-only in "
|
f"BOT_BOTTLE_BACKEND=smolmachines is not supported "
|
||||||
"v1 (libkrun TSI)"
|
f"on {sys.platform} (macOS and Linux only)"
|
||||||
)
|
)
|
||||||
if shutil.which("smolvm") is None:
|
if shutil.which("smolvm") is None:
|
||||||
raise unittest.SkipTest(
|
raise unittest.SkipTest(
|
||||||
|
|||||||
@@ -21,7 +21,7 @@ security properties the design pivot was about:
|
|||||||
bind-address mitigation is what closes TSI's port-granularity
|
bind-address mitigation is what closes TSI's port-granularity
|
||||||
gap.
|
gap.
|
||||||
|
|
||||||
Gated on macOS + smolvm + docker + not GITEA_ACTIONS — the
|
Gated on macOS/Linux + smolvm + docker + not GITEA_ACTIONS — the
|
||||||
runner can't host libkrun-backed VMs."""
|
runner can't host libkrun-backed VMs."""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
@@ -65,8 +65,8 @@ def _minimal_manifest() -> ManifestIndex:
|
|||||||
|
|
||||||
@skip_unless_docker()
|
@skip_unless_docker()
|
||||||
@unittest.skipUnless(
|
@unittest.skipUnless(
|
||||||
platform.system() == "Darwin",
|
platform.system() in ("Darwin", "Linux"),
|
||||||
"smolvm is macOS-only for v1; Linux+KVM path is a future PRD",
|
"smolvm requires macOS or Linux",
|
||||||
)
|
)
|
||||||
@unittest.skipUnless(
|
@unittest.skipUnless(
|
||||||
_smolvm_available(),
|
_smolvm_available(),
|
||||||
@@ -141,7 +141,15 @@ class TestSmolmachinesLaunch(unittest.TestCase):
|
|||||||
proxies = [line.strip() for line in r.stdout.splitlines()]
|
proxies = [line.strip() for line in r.stdout.splitlines()]
|
||||||
self.assertEqual(2, len(proxies), proxies)
|
self.assertEqual(2, len(proxies), proxies)
|
||||||
self.assertEqual(proxies[0], proxies[1], proxies)
|
self.assertEqual(proxies[0], proxies[1], proxies)
|
||||||
self.assertTrue(proxies[0].startswith("http://127."), proxies[0])
|
# macOS: proxy binds to the per-bottle loopback alias (127.x.x.x) so
|
||||||
|
# TSI can intercept guest connections to it. Linux: the guest kernel
|
||||||
|
# routes 127.0.0.0/8 to its own loopback (TSI never sees those), so
|
||||||
|
# the proxy instead binds to the per-bottle bridge gateway (192.168.x.1)
|
||||||
|
# which routes via eth0 and is intercepted by TSI normally.
|
||||||
|
self.assertRegex(
|
||||||
|
proxies[0], r"^http://\d+\.\d+\.\d+\.\d+:\d+$",
|
||||||
|
"expected proxy URL to be an http://IP:port address",
|
||||||
|
)
|
||||||
|
|
||||||
r = self.bottle.exec(
|
r = self.bottle.exec(
|
||||||
"curl -fsS --max-time 20 https://example.com >/dev/null && echo OK"
|
"curl -fsS --max-time 20 https://example.com >/dev/null && echo OK"
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
exercised against the real binary.
|
exercised against the real binary.
|
||||||
|
|
||||||
The full machine-lifecycle round trip (create → start → exec →
|
The full machine-lifecycle round trip (create → start → exec →
|
||||||
delete) is gated behind macOS + Darwin platform check and lives
|
delete) is gated behind macOS/Linux platform check and lives
|
||||||
in chunk 2d's smoke. This file just verifies `is_available()`
|
in chunk 2d's smoke. This file just verifies `is_available()`
|
||||||
correctly reports presence and `_smolvm()` can run a no-op
|
correctly reports presence and `_smolvm()` can run a no-op
|
||||||
subcommand without errors — enough to flag wrapper drift if
|
subcommand without errors — enough to flag wrapper drift if
|
||||||
@@ -23,8 +23,8 @@ from bot_bottle.backend.smolmachines.smolvm import is_available
|
|||||||
"skipped under act_runner: smolvm not installed on the runner",
|
"skipped under act_runner: smolvm not installed on the runner",
|
||||||
)
|
)
|
||||||
@unittest.skipUnless(
|
@unittest.skipUnless(
|
||||||
platform.system() == "Darwin",
|
platform.system() in ("Darwin", "Linux"),
|
||||||
"smolvm is macOS-only for v1; Linux+KVM path is a future PRD",
|
"smolvm requires macOS or Linux",
|
||||||
)
|
)
|
||||||
@unittest.skipUnless(
|
@unittest.skipUnless(
|
||||||
is_available(),
|
is_available(),
|
||||||
|
|||||||
@@ -35,7 +35,10 @@ class TestAgentProviderRuntime(unittest.TestCase):
|
|||||||
)
|
)
|
||||||
config = Path(tmp, "codex-config.toml").read_text()
|
config = Path(tmp, "codex-config.toml").read_text()
|
||||||
self.assertEqual("codex", plan.template)
|
self.assertEqual("codex", plan.template)
|
||||||
self.assertEqual("codex", plan.command)
|
self.assertEqual(
|
||||||
|
"/home/node/.codex/packages/standalone/current/bin/codex",
|
||||||
|
plan.command,
|
||||||
|
)
|
||||||
self.assertEqual("read_prompt_file", plan.prompt_mode)
|
self.assertEqual("read_prompt_file", plan.prompt_mode)
|
||||||
self.assertEqual("/tmp/Dockerfile.codex", plan.dockerfile)
|
self.assertEqual("/tmp/Dockerfile.codex", plan.dockerfile)
|
||||||
self.assertEqual(
|
self.assertEqual(
|
||||||
|
|||||||
@@ -20,7 +20,7 @@ from bot_bottle.store_manager import StoreManager
|
|||||||
|
|
||||||
class TestMainDispatch(unittest.TestCase):
|
class TestMainDispatch(unittest.TestCase):
|
||||||
def setUp(self) -> None:
|
def setUp(self) -> None:
|
||||||
patcher = patch.object(DbStore, "check_migrations", return_value=True)
|
patcher = patch.object(DbStore, "is_migrated", return_value=True)
|
||||||
self._mock_check = patcher.start()
|
self._mock_check = patcher.start()
|
||||||
self.addCleanup(patcher.stop)
|
self.addCleanup(patcher.stop)
|
||||||
self.addCleanup(StoreManager.reset)
|
self.addCleanup(StoreManager.reset)
|
||||||
@@ -64,6 +64,15 @@ class TestMainDispatch(unittest.TestCase):
|
|||||||
main(["x", "a", "b"])
|
main(["x", "a", "b"])
|
||||||
self.assertEqual([["a", "b"]], seen)
|
self.assertEqual([["a", "b"]], seen)
|
||||||
|
|
||||||
|
def test_missing_env_var_error_maps_to_1(self) -> None:
|
||||||
|
from bot_bottle.errors import MissingEnvVarError
|
||||||
|
|
||||||
|
def boom(_rest: list[str]) -> int:
|
||||||
|
raise MissingEnvVarError("MY_VAR", "MY_VAR is not set")
|
||||||
|
|
||||||
|
with patch.dict(climod.COMMANDS, {"x": boom}), patch("sys.stderr", io.StringIO()):
|
||||||
|
self.assertEqual(1, main(["x"]))
|
||||||
|
|
||||||
def test_manifest_error_maps_to_1(self) -> None:
|
def test_manifest_error_maps_to_1(self) -> None:
|
||||||
def boom(_rest: list[str]) -> int:
|
def boom(_rest: list[str]) -> int:
|
||||||
raise ManifestError("bad manifest")
|
raise ManifestError("bad manifest")
|
||||||
|
|||||||
@@ -9,6 +9,7 @@ is created.
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import io
|
||||||
import os
|
import os
|
||||||
import unittest
|
import unittest
|
||||||
from unittest.mock import MagicMock, patch
|
from unittest.mock import MagicMock, patch
|
||||||
@@ -184,5 +185,55 @@ class TestCmdStartHeadless(unittest.TestCase):
|
|||||||
self.assertEqual("docker", self._launch_mock.call_args[1]["backend_name"])
|
self.assertEqual("docker", self._launch_mock.call_args[1]["backend_name"])
|
||||||
|
|
||||||
|
|
||||||
|
class TestPrepareWithPreflight(unittest.TestCase):
|
||||||
|
"""prepare_with_preflight calls render_preflight with the plan and backend name."""
|
||||||
|
|
||||||
|
def test_render_preflight_called(self):
|
||||||
|
from pathlib import Path
|
||||||
|
mock_backend = MagicMock()
|
||||||
|
mock_plan = MagicMock()
|
||||||
|
mock_backend.prepare.return_value = mock_plan
|
||||||
|
mock_backend.name = "test-backend"
|
||||||
|
render = MagicMock()
|
||||||
|
|
||||||
|
with patch("bot_bottle.cli.start.get_bottle_backend", return_value=mock_backend), \
|
||||||
|
patch("bot_bottle.cli.start._identity_from_plan", return_value="id"), \
|
||||||
|
patch("bot_bottle.cli.start.info"):
|
||||||
|
start_mod.prepare_with_preflight(
|
||||||
|
MagicMock(),
|
||||||
|
stage_dir=Path("/tmp"),
|
||||||
|
render_preflight=render,
|
||||||
|
prompt_yes=MagicMock(),
|
||||||
|
dry_run=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
render.assert_called_once_with(mock_plan, "test-backend")
|
||||||
|
|
||||||
|
|
||||||
|
class TestNoAgentsDefined(unittest.TestCase):
|
||||||
|
"""cmd_start prints an error and returns 1 when the manifest has no agents."""
|
||||||
|
|
||||||
|
def test_no_agents_defined_returns_1(self):
|
||||||
|
manifest = _make_manifest([], [])
|
||||||
|
with patch(
|
||||||
|
"bot_bottle.cli.start.ManifestIndex.resolve", return_value=manifest
|
||||||
|
), patch("sys.stderr", io.StringIO()) as err:
|
||||||
|
rc = start_mod.cmd_start([])
|
||||||
|
self.assertEqual(1, rc)
|
||||||
|
self.assertIn("no agents defined", err.getvalue())
|
||||||
|
|
||||||
|
|
||||||
|
class TestTextRenderPreflight(unittest.TestCase):
|
||||||
|
"""_text_render_preflight returns a renderer that prints the backend name."""
|
||||||
|
|
||||||
|
def test_backend_name_in_output(self):
|
||||||
|
render = start_mod._text_render_preflight()
|
||||||
|
plan = MagicMock()
|
||||||
|
with patch("bot_bottle.cli.start._manifest_to_yaml", return_value=""), \
|
||||||
|
patch("sys.stderr", io.StringIO()) as err:
|
||||||
|
render(plan, "my-backend")
|
||||||
|
self.assertIn("my-backend", err.getvalue())
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
unittest.main()
|
unittest.main()
|
||||||
|
|||||||
@@ -261,6 +261,36 @@ class TestCodexProvision(unittest.TestCase):
|
|||||||
self.assertTrue(any("find" in s and "-delete" in s for s in scripts))
|
self.assertTrue(any("find" in s and "-delete" in s for s in scripts))
|
||||||
self.assertTrue(any("runuser" in s and "codex login status" in s for s in scripts))
|
self.assertTrue(any("runuser" in s and "codex login status" in s for s in scripts))
|
||||||
|
|
||||||
|
def test_forwarded_credentials_verify_sets_codex_path(self):
|
||||||
|
with tempfile.TemporaryDirectory(prefix="bb-codex-auth.") as tmp:
|
||||||
|
state_dir = Path(tmp)
|
||||||
|
prompt_file = state_dir / "prompt.txt"
|
||||||
|
prompt_file.write_text("")
|
||||||
|
with patch(
|
||||||
|
"bot_bottle.contrib.codex.agent_provider.codex_host_access_token",
|
||||||
|
return_value="token",
|
||||||
|
), patch(
|
||||||
|
"bot_bottle.contrib.codex.agent_provider.write_codex_dummy_auth_file",
|
||||||
|
):
|
||||||
|
provision = CodexAgentProvider().provision_plan(
|
||||||
|
dockerfile="",
|
||||||
|
state_dir=state_dir,
|
||||||
|
instance_name="bot-bottle-demo-abc12",
|
||||||
|
prompt_file=prompt_file,
|
||||||
|
forward_host_credentials=True,
|
||||||
|
host_env={},
|
||||||
|
)
|
||||||
|
|
||||||
|
verify_argv = provision.verify[0].argv
|
||||||
|
self.assertTrue(any(
|
||||||
|
item.startswith("PATH=/home/node/.local/bin:")
|
||||||
|
for item in verify_argv
|
||||||
|
))
|
||||||
|
self.assertIn(
|
||||||
|
"/home/node/.codex/packages/standalone/current/bin/codex",
|
||||||
|
verify_argv,
|
||||||
|
)
|
||||||
|
|
||||||
def test_dies_when_dir_creation_fails(self):
|
def test_dies_when_dir_creation_fails(self):
|
||||||
provision = AgentProvisionPlan(
|
provision = AgentProvisionPlan(
|
||||||
template="codex", command="codex",
|
template="codex", command="codex",
|
||||||
@@ -301,7 +331,8 @@ class TestCodexSuperviseMcp(unittest.TestCase):
|
|||||||
script = bottle.exec.call_args.args[0]
|
script = bottle.exec.call_args.args[0]
|
||||||
self.assertEqual("node", bottle.exec.call_args.kwargs.get("user"))
|
self.assertEqual("node", bottle.exec.call_args.kwargs.get("user"))
|
||||||
self.assertEqual(
|
self.assertEqual(
|
||||||
f"codex mcp add supervise --url {_URL}",
|
"/home/node/.codex/packages/standalone/current/bin/codex "
|
||||||
|
f"mcp add supervise --url {_URL}",
|
||||||
script,
|
script,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
@@ -19,7 +19,7 @@ from bot_bottle.egress import (
|
|||||||
egress_sidecar_env_entries,
|
egress_sidecar_env_entries,
|
||||||
egress_token_env_map,
|
egress_token_env_map,
|
||||||
)
|
)
|
||||||
from bot_bottle.log import Die
|
from bot_bottle.errors import MissingEnvVarError
|
||||||
from bot_bottle.manifest import ManifestIndex
|
from bot_bottle.manifest import ManifestIndex
|
||||||
from bot_bottle.yaml_subset import parse_yaml_subset
|
from bot_bottle.yaml_subset import parse_yaml_subset
|
||||||
|
|
||||||
@@ -499,14 +499,14 @@ class TestResolveTokenValues(unittest.TestCase):
|
|||||||
self.assertEqual({"EGRESS_TOKEN_0": "the-value"}, out)
|
self.assertEqual({"EGRESS_TOKEN_0": "the-value"}, out)
|
||||||
|
|
||||||
def test_missing_token_ref_dies(self):
|
def test_missing_token_ref_dies(self):
|
||||||
with self.assertRaises(Die):
|
with self.assertRaises(MissingEnvVarError):
|
||||||
egress_resolve_token_values(
|
egress_resolve_token_values(
|
||||||
{"EGRESS_TOKEN_0": "GH_PAT"},
|
{"EGRESS_TOKEN_0": "GH_PAT"},
|
||||||
{},
|
{},
|
||||||
)
|
)
|
||||||
|
|
||||||
def test_empty_token_ref_dies(self):
|
def test_empty_token_ref_dies(self):
|
||||||
with self.assertRaises(Die):
|
with self.assertRaises(MissingEnvVarError):
|
||||||
egress_resolve_token_values(
|
egress_resolve_token_values(
|
||||||
{"EGRESS_TOKEN_0": "GH_PAT"},
|
{"EGRESS_TOKEN_0": "GH_PAT"},
|
||||||
{"GH_PAT": ""},
|
{"GH_PAT": ""},
|
||||||
|
|||||||
@@ -19,6 +19,7 @@ from bot_bottle.git_gate import (
|
|||||||
_resolve_identity_file,
|
_resolve_identity_file,
|
||||||
git_gate_upstreams_for_bottle,
|
git_gate_upstreams_for_bottle,
|
||||||
)
|
)
|
||||||
|
from bot_bottle.errors import MissingEnvVarError
|
||||||
from bot_bottle.manifest import ManifestIndex
|
from bot_bottle.manifest import ManifestIndex
|
||||||
from tests.fixtures import fixture_minimal, fixture_with_git
|
from tests.fixtures import fixture_minimal, fixture_with_git
|
||||||
|
|
||||||
@@ -217,6 +218,16 @@ class TestHookRender(unittest.TestCase):
|
|||||||
self.assertIn("supervisor approved # gitleaks:allow", hook)
|
self.assertIn("supervisor approved # gitleaks:allow", hook)
|
||||||
self.assertIn("supervisor rejected # gitleaks:allow", hook)
|
self.assertIn("supervisor rejected # gitleaks:allow", hook)
|
||||||
|
|
||||||
|
def test_inline_gitleaks_allow_python_imports_work_in_sidecar_layout(self):
|
||||||
|
hook = git_gate_render_hook()
|
||||||
|
# The sidecar image copies supervise.py flat under /app, while
|
||||||
|
# host-side tests import it through the bot_bottle package.
|
||||||
|
# Hooks execute from the bare repo directory, so the embedded
|
||||||
|
# Python must include /app and support both import layouts.
|
||||||
|
self.assertIn('PYTHONPATH="/app${PYTHONPATH:+:$PYTHONPATH}"', hook)
|
||||||
|
self.assertIn("import supervise as _sv", hook)
|
||||||
|
self.assertIn("from bot_bottle import supervise as _sv", hook)
|
||||||
|
|
||||||
def test_inline_gitleaks_allow_fails_closed_without_supervisor(self):
|
def test_inline_gitleaks_allow_fails_closed_without_supervisor(self):
|
||||||
hook = git_gate_render_hook()
|
hook = git_gate_render_hook()
|
||||||
self.assertIn(
|
self.assertIn(
|
||||||
@@ -411,7 +422,7 @@ class TestDynamicKeyProvisioning(unittest.TestCase):
|
|||||||
def test_revoke_missing_token_raises(self):
|
def test_revoke_missing_token_raises(self):
|
||||||
bottle = self._gitea_manifest().bottles["dev"]
|
bottle = self._gitea_manifest().bottles["dev"]
|
||||||
(self.stage / "repo-deploy-key-id").write_text("123\n")
|
(self.stage / "repo-deploy-key-id").write_text("123\n")
|
||||||
with patch.dict("os.environ", {}, clear=True), self.assertRaises(RuntimeError) as cm:
|
with patch.dict("os.environ", {}, clear=True), self.assertRaises(MissingEnvVarError) as cm:
|
||||||
revoke_git_gate_provisioned_keys(bottle, self.stage)
|
revoke_git_gate_provisioned_keys(bottle, self.stage)
|
||||||
self.assertIn("env var is not set", str(cm.exception))
|
self.assertIn("env var is not set", str(cm.exception))
|
||||||
|
|
||||||
|
|||||||
@@ -13,6 +13,7 @@ from pathlib import Path
|
|||||||
from typing import Any, cast
|
from typing import Any, cast
|
||||||
from unittest.mock import patch
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
from bot_bottle.errors import MissingEnvVarError
|
||||||
from bot_bottle.git_gate import (
|
from bot_bottle.git_gate import (
|
||||||
_gitconfig_validate_value,
|
_gitconfig_validate_value,
|
||||||
_provision_dynamic_key,
|
_provision_dynamic_key,
|
||||||
@@ -120,7 +121,7 @@ class TestProvisionDynamicKey(unittest.TestCase):
|
|||||||
patch.dict("os.environ", {}, clear=False):
|
patch.dict("os.environ", {}, clear=False):
|
||||||
import os
|
import os
|
||||||
os.environ.pop("GITEA_TOK", None)
|
os.environ.pop("GITEA_TOK", None)
|
||||||
with self.assertRaises(RuntimeError):
|
with self.assertRaises(MissingEnvVarError):
|
||||||
_provision_dynamic_key(_gitea_entry(), "s", Path(d))
|
_provision_dynamic_key(_gitea_entry(), "s", Path(d))
|
||||||
|
|
||||||
|
|
||||||
@@ -166,7 +167,7 @@ class TestRevokeProvisionedKeys(unittest.TestCase):
|
|||||||
import os
|
import os
|
||||||
os.environ.pop("GITEA_TOK", None)
|
os.environ.pop("GITEA_TOK", None)
|
||||||
(Path(d) / "repo-deploy-key-id").write_text("kid123")
|
(Path(d) / "repo-deploy-key-id").write_text("kid123")
|
||||||
with self.assertRaises(RuntimeError):
|
with self.assertRaises(MissingEnvVarError):
|
||||||
revoke_git_gate_provisioned_keys(_bottle(_gitea_entry()), Path(d))
|
revoke_git_gate_provisioned_keys(_bottle(_gitea_entry()), Path(d))
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -30,6 +30,19 @@ class TestMacosContainerBottle(unittest.TestCase):
|
|||||||
argv,
|
argv,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
def test_agent_argv_accepts_absolute_provider_command(self):
|
||||||
|
command = "/home/node/.codex/packages/standalone/current/bin/codex"
|
||||||
|
bottle = MacosContainerBottle(
|
||||||
|
"bot-bottle-dev-abc",
|
||||||
|
lambda: None,
|
||||||
|
None,
|
||||||
|
agent_command=command,
|
||||||
|
)
|
||||||
|
with patch.dict(bottle_mod.os.environ, {}, clear=True):
|
||||||
|
argv = bottle.agent_argv(["run"])
|
||||||
|
self.assertIn(command, argv)
|
||||||
|
self.assertNotIn("codex", argv)
|
||||||
|
|
||||||
def test_agent_argv_includes_workdir(self):
|
def test_agent_argv_includes_workdir(self):
|
||||||
bottle = MacosContainerBottle(
|
bottle = MacosContainerBottle(
|
||||||
"bot-bottle-dev-abc",
|
"bot-bottle-dev-abc",
|
||||||
|
|||||||
@@ -139,7 +139,7 @@ class TestMacosContainerLaunchArgv(unittest.TestCase):
|
|||||||
argv,
|
argv,
|
||||||
)
|
)
|
||||||
self.assertIn(
|
self.assertIn(
|
||||||
"type=bind,source=/state/bot-bottle.db,target=/run/supervise/bot-bottle.db",
|
"type=bind,source=/state,target=/run/supervise",
|
||||||
argv,
|
argv,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
@@ -9,7 +9,11 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import unittest
|
import unittest
|
||||||
|
|
||||||
from bot_bottle.manifest import ManifestBottle, ManifestIndex
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
from bot_bottle.env import resolve_env
|
||||||
|
from bot_bottle.errors import MissingEnvVarError
|
||||||
|
from bot_bottle.manifest import Manifest, ManifestBottle, ManifestIndex
|
||||||
from bot_bottle.manifest_agent import (
|
from bot_bottle.manifest_agent import (
|
||||||
ManifestAgent,
|
ManifestAgent,
|
||||||
ManifestAgentProvider,
|
ManifestAgentProvider,
|
||||||
@@ -238,5 +242,26 @@ class TestEagerIndexLookups(unittest.TestCase):
|
|||||||
self.assertIsNone(m.git_identity_summary())
|
self.assertIsNone(m.git_identity_summary())
|
||||||
|
|
||||||
|
|
||||||
|
class TestResolveEnv(unittest.TestCase):
|
||||||
|
"""resolve_env raises MissingEnvVarError when an interpolated entry's
|
||||||
|
host var is unset."""
|
||||||
|
|
||||||
|
def _manifest(self, env_dict: dict[str, str]) -> Manifest:
|
||||||
|
idx = ManifestIndex.from_json_obj({
|
||||||
|
"bottles": {"dev": {"env": env_dict}},
|
||||||
|
"agents": {"demo": {"bottle": "dev"}},
|
||||||
|
})
|
||||||
|
return idx.load_for_agent("demo")
|
||||||
|
|
||||||
|
def test_missing_interpolated_host_var_raises(self):
|
||||||
|
manifest = self._manifest({"API_KEY": "${HOST_API_KEY}"})
|
||||||
|
with patch.dict("os.environ", {}, clear=False):
|
||||||
|
import os
|
||||||
|
os.environ.pop("HOST_API_KEY", None)
|
||||||
|
with self.assertRaises(MissingEnvVarError) as cm:
|
||||||
|
resolve_env(manifest)
|
||||||
|
self.assertIn("HOST_API_KEY", str(cm.exception))
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
unittest.main()
|
unittest.main()
|
||||||
|
|||||||
@@ -79,6 +79,11 @@ class TestClaudeArgvWrapped(unittest.TestCase):
|
|||||||
"--",
|
"--",
|
||||||
"runuser", "-u", "node", "--",
|
"runuser", "-u", "node", "--",
|
||||||
"env", "HOME=/home/node", "USER=node",
|
"env", "HOME=/home/node", "USER=node",
|
||||||
|
(
|
||||||
|
"PATH=/home/node/.local/bin:"
|
||||||
|
"/home/node/.codex/packages/standalone/current/bin:"
|
||||||
|
"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
|
||||||
|
),
|
||||||
"claude",
|
"claude",
|
||||||
],
|
],
|
||||||
argv,
|
argv,
|
||||||
@@ -127,16 +132,24 @@ class TestClaudeArgvWrapped(unittest.TestCase):
|
|||||||
self.assertIn("HTTPS_PROXY=http://127.0.0.1:1234", argv)
|
self.assertIn("HTTPS_PROXY=http://127.0.0.1:1234", argv)
|
||||||
self.assertIn("NO_PROXY=localhost", argv)
|
self.assertIn("NO_PROXY=localhost", argv)
|
||||||
|
|
||||||
|
def test_guest_env_path_overrides_default_path(self):
|
||||||
|
argv = _unwrap(_bottle(None, PATH="/custom/bin").agent_argv([]))
|
||||||
|
self.assertIn("PATH=/custom/bin", argv)
|
||||||
|
self.assertFalse(any(
|
||||||
|
item.startswith("PATH=/home/node/.local/bin")
|
||||||
|
for item in argv
|
||||||
|
))
|
||||||
|
|
||||||
def test_runuser_switch_precedes_claude(self):
|
def test_runuser_switch_precedes_claude(self):
|
||||||
# The dashboard's `_build_resume_argv_with_fallback` finds
|
# The dashboard's `_build_resume_argv_with_fallback` finds
|
||||||
# the `claude` token to split exec-framing from the claude
|
# the `claude` token to split exec-framing from the claude
|
||||||
# tail. `runuser -u node --` must sit on the prefix side so
|
# tail. `runuser -u node --` must sit on the prefix side so
|
||||||
# the shell wrap inherits the UID switch.
|
# the shell wrap inherits the UID switch.
|
||||||
argv = _bottle().agent_argv([])
|
argv = _bottle().agent_argv([])
|
||||||
agent_idx = argv.index("claude")
|
runuser_idx = argv.index("runuser")
|
||||||
self.assertEqual(
|
self.assertEqual(
|
||||||
["runuser", "-u", "node", "--", "env"],
|
["runuser", "-u", "node", "--", "env"],
|
||||||
argv[agent_idx - 7:agent_idx - 2],
|
argv[runuser_idx:runuser_idx + 5],
|
||||||
)
|
)
|
||||||
|
|
||||||
def test_pi_provider_appends_system_prompt_without_print_mode(self):
|
def test_pi_provider_appends_system_prompt_without_print_mode(self):
|
||||||
|
|||||||
@@ -94,6 +94,10 @@ class TestEnsureSmolmachine(unittest.TestCase):
|
|||||||
calls.append(name)
|
calls.append(name)
|
||||||
return _f
|
return _f
|
||||||
|
|
||||||
|
def save_and_record(image_ref: str, path: str) -> None:
|
||||||
|
Path(path).touch()
|
||||||
|
calls.append("save")
|
||||||
|
|
||||||
with patch.object(
|
with patch.object(
|
||||||
_launch_mod.docker_mod, "build_image",
|
_launch_mod.docker_mod, "build_image",
|
||||||
side_effect=record("build"),
|
side_effect=record("build"),
|
||||||
@@ -102,7 +106,7 @@ class TestEnsureSmolmachine(unittest.TestCase):
|
|||||||
return_value=f"sha256:{digest}fffffffffffffffff",
|
return_value=f"sha256:{digest}fffffffffffffffff",
|
||||||
), patch.object(
|
), patch.object(
|
||||||
_launch_mod.docker_mod, "save",
|
_launch_mod.docker_mod, "save",
|
||||||
side_effect=record("save"),
|
side_effect=save_and_record,
|
||||||
) as save, patch.object(
|
) as save, patch.object(
|
||||||
_launch_mod, "ephemeral_registry",
|
_launch_mod, "ephemeral_registry",
|
||||||
return_value=_Reg(),
|
return_value=_Reg(),
|
||||||
|
|||||||
@@ -8,6 +8,7 @@ inspecting running bundle containers' port bindings."""
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import json
|
import json
|
||||||
|
import os
|
||||||
import sqlite3
|
import sqlite3
|
||||||
import subprocess
|
import subprocess
|
||||||
import tempfile
|
import tempfile
|
||||||
@@ -112,9 +113,16 @@ class TestEnsurePool(unittest.TestCase):
|
|||||||
|
|
||||||
|
|
||||||
class TestAllocate(unittest.TestCase):
|
class TestAllocate(unittest.TestCase):
|
||||||
def test_returns_loopback_on_linux(self):
|
def test_per_bottle_alias_on_linux(self):
|
||||||
with patch.object(loopback_alias, "_is_macos", return_value=False):
|
# Linux gets the same per-bottle scoping as macOS (127/8 is
|
||||||
self.assertEqual("127.0.0.1", loopback_alias.allocate("demo"))
|
# already loopback, so no ifconfig is needed). A fresh host
|
||||||
|
# with no running bundles allocates the first pool entry.
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
lock_path = Path(tmp) / "smolmachines.lock"
|
||||||
|
with patch.object(loopback_alias, "_is_macos", return_value=False), \
|
||||||
|
patch.object(loopback_alias, "_ALLOC_LOCK_PATH", lock_path), \
|
||||||
|
patch.object(loopback_alias, "_aliases_in_use", return_value=set()):
|
||||||
|
self.assertEqual("127.0.0.16", loopback_alias.allocate("demo"))
|
||||||
|
|
||||||
def test_picks_lowest_unused_on_macos(self):
|
def test_picks_lowest_unused_on_macos(self):
|
||||||
# No bundles running -> first pool entry.
|
# No bundles running -> first pool entry.
|
||||||
@@ -166,12 +174,25 @@ class TestAllocateLock(unittest.TestCase):
|
|||||||
|
|
||||||
self.assertIn(fcntl_mod.LOCK_EX, flock_calls)
|
self.assertIn(fcntl_mod.LOCK_EX, flock_calls)
|
||||||
|
|
||||||
def test_no_lock_on_linux(self):
|
def test_acquires_exclusive_lock_on_linux(self):
|
||||||
# Linux early-returns before touching the lock file.
|
# Linux allocates per-bottle too, so it must take the same
|
||||||
|
# lock to serialise concurrent launches.
|
||||||
|
import fcntl as fcntl_mod
|
||||||
|
flock_calls: list[int] = []
|
||||||
|
|
||||||
|
def record_flock(fd, op): # type: ignore
|
||||||
|
flock_calls.append(op)
|
||||||
|
|
||||||
|
with tempfile.TemporaryDirectory() as tmp:
|
||||||
|
lock_path = Path(tmp) / "smolmachines.lock"
|
||||||
with patch.object(loopback_alias, "_is_macos", return_value=False), \
|
with patch.object(loopback_alias, "_is_macos", return_value=False), \
|
||||||
patch.object(loopback_alias.fcntl, "flock") as flock:
|
patch.object(loopback_alias, "_ALLOC_LOCK_PATH", lock_path), \
|
||||||
|
patch.object(loopback_alias, "_aliases_in_use", return_value=set()), \
|
||||||
|
patch.object(loopback_alias.fcntl, "flock",
|
||||||
|
side_effect=record_flock):
|
||||||
loopback_alias.allocate("demo")
|
loopback_alias.allocate("demo")
|
||||||
flock.assert_not_called()
|
|
||||||
|
self.assertIn(fcntl_mod.LOCK_EX, flock_calls)
|
||||||
|
|
||||||
def test_sequential_allocations_with_shared_lock_are_serialised(self):
|
def test_sequential_allocations_with_shared_lock_are_serialised(self):
|
||||||
# Two sequential calls share the same lock file. The second
|
# Two sequential calls share the same lock file. The second
|
||||||
@@ -241,10 +262,12 @@ class TestAliasInUseDetection(unittest.TestCase):
|
|||||||
|
|
||||||
|
|
||||||
class TestForceAllowlist(unittest.TestCase):
|
class TestForceAllowlist(unittest.TestCase):
|
||||||
"""Smolvm 0.8.0 silently drops `--allow-cidr` with `--from`,
|
"""Smolvm 0.8.0 silently drops `--allow-cidr` with `--from`, so
|
||||||
so `force_allowlist` opens the state DB directly and sets
|
`force_allowlist` opens the state DB directly and sets the row's
|
||||||
the row's `allowed_cidrs` field. Round-trip tests against a
|
`allowed_cidrs` field — on both macOS and Linux. It is
|
||||||
real SQLite DB to lock down the BLOB encoding."""
|
fail-closed: it dies rather than launching a VM whose allowlist
|
||||||
|
it can't confirm. Round-trip tests against a real SQLite DB to
|
||||||
|
lock down the BLOB encoding."""
|
||||||
|
|
||||||
def setUp(self):
|
def setUp(self):
|
||||||
self._tmp = tempfile.TemporaryDirectory(prefix="smolvm-db.")
|
self._tmp = tempfile.TemporaryDirectory(prefix="smolvm-db.")
|
||||||
@@ -290,17 +313,67 @@ class TestForceAllowlist(unittest.TestCase):
|
|||||||
self.assertEqual(4, cfg["cpus"])
|
self.assertEqual(4, cfg["cpus"])
|
||||||
self.assertTrue(cfg["network"])
|
self.assertTrue(cfg["network"])
|
||||||
|
|
||||||
def test_noop_on_linux(self):
|
def test_patches_on_linux_too(self):
|
||||||
|
# force_allowlist no longer no-ops on Linux — the TSI
|
||||||
|
# allowlist must be enforced there as well.
|
||||||
with patch.object(loopback_alias, "_is_macos", return_value=False), \
|
with patch.object(loopback_alias, "_is_macos", return_value=False), \
|
||||||
patch.object(loopback_alias, "_SMOLVM_DB_PATH", self.db):
|
patch.object(loopback_alias, "_SMOLVM_DB_PATH", self.db):
|
||||||
loopback_alias.force_allowlist("demo-vm", ["127.0.0.16/32"])
|
loopback_alias.force_allowlist("demo-vm", ["127.0.0.16/32"])
|
||||||
# DB row should be untouched.
|
|
||||||
con = sqlite3.connect(str(self.db))
|
con = sqlite3.connect(str(self.db))
|
||||||
cfg = json.loads(con.execute(
|
cfg = json.loads(con.execute(
|
||||||
"SELECT data FROM vms WHERE name='demo-vm'",
|
"SELECT data FROM vms WHERE name='demo-vm'",
|
||||||
).fetchone()[0])
|
).fetchone()[0])
|
||||||
con.close()
|
con.close()
|
||||||
self.assertIsNone(cfg["allowed_cidrs"])
|
self.assertEqual(["127.0.0.16/32"], cfg["allowed_cidrs"])
|
||||||
|
|
||||||
|
def test_skips_write_when_already_matching(self):
|
||||||
|
# A newer smolvm that honors --allow-cidr at create leaves the
|
||||||
|
# row already correct; force_allowlist must not rewrite it. We
|
||||||
|
# detect a no-write by comparing the raw BLOB byte-for-byte
|
||||||
|
# (a rewrite re-serialises the JSON, changing key order/bytes
|
||||||
|
# is not guaranteed, but mtime/identity isn't observable — so
|
||||||
|
# we assert the stored bytes are exactly what we pre-seeded).
|
||||||
|
seeded = json.dumps({
|
||||||
|
"name": "demo-vm", "cpus": 4, "mem": 8192,
|
||||||
|
"network": True, "allowed_cidrs": ["127.0.0.16/32"],
|
||||||
|
}).encode()
|
||||||
|
con = sqlite3.connect(str(self.db))
|
||||||
|
con.execute(
|
||||||
|
"UPDATE vms SET data=? WHERE name='demo-vm'",
|
||||||
|
(sqlite3.Binary(seeded),),
|
||||||
|
)
|
||||||
|
con.commit()
|
||||||
|
con.close()
|
||||||
|
|
||||||
|
with patch.object(loopback_alias, "_is_macos", return_value=True), \
|
||||||
|
patch.object(loopback_alias, "_SMOLVM_DB_PATH", self.db):
|
||||||
|
loopback_alias.force_allowlist("demo-vm", ["127.0.0.16/32"])
|
||||||
|
|
||||||
|
con = sqlite3.connect(str(self.db))
|
||||||
|
stored = con.execute(
|
||||||
|
"SELECT data FROM vms WHERE name='demo-vm'").fetchone()[0]
|
||||||
|
con.close()
|
||||||
|
self.assertEqual(seeded, bytes(stored))
|
||||||
|
|
||||||
|
def test_dies_when_patch_does_not_take(self):
|
||||||
|
# If the persisted allowlist still doesn't match after the
|
||||||
|
# patch (e.g. wrong schema / smolvm stores it elsewhere),
|
||||||
|
# force_allowlist must fail closed rather than boot the VM.
|
||||||
|
original = loopback_alias._read_machine_cfg
|
||||||
|
|
||||||
|
def stale_cfg(con: sqlite3.Connection, name: str) -> dict[str, object]:
|
||||||
|
# Always report the un-patched row so the post-write
|
||||||
|
# verification never sees the requested cidrs.
|
||||||
|
cfg = original(con, name)
|
||||||
|
cfg["allowed_cidrs"] = None
|
||||||
|
return cfg
|
||||||
|
|
||||||
|
with patch.object(loopback_alias, "_is_macos", return_value=True), \
|
||||||
|
patch.object(loopback_alias, "_SMOLVM_DB_PATH", self.db), \
|
||||||
|
patch.object(loopback_alias, "_read_machine_cfg", side_effect=stale_cfg), \
|
||||||
|
patch.object(loopback_alias, "die", side_effect=SystemExit("die")):
|
||||||
|
with self.assertRaises(SystemExit):
|
||||||
|
loopback_alias.force_allowlist("demo-vm", ["127.0.0.16/32"])
|
||||||
|
|
||||||
def test_dies_on_missing_db(self):
|
def test_dies_on_missing_db(self):
|
||||||
with patch.object(loopback_alias, "_is_macos", return_value=True), \
|
with patch.object(loopback_alias, "_is_macos", return_value=True), \
|
||||||
@@ -323,5 +396,35 @@ class TestForceAllowlist(unittest.TestCase):
|
|||||||
loopback_alias.force_allowlist("not-in-db", ["127.0.0.16/32"])
|
loopback_alias.force_allowlist("not-in-db", ["127.0.0.16/32"])
|
||||||
|
|
||||||
|
|
||||||
|
class TestSmolvmDbPath(unittest.TestCase):
|
||||||
|
"""The smolvm state-DB path is platform-derived: Application
|
||||||
|
Support on macOS, XDG data dir on Linux."""
|
||||||
|
|
||||||
|
def test_macos_path(self):
|
||||||
|
with patch.object(loopback_alias.platform, "system", return_value="Darwin"):
|
||||||
|
p = loopback_alias._smolvm_db_path()
|
||||||
|
self.assertEqual(
|
||||||
|
("Library", "Application Support", "smolvm", "server", "smolvm.db"),
|
||||||
|
p.parts[-5:],
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_linux_default_xdg_path(self):
|
||||||
|
env = {k: v for k, v in os.environ.items() if k != "XDG_DATA_HOME"}
|
||||||
|
with patch.object(loopback_alias.platform, "system", return_value="Linux"), \
|
||||||
|
patch.dict(loopback_alias.os.environ, env, clear=True):
|
||||||
|
p = loopback_alias._smolvm_db_path()
|
||||||
|
self.assertEqual(
|
||||||
|
(".local", "share", "smolvm", "server", "smolvm.db"),
|
||||||
|
p.parts[-5:],
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_linux_respects_xdg_data_home(self):
|
||||||
|
with patch.object(loopback_alias.platform, "system", return_value="Linux"), \
|
||||||
|
patch.dict(loopback_alias.os.environ,
|
||||||
|
{"XDG_DATA_HOME": "/custom/data"}, clear=False):
|
||||||
|
p = loopback_alias._smolvm_db_path()
|
||||||
|
self.assertEqual(Path("/custom/data/smolvm/server/smolvm.db"), p)
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
unittest.main()
|
unittest.main()
|
||||||
|
|||||||
@@ -276,8 +276,8 @@ class TestProvisionCA(unittest.TestCase):
|
|||||||
str(self.egress_ca),
|
str(self.egress_ca),
|
||||||
AGENT_CA_PATH,
|
AGENT_CA_PATH,
|
||||||
)
|
)
|
||||||
bottle.exec.assert_called_once()
|
self.assertEqual(2, bottle.exec.call_count)
|
||||||
script = bottle.exec.call_args.args[0]
|
script = bottle.exec.call_args_list[1].args[0]
|
||||||
self.assertIn("chmod 644", script)
|
self.assertIn("chmod 644", script)
|
||||||
self.assertIn("update-ca-certificates", script)
|
self.assertIn("update-ca-certificates", script)
|
||||||
self.assertEqual("root", bottle.exec.call_args.kwargs.get("user"))
|
self.assertEqual("root", bottle.exec.call_args.kwargs.get("user"))
|
||||||
@@ -506,5 +506,54 @@ class TestProvisionGitUser(unittest.TestCase):
|
|||||||
self.assertIn("bot@example.com", calls[0][0])
|
self.assertIn("bot@example.com", calls[0][0])
|
||||||
|
|
||||||
|
|
||||||
|
class TestProxyHost(unittest.TestCase):
|
||||||
|
"""_proxy_host returns the bridge gateway on Linux and the loopback
|
||||||
|
alias on other platforms."""
|
||||||
|
|
||||||
|
def test_linux_returns_bundle_gateway(self):
|
||||||
|
plan = _plan()
|
||||||
|
with patch("bot_bottle.backend.smolmachines.launch.platform.system",
|
||||||
|
return_value="Linux"):
|
||||||
|
result = _launch._proxy_host(plan, "127.0.0.16")
|
||||||
|
self.assertEqual(plan.bundle_gateway, result)
|
||||||
|
|
||||||
|
def test_non_linux_returns_loopback(self):
|
||||||
|
plan = _plan()
|
||||||
|
with patch("bot_bottle.backend.smolmachines.launch.platform.system",
|
||||||
|
return_value="Darwin"):
|
||||||
|
result = _launch._proxy_host(plan, "127.0.0.16")
|
||||||
|
self.assertEqual("127.0.0.16", result)
|
||||||
|
|
||||||
|
|
||||||
|
class TestDiscoverUrls(unittest.TestCase):
|
||||||
|
"""_discover_urls stamps git-gate host + supervise URL into the plan."""
|
||||||
|
|
||||||
|
def test_git_gate_host_set_when_upstreams_present(self):
|
||||||
|
plan = _plan()
|
||||||
|
plan = replace(
|
||||||
|
plan,
|
||||||
|
git_gate_plan=replace(
|
||||||
|
plan.git_gate_plan,
|
||||||
|
upstreams=(GitGateUpstream(
|
||||||
|
name="bot-bottle",
|
||||||
|
upstream_url="ssh://git@host/repo.git",
|
||||||
|
upstream_host="host",
|
||||||
|
upstream_port="22",
|
||||||
|
identity_file="/tmp/key",
|
||||||
|
known_host_key="",
|
||||||
|
),),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
with patch.object(_launch._bundle, "bundle_host_port", return_value="9420"):
|
||||||
|
stamped = _launch._discover_urls(plan, "127.0.0.16")
|
||||||
|
self.assertEqual("127.0.0.16:9420", stamped.agent_git_gate_host)
|
||||||
|
|
||||||
|
def test_supervise_url_set_when_supervise_present(self):
|
||||||
|
plan = _plan(supervise=True)
|
||||||
|
with patch.object(_launch._bundle, "bundle_host_port", return_value="55556"):
|
||||||
|
stamped = _launch._discover_urls(plan, "127.0.0.16")
|
||||||
|
self.assertEqual("http://127.0.0.16:55556/", stamped.agent_supervise_url)
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
unittest.main()
|
unittest.main()
|
||||||
|
|||||||
@@ -79,7 +79,7 @@ class TestArgvShapes(unittest.TestCase):
|
|||||||
with self._patch_run() as m:
|
with self._patch_run() as m:
|
||||||
machine_create("agent-xyz")
|
machine_create("agent-xyz")
|
||||||
self.assertEqual(
|
self.assertEqual(
|
||||||
["smolvm", "machine", "create", "agent-xyz"],
|
["smolvm", "machine", "create", "--name", "agent-xyz"],
|
||||||
m.call_args.args[0],
|
m.call_args.args[0],
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -103,7 +103,8 @@ class TestArgvShapes(unittest.TestCase):
|
|||||||
self.assertIn("192.168.50.2/32", argv)
|
self.assertIn("192.168.50.2/32", argv)
|
||||||
self.assertIn("-e", argv)
|
self.assertIn("-e", argv)
|
||||||
self.assertIn("HTTPS_PROXY=http://192.168.50.2:8888", argv)
|
self.assertIn("HTTPS_PROXY=http://192.168.50.2:8888", argv)
|
||||||
self.assertEqual("agent-xyz", argv[-1])
|
self.assertIn("--name", argv)
|
||||||
|
self.assertIn("agent-xyz", argv)
|
||||||
|
|
||||||
def test_machine_create_omits_net_when_no_allow_cidrs(self):
|
def test_machine_create_omits_net_when_no_allow_cidrs(self):
|
||||||
with self._patch_run() as m:
|
with self._patch_run() as m:
|
||||||
@@ -127,13 +128,13 @@ class TestArgvShapes(unittest.TestCase):
|
|||||||
m.call_args.args[0],
|
m.call_args.args[0],
|
||||||
)
|
)
|
||||||
|
|
||||||
def test_machine_delete_uses_positional_name_and_force(self):
|
def test_machine_delete_uses_name_flag_and_force(self):
|
||||||
# delete NAME is positional; -f required so no interactive
|
# delete uses --name flag; -f required so no interactive
|
||||||
# confirmation blocks teardown.
|
# confirmation blocks teardown.
|
||||||
with self._patch_run() as m:
|
with self._patch_run() as m:
|
||||||
machine_delete("agent-xyz")
|
machine_delete("agent-xyz")
|
||||||
self.assertEqual(
|
self.assertEqual(
|
||||||
["smolvm", "machine", "delete", "-f", "agent-xyz"],
|
["smolvm", "machine", "delete", "--name", "agent-xyz", "-f"],
|
||||||
m.call_args.args[0],
|
m.call_args.args[0],
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
@@ -56,9 +56,14 @@ class TestBundleSubnet(unittest.TestCase):
|
|||||||
|
|
||||||
class TestPreflight(unittest.TestCase):
|
class TestPreflight(unittest.TestCase):
|
||||||
def test_smolvm_present_returns_none(self):
|
def test_smolvm_present_returns_none(self):
|
||||||
|
# Pin macOS so the Linux KVM gate doesn't fire on a CI runner
|
||||||
|
# (ubuntu, no /dev/kvm) — this test isolates the PATH check.
|
||||||
with patch(
|
with patch(
|
||||||
"bot_bottle.backend.smolmachines.util.shutil.which",
|
"bot_bottle.backend.smolmachines.util.shutil.which",
|
||||||
return_value="/usr/local/bin/smolvm",
|
return_value="/usr/local/bin/smolvm",
|
||||||
|
), patch(
|
||||||
|
"bot_bottle.backend.smolmachines.util.platform.system",
|
||||||
|
return_value="Darwin",
|
||||||
):
|
):
|
||||||
self.assertIsNone(smolmachines_preflight())
|
self.assertIsNone(smolmachines_preflight())
|
||||||
|
|
||||||
@@ -88,5 +93,63 @@ class TestPreflight(unittest.TestCase):
|
|||||||
self.assertIn("BOT_BOTTLE_BACKEND=docker", msg)
|
self.assertIn("BOT_BOTTLE_BACKEND=docker", msg)
|
||||||
|
|
||||||
|
|
||||||
|
class TestKvmPreflight(unittest.TestCase):
|
||||||
|
"""Linux-only KVM gate: smolvm needs /dev/kvm present and
|
||||||
|
accessible. macOS skips this entirely (Hypervisor.framework)."""
|
||||||
|
|
||||||
|
def _run(self, *, system: str, exists: bool, access: bool) -> None:
|
||||||
|
with patch(
|
||||||
|
"bot_bottle.backend.smolmachines.util.shutil.which",
|
||||||
|
return_value="/usr/bin/smolvm",
|
||||||
|
), patch(
|
||||||
|
"bot_bottle.backend.smolmachines.util.platform.system",
|
||||||
|
return_value=system,
|
||||||
|
), patch(
|
||||||
|
"bot_bottle.backend.smolmachines.util.os.path.exists",
|
||||||
|
return_value=exists,
|
||||||
|
), patch(
|
||||||
|
"bot_bottle.backend.smolmachines.util.os.access",
|
||||||
|
return_value=access,
|
||||||
|
):
|
||||||
|
return smolmachines_preflight()
|
||||||
|
|
||||||
|
def test_macos_skips_kvm_check(self):
|
||||||
|
# Even with /dev/kvm absent, macOS must not run the gate.
|
||||||
|
self.assertIsNone(self._run(system="Darwin", exists=False, access=False))
|
||||||
|
|
||||||
|
def test_linux_ok_returns_none(self):
|
||||||
|
self.assertIsNone(self._run(system="Linux", exists=True, access=True))
|
||||||
|
|
||||||
|
def test_linux_missing_device_dies(self):
|
||||||
|
with self.assertRaises(SystemExit):
|
||||||
|
self._run(system="Linux", exists=False, access=False)
|
||||||
|
|
||||||
|
def test_linux_no_access_dies(self):
|
||||||
|
with self.assertRaises(SystemExit):
|
||||||
|
self._run(system="Linux", exists=True, access=False)
|
||||||
|
|
||||||
|
def test_linux_missing_device_message(self):
|
||||||
|
import io
|
||||||
|
import sys
|
||||||
|
captured = io.StringIO()
|
||||||
|
with patch.object(sys, "stderr", captured):
|
||||||
|
with self.assertRaises(SystemExit):
|
||||||
|
self._run(system="Linux", exists=False, access=False)
|
||||||
|
msg = captured.getvalue()
|
||||||
|
self.assertIn("/dev/kvm", msg)
|
||||||
|
self.assertIn("kvm-intel", msg)
|
||||||
|
|
||||||
|
def test_linux_no_access_message(self):
|
||||||
|
import io
|
||||||
|
import sys
|
||||||
|
captured = io.StringIO()
|
||||||
|
with patch.object(sys, "stderr", captured):
|
||||||
|
with self.assertRaises(SystemExit):
|
||||||
|
self._run(system="Linux", exists=True, access=False)
|
||||||
|
msg = captured.getvalue()
|
||||||
|
self.assertIn("kvm", msg)
|
||||||
|
self.assertIn("group", msg)
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
unittest.main()
|
unittest.main()
|
||||||
|
|||||||
Reference in New Issue
Block a user