Compare commits

..

17 Commits

Author SHA1 Message Date
didericis-claude 784c7ac152 fix(lint): remove unused BottleImages imports from two test files
lint / lint (push) Successful in 2m5s
test / unit (pull_request) Failing after 1m0s
test / integration (pull_request) Successful in 18s
test / coverage (pull_request) Failing after 1m2s
pyright strict reportUnusedImport flagged BottleImages in
test_docker_launch_committed_image.py and test_macos_container_launch.py;
neither file references the type by name (they only use the returned
value's attributes).
2026-07-10 20:57:00 +00:00
didericis-claude 4a4d3b2455 fix: remove redundant quoted annotations in BottleImages and abstract methods
`from __future__ import annotations` already defers all annotation
evaluation, so quoting `str | Path`, `BottleImages` inside the same
module was redundant and tripped pyright strict mode.
2026-07-10 20:57:00 +00:00
didericis-claude 715e474306 refactor: BottleImages dataclass, prelaunch_checks, build_or_load_images
Addresses review comments 3098, 3099, 3100 on PR #336:

- Add BottleImages(agent, sidecar) dataclass to backend/__init__.py.
  Docker/macOS backends use str image refs; smolmachines uses Path
  artifacts. Replaces the singular `image` variable from the canonical
  pattern in comment 3100.

- Replace _image_stale_checks/skip_stale with public prelaunch_checks().
  CLI now calls backend.prelaunch_checks(plan) before backend.launch(plan);
  if StaleImageError is raised and the operator confirms, launch proceeds
  without re-checking. Removes the while-True/skip_stale retry loop.

- Add abstract _build_or_load_images(plan) -> BottleImages to
  BottleBackend. launch() calls it then passes images to _launch_impl.
  Each backend implements both methods.

- Fix comment 3098 (macos-container): _build_images is removed.
  build_or_load_images() has separate fresh/cached code paths — the
  cached path never calls a build helper.

- Update _start_bundle (smolmachines) to accept sidecar_artifact: Path
  directly. Sidecar artifact resolution moves to _sidecar_from_path(),
  called by build_or_load_images alongside _agent_from_path().
2026-07-10 20:57:00 +00:00
didericis-claude ecd260b5ef fix(tests): resolve pyright errors in stale-check test files
Remove unused imports, add missing type annotations, fix Die()
constructor calls (int code, not str), replace fake backend
subclass with patch.object approach to avoid reportMissingParameterType
and override-incompatibility errors in strict pyright mode.
2026-07-10 20:57:00 +00:00
didericis-claude 7302ed6518 test: add unit tests for stale-image checks to reach ≥90% diff-coverage
Covers StaleImageError / check_stale / check_stale_path, the
BottleBackend.launch template method (skip_stale flag), stale_checks
functions across docker / smolmachines / macos-container backends,
_build_images cached paths in the macOS backend, image_created_at edge
cases in both docker and container util modules, the CLI stale loop
(headless die, interactive decline, interactive confirm + retry), and
ConfigStore.cached_image_stale_warning_days fallback paths.

Diff-coverage: 488/507 changed lines covered (96.3%).
2026-07-10 20:57:00 +00:00
didericis-claude 220f92af5b refactor(image-cache): replace warn_if_stale with StaleImageError; add launch template
- image_cache: StaleImageError exception + check_stale/check_stale_path (raise instead of warn)
- BottleBackend.launch: template method (skip_stale flag) that calls _image_stale_checks then _launch_impl
- Each backend: _image_stale_checks delegates to a stale_checks() function in its launch module; _launch_impl replaces launch override
- macos_container: adds image_created_at to util, cached-image support in _build_images, stale_checks
- cli/start.py: catches StaleImageError, prompts interactively, retries with skip_stale=True; headless mode dies on it
2026-07-10 20:57:00 +00:00
didericis-claude 6a086acb4a fix: pyright error in test_compose — annotate kwargs as dict[str, Any] 2026-07-10 20:57:00 +00:00
didericis-codex c4f16c168c fix: make config store schema explicit 2026-07-10 20:57:00 +00:00
didericis-codex d5ac323d9c Add cached image quickstart 2026-07-10 20:57:00 +00:00
didericis 9c4400cce2 fix(smolmachines): consolidate virtiofs mounts to stay within libkrun limit
lint / lint (push) Successful in 2m1s
test / unit (pull_request) Failing after 54s
test / integration (pull_request) Successful in 17s
test / coverage (pull_request) Failing after 54s
libkrun limits total mounts + port-mappings to 5. With all three
sidecar daemons active (egress, git-gate, supervise), the prior
layout used 3 mounts + 3 ports = 6, causing the VM to crash at boot
with krun_start_enter returned -22.

Merge the egress confdir and git-gate scripts into a single staging
directory mounted at /bot-bottle-data/ with egress/ and git-gate/
subdirectories, reducing to 2 mounts + 3 ports = 5. Also clean up
leftover VMs before creating new ones to handle interrupted teardowns.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-07-10 16:14:03 -04:00
didericis 62d2e86e7e fix(smolmachines): mount git-gate files at boot via virtiofs staging dirs
Two issues caused krun_start_enter -22 (VM boot crash):

1. git-gate entrypoint missing at boot: sidecar_init.py starts all
   daemons (including git-gate) immediately as PID 1. The entrypoint
   was copied in via machine_cp which only runs after machine_start
   returns — too late. virtiofs also can't mount files at root (/).

   Fix: bake a static /git-gate-entrypoint.sh wrapper into the
   Dockerfile.sidecars image that delegates to /etc/git-gate/entrypoint.sh.
   For smolmachines, stage per-bottle scripts with correct in-VM names
   under the git-gate state dir and virtiofs-mount to /etc/git-gate/ at
   VM creation time. docker/macOS backends are unchanged (their file
   bind-mount/docker cp still overrides the static wrapper).

2. Egress confdir mounted read-only: egress_entrypoint.sh writes
   combined-trust.pem to confdir under set -e; mitmproxy also needs
   to write its per-host cert cache there. Both fail fatally on a
   read-only virtiofs mount.

   Fix: change the egress confdir virtiofs mount to writable.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-10 16:14:03 -04:00
didericis-codex 2b53c36608 refactor(smolmachines): drop obsolete docker-sidecar helpers
test / unit (pull_request) Successful in 1m0s
test / integration (pull_request) Successful in 17s
test / coverage (pull_request) Successful in 1m8s
2026-07-10 15:31:41 -04:00
didericis-codex bb9cca48fd refactor(smolmachines): inline sidecar proxy host 2026-07-10 15:31:41 -04:00
didericis-codex 5828668e21 test(smolmachines): cover sidecar vm launch paths 2026-07-10 15:31:41 -04:00
didericis-codex 869a8bbc1f feat: run smolmachines sidecar as vm 2026-07-10 15:31:41 -04:00
didericis-codex 4863e81cd3 docs: draft smolmachines sidecar vm prd 2026-07-10 15:31:41 -04:00
didericis-claude 814c7338a1 docs(research): update landscape doc — SuperHQ, multi-backend, multi-provider, in-flight directions
- Broaden scope from "Claude Code in Docker" to "AI coding agents in isolated sandboxes"
- Add SuperHQ (superhq.ai, v0.4.4) as a new adjacent-competitor entry: macOS-only
  microVM desktop app, overlaps on isolation/credential-proxy/multi-provider but has
  no manifest layer and no audit logging
- Note SuperHQ's user-voiced audit gap (Brian Cheong, Dunialabs.io) and that
  bot-bottle already covers it
- Update differentiation list to reflect three backends (Docker, Apple container,
  smolmachines) and three built-in providers (Claude Code, Codex, Pi) plus plugin system
- Add in-flight directions for forge-native dispatch (#317) and paid web control plane
  (#327) with honest framing: lifecycle concept is not novel vs. cloud services (Devin,
  Copilot Workspace); differentiation is self-hosted + manifest-driven + stronger isolation
2026-07-09 18:55:15 +00:00
25 changed files with 519 additions and 796 deletions
+19 -4
View File
@@ -14,9 +14,10 @@
# /app/supervise_server.py + .py supervise MCP server
# /app/sidecar_init.py PID 1 supervisor
# /etc/egress/routes.yaml bind-mounted at run time
# /etc/git-gate/pre-receive docker-cp'd at start time
# /git-gate-entrypoint.sh docker-cp'd at start time
# /git-gate/creds/* docker-cp'd at start time
# /etc/git-gate/entrypoint.sh per-bottle (docker-cp or virtiofs mount)
# /etc/git-gate/pre-receive per-bottle (docker-cp or virtiofs mount)
# /git-gate-entrypoint.sh static wrapper → /etc/git-gate/entrypoint.sh
# /git-gate/creds/* per-bottle (docker-cp or virtiofs mount)
# /git/* bare repos, populated at runtime
# /run/supervise/bot-bottle.db bind-mounted at run time
# /home/mitmproxy/.mitmproxy/ mitmproxy CA dir
@@ -88,7 +89,21 @@ RUN mkdir -p \
/git-gate/creds \
/git \
/run/supervise \
/home/mitmproxy/.mitmproxy
/home/mitmproxy/.mitmproxy \
/bot-bottle-data/egress \
/bot-bottle-data/git-gate
# Static wrapper for the git-gate entrypoint. The per-bottle
# entrypoint script is either:
# - docker-cp'd to /git-gate-entrypoint.sh (docker/macOS backends),
# which overwrites this wrapper; or
# - virtiofs-mounted at /bot-bottle-data/git-gate/entrypoint.sh
# (smolmachines), where this wrapper delegates to it at runtime.
# Fallback to /etc/git-gate/ for backwards compatibility.
# Either way sidecar_init.py calls `/bin/sh /git-gate-entrypoint.sh`.
RUN printf '#!/bin/sh\nif [ -x /bot-bottle-data/git-gate/entrypoint.sh ]; then\n exec /bot-bottle-data/git-gate/entrypoint.sh "$@"\nfi\nexec /etc/git-gate/entrypoint.sh "$@"\n' \
> /git-gate-entrypoint.sh \
&& chmod 755 /git-gate-entrypoint.sh
# Documentation only — the compose renderer publishes whichever
# subset the bottle uses.
+28 -15
View File
@@ -276,6 +276,18 @@ PlanT = TypeVar("PlanT", bound=BottlePlan)
CleanupT = TypeVar("CleanupT", bound=BottleCleanupPlan)
@dataclass(frozen=True)
class BottleImages:
"""Resolved image references (or artifact paths) for a bottle launch.
For Docker/macOS-container backends, `agent` and `sidecar` are string
image refs. For the smolmachines backend they are Path objects pointing
to pre-built `.smolmachine` artifacts."""
agent: str | Path
sidecar: str | Path
class BottleBackend(ABC, Generic[PlanT, CleanupT]):
"""Abstract base for selectable bottle backends. Concrete subclasses
(e.g. DockerBottleBackend) own their own prepare/launch impls.
@@ -435,26 +447,27 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
prompt file, Dockerfile path, and guest home all live on
`agent_provision_plan` — the source of truth."""
def prelaunch_checks(self, plan: PlanT) -> None:
"""Raise StaleImageError if any cached image used by this plan is stale.
No-op default; backends override to call the shared check_stale*
helpers on their image/artifact timestamps. Called by the CLI before
launch so the operator can be prompted outside the launch context."""
@contextmanager
def launch(
self, plan: PlanT, *, skip_stale: bool = False
) -> Generator[Bottle, None, None]:
"""Template: optionally check for stale cached images, then delegate
to `_launch_impl`. Pass `skip_stale=True` to bypass the stale check
(used by the interactive CLI after the operator confirms)."""
if not skip_stale:
self._image_stale_checks(plan)
with self._launch_impl(plan) as bottle:
def launch(self, plan: PlanT) -> Generator[Bottle, None, None]:
"""Template: build or load images, then delegate to _launch_impl."""
images = self._build_or_load_images(plan)
with self._launch_impl(plan, images) as bottle:
yield bottle
def _image_stale_checks(self, plan: PlanT) -> None:
"""Raise StaleImageError if any cached image used by this plan is stale.
No-op default; backends override to call the shared `check_stale*`
helpers on their image/artifact timestamps."""
@abstractmethod
def _build_or_load_images(self, plan: PlanT) -> BottleImages:
"""Return the agent and sidecar image references (or artifact paths)
for this plan, building fresh images when the policy requires it."""
@abstractmethod
def _launch_impl(self, plan: PlanT) -> AbstractContextManager[Bottle]:
"""Build/run the bottle and yield a handle; tear down on exit."""
def _launch_impl(self, plan: PlanT, images: BottleImages) -> AbstractContextManager[Bottle]:
"""Bring up the bottle using pre-resolved images; yield a handle; tear down on exit."""
def provision(self, plan: PlanT, bottle: "Bottle") -> str | None:
"""Copy host-side files (CA cert, prompt, skills, .git) into
+7 -4
View File
@@ -31,7 +31,7 @@ from ...env import ResolvedEnv
from ...git_gate import GitGatePlan
from ...supervise import SupervisePlan
from ...manifest import Manifest
from .. import ActiveAgent, BottleBackend, BottleSpec
from .. import ActiveAgent, BottleBackend, BottleImages, BottleSpec
from . import cleanup as _cleanup
from . import enumerate as _enumerate
from . import launch as _launch
@@ -85,12 +85,15 @@ class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanup
stage_dir=stage_dir,
)
def _image_stale_checks(self, plan: DockerBottlePlan) -> None:
def prelaunch_checks(self, plan: DockerBottlePlan) -> None:
_launch.stale_checks(plan)
def _build_or_load_images(self, plan: DockerBottlePlan) -> BottleImages:
return _launch.build_or_load_images(plan)
@contextmanager
def _launch_impl(self, plan: DockerBottlePlan) -> Generator[DockerBottle, None, None]:
with _launch.launch(plan, provision=self.provision) as bottle:
def _launch_impl(self, plan: DockerBottlePlan, images: BottleImages) -> Generator[DockerBottle, None, None]:
with _launch.launch(plan, images, provision=self.provision) as bottle:
yield bottle
def supervise_mcp_url(self, plan: DockerBottlePlan) -> str:
+37 -32
View File
@@ -43,6 +43,7 @@ from ...git_gate import (
)
from ...image_cache import check_stale
from ...log import die, info, warn
from .. import BottleImages
from . import network as network_mod
from . import util as docker_mod
from .bottle import DockerBottle
@@ -71,16 +72,50 @@ from .sidecar_bundle import SIDECAR_BUNDLE_IMAGE
_REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent)
def build_or_load_images(plan: DockerBottlePlan) -> BottleImages:
"""Resolve the agent and sidecar image refs for this plan.
Returns the committed snapshot if one exists, the local cached images
when the policy is 'cached', or builds fresh images and returns those."""
committed = read_committed_image(plan.slug)
if committed and docker_mod.image_exists(committed):
info(f"using committed image {committed!r}")
return BottleImages(agent=committed, sidecar=SIDECAR_BUNDLE_IMAGE)
if plan.spec.image_policy == "cached":
if not docker_mod.image_exists(plan.image):
die(
f"cached agent image {plan.image!r} not found; "
"run without --cached-images to build it"
)
if not docker_mod.image_exists(SIDECAR_BUNDLE_IMAGE):
die(
f"cached sidecar image {SIDECAR_BUNDLE_IMAGE!r} not found; "
"run without --cached-images to build it"
)
info(f"using cached agent image {plan.image!r}")
info(f"using cached sidecar image {SIDECAR_BUNDLE_IMAGE!r}")
return BottleImages(agent=plan.image, sidecar=SIDECAR_BUNDLE_IMAGE)
docker_mod.build_image(plan.image, _REPO_DIR, dockerfile=plan.dockerfile_path)
return BottleImages(agent=plan.image, sidecar=SIDECAR_BUNDLE_IMAGE)
@contextmanager
def launch(
plan: DockerBottlePlan,
images: BottleImages,
*,
provision: Callable[[DockerBottlePlan, "DockerBottle"], str | None],
) -> Generator[DockerBottle, None, None]:
"""Build, launch, and provision a Docker bottle via compose.
Teardown on exit."""
"""Launch and provision a Docker bottle via compose. Teardown on exit."""
stack = ExitStack()
# Stamp the resolved agent image ref into the plan so compose rendering
# picks up the right image (may be a committed snapshot or cached ref).
plan = dataclasses.replace(
plan,
agent_provision=dataclasses.replace(plan.agent_provision, image=str(images.agent)),
)
_bottle_for_revoke = plan.manifest.bottle
_git_gate_dir_for_revoke = git_gate_state_dir(plan.slug)
@@ -97,36 +132,6 @@ def launch(
)
try:
# Step 1: agent image. Use a committed snapshot when one exists
# and is present in the local daemon; otherwise build from the
# Dockerfile. Sidecar images get built lazily by `docker compose
# up` via the renderer's `build:` directives.
committed = read_committed_image(plan.slug)
cached_policy = plan.spec.image_policy == "cached"
if committed and docker_mod.image_exists(committed):
info(f"using committed image {committed!r}")
plan = dataclasses.replace(
plan,
agent_provision=dataclasses.replace(plan.agent_provision, image=committed),
)
elif cached_policy:
if not docker_mod.image_exists(plan.image):
die(
f"cached agent image {plan.image!r} not found; "
"run without --cached-images to build it"
)
if not docker_mod.image_exists(SIDECAR_BUNDLE_IMAGE):
die(
f"cached sidecar image {SIDECAR_BUNDLE_IMAGE!r} not found; "
"run without --cached-images to build it"
)
info(f"using cached agent image {plan.image!r}")
info(f"using cached sidecar image {SIDECAR_BUNDLE_IMAGE!r}")
else:
docker_mod.build_image(
plan.image, _REPO_DIR,
dockerfile=plan.dockerfile_path,
)
internal_network = network_mod.network_name_for_slug(plan.slug)
egress_network = network_mod.network_egress_name_for_slug(plan.slug)
@@ -12,7 +12,7 @@ from ...env import ResolvedEnv
from ...git_gate import GitGatePlan
from ...supervise import SupervisePlan
from ...manifest import Manifest
from .. import ActiveAgent, BottleBackend, BottleSpec
from .. import ActiveAgent, BottleBackend, BottleImages, BottleSpec
from . import cleanup as _cleanup
from . import enumerate as _enumerate
from . import launch as _launch
@@ -67,14 +67,17 @@ class MacosContainerBottleBackend(
stage_dir=stage_dir,
)
def _image_stale_checks(self, plan: MacosContainerBottlePlan) -> None:
def prelaunch_checks(self, plan: MacosContainerBottlePlan) -> None:
_launch.stale_checks(plan)
def _build_or_load_images(self, plan: MacosContainerBottlePlan) -> BottleImages:
return _launch.build_or_load_images(plan)
@contextmanager
def _launch_impl(
self, plan: MacosContainerBottlePlan
self, plan: MacosContainerBottlePlan, images: BottleImages
) -> Generator[MacosContainerBottle, None, None]:
with _launch.launch(plan, provision=self.provision) as bottle:
with _launch.launch(plan, images, provision=self.provision) as bottle:
yield bottle
def prepare_cleanup(self) -> MacosContainerBottleCleanupPlan:
+36 -31
View File
@@ -34,6 +34,7 @@ from ...git_gate import (
)
from ...image_cache import check_stale
from ...log import die, info, warn
from .. import BottleImages
from ...supervise import DB_PATH_IN_CONTAINER, SUPERVISE_PORT
from ...util import expand_tilde
from ..docker.egress import EGRESS_CA_IN_CONTAINER, EGRESS_PORT
@@ -72,17 +73,51 @@ def sidecar_container_name(slug: str) -> str:
return f"bot-bottle-sidecars-{slug}"
def build_or_load_images(plan: MacosContainerBottlePlan) -> BottleImages:
"""Resolve agent and sidecar image refs. Builds the sidecar when needed,
but never builds the agent when policy is 'cached'."""
committed = read_committed_image(plan.slug)
if committed and container_mod.image_exists(committed):
info(f"using committed image {committed!r}")
if plan.spec.image_policy != "cached":
container_mod.build_image(SIDECAR_BUNDLE_IMAGE, _REPO_DIR, dockerfile=SIDECAR_BUNDLE_DOCKERFILE)
return BottleImages(agent=committed, sidecar=SIDECAR_BUNDLE_IMAGE)
if plan.spec.image_policy == "cached":
if not container_mod.image_exists(plan.image):
die(
f"cached agent image {plan.image!r} not found; "
"run without --cached-images to build it"
)
if not container_mod.image_exists(SIDECAR_BUNDLE_IMAGE):
die(
f"cached sidecar image {SIDECAR_BUNDLE_IMAGE!r} not found; "
"run without --cached-images to build it"
)
info(f"using cached agent image {plan.image!r}")
info(f"using cached sidecar image {SIDECAR_BUNDLE_IMAGE!r}")
return BottleImages(agent=plan.image, sidecar=SIDECAR_BUNDLE_IMAGE)
container_mod.build_image(SIDECAR_BUNDLE_IMAGE, _REPO_DIR, dockerfile=SIDECAR_BUNDLE_DOCKERFILE)
container_mod.build_image(plan.image, _REPO_DIR, dockerfile=plan.dockerfile_path)
return BottleImages(agent=plan.image, sidecar=SIDECAR_BUNDLE_IMAGE)
@contextmanager
def launch(
plan: MacosContainerBottlePlan,
images: BottleImages,
*,
provision: Callable[[MacosContainerBottlePlan, "MacosContainerBottle"], str | None],
) -> Generator[MacosContainerBottle, None, None]:
"""Build, run, provision, and yield an Apple Container bottle."""
"""Run, provision, and yield an Apple Container bottle."""
stack = ExitStack()
bottle_for_revoke = plan.manifest.bottle
git_gate_dir_for_revoke = git_gate_state_dir(plan.slug)
plan = dataclasses.replace(
plan,
agent_provision=dataclasses.replace(plan.agent_provision, image=str(images.agent)),
)
def teardown() -> None:
teardown_exc: BaseException | None = None
try:
@@ -96,7 +131,6 @@ def launch(
try:
plan = _mint_certs(plan)
plan = _build_images(plan)
internal_network = internal_network_name(plan.slug)
egress_network = egress_network_name(plan.slug)
@@ -149,35 +183,6 @@ def _mint_certs(plan: MacosContainerBottlePlan) -> MacosContainerBottlePlan:
return dataclasses.replace(plan, egress_plan=egress_plan)
def _build_images(plan: MacosContainerBottlePlan) -> MacosContainerBottlePlan:
cached = plan.spec.image_policy == "cached"
committed = read_committed_image(plan.slug)
if committed and container_mod.image_exists(committed):
info(f"using committed image {committed!r}")
if not cached:
container_mod.build_image(SIDECAR_BUNDLE_IMAGE, _REPO_DIR, dockerfile=SIDECAR_BUNDLE_DOCKERFILE)
return dataclasses.replace(
plan,
agent_provision=dataclasses.replace(plan.agent_provision, image=committed),
)
if cached:
if not container_mod.image_exists(plan.image):
die(
f"cached agent image {plan.image!r} not found; "
"run without --cached-images to build it"
)
if not container_mod.image_exists(SIDECAR_BUNDLE_IMAGE):
die(
f"cached sidecar image {SIDECAR_BUNDLE_IMAGE!r} not found; "
"run without --cached-images to build it"
)
info(f"using cached agent image {plan.image!r}")
info(f"using cached sidecar image {SIDECAR_BUNDLE_IMAGE!r}")
return plan
container_mod.build_image(SIDECAR_BUNDLE_IMAGE, _REPO_DIR, dockerfile=SIDECAR_BUNDLE_DOCKERFILE)
container_mod.build_image(plan.image, _REPO_DIR, dockerfile=plan.dockerfile_path)
return plan
def stale_checks(plan: MacosContainerBottlePlan) -> None:
"""Raise StaleImageError if a cached image is older than the configured
+7 -4
View File
@@ -19,7 +19,7 @@ from ...env import ResolvedEnv
from ...git_gate import GitGatePlan
from ...supervise import SupervisePlan
from ...manifest import Manifest
from .. import ActiveAgent, BottleBackend, BottleSpec
from .. import ActiveAgent, BottleBackend, BottleImages, BottleSpec
from . import cleanup as _cleanup
from . import enumerate as _enumerate
from . import launch as _launch
@@ -77,14 +77,17 @@ class SmolmachinesBottleBackend(
stage_dir=stage_dir,
)
def _image_stale_checks(self, plan: SmolmachinesBottlePlan) -> None:
def prelaunch_checks(self, plan: SmolmachinesBottlePlan) -> None:
_launch.stale_checks(plan)
def _build_or_load_images(self, plan: SmolmachinesBottlePlan) -> BottleImages:
return _launch.build_or_load_images(plan)
@contextmanager
def _launch_impl(
self, plan: SmolmachinesBottlePlan
self, plan: SmolmachinesBottlePlan, images: BottleImages
) -> Generator[SmolmachinesBottle, None, None]:
with _launch.launch(plan, provision=self.provision) as bottle:
with _launch.launch(plan, images, provision=self.provision) as bottle:
yield bottle
def supervise_mcp_url(self, plan: SmolmachinesBottlePlan) -> str:
+10 -23
View File
@@ -1,10 +1,8 @@
"""SmolmachinesBottlePlan — concrete BottlePlan for the smolmachines
backend (PRD 0023).
Slug + bundle docker subnet / gateway / pinned IP + smolvm
machine name + agent `.smolmachine` artifact + per-bottle guest
env. Provisioning fields (CA cert path, prompt path, etc.) land
in chunk 4."""
Slug + legacy bundle network coordinates + smolvm machine name +
agent `.smolmachine` artifact + per-bottle guest env."""
from __future__ import annotations
@@ -23,9 +21,10 @@ class SmolmachinesBottlePlan(BottlePlan):
`supervise_plan`, and `agent_provision` from BottlePlan."""
slug: str
# Per-bottle docker subnet for the sidecar bundle container.
# The bundle runs at `bundle_ip` (always `.2`); the gateway is
# at `.1`. smolvm's TSI allowlist is set to `bundle_ip/32`.
# Legacy per-bottle bundle network coordinates. These remain on
# the plan while BundleLaunchSpec still carries the original shape,
# but the smolmachines launch path exposes the sidecar VM through
# host-loopback forwarders instead of a Docker bridge IP.
bundle_subnet: str
bundle_gateway: str
bundle_ip: str
@@ -36,22 +35,10 @@ class SmolmachinesBottlePlan(BottlePlan):
# `--smolfile` is mutually exclusive with `--from`, and
# `--from` is the path that avoids the registry-pull race).
guest_env: dict[str, str]
# Inner Plans for the sidecar bundle daemons. The same shape the
# docker backend uses — same `.prepare()` calls produced
# them — but our launch step doesn't populate the
# docker-specific network fields (internal_network,
# egress_network) because the smolmachines bundle isn't on
# docker's `--internal` + egress bridge topology; it's on a
# per-bottle bridge with a pinned IP. The unused fields stay
# at their dataclass defaults.
# Agent-side endpoints. On Docker Desktop the docker bridge
# IPs aren't reachable from the smolvm guest (TSI uses macOS
# networking; docker container IPs live in the daemon's VM),
# so the agent dials the bundle via host loopback +
# docker-published random ports. Empty at prepare time;
# launch populates these after bundle bringup via
# `dataclasses.replace`. Format: a `host:port` for git-gate
# (insteadOf URL prefix) + full URLs for proxy / supervise.
# Agent-side endpoints. Empty at prepare time; launch populates
# these after sidecar VM bringup via `dataclasses.replace`.
# Format: a `host:port` for git-gate (insteadOf URL prefix) +
# full URLs for proxy / supervise.
agent_proxy_url: str = ""
agent_git_gate_host: str = ""
agent_supervise_url: str = ""
@@ -7,16 +7,23 @@ exec`` instead of Docker.
from __future__ import annotations
from ...egress import EGRESS_ROUTES_IN_CONTAINER
from pathlib import Path
from ...bottle_state import egress_state_dir
from ...log import warn
from ..egress_apply import EgressApplicator, EgressApplyError
from . import sidecar_bundle as _bundle
from . import smolvm as _smolvm
# Routes file path inside the sidecar VM. Set via EGRESS_ROUTES env var
# at launch so the addon reads from the virtiofs-mounted confdir instead
# of the default /etc/egress/routes.yaml.
_EGRESS_ROUTES_IN_SIDECAR_VM = "/bot-bottle-data/egress/routes.yaml"
def fetch_current_routes(slug: str) -> str:
machine = _bundle.bundle_machine_name(slug)
result = _smolvm.machine_exec(machine, ["cat", EGRESS_ROUTES_IN_CONTAINER])
result = _smolvm.machine_exec(machine, ["cat", _EGRESS_ROUTES_IN_SIDECAR_VM])
if result.returncode != 0:
raise EgressApplyError(
f"could not read routes.yaml from {machine}: "
@@ -26,6 +33,14 @@ def fetch_current_routes(slug: str) -> str:
class SmolmachinesEgressApplicator(EgressApplicator):
@staticmethod
def _routes_path(slug: str) -> Path:
# Routes live in the smolvm-sidecar-data staging dir, which is
# virtiofs-mounted at /bot-bottle-data/ in the sidecar VM. Writes
# here are visible inside the VM immediately; a SIGHUP causes the
# addon to reload from _EGRESS_ROUTES_IN_SIDECAR_VM.
return egress_state_dir(slug) / "smolvm-sidecar-data" / "egress" / "routes.yaml"
def _signal_bundle_reload(self, slug: str) -> None:
machine = _bundle.bundle_machine_name(slug)
result = _smolvm.machine_exec(machine, ["sh", "-c", "kill -HUP 1"])
+138 -60
View File
@@ -1,12 +1,9 @@
"""End-to-end launch flow for the smolmachines backend
(PRD 0023 chunks 2d + 4b).
"""End-to-end launch flow for the smolmachines backend.
Brings up the per-bottle docker bridge + sidecar bundle (with
real daemons + their config files), creates + starts the smolvm
guest pointed at the bundle's pinned IP via TSI's
`--allow-cidr <bundle-ip>/32` allowlist, yields a
`SmolmachinesBottle` handle, tears everything down on context
exit.
Builds the sidecar bundle smolmachine, starts it as a sidecar VM
with real daemons + their config files, creates + starts the agent
smolVM, yields a `SmolmachinesBottle` handle, and tears everything
down on context exit.
The bundle's daemons consume the inner Plans the docker backend
already produces: egress reads routes + CAs from the EgressPlan.
@@ -17,12 +14,12 @@ from __future__ import annotations
import dataclasses
import os
import shutil
from contextlib import ExitStack, contextmanager
from pathlib import Path
from typing import Callable, Generator
from ...egress import (
EGRESS_ROUTES_IN_CONTAINER,
egress_agent_env_entries,
egress_resolve_token_values,
egress_sidecar_env_entries,
@@ -31,16 +28,9 @@ from ...supervise import DB_PATH_IN_CONTAINER, SUPERVISE_PORT
from ...util import expand_tilde
from ..docker import util as docker_mod
from ..docker.egress import (
EGRESS_CA_IN_CONTAINER,
EGRESS_PORT as _EGRESS_PORT,
egress_tls_init,
)
from ..docker.git_gate import (
GIT_GATE_ACCESS_HOOK_IN_CONTAINER,
GIT_GATE_CREDS_DIR_IN_CONTAINER,
GIT_GATE_ENTRYPOINT_IN_CONTAINER,
GIT_GATE_HOOK_IN_CONTAINER,
)
from ...git_gate import (
provision_git_gate_dynamic_keys,
revoke_git_gate_provisioned_keys,
@@ -52,6 +42,7 @@ from ...bottle_state import (
git_gate_state_dir,
read_committed_image,
)
from .. import BottleImages
from . import loopback_alias as _loopback
from . import port_forward as _forward
from . import sidecar_bundle as _bundle
@@ -65,6 +56,18 @@ from .local_registry import crane_push_tarball, ephemeral_registry
_REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent)
# Single virtiofs mount for egress + git-gate files. libkrun limits
# the total of mounts + port-mappings to 5; with 3 daemon ports the
# sidecar VM can carry at most 2 mounts. Egress CA/routes and
# git-gate scripts/creds are staged into subdirectories of one host
# dir and mounted here. Env vars (EGRESS_CONFDIR, EGRESS_ROUTES,
# and the Dockerfile's git-gate wrapper) point each daemon at its
# subdirectory.
_SIDECAR_DATA_DIR_IN_VM = "/bot-bottle-data"
_EGRESS_CONFDIR_IN_VM = f"{_SIDECAR_DATA_DIR_IN_VM}/egress"
_GIT_GATE_SCRIPTS_DIR_IN_VM = f"{_SIDECAR_DATA_DIR_IN_VM}/git-gate"
# Per-host cache for `smolvm pack create` outputs. Keyed by the
# docker image ID so a Dockerfile change automatically invalidates
# the cache. `pack create` is idempotent on the smolvm side but
@@ -73,20 +76,38 @@ _SMOLMACHINE_CACHE_DIR = Path.home() / ".cache" / "bot-bottle" / "smolmachines"
# Container-internal listening ports for each bundle daemon. The
# bundle publishes each one on a random host loopback port (see
# `_bundle.start_bundle`), and `_bundle.bundle_host_port` looks
# them up post-start.
# sidecar VM publishes each one on a random host loopback port, and
# the launch flow wraps those raw ports with per-bottle forwarders.
_GIT_HTTP_PORT = 9420
_SUPERVISE_PORT = SUPERVISE_PORT
def build_or_load_images(plan: SmolmachinesBottlePlan) -> BottleImages:
"""Return pre-built or freshly built smolmachine artifact paths."""
return BottleImages(
agent=_agent_from_path(plan),
sidecar=_sidecar_from_path(plan),
)
def _sidecar_from_path(plan: SmolmachinesBottlePlan) -> Path:
"""Return the sidecar bundle artifact path, building it if needed."""
if _image_policy(plan) == "cached":
return _cached_smolmachine(_bundle.SIDECAR_BUNDLE_IMAGE, label="sidecar")
return _ensure_smolmachine(
_bundle.SIDECAR_BUNDLE_IMAGE,
dockerfile=_bundle.SIDECAR_BUNDLE_DOCKERFILE,
)
@contextmanager
def launch(
plan: SmolmachinesBottlePlan,
images: BottleImages,
*,
provision: Callable[[SmolmachinesBottlePlan, "SmolmachinesBottle"], str | None],
) -> Generator[SmolmachinesBottle, None, None]:
"""Build + run the bottle and yield a handle; tear everything
"""Run the bottle from pre-built images and yield a handle; tear everything
down on exit. Errors during bringup unwind any partial state
via the ExitStack."""
stack = ExitStack()
@@ -94,11 +115,9 @@ def launch(
loopback_ip, network = _allocate_resources(plan, stack)
plan = _mint_certs(plan)
proxy_host = loopback_ip
plan = _start_bundle(plan, network, proxy_host, stack)
plan = _start_bundle(plan, network, proxy_host, Path(images.sidecar), stack)
agent_from_path = _agent_from_path(plan)
_launch_vm(plan, agent_from_path, proxy_host, stack)
_launch_vm(plan, Path(images.agent), proxy_host, stack)
_init_vm(plan)
bottle = SmolmachinesBottle(
@@ -175,24 +194,18 @@ def _start_bundle(
plan: SmolmachinesBottlePlan,
network: str,
proxy_host: str,
sidecar_artifact: Path,
stack: ExitStack,
) -> SmolmachinesBottlePlan:
"""Build the BundleLaunchSpec, start the sidecar VM, wrap its raw
smolVM-published loopback ports with per-bottle forwarders, stamp
agent URLs from those forwarder ports, and register teardown."""
"""Build the BundleLaunchSpec, start the sidecar VM from the pre-resolved
artifact, wrap its raw smolVM-published loopback ports with per-bottle
forwarders, stamp agent URLs from those forwarder ports, and register teardown."""
plan = _provision_git_gate_keys(plan)
bundle_spec = _bundle_launch_spec(plan, network, proxy_host)
token_env = _resolve_token_env(plan, dict(os.environ))
if _image_policy(plan) == "cached":
artifact = _cached_smolmachine(bundle_spec.image, label="sidecar")
else:
artifact = _ensure_smolmachine(
bundle_spec.image,
dockerfile=_bundle.SIDECAR_BUNDLE_DOCKERFILE,
)
launch = _bundle.start_bundle_vm(
bundle_spec,
from_path=artifact,
from_path=sidecar_artifact,
host_env={**os.environ, **token_env},
)
stack.callback(_bundle.stop_bundle_vm, plan.slug)
@@ -299,6 +312,16 @@ def _launch_vm(
fails closed if it can't. Smolfile isn't usable here smolvm 0.8.0
makes --from and --smolfile mutually exclusive."""
tsi_cidr = f"{proxy_host}/32"
# Destroy any leftover machine from a previous run that didn't
# clean up (e.g. crash, interrupted teardown).
try:
_smolvm.machine_stop(plan.machine_name)
except _smolvm.SmolvmError:
pass
try:
_smolvm.machine_delete(plan.machine_name)
except _smolvm.SmolvmError:
pass
_smolvm.machine_create(
plan.machine_name,
from_path=agent_from_path,
@@ -366,6 +389,64 @@ def _port_for_label(label: str) -> int:
raise ValueError(f"unknown sidecar forward label: {label}")
def _stage_sidecar_data(plan: SmolmachinesBottlePlan) -> Path:
"""Stage egress + git-gate files into one virtiofs-mountable dir.
libkrun limits total mounts + port-mappings to 5. With 3 daemon
ports the sidecar VM can carry at most 2 mounts (the second is
the supervise DB). Egress and git-gate share a single mount:
<staging>/egress/ _EGRESS_CONFDIR_IN_VM
<staging>/git-gate/ _GIT_GATE_SCRIPTS_DIR_IN_VM
The mount is writable so mitmproxy can write combined-trust.pem
and cache per-host certs under the egress subdir."""
staging = egress_state_dir(plan.slug) / "smolvm-sidecar-data"
# --- egress subdir ---
confdir = staging / "egress"
confdir.mkdir(parents=True, exist_ok=True)
ep = plan.egress_plan
shutil.copy2(str(ep.mitmproxy_ca_host_path), str(confdir / "mitmproxy-ca.pem"))
if ep.routes:
shutil.copy2(str(ep.routes_path), str(confdir / "routes.yaml"))
# --- git-gate subdir (only when upstreams are configured) ---
gp = plan.git_gate_plan
if gp.upstreams:
scripts_dir = staging / "git-gate"
scripts_dir.mkdir(parents=True, exist_ok=True)
shutil.copy2(str(gp.entrypoint_script), str(scripts_dir / "entrypoint.sh"))
shutil.copy2(str(gp.hook_script), str(scripts_dir / "pre-receive"))
shutil.copy2(str(gp.access_hook_script), str(scripts_dir / "access-hook"))
for name in ("entrypoint.sh", "pre-receive", "access-hook"):
(scripts_dir / name).chmod(0o755)
# Patch credential paths: the rendered entrypoint hardcodes
# /git-gate/creds/; rewrite to the in-VM git-gate subdir.
ep_path = scripts_dir / "entrypoint.sh"
ep_path.write_text(
ep_path.read_text().replace(
"/git-gate/creds/",
f"{_GIT_GATE_SCRIPTS_DIR_IN_VM}/creds/",
)
)
creds_dir = scripts_dir / "creds"
creds_dir.mkdir(exist_ok=True)
for u in gp.upstreams:
keypath = Path(expand_tilde(u.identity_file))
dest_key = creds_dir / f"{u.name}-key"
shutil.copy2(str(keypath), str(dest_key))
dest_key.chmod(0o600)
if u.known_hosts_file:
dest_kh = creds_dir / f"{u.name}-known_hosts"
shutil.copy2(str(u.known_hosts_file), str(dest_kh))
dest_kh.chmod(0o600)
return staging
def _bundle_launch_spec(
plan: SmolmachinesBottlePlan, network: str, proxy_host: str,
) -> _bundle.BundleLaunchSpec:
@@ -384,35 +465,26 @@ def _bundle_launch_spec(
env: list[str] = []
volumes: list[tuple[str, str, bool]] = []
# --- egress -----------------------------------------------
# --- egress + git-gate (single mount) ---------------------
# Stage both into one dir and mount it at _SIDECAR_DATA_DIR_IN_VM.
# libkrun limits mounts + port-mappings to 5; with 3 daemon ports
# we can carry at most 2 mounts (this one + supervise DB).
# Writable so egress_entrypoint.sh can write combined-trust.pem
# and mitmproxy can create its per-host cert cache.
ep = plan.egress_plan
volumes.append((str(ep.mitmproxy_ca_host_path), EGRESS_CA_IN_CONTAINER, True))
if ep.routes:
volumes.append((str(ep.routes_path.parent), str(Path(EGRESS_ROUTES_IN_CONTAINER).parent), True))
gp = plan.git_gate_plan
staging = _stage_sidecar_data(plan)
volumes.append((str(staging), _SIDECAR_DATA_DIR_IN_VM, False))
# Tell the egress entrypoint where to find its CA + routes.
env.append(f"EGRESS_CONFDIR={_EGRESS_CONFDIR_IN_VM}")
# Always set EGRESS_ROUTES so the addon reads from the confdir path
# even when no routes were configured at launch (apply_routes_change
# writes here and a SIGHUP causes the addon to pick them up).
env.append(f"EGRESS_ROUTES={_EGRESS_CONFDIR_IN_VM}/routes.yaml")
env.extend(egress_sidecar_env_entries(ep))
# --- git-gate ---------------------------------------------
gp = plan.git_gate_plan
if gp.upstreams:
daemons += ["git-gate", "git-http"]
volumes += [
(str(gp.entrypoint_script), GIT_GATE_ENTRYPOINT_IN_CONTAINER, True),
(str(gp.hook_script), GIT_GATE_HOOK_IN_CONTAINER, True),
(str(gp.access_hook_script), GIT_GATE_ACCESS_HOOK_IN_CONTAINER, True),
]
for u in gp.upstreams:
keypath = expand_tilde(u.identity_file)
volumes.append((
keypath,
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{u.name}-key",
True,
))
if u.known_hosts_file:
volumes.append((
str(u.known_hosts_file),
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{u.name}-known_hosts",
True,
))
# --- supervise --------------------------------------------
sp = plan.supervise_plan
@@ -423,7 +495,13 @@ def _bundle_launch_spec(
f"SUPERVISE_DB_PATH={DB_PATH_IN_CONTAINER}",
f"SUPERVISE_PORT={SUPERVISE_PORT}",
]
volumes.append((str(sp.db_path), DB_PATH_IN_CONTAINER, False))
# virtiofs requires directory mount — mount the DB's parent
# dir so bot-bottle.db lands at the right in-VM path.
volumes.append((
str(sp.db_path.parent),
str(Path(DB_PATH_IN_CONTAINER).parent),
False,
))
# Container ports the agent reaches from the smolvm guest —
# published on `proxy_host` so the TSI allowlist and the docker
@@ -56,13 +56,11 @@ def resolve_plan(
git_gate_plan: GitGatePlan,
stage_dir: Path,
) -> SmolmachinesBottlePlan:
"""Materialize the smolmachines plan. The bundle's docker
subnet + pinned IP are derived from the slug; the agent's
`.smolmachine` artifact is built (or cache-hit) here so
launch's `machine create --from` boots without a registry
pull. Per-bottle guest env + the TSI allow_cidrs land on the
plan for launch to pass straight through to
`machine create` flags."""
"""Materialize the smolmachines plan. The agent `.smolmachine`
artifact is built (or cache-hit) here so launch's
`machine create --from` boots without a registry pull. Per-bottle
guest env lands on the plan for launch to pass straight through
to `machine create` flags."""
# ==== smolmachines specific setup ====
subnet, gateway, bundle_ip = smolmachines_bundle_subnet(slug)
+17 -162
View File
@@ -1,36 +1,18 @@
"""Per-bottle sidecar bundle bringup for the smolmachines backend
(PRD 0023).
"""Per-bottle sidecar bundle bringup for the smolmachines backend.
Two docker resources per bottle live here:
- **A dedicated bridge network**, subnet derived from the slug.
The bundle container gets a pinned IP at `<subnet>.2` so the
smolvm guest's TSI allowlist (`<bundle-ip>/32`) has a stable
target. Without pinning, we'd have to inspect the container's
assigned IP after start and feed it back into the Smolfile
a race we can sidestep with `--ip`.
- **The bundle container itself**, running the PRD 0024 bundle
image (`bot-bottle-sidecars:latest` by default). Same
image, same daemons, same daemon-private env / bind-mounts
as the docker backend.
This module ships the lifecycle primitives only create
network, start bundle, stop bundle, remove network wrapped
around `subprocess.run(["docker", ...])`. Wiring them into the
launch flow + populating the `BundleLaunchSpec` from the inner
Plans (EgressPlan, ) lands in chunk 2d."""
The sidecar bundle runs as its own smolVM. The agent VM reaches
bundle daemons through host-loopback ports published by that sidecar
VM and wrapped by per-bottle address-bound forwarders."""
from __future__ import annotations
import os
import socket
import subprocess
from dataclasses import dataclass, field
from pathlib import Path
from typing import Sequence
from ...log import die, warn
from ...log import warn
from ..docker import util as docker_mod
from ..docker.sidecar_bundle import (
SIDECAR_BUNDLE_DOCKERFILE,
@@ -78,10 +60,8 @@ class BundleLaunchSpec:
# supervisor inside the bundle reads it to skip
# bottle-irrelevant daemons (e.g. supervise=False bottles).
daemons_csv: str = "egress"
# Plain "KEY=VALUE" strings + "KEY" bare names (the bare-name
# form inherits the value from the docker-run subprocess env,
# matching the docker backend's compose-up secret-forwarding
# pattern).
# Plain "KEY=VALUE" strings + "KEY" bare names. Bare names inherit
# from the host env passed to the sidecar VM launch.
environment: Sequence[str] = field(default_factory=tuple)
# (host_path, container_path, read_only) bind mounts.
volumes: Sequence[tuple[str, str, bool]] = field(default_factory=tuple)
@@ -155,6 +135,16 @@ def start_bundle_vm(
elif entry in effective_host_env:
env[entry] = effective_host_env[entry]
name = bundle_machine_name(spec.slug)
# Destroy any leftover machine from a previous run that didn't
# clean up (e.g. crash, interrupted teardown).
try:
_smolvm.machine_stop(name)
except _smolvm.SmolvmError:
pass
try:
_smolvm.machine_delete(name)
except _smolvm.SmolvmError:
pass
_smolvm.machine_create(
name,
from_path=from_path,
@@ -182,138 +172,3 @@ def stop_bundle_vm(slug: str) -> None:
_smolvm.machine_delete(name)
except _smolvm.SmolvmError as exc:
warn(f"smolvm machine delete {name} failed: {exc}")
def create_bundle_network(network_name: str, subnet: str, gateway: str) -> None:
"""`docker network create` with an explicit subnet + gateway
so the bundle's `--ip` lands on the address the Smolfile's
TSI allowlist points at. Idempotent on the caller's side —
`start_bundle` catches the "network exists" error and treats
it as success (chunk-2d teardown is paired with each create).
"""
result = subprocess.run(
["docker", "network", "create",
"--subnet", subnet, "--gateway", gateway,
network_name],
capture_output=True, text=True, check=False,
)
if result.returncode != 0:
# Already-exists is fine on a resume path; everything else
# is fatal — the bundle won't have an addressable network.
if "already exists" in (result.stderr or "").lower():
return
die(
f"docker network create {network_name} failed: "
f"{(result.stderr or '').strip()}"
)
def remove_bundle_network(network_name: str) -> None:
"""Idempotent: a missing network returns success."""
result = subprocess.run(
["docker", "network", "rm", network_name],
capture_output=True, text=True, check=False,
)
if result.returncode == 0:
return
if "no such network" in (result.stderr or "").lower():
return
# Network with attached containers is the common non-fatal
# case during a partial teardown — warn but don't die.
warn(
f"docker network rm {network_name} failed: "
f"{(result.stderr or '').strip()}"
)
def start_bundle(spec: BundleLaunchSpec, *,
env: dict[str, str] | None = None) -> None:
"""Bring the bundle container up on the per-bottle bridge with
the pinned IP. Argv is built deterministically from `spec`;
`env` is the host subprocess env (forwarded values for any
bare-name entries in `spec.environment`)."""
container = bundle_container_name(spec.slug)
argv = [
"docker", "run",
"--name", container,
"--detach",
"--rm",
"--network", spec.network_name,
"--ip", spec.bundle_ip,
"-e", f"BOT_BOTTLE_SIDECAR_DAEMONS={spec.daemons_csv}",
]
for entry in spec.environment:
argv += ["-e", entry]
for host_path, container_path, read_only in spec.volumes:
suffix = ":ro" if read_only else ""
argv += ["-v", f"{host_path}:{container_path}{suffix}"]
# Loopback-only host port-forwards — the smolvm guest's TSI
# uses macOS networking, and macOS loopback is the only host
# surface that round-trips into Docker Desktop's daemon VM.
# Binds to the per-bottle alias so TSI's IP-only allowlist
# narrows reachability to this bottle's bundle only.
for port in spec.ports_to_publish:
argv += ["-p", f"{spec.publish_host_ip}::{port}"]
argv.append(spec.image)
result = subprocess.run(
argv, capture_output=True, text=True,
env=dict(env) if env is not None else None, check=False,
)
if result.returncode != 0:
die(
f"docker run for bundle {container} failed: "
f"{(result.stderr or '').strip()}"
)
def bundle_host_port(
slug: str, container_port: int, *, host_ip: str = "127.0.0.1",
) -> int:
"""`docker port <bundle> <container_port>/tcp` → the random
host-side port docker assigned for the binding on `host_ip`.
Called after `start_bundle` on each container port listed in
`BundleLaunchSpec.ports_to_publish` so the launch step can
build the agent's HTTPS_PROXY / GIT_GATE / SUPERVISE URLs in
`<host_ip>:<host port>` form."""
container = bundle_container_name(slug)
result = subprocess.run(
["docker", "port", container, f"{container_port}/tcp"],
capture_output=True, text=True, check=False,
)
if result.returncode != 0:
die(
f"docker port {container} {container_port}/tcp failed: "
f"{(result.stderr or '').strip() or '<no stderr>'}"
)
# Each line looks like `127.0.0.16:54321` — one per address
# family / host IP. Match on the expected host_ip prefix so
# bottles bound to per-bottle aliases pick the right line.
for raw in (result.stdout or "").splitlines():
line = raw.strip()
if line.startswith(f"{host_ip}:"):
_, _, port_str = line.rpartition(":")
try:
return int(port_str)
except ValueError:
die(f"unexpected `docker port` output: {line!r}")
die(
f"no port mapping on {host_ip} for {container} "
f"{container_port}/tcp; got: {(result.stdout or '').strip()!r}"
)
def stop_bundle(slug: str) -> None:
"""Idempotent: a missing container returns success."""
container = bundle_container_name(slug)
result = subprocess.run(
["docker", "rm", "-f", container],
capture_output=True, text=True, check=False,
)
if result.returncode == 0:
return
if "no such container" in (result.stderr or "").lower():
return
warn(
f"docker rm -f {container} failed: "
f"{(result.stderr or '').strip()}"
)
+28 -31
View File
@@ -556,38 +556,35 @@ def _launch_bottle(
return 0
backend = get_bottle_backend(backend_name)
skip_stale = False
while True:
try:
with backend.launch(plan, skip_stale=skip_stale) as bottle:
agent_provider_template = getattr(plan, "agent_provider_template", "claude")
extra_args: tuple[str, ...] = ()
if headless_prompt_text:
extra_args = tuple(
get_provider(agent_provider_template).headless_prompt(
headless_prompt_text
)
)
exit_code = attach_agent(
bottle,
agent_provider_template=agent_provider_template,
startup_args=plan.agent_provision.startup_args + extra_args,
try:
backend.prelaunch_checks(plan)
except StaleImageError as exc:
if assume_yes:
die(str(exc))
sys.stderr.write(f"bot-bottle: {exc}\nLaunch anyway? [y/N] ")
sys.stderr.flush()
if read_tty_line() not in ("y", "Y", "yes", "YES"):
return 0
with backend.launch(plan) as bottle:
agent_provider_template = getattr(plan, "agent_provider_template", "claude")
extra_args: tuple[str, ...] = ()
if headless_prompt_text:
extra_args = tuple(
get_provider(agent_provider_template).headless_prompt(
headless_prompt_text
)
info(
f"session ended (exit {exit_code}); "
f"container {bottle.name} will be removed"
)
if agent_provider_template == "claude":
capture_claude_session_state(identity, exit_code)
break
except StaleImageError as exc:
if assume_yes:
die(str(exc))
sys.stderr.write(f"bot-bottle: {exc}\nLaunch anyway? [y/N] ")
sys.stderr.flush()
if read_tty_line() not in ("y", "Y", "yes", "YES"):
break
skip_stale = True
)
exit_code = attach_agent(
bottle,
agent_provider_template=agent_provider_template,
startup_args=plan.agent_provision.startup_args + extra_args,
)
info(
f"session ended (exit {exit_code}); "
f"container {bottle.name} will be removed"
)
if agent_provider_template == "claude":
capture_claude_session_state(identity, exit_code)
return 0
finally:
# PRD 0018 chunk 2: prepare now writes the bottle's bind-mount
+1 -1
View File
@@ -28,7 +28,7 @@ set -e
# flag mitmdump would generate a fresh CA on the wrong path and
# the agent's installed trust anchor would no longer match the
# bumped leaf certs.
CONFDIR=/home/mitmproxy/.mitmproxy
CONFDIR="${EGRESS_CONFDIR:-/home/mitmproxy/.mitmproxy}"
CONFDIR_FLAG="--set confdir=$CONFDIR"
MODE="--mode regular@9099"
+69 -13
View File
@@ -1,14 +1,20 @@
# Landscape: containerized Claude Code agent tools
# Landscape: containerized AI coding agent tools
Research into whether bot-bottle is redundant with existing projects, and
whether it's worth publishing.
## Summary
The "Claude Code in Docker" space is active but not saturated. bot-bottle
occupies a distinct position: no surveyed project combines all five of its
defining features. Publishing is likely worthwhile, with the main risk being
claudebox expanding to absorb the same niche.
The "AI coding agents in isolated sandboxes" space is active but not saturated.
bot-bottle occupies a distinct position: no surveyed project combines all five
of its defining features. Publishing is likely worthwhile, with the main risk
being claudebox expanding to absorb the same niche.
**Updated 2026-07-09:** bot-bottle now supports three isolation backends
(Docker, Apple `container`, smolmachines/libkrun microVMs) and three built-in
agent providers (Claude Code, OpenAI Codex, Pi) with an open plugin system for
arbitrary providers. This meaningfully strengthens the differentiation against
all surveyed competitors.
## Closest competitor: claudebox
@@ -43,28 +49,78 @@ manifest merge.
Still marked early-development.
- **E2B, Northflank, Cloudflare Sandbox SDK** — cloud-hosted SaaS sandbox
runtimes; fundamentally different architecture.
- **superhq.ai / SuperHQ** (v0.4.4, April 2026) — macOS desktop app (Rust/GPUI)
that runs Claude Code, Codex, and Pi inside microVMs via Apple's
Virtualization.framework (their own shuru-sdk / libkrun). Auth gateway
injects API keys on the wire so the sandbox never sees them; tmpfs overlay
stages agent writes for diff-and-accept review; mobile remote access via
remote.superhq.ai. Early alpha, free on launch, Apple Silicon only.
Overlap: both projects cover agent isolation, credential proxying, and
multi-provider support (Claude Code / Codex / Pi). Differences: SuperHQ is a
GUI desktop app with no manifest layer; bot-bottle is a CLI fleet manager with
named agents, skills injection, per-agent system prompts, and cross-platform
backends (Docker, Apple `container`, smolmachines). SuperHQ's microVM
isolation story is now partially matched by bot-bottle's `macos_container` and
smolmachines backends. Worth watching — it targets the same security-minded
power-user audience and moves fast.
**Known gap in SuperHQ (user-requested, as of 2026-07-09):** A named user
(Brian Cheong, Founder, Dunialabs.io) explicitly called out the absence of
per-run audit logging: tool calls and network egress. Bot-bottle covers both:
network egress is logged by pipelock/mitmproxy, and per-run op-log/audit state
is persisted to SQLite.
## What no found project does
None combine:
1. Named-agent JSON manifest with per-agent env resolution (prompt / host-forward / literal)
2. Claude Code skills directory injection
1. Named-agent manifest with per-agent env resolution (prompt / host-forward / literal), supporting multiple providers (Claude Code, Codex, Pi, arbitrary plugins)
2. Skills directory injection
3. Per-agent system prompts
4. SSH-agent key forwarding without copying private keys into the container
5. Home + project manifest merge
6. Pluggable isolation backends: Docker (Linux/macOS), Apple `container` (macOS microVMs), smolmachines/libkrun microVMs
7. Per-run audit log: network egress via pipelock/mitmproxy + op-log persisted to SQLite
**In-flight directions (not yet shipped):**
- **Forge-native dispatch (issue #317):** Gitea webhook → orchestrator spins up a bottle
with the issue body as prompt → agent works → bottle freezes awaiting review comment →
rehydrates on comment → tears down on PR close. The issue-to-PR lifecycle concept is not
novel (Devin, Copilot Workspace, SWE-agent all do this as cloud services); what's
distinct is doing it self-hosted, manifest-driven, inside bot-bottle's isolation
primitives.
- **Paid web control plane (issue #327):** Browser-based multi-host agent launch and
monitoring; account-scoped bottle and agent definitions; secret custody (encrypted at
rest, injected into the sidecar at launch, never exposed to the agent or returned by any
read API). Monetization model: OSS runtime free, control plane paid — a standard split
(HashiCorp, Grafana) applied to a self-hosted agent sandbox. The principled secret
custody model (agent never sees real credentials, even via printenv) is more rigorous
than most surveyed tools but not unprecedented.
## Publishing verdict
Worth publishing. Differentiators that matter to the target audience (power
users running parallel Claude Code sessions with distinct personas/tooling):
users running parallel AI coding agent sessions with distinct personas/tooling):
- The Python-stdlib-first, low-dependency design — competitors are npm-based or
Kubernetes-native.
- The Python-stdlib-first, low-dependency design — competitors are npm-based,
Rust/GUI, or Kubernetes-native.
- Named agents with distinct skills and system prompts, not just language profiles.
- Multi-backend isolation: Docker, Apple `container` microVMs, and
smolmachines/libkrun — single manifest works across all three.
- Multi-provider: Claude Code, Codex, Pi, plus an open plugin system for
arbitrary providers.
- SSH forwarding without key copying.
- Per-run audit log (tool calls + network egress) — an explicitly requested gap
in SuperHQ as of 2026-07-09.
- Forge-native dispatch and a paid control plane (in flight) bring bot-bottle
into the same product category as cloud services like Devin and Copilot
Workspace — but self-hosted, with stronger isolation guarantees and a
manifest-driven fleet model those services don't have.
Main risk: claudebox adds manifest/agent config. The space is moving fast
enough that publishing sooner is better if establishing prior art matters.
Main risk: claudebox adds manifest/agent config; SuperHQ is moving fast on the
GUI / microVM side. The space is moving fast enough that publishing sooner is
better if establishing prior art matters.
Discovery will be slow without active promotion; an Anthropic Discord post or
HN "Show HN" would do most of the work.
@@ -73,4 +129,4 @@ HN "Show HN" would do most of the work.
- GitHub search cannot surface private or very new repos comprehensively.
- Counts (stars, forks) were not confirmed for every project.
- Research conducted 2026-05-07; the space moves fast.
- Initial research conducted 2026-05-07; SuperHQ entry added 2026-07-09; the space moves fast.
@@ -1,111 +0,0 @@
"""Integration: PRD 0023 chunk 2c — bundle bringup on a per-bottle
docker bridge with the pinned IP.
End-to-end against the real docker daemon. Brings up just the
sidecar bundle on its own bridge, confirms the container lands at
the pinned IP, then tears down. Skipped under act_runner (docker
socket mount topology breaks bridge visibility) and when the
bundle image isn't available.
Full launch flow (smolvm + bundle + provisioning + the
localhost-reach / egress-port-bypass probes) lives in chunk 2d."""
from __future__ import annotations
import os
import subprocess
import time
import unittest
from bot_bottle.backend.smolmachines.sidecar_bundle import (
BundleLaunchSpec,
bundle_container_name,
bundle_network_name,
create_bundle_network,
remove_bundle_network,
start_bundle,
stop_bundle,
)
from tests._docker import skip_unless_docker
@skip_unless_docker()
@unittest.skipIf(
os.environ.get("GITEA_ACTIONS") == "true",
"skipped under act_runner: docker socket mount topology breaks "
"in-process visibility of networks created on the host daemon",
)
class TestBundleBringup(unittest.TestCase):
def setUp(self):
self.slug = f"cb-test-bundle-{os.getpid()}-{int(time.time())}"
self.network = bundle_network_name(self.slug)
self.container = bundle_container_name(self.slug)
def tearDown(self):
stop_bundle(self.slug)
remove_bundle_network(self.network)
def _bundle_image_built(self) -> bool:
"""The bundle image (`bot-bottle-sidecars:latest`) is
built lazily by the docker backend's compose. If a
smolmachines-only operator hasn't run the docker backend
first, the image won't exist locally. Skip rather than
fail."""
r = subprocess.run(
["docker", "image", "inspect", "bot-bottle-sidecars:latest"],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL,
check=False,
)
return r.returncode == 0
def test_create_network_then_start_bundle_pins_ip(self):
if not self._bundle_image_built():
self.skipTest(
"bot-bottle-sidecars:latest not built; run a docker "
"bottle first or `docker build -f Dockerfile.sidecars .`"
)
# Pick a subnet unlikely to collide on the host. Last
# octet of the slug hash isn't deterministic across runs;
# we hardcode a high octet (.211) that the docker default
# bridges almost never use.
subnet = "192.168.211.0/24"
gateway = "192.168.211.1"
bundle_ip = "192.168.211.2"
create_bundle_network(self.network, subnet, gateway)
spec = BundleLaunchSpec(
slug=self.slug,
network_name=self.network,
subnet=subnet,
gateway=gateway,
bundle_ip=bundle_ip,
# Empty daemons_csv → init exits "no daemons selected"
# immediately. We just need the container to land on
# the network at the right IP before it exits.
daemons_csv="", # empty → init exits "no daemons selected"
)
start_bundle(spec)
# Inspect the container's IP on the per-bottle network.
r = subprocess.run(
["docker", "inspect",
"--format",
"{{(index .NetworkSettings.Networks \"" + self.network + "\").IPAddress}}",
self.container],
capture_output=True, text=True, check=False,
)
# Container may have exited (no daemons selected → exit 0).
# The inspect still works on exited containers as long as
# `--rm` hasn't fired yet, which is a race. Even if it has,
# the launch succeeded — the container existed, on the
# right network, at the right IP. We don't fail here on
# missing inspect.
if r.returncode == 0 and r.stdout.strip():
self.assertEqual(bundle_ip, r.stdout.strip(),
f"bundle landed at wrong IP: {r.stdout!r}")
if __name__ == "__main__":
unittest.main()
+7 -37
View File
@@ -1,10 +1,9 @@
"""Integration: PRD 0023 chunk 2d — end-to-end launch + exec
round trip + the acceptance probes.
"""Integration: end-to-end smolmachines launch + exec round trip.
The smoke confirms the launch flow (per-bottle docker bridge
sidecar bundle with host-loopback published ports smolvm guest
with TSI allowlist exec) plumbs together end to end. The probes confirm the
security properties the design pivot was about:
The smoke confirms the launch flow (sidecar bundle smolVM
host-loopback forwarders agent smolVM with TSI allowlist exec)
plumbs together end to end. The probes confirm the security
properties the design pivot was about:
- **localhost-reach probe** guest tries to dial a service
bound on the host's `127.0.0.1`. TSI's per-bottle loopback
@@ -14,13 +13,6 @@ security properties the design pivot was about:
the injected `HTTPS_PROXY`/`HTTP_PROXY` URL on the per-bottle
loopback alias, while direct egress with proxy vars unset fails.
- **egress-port-bypass probe** guest tries to dial
`<bundle-ip>:9099` (egress's port). TSI permits the IP but
the bundle's egress daemon binds `127.0.0.1` inside its
container, so the connect refuses at the socket level. The
bind-address mitigation is what closes TSI's port-granularity
gap.
Gated on macOS/Linux + smolvm + docker + not GITEA_ACTIONS the
runner can't host libkrun-backed VMs."""
@@ -114,8 +106,8 @@ class TestSmolmachinesLaunch(unittest.TestCase):
def test_localhost_reach_probe(self):
# Agent dials a 127.0.0.1 service on the host. TSI's
# allowlist contains only <bundle-ip>/32, so this must
# refuse. We use a port unlikely to be bound on the host
# allowlist contains only the per-bottle loopback alias, so
# this must refuse. We use a port unlikely to be bound on the host
# (high-numbered) so we're confirming TSI refusal, not
# just "no service listening."
r = self.bottle.exec(
@@ -196,28 +188,6 @@ class TestSmolmachinesLaunch(unittest.TestCase):
self.assertEqual(0, r.returncode, msg=r.stderr)
self.assertEqual(_AGENT_PROMPT, r.stdout.rstrip("\n"))
def test_egress_port_bypass_probe(self):
# Agent dials <bundle-ip>:9099 (egress's port). TSI
# permits the IP, but egress will bind 127.0.0.1:9099
# inside the bundle in chunk 3, so the connect refuses
# at the socket level. NOTE: in chunk 2d the bundle's
# daemons aren't running (daemons_csv=""), so nothing
# is listening on :9099 anyway — this test asserts the
# connect fails, which is the property chunk 3 will
# preserve once egress is actually running.
r = self.bottle.exec(
"env -u HTTPS_PROXY -u HTTP_PROXY -u https_proxy -u http_proxy "
f"curl -s --show-error --max-time 3 http://{self.plan.bundle_ip}:9099 "
"2>&1 || true"
)
self.assertTrue(
"refused" in r.stdout.lower()
or "timed out" in r.stdout.lower()
or "unreachable" in r.stdout.lower()
or "failed" in r.stdout.lower(),
f"expected egress port refusal; got: {r.stdout!r}",
)
if __name__ == "__main__":
unittest.main()
+19 -37
View File
@@ -1,9 +1,9 @@
"""Unit: _launch_bottle StaleImageError handling.
Exercises the while-True / try-except loop around backend.launch:
Exercises prelaunch_checks / backend.launch flow:
- headless mode die on stale
- interactive mode, user declines stop without launching
- interactive mode, user confirms retry with skip_stale=True
- interactive mode, user confirms skip stale check and launch once
"""
from __future__ import annotations
@@ -28,14 +28,6 @@ def _fake_plan() -> Any:
))
def _stale_cm() -> MagicMock:
"""Return a context-manager mock whose __enter__ raises StaleImageError."""
cm = MagicMock()
cm.__enter__ = MagicMock(side_effect=StaleImageError("image is 5 day(s) old"))
cm.__exit__ = MagicMock(return_value=False)
return cm
def _ok_cm(bottle: Any) -> MagicMock:
"""Return a context-manager mock that yields `bottle`."""
cm = MagicMock()
@@ -78,23 +70,25 @@ class TestLaunchBottleStaleHandling(unittest.TestCase):
)
def test_headless_stale_calls_die(self) -> None:
"""In headless mode (assume_yes=True), a StaleImageError must call die()."""
"""In headless mode (assume_yes=True), a StaleImageError from prelaunch_checks must call die()."""
import bot_bottle.cli.start as start_mod
backend_mock = MagicMock()
backend_mock.launch.return_value = _stale_cm()
backend_mock.prelaunch_checks.side_effect = StaleImageError("image is 5 day(s) old")
with patch.object(start_mod, "get_bottle_backend", return_value=backend_mock), \
patch.object(start_mod, "die", side_effect=Die()):
with self.assertRaises(Die):
self._run_launch(assume_yes=True)
def test_interactive_user_declines_stops_loop(self) -> None:
"""Interactive user answering 'n' → loop exits without a second launch."""
backend_mock.launch.assert_not_called()
def test_interactive_user_declines_stops_before_launch(self) -> None:
"""Interactive user answering 'n' → launch is never called."""
import bot_bottle.cli.start as start_mod
backend_mock = MagicMock()
backend_mock.launch.return_value = _stale_cm()
backend_mock.prelaunch_checks.side_effect = StaleImageError("image is 5 day(s) old")
with patch.object(start_mod, "get_bottle_backend", return_value=backend_mock), \
patch.object(start_mod, "read_tty_line", return_value="n"), \
@@ -102,23 +96,18 @@ class TestLaunchBottleStaleHandling(unittest.TestCase):
rc = self._run_launch(assume_yes=False)
self.assertEqual(0, rc)
# launch was called once (stale), then the user declined → no second call.
backend_mock.launch.assert_called_once()
backend_mock.launch.assert_not_called()
def test_interactive_user_confirms_retries_with_skip_stale(self) -> None:
"""Interactive user answering 'y'second launch with skip_stale=True."""
def test_interactive_user_confirms_launches_once(self) -> None:
"""Interactive user answering 'y'prelaunch stale error is bypassed; launch called once."""
import bot_bottle.cli.start as start_mod
bottle_mock = MagicMock()
bottle_mock.name = "dev-abc"
def launch_side_effect(plan: Any, *, skip_stale: bool = False) -> Any:
if not skip_stale:
return _stale_cm()
return _ok_cm(bottle_mock)
backend_mock = MagicMock()
backend_mock.launch.side_effect = launch_side_effect
backend_mock.prelaunch_checks.side_effect = StaleImageError("image is 5 day(s) old")
backend_mock.launch.return_value = _ok_cm(bottle_mock)
with patch.object(start_mod, "get_bottle_backend", return_value=backend_mock), \
patch.object(start_mod, "read_tty_line", return_value="y"), \
@@ -128,11 +117,8 @@ class TestLaunchBottleStaleHandling(unittest.TestCase):
rc = self._run_launch(assume_yes=False)
self.assertEqual(0, rc)
# First call: skip_stale=False → stale. Second call: skip_stale=True → ok.
self.assertEqual(2, backend_mock.launch.call_count)
calls = backend_mock.launch.call_args_list
self.assertFalse(calls[0].kwargs.get("skip_stale", False))
self.assertTrue(calls[1].kwargs.get("skip_stale", False))
backend_mock.prelaunch_checks.assert_called_once()
backend_mock.launch.assert_called_once()
def test_interactive_yes_uppercase_also_accepted(self) -> None:
"""'Y' or 'YES' should also be accepted as confirmation."""
@@ -141,13 +127,9 @@ class TestLaunchBottleStaleHandling(unittest.TestCase):
bottle_mock = MagicMock()
bottle_mock.name = "dev-abc"
def launch_side_effect(plan: Any, *, skip_stale: bool = False) -> Any:
if not skip_stale:
return _stale_cm()
return _ok_cm(bottle_mock)
backend_mock = MagicMock()
backend_mock.launch.side_effect = launch_side_effect
backend_mock.prelaunch_checks.side_effect = StaleImageError("image is 5 day(s) old")
backend_mock.launch.return_value = _ok_cm(bottle_mock)
with patch.object(start_mod, "get_bottle_backend", return_value=backend_mock), \
patch.object(start_mod, "read_tty_line", return_value="YES"), \
@@ -157,7 +139,7 @@ class TestLaunchBottleStaleHandling(unittest.TestCase):
rc = self._run_launch(assume_yes=False)
self.assertEqual(0, rc)
self.assertEqual(2, backend_mock.launch.call_count)
backend_mock.launch.assert_called_once()
if __name__ == "__main__":
@@ -90,7 +90,7 @@ class TestLaunchCommittedImage(unittest.TestCase):
committed_tag: str | None = None,
image_present: bool = True,
) -> list[str]:
"""Drive launch() through its full sequence with the committed-image
"""Drive build_or_load_images() + launch() with the committed-image
behaviour controlled by the arguments. Returns the images that were
passed to `build_image` (empty list if it was never called)."""
built: list[str] = []
@@ -124,8 +124,9 @@ class TestLaunchCommittedImage(unittest.TestCase):
mock.patch.object(launch_mod, "compose_dump_logs"), \
mock.patch.object(launch_mod, "compose_down"), \
contextlib.redirect_stderr(io.StringIO()):
images = launch_mod.build_or_load_images(plan)
provision = mock.Mock(return_value=None)
with launch_mod.launch(plan, provision=provision):
with launch_mod.launch(plan, images, provision=provision):
pass
return built
@@ -170,8 +171,9 @@ class TestLaunchCommittedImage(unittest.TestCase):
mock.patch.object(launch_mod, "compose_dump_logs"), \
mock.patch.object(launch_mod, "compose_down"), \
contextlib.redirect_stderr(io.StringIO()):
images = launch_mod.build_or_load_images(plan)
provision = mock.Mock(return_value=None)
with launch_mod.launch(plan, provision=provision):
with launch_mod.launch(plan, images, provision=provision):
pass
self.assertEqual(1, len(captured_plans))
+4 -2
View File
@@ -16,7 +16,7 @@ from pathlib import Path
from unittest import mock
from bot_bottle.agent_provider import AgentProvisionPlan
from bot_bottle.backend import BottleSpec
from bot_bottle.backend import BottleImages, BottleSpec
from bot_bottle.backend.docker import launch as launch_mod
from bot_bottle.backend.docker.bottle_plan import DockerBottlePlan
from bot_bottle.egress import EgressPlan
@@ -86,6 +86,8 @@ class TestTeardownWarning(unittest.TestCase):
plan = _plan(self._tmp)
buf = io.StringIO()
images = BottleImages(agent="bot-bottle-claude:latest", sidecar="bot-bottle-sidecars:latest")
with mock.patch.object(launch_mod.docker_mod, "build_image"), \
mock.patch.object(
launch_mod, "egress_tls_init",
@@ -115,7 +117,7 @@ class TestTeardownWarning(unittest.TestCase):
), \
contextlib.redirect_stderr(buf):
provision = mock.Mock(return_value=None)
with launch_mod.launch(plan, provision=provision):
with launch_mod.launch(plan, images, provision=provision):
pass
output = buf.getvalue()
+10 -10
View File
@@ -339,9 +339,9 @@ class TestMacosContainerLaunchCommittedImage(unittest.TestCase):
), patch.object(
launch.container_mod, "build_image", side_effect=fake_build,
), patch.object(launch, "info"):
updated = launch._build_images(plan)
images = launch.build_or_load_images(plan)
self.assertEqual("bot-bottle-committed-dev-abc:latest", updated.image)
self.assertEqual("bot-bottle-committed-dev-abc:latest", images.agent)
self.assertEqual(1, len(calls))
self.assertEqual(launch.SIDECAR_BUNDLE_IMAGE, calls[0][0])
@@ -360,9 +360,9 @@ class TestMacosContainerLaunchCommittedImage(unittest.TestCase):
), patch.object(
launch.container_mod, "build_image", side_effect=fake_build,
):
updated = launch._build_images(plan)
images = launch.build_or_load_images(plan)
self.assertEqual("bot-bottle-agent:latest", updated.image)
self.assertEqual("bot-bottle-agent:latest", images.agent)
self.assertEqual(2, len(calls))
self.assertEqual("bot-bottle-agent:latest", calls[1][0])
@@ -395,7 +395,7 @@ class TestMacosContainerLaunchCachedImages(unittest.TestCase):
launch, "die", side_effect=Die(),
):
with self.assertRaises(Die):
launch._build_images(plan)
launch.build_or_load_images(plan)
def test_cached_mode_dies_when_sidecar_image_missing(self) -> None:
plan = self._cached_plan()
@@ -411,7 +411,7 @@ class TestMacosContainerLaunchCachedImages(unittest.TestCase):
launch, "die", side_effect=Die(),
):
with self.assertRaises(Die):
launch._build_images(plan)
launch.build_or_load_images(plan)
def test_cached_mode_both_present_returns_unchanged_plan(self) -> None:
plan = self._cached_plan()
@@ -422,10 +422,10 @@ class TestMacosContainerLaunchCachedImages(unittest.TestCase):
), patch.object(
launch.container_mod, "build_image",
) as build, patch.object(launch, "info"):
result = launch._build_images(plan)
images = launch.build_or_load_images(plan)
build.assert_not_called()
self.assertEqual(plan.image, result.image)
self.assertEqual(plan.image, images.agent)
def test_committed_image_plus_cached_skips_sidecar_build(self) -> None:
plan = self._cached_plan()
@@ -437,11 +437,11 @@ class TestMacosContainerLaunchCachedImages(unittest.TestCase):
), patch.object(
launch.container_mod, "build_image",
) as build, patch.object(launch, "info"):
result = launch._build_images(plan)
images = launch.build_or_load_images(plan)
# In cached mode with a committed image, no builds should fire.
build.assert_not_called()
self.assertEqual("bot-bottle-committed-dev-abc:latest", result.image)
self.assertEqual("bot-bottle-committed-dev-abc:latest", images.agent)
if __name__ == "__main__":
+1 -1
View File
@@ -18,7 +18,7 @@ class TestFetchCurrentRoutes(unittest.TestCase):
self.assertEqual("routes", egress_apply.fetch_current_routes("dev-abc"))
exec_.assert_called_once_with(
"bot-bottle-sidecars-dev-abc",
["cat", "/etc/egress/routes.yaml"],
["cat", "/bot-bottle-data/egress/routes.yaml"],
)
def test_read_failure_raises_apply_error(self):
+25 -10
View File
@@ -404,7 +404,11 @@ class TestBundleLaunchSpec(unittest.TestCase):
),
)
spec = _bundle_launch_spec(plan, "net", "127.0.0.16")
with patch(
"bot_bottle.backend.smolmachines.launch._stage_sidecar_data",
return_value=Path("/tmp/smolvm-sidecar-data"),
):
spec = _bundle_launch_spec(plan, "net", "127.0.0.16")
self.assertEqual(
"egress,git-gate,git-http",
@@ -416,7 +420,11 @@ class TestBundleLaunchSpec(unittest.TestCase):
def test_canary_env_registered_as_sensitive_in_bundle(self):
plan = _plan(canary=True)
spec = _bundle_launch_spec(plan, "net", "127.0.0.16")
with patch(
"bot_bottle.backend.smolmachines.launch._stage_sidecar_data",
return_value=Path("/tmp/smolvm-sidecar-data"),
):
spec = _bundle_launch_spec(plan, "net", "127.0.0.16")
self.assertIn("CANON_ALPHA_SECRET=fake-canary-value", spec.environment)
self.assertIn(
@@ -427,10 +435,16 @@ class TestBundleLaunchSpec(unittest.TestCase):
def test_supervise_adds_daemon_volume_and_env(self):
from bot_bottle.supervise import DB_PATH_IN_CONTAINER
plan = _plan(supervise=True)
spec = _bundle_launch_spec(plan, "net", "127.0.0.16")
with patch(
"bot_bottle.backend.smolmachines.launch._stage_sidecar_data",
return_value=Path("/tmp/smolvm-sidecar-data"),
):
spec = _bundle_launch_spec(plan, "net", "127.0.0.16")
self.assertIn("supervise", spec.daemons_csv)
self.assertIn(f"SUPERVISE_DB_PATH={DB_PATH_IN_CONTAINER}", spec.environment)
self.assertIn(("/tmp/bot-bottle.db", DB_PATH_IN_CONTAINER, False), spec.volumes)
# virtiofs requires directory mounts; the DB's parent dir is
# mounted so bot-bottle.db lands at the right in-VM path.
self.assertIn(("/tmp", str(Path(DB_PATH_IN_CONTAINER).parent), False), spec.volumes)
def test_canary_env_visible_to_smolvm_guest(self):
plan = _plan(canary=True)
@@ -546,24 +560,25 @@ class TestLaunchResourceWiring(unittest.TestCase):
),
)
sidecar_artifact = Path("/cache/sidecar.smolmachine")
with patch(
"bot_bottle.backend.smolmachines.launch._provision_git_gate_keys",
return_value=plan,
), patch(
"bot_bottle.backend.smolmachines.launch._ensure_smolmachine",
return_value=Path("/cache/sidecar.smolmachine"),
) as ensure, patch(
"bot_bottle.backend.smolmachines.launch._bundle.start_bundle_vm",
return_value=raw_launch,
) as start_vm, patch(
"bot_bottle.backend.smolmachines.launch._stage_sidecar_data",
return_value=Path("/tmp/smolvm-sidecar-data"),
), patch(
"bot_bottle.backend.smolmachines.launch._forward.start_forwarder",
return_value=handle,
) as start_forwarder:
stamped = _launch._start_bundle(plan, "net", "127.0.0.16", stack)
stamped = _launch._start_bundle(plan, "net", "127.0.0.16", sidecar_artifact, stack)
ensure.assert_called_once()
start_vm.assert_called_once()
self.assertEqual(Path("/cache/sidecar.smolmachine"), start_vm.call_args.kwargs["from_path"])
self.assertEqual(sidecar_artifact, start_vm.call_args.kwargs["from_path"])
specs = start_forwarder.call_args.args[0]
self.assertEqual(
(
+1 -161
View File
@@ -1,9 +1,4 @@
"""Unit: bundle bringup primitives for the smolmachines backend
(PRD 0023 chunk 2c).
Tests mock `subprocess.run` and assert on the docker argv shape.
The end-to-end integration smoke (real docker daemon, real
bundle image) lands in chunk 2d."""
"""Unit: bundle bringup primitives for the smolmachines backend."""
from __future__ import annotations
@@ -17,28 +12,12 @@ from bot_bottle.backend.smolmachines.sidecar_bundle import (
allocate_raw_host_ports,
bundle_container_name,
bundle_network_name,
create_bundle_network,
ensure_bundle_image,
start_bundle_vm,
remove_bundle_network,
start_bundle,
stop_bundle,
stop_bundle_vm,
)
def _ok(stdout: str = "", stderr: str = "") -> subprocess.CompletedProcess: # type: ignore
return subprocess.CompletedProcess(
args=[], returncode=0, stdout=stdout, stderr=stderr,
)
def _fail(stderr: str = "boom") -> subprocess.CompletedProcess: # type: ignore
return subprocess.CompletedProcess(
args=[], returncode=1, stdout="", stderr=stderr,
)
def _spec(**kwargs) -> BundleLaunchSpec: # type: ignore
defaults = dict(
slug="demo-abc12",
@@ -71,123 +50,6 @@ class TestNamingHelpers(unittest.TestCase):
)
class TestNetworkLifecycle(unittest.TestCase):
def _patch_run(self, **kwargs): # type: ignore
return patch(
"bot_bottle.backend.smolmachines.sidecar_bundle.subprocess.run",
**kwargs,
)
def test_create_argv_explicit_subnet_and_gateway(self):
with self._patch_run(return_value=_ok()) as m:
create_bundle_network("nn", "192.168.50.0/24", "192.168.50.1")
self.assertEqual(
["docker", "network", "create",
"--subnet", "192.168.50.0/24",
"--gateway", "192.168.50.1",
"nn"],
m.call_args.args[0],
)
def test_create_treats_existing_network_as_success(self):
with self._patch_run(return_value=_fail("network nn already exists")):
# No SystemExit.
create_bundle_network("nn", "192.168.50.0/24", "192.168.50.1")
def test_create_other_failure_is_fatal(self):
with self._patch_run(return_value=_fail("invalid subnet")):
with self.assertRaises(SystemExit):
create_bundle_network("nn", "bogus", "bogus")
def test_remove_missing_network_is_idempotent(self):
# No SystemExit / no warn-and-continue noise; missing
# network is the expected case during a partial teardown.
with self._patch_run(return_value=_fail("Error: No such network: nn")):
remove_bundle_network("nn")
def test_remove_clean_returns_success(self):
with self._patch_run(return_value=_ok()):
remove_bundle_network("nn")
class TestStartBundle(unittest.TestCase):
def _patch_run(self):
return patch(
"bot_bottle.backend.smolmachines.sidecar_bundle.subprocess.run",
return_value=_ok(),
)
def test_argv_pins_ip_on_network(self):
with self._patch_run() as m:
start_bundle(_spec())
argv = m.call_args.args[0]
# --network NETNAME --ip <bundle-ip> on the docker run.
self.assertIn("--network", argv)
self.assertIn("bot-bottle-bundle-demo-abc12", argv)
self.assertIn("--ip", argv)
self.assertIn("192.168.50.2", argv)
# Detached and auto-removed.
self.assertIn("--detach", argv)
self.assertIn("--rm", argv)
# Container name uses the per-slug bundle prefix.
i = argv.index("--name")
self.assertEqual("bot-bottle-sidecars-demo-abc12", argv[i + 1])
# Image at the end.
self.assertEqual("bot-bottle-sidecars:latest", argv[-1])
def test_daemons_env_passed_in(self):
with self._patch_run() as m:
start_bundle(_spec(daemons_csv="egress,supervise"))
argv = m.call_args.args[0]
self.assertIn("-e", argv)
self.assertIn(
"BOT_BOTTLE_SIDECAR_DAEMONS=egress,supervise",
argv,
)
def test_environment_entries_pass_through(self):
with self._patch_run() as m:
start_bundle(_spec(environment=(
"SUPERVISE_BOTTLE_SLUG=demo-abc12",
"EGRESS_TOKEN_0", # bare-name → host env inherit
)))
argv = m.call_args.args[0]
self.assertIn("SUPERVISE_BOTTLE_SLUG=demo-abc12", argv)
self.assertIn("EGRESS_TOKEN_0", argv)
def test_volumes_render_with_ro_flag(self):
with self._patch_run() as m:
start_bundle(_spec(volumes=(
("/host/egress-ca.pem", "/home/mitmproxy/.mitmproxy/mitmproxy-ca.pem", True),
("/host/queue", "/run/supervise/queue", False),
)))
argv = m.call_args.args[0]
self.assertIn(
"/host/egress-ca.pem:/home/mitmproxy/.mitmproxy/mitmproxy-ca.pem:ro",
argv,
)
self.assertIn("/host/queue:/run/supervise/queue", argv)
def test_failure_dies(self):
with patch(
"bot_bottle.backend.smolmachines.sidecar_bundle.subprocess.run",
return_value=_fail("invalid mount"),
):
with self.assertRaises(SystemExit):
start_bundle(_spec())
def test_host_env_inherited_to_subprocess(self):
# Bare-name entries in spec.environment rely on the docker
# subprocess being run with the host env. Confirm `env=`
# threads through.
with patch(
"bot_bottle.backend.smolmachines.sidecar_bundle.subprocess.run",
return_value=_ok(),
) as m:
start_bundle(_spec(), env={"FOO": "bar"})
self.assertEqual({"FOO": "bar"}, m.call_args.kwargs["env"])
class TestEnsureBundleImage(unittest.TestCase):
def test_builds_sidecar_dockerfile_before_plain_docker_run(self):
with patch(
@@ -263,28 +125,6 @@ class TestStartBundleVm(unittest.TestCase):
)
class TestStopBundle(unittest.TestCase):
def _patch_run(self, **kwargs): # type: ignore
return patch(
"bot_bottle.backend.smolmachines.sidecar_bundle.subprocess.run",
**kwargs,
)
def test_argv_force_removes(self):
with self._patch_run(return_value=_ok()) as m:
stop_bundle("demo-abc12")
self.assertEqual(
["docker", "rm", "-f", "bot-bottle-sidecars-demo-abc12"],
m.call_args.args[0],
)
def test_missing_container_is_idempotent(self):
with self._patch_run(return_value=_fail(
"Error: No such container: bot-bottle-sidecars-demo-abc12"
)):
stop_bundle("demo-abc12") # no raise
class TestStopBundleVm(unittest.TestCase):
def test_stops_then_deletes_sidecar_vm(self):
with patch(
+21 -31
View File
@@ -1,5 +1,5 @@
"""Unit: stale-image check functions across backends, and the
BottleBackend.launch template method (skip_stale flag).
BottleBackend.launch template method (prelaunch_checks + build_or_load_images).
No real images or containers are used all Docker/container/smolmachine
calls are mocked at the module boundary."""
@@ -23,51 +23,41 @@ def _bottle_cm(bottle: Any) -> MagicMock:
# ---------------------------------------------------------------------------
# BottleBackend.launch template — _image_stale_checks + skip_stale
# BottleBackend.launch template — prelaunch_checks + _build_or_load_images
# ---------------------------------------------------------------------------
class TestBottleBackendLaunchTemplate(unittest.TestCase):
"""Verify the concrete launch() method on BottleBackend calls
_image_stale_checks unless skip_stale=True, then delegates to _launch_impl."""
_build_or_load_images and _launch_impl, and that prelaunch_checks is a no-op
on the base class."""
def _make_backend(self) -> Any:
from bot_bottle.backend.docker.backend import DockerBottleBackend
return DockerBottleBackend()
def test_stale_checks_called_by_default(self) -> None:
def test_launch_delegates_to_build_or_load_and_launch_impl(self) -> None:
from bot_bottle.backend import BottleImages
backend = self._make_backend()
plan = cast(Any, SimpleNamespace())
bottle = MagicMock()
images = BottleImages(agent="agent:latest", sidecar="sidecar:latest")
with patch.object(
backend, "_image_stale_checks",
) as stale_mock, patch.object(
backend, "_build_or_load_images", return_value=images,
) as build_mock, patch.object(
backend, "_launch_impl",
return_value=_bottle_cm(bottle),
):
) as impl_mock:
with backend.launch(plan):
pass
stale_mock.assert_called_once_with(plan)
build_mock.assert_called_once_with(plan)
impl_mock.assert_called_once_with(plan, images)
def test_skip_stale_bypasses_stale_checks(self) -> None:
backend = self._make_backend()
plan = cast(Any, SimpleNamespace())
bottle = MagicMock()
with patch.object(
backend, "_image_stale_checks",
) as stale_mock, patch.object(
backend, "_launch_impl",
return_value=_bottle_cm(bottle),
):
with backend.launch(plan, skip_stale=True):
pass
stale_mock.assert_not_called()
def test_noop_default_image_stale_checks(self) -> None:
def test_noop_default_prelaunch_checks(self) -> None:
from bot_bottle.backend.docker.backend import DockerBottleBackend
from bot_bottle.backend import BottleBackend
backend = DockerBottleBackend()
# Call the base-class _image_stale_checks directly to verify it's a no-op.
BottleBackend._image_stale_checks(backend, cast(Any, SimpleNamespace())) # type: ignore[arg-type]
# Base-class prelaunch_checks is a no-op — must not raise.
BottleBackend.prelaunch_checks(backend, cast(Any, SimpleNamespace())) # type: ignore[arg-type]
# ---------------------------------------------------------------------------
@@ -142,13 +132,13 @@ class TestDockerStaleChecks(unittest.TestCase):
self.assertEqual(1, cs.call_count)
self.assertIn(plan.image, cs.call_args.args[0])
def test_backend_image_stale_checks_delegates(self) -> None:
def test_backend_prelaunch_checks_delegates(self) -> None:
from bot_bottle.backend.docker.backend import DockerBottleBackend
from bot_bottle.backend.docker import launch as mod
backend = DockerBottleBackend()
plan = self._plan()
with patch.object(mod, "stale_checks") as sc:
backend._image_stale_checks(plan) # type: ignore[arg-type]
backend.prelaunch_checks(plan) # type: ignore[arg-type]
sc.assert_called_once_with(plan)
@@ -227,13 +217,13 @@ class TestSmolmachinesStaleChecks(unittest.TestCase):
sidecar_calls = [c for c in csp.call_args_list if "sidecar" in c.args[0]]
self.assertGreaterEqual(len(sidecar_calls), 1)
def test_backend_image_stale_checks_delegates(self) -> None:
def test_backend_prelaunch_checks_delegates(self) -> None:
from bot_bottle.backend.smolmachines.backend import SmolmachinesBottleBackend
from bot_bottle.backend.smolmachines import launch as mod
backend = SmolmachinesBottleBackend()
plan = self._plan()
with patch.object(mod, "stale_checks") as sc:
backend._image_stale_checks(plan) # type: ignore[arg-type]
backend.prelaunch_checks(plan) # type: ignore[arg-type]
sc.assert_called_once_with(plan)
@@ -288,13 +278,13 @@ class TestMacosContainerStaleChecks(unittest.TestCase):
mod.stale_checks(self._plan())
cs.assert_not_called()
def test_backend_image_stale_checks_delegates(self) -> None:
def test_backend_prelaunch_checks_delegates(self) -> None:
from bot_bottle.backend.macos_container.backend import MacosContainerBottleBackend
from bot_bottle.backend.macos_container import launch as mod
backend = MacosContainerBottleBackend()
plan = self._plan()
with patch.object(mod, "stale_checks") as sc:
backend._image_stale_checks(plan) # type: ignore[arg-type]
backend.prelaunch_checks(plan) # type: ignore[arg-type]
sc.assert_called_once_with(plan)