Compare commits
3 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 898b6350bc | |||
| d2081839c9 | |||
| 23015f7fd8 |
+2
-8
@@ -18,7 +18,7 @@
|
||||
# /git-gate-entrypoint.sh docker-cp'd at start time
|
||||
# /git-gate/creds/* docker-cp'd at start time
|
||||
# /git/* bare repos, populated at runtime
|
||||
# /run/supervise/bot-bottle.db bind-mounted at run time
|
||||
# /run/supervise/queue/ bind-mounted at run time
|
||||
# /home/mitmproxy/.mitmproxy/ mitmproxy CA dir
|
||||
#
|
||||
# Exposed ports inside the container:
|
||||
@@ -66,12 +66,6 @@ COPY bot_bottle/egress_dlp_config.py /app/egress_dlp_config.py
|
||||
COPY bot_bottle/egress_addon.py /app/egress_addon.py
|
||||
COPY bot_bottle/dlp_detectors.py /app/dlp_detectors.py
|
||||
COPY bot_bottle/yaml_subset.py /app/yaml_subset.py
|
||||
COPY bot_bottle/migrations.py /app/migrations.py
|
||||
COPY bot_bottle/db_store.py /app/db_store.py
|
||||
COPY bot_bottle/supervise_types.py /app/supervise_types.py
|
||||
COPY bot_bottle/queue_store.py /app/queue_store.py
|
||||
COPY bot_bottle/audit_store.py /app/audit_store.py
|
||||
COPY bot_bottle/store_manager.py /app/store_manager.py
|
||||
COPY bot_bottle/supervise.py /app/supervise.py
|
||||
COPY bot_bottle/supervise_server.py /app/supervise_server.py
|
||||
COPY bot_bottle/sidecar_init.py /app/sidecar_init.py
|
||||
@@ -87,7 +81,7 @@ RUN mkdir -p \
|
||||
/etc/git-gate \
|
||||
/git-gate/creds \
|
||||
/git \
|
||||
/run/supervise \
|
||||
/run/supervise/queue \
|
||||
/home/mitmproxy/.mitmproxy
|
||||
|
||||
# Documentation only — the compose renderer publishes whichever
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
|
||||
[](https://gitea.dideric.is/didericis/bot-bottle/actions?workflow=test.yml)
|
||||
[](https://coverage.readthedocs.io/)
|
||||
[](https://gitea.dideric.is/didericis/bot-bottle/src/branch/main/docs/decisions/0004-coverage-policy.md)
|
||||
[](https://gitea.dideric.is/didericis/bot-bottle/src/branch/main/docs/decisions/0004-coverage-policy.md)
|
||||
|
||||
**Problem:** Developer wants to run a coding agent without supervision, but they don't want a prompt injected or misbehaving agent wrecking their environment or exfiltrating sensitive data.
|
||||
|
||||
@@ -25,7 +25,7 @@
|
||||
- **Provider templates (Claude, Codex)** — `Dockerfile.claude` / `Dockerfile.codex`, or a bottle-supplied Dockerfile. Claude auth via long-lived OAuth token; Codex via opt-in host device-auth forwarding.
|
||||
- **gVisor auto-detect** — on Linux hosts where `runsc` is registered with Docker, every bottle launches under it for a userspace syscall barrier; no manifest config required.
|
||||
- **Apple Container backend (macOS default when available)** — runs the agent and sidecar bundle with Apple's `container` CLI, using a host-only agent network plus a separate sidecar egress network.
|
||||
- **Smolmachines backend** — runs the agent in a libkrun micro-VM while the sidecar bundle stays in Docker. TSI and smolmachines DNS filtering close the raw DNS exfiltration gap that exists in the legacy Docker backend. Runs on macOS (Hypervisor.framework) and Linux (KVM, `/dev/kvm`).
|
||||
- **Smolmachines backend** — runs the agent in a libkrun micro-VM while the sidecar bundle stays in Docker. TSI and smolmachines DNS filtering close the raw DNS exfiltration gap that exists in the legacy Docker backend.
|
||||
- **Legacy Docker backend** — still available for examples, CI, and hosts without Apple Container via `BOT_BOTTLE_BACKEND=docker` or `--backend=docker`.
|
||||
|
||||
## Architecture
|
||||
@@ -71,26 +71,10 @@ When the agent exits, `cli.py` tears down every sidecar and both networks; nothi
|
||||
|
||||
## Quickstart
|
||||
|
||||
On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The smolmachines backend requires Docker on the host for the sidecar bundle plus `smolvm` (macOS or Linux). The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
|
||||
On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The smolmachines backend requires Docker on the host for the sidecar bundle plus smolvm. The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
|
||||
|
||||
Use `BOT_BOTTLE_BACKEND=docker ./cli.py start <agent>` on hosts where Apple Container is not installed and Docker is the desired backend.
|
||||
|
||||
### smolmachines on Linux
|
||||
|
||||
The smolmachines backend runs on Linux as well as macOS. On Linux, `smolvm`/libkrun use KVM, so the host needs:
|
||||
|
||||
- **`/dev/kvm`** present and accessible. Load `kvm-intel` or `kvm-amd` (and enable virtualization in BIOS/firmware). The invoking user must be in the `kvm` group: `sudo usermod -aG kvm "$USER"` then re-login. bot-bottle preflights this and reports exactly what's missing.
|
||||
- **`smolvm`** on `PATH`: `curl -sSL https://smolmachines.com/install.sh | sh`.
|
||||
- **Docker** for the sidecar bundle and image build, same as macOS.
|
||||
|
||||
Per-bottle isolation works the same as macOS without any `ifconfig`/sudo step — all of `127.0.0.0/8` is already loopback on Linux, so each bottle's sidecar bundle is published on its own `127.0.0.<N>` and TSI's allowlist is scoped to that `/32`.
|
||||
|
||||
```sh
|
||||
BOT_BOTTLE_BACKEND=smolmachines ./cli.py start <agent>
|
||||
```
|
||||
|
||||
> **NixOS:** enable `virtualisation.docker`, ensure the KVM module is loaded (`boot.kernelModules = [ "kvm-intel" ];` or `kvm-amd`), and add your user to the `kvm` and `docker` groups. If you run bottles from a Gitea Actions runner, use a `host`-label runner so Docker, `smolvm`, and `/dev/kvm` are all reachable from the job. `smolvm` isn't in nixpkgs — install the release binary (pin the version) and put it on the runner's `PATH`.
|
||||
|
||||
```sh
|
||||
./cli.py start <agent> # builds the image on first run, drops you into claude
|
||||
```
|
||||
|
||||
@@ -209,15 +209,6 @@ class AgentProvider(ABC):
|
||||
the supervise sidecar is reachable. No-op when
|
||||
`plan.supervise_plan is None`."""
|
||||
|
||||
@abstractmethod
|
||||
def headless_prompt(self, prompt: str) -> list[str]:
|
||||
"""Return the agent CLI args that deliver `prompt` as the
|
||||
initial task in a non-interactive (headless) session.
|
||||
|
||||
Called only when ``--prompt`` is passed to
|
||||
``./cli.py start --headless``; the returned args are appended
|
||||
after the provider's ``bypass_args`` and ``startup_args``."""
|
||||
|
||||
def provision_ca(self, bottle: "Bottle", plan: "BottlePlan") -> None:
|
||||
"""Install the egress MITM CA into the agent's trust store.
|
||||
|
||||
@@ -227,10 +218,6 @@ class AgentProvider(ABC):
|
||||
from .backend.util import AGENT_CA_PATH, log_ca_fingerprint, select_ca_cert
|
||||
from .log import die
|
||||
cert_host_path, label = select_ca_cert(plan.egress_plan)
|
||||
# Ensure the target directory exists. smolvm's pack step may not
|
||||
# preserve the empty /usr/local/share/ca-certificates/ directory
|
||||
# on Linux; mkdir -p is idempotent and safe for all backends.
|
||||
bottle.exec("mkdir -p /usr/local/share/ca-certificates", user="root")
|
||||
bottle.cp_in(str(cert_host_path), AGENT_CA_PATH)
|
||||
r = bottle.exec(
|
||||
f"chmod 644 {AGENT_CA_PATH} && update-ca-certificates",
|
||||
|
||||
@@ -1,91 +0,0 @@
|
||||
"""SQLite-backed audit store for supervise (PRD 0013)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sqlite3
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
from .supervise_types import AuditEntry, host_db_path
|
||||
from .db_store import DbStore
|
||||
from .migrations import TableMigrations
|
||||
except ImportError:
|
||||
from supervise_types import AuditEntry, host_db_path # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
|
||||
from db_store import DbStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
|
||||
from migrations import TableMigrations # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
|
||||
|
||||
|
||||
class AuditStore(DbStore):
|
||||
"""SQLite-backed persistent store for supervise audit entries."""
|
||||
|
||||
def __init__(self, db_path: Path | None = None) -> None:
|
||||
# One entry per schema version: migrations[0] brings a fresh DB to
|
||||
# version 1, [1] to version 2, etc. Add new entries at the end; never
|
||||
# edit existing ones.
|
||||
migrations = TableMigrations("audit_store", [
|
||||
# v1 — initial schema
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS supervise_audit_entries (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
timestamp TEXT NOT NULL,
|
||||
bottle_slug TEXT NOT NULL,
|
||||
component TEXT NOT NULL,
|
||||
operator_action TEXT NOT NULL,
|
||||
operator_notes TEXT NOT NULL,
|
||||
justification TEXT NOT NULL,
|
||||
diff TEXT NOT NULL
|
||||
)
|
||||
""",
|
||||
])
|
||||
super().__init__(db_path or host_db_path(), migrations)
|
||||
|
||||
def write_audit_entry(self, entry: AuditEntry) -> Path:
|
||||
with self._connect() as conn:
|
||||
conn.execute(
|
||||
"""
|
||||
INSERT INTO supervise_audit_entries (
|
||||
timestamp, bottle_slug, component, operator_action,
|
||||
operator_notes, justification, diff
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(
|
||||
entry.timestamp,
|
||||
entry.bottle_slug,
|
||||
entry.component,
|
||||
entry.operator_action,
|
||||
entry.operator_notes,
|
||||
entry.justification,
|
||||
entry.diff,
|
||||
),
|
||||
)
|
||||
self._chmod()
|
||||
return self.db_path
|
||||
|
||||
def read_audit_entries(self, component: str, slug: str) -> list[AuditEntry]:
|
||||
if not self.db_path.is_file():
|
||||
return []
|
||||
with self._connect() as conn:
|
||||
rows = conn.execute(
|
||||
"""
|
||||
SELECT * FROM supervise_audit_entries
|
||||
WHERE component = ? AND bottle_slug = ?
|
||||
ORDER BY id
|
||||
""",
|
||||
(component, slug),
|
||||
).fetchall()
|
||||
return [self._row_to_entry(row) for row in rows]
|
||||
|
||||
@staticmethod
|
||||
def _row_to_entry(row: sqlite3.Row) -> AuditEntry:
|
||||
return AuditEntry(
|
||||
timestamp=row["timestamp"],
|
||||
bottle_slug=row["bottle_slug"],
|
||||
component=row["component"],
|
||||
operator_action=row["operator_action"],
|
||||
operator_notes=row["operator_notes"],
|
||||
justification=row["justification"],
|
||||
diff=row["diff"],
|
||||
)
|
||||
|
||||
|
||||
__all__ = ["AuditStore"]
|
||||
@@ -36,10 +36,10 @@ import os
|
||||
import shlex
|
||||
import sys
|
||||
from abc import ABC, abstractmethod
|
||||
from contextlib import AbstractContextManager, contextmanager
|
||||
from contextlib import AbstractContextManager
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any, Generator, Generic, Sequence, TypeVar
|
||||
from typing import Any, Generic, Sequence, TypeVar
|
||||
|
||||
from ..agent_provider import AgentProvisionPlan, get_provider, build_agent_provision_plan
|
||||
from ..egress import EgressPlan
|
||||
@@ -75,12 +75,6 @@ class BottleSpec:
|
||||
# Ordered bottle names selected at launch (issue #269). When non-empty
|
||||
# they are merged in order and replace the agent's `bottle:` field.
|
||||
bottle_names: tuple[str, ...] = ()
|
||||
# True when launched via --headless (no TTY, no interactive prompts).
|
||||
# The git-gate host-key preflight uses this to error rather than prompt.
|
||||
headless: bool = False
|
||||
# Image startup policy. "fresh" preserves the normal build path;
|
||||
# "cached" reuses the current local image/artifact without rebuilding.
|
||||
image_policy: str = "fresh"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -306,13 +300,6 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
|
||||
|
||||
self._preflight()
|
||||
|
||||
from ..git_gate_host_key import preflight_host_keys
|
||||
manifest = preflight_host_keys(
|
||||
manifest,
|
||||
headless=spec.headless,
|
||||
home_md=spec.manifest.home_md,
|
||||
)
|
||||
|
||||
manifest_bottle = manifest.bottle
|
||||
manifest_agent_provider = manifest_bottle.agent_provider
|
||||
agent_provider = get_provider(manifest_agent_provider.template)
|
||||
@@ -435,25 +422,8 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
|
||||
prompt file, Dockerfile path, and guest home all live on
|
||||
`agent_provision_plan` — the source of truth."""
|
||||
|
||||
@contextmanager
|
||||
def launch(
|
||||
self, plan: PlanT, *, skip_stale: bool = False
|
||||
) -> Generator[Bottle, None, None]:
|
||||
"""Template: optionally check for stale cached images, then delegate
|
||||
to `_launch_impl`. Pass `skip_stale=True` to bypass the stale check
|
||||
(used by the interactive CLI after the operator confirms)."""
|
||||
if not skip_stale:
|
||||
self._image_stale_checks(plan)
|
||||
with self._launch_impl(plan) as bottle:
|
||||
yield bottle
|
||||
|
||||
def _image_stale_checks(self, plan: PlanT) -> None:
|
||||
"""Raise StaleImageError if any cached image used by this plan is stale.
|
||||
No-op default; backends override to call the shared `check_stale*`
|
||||
helpers on their image/artifact timestamps."""
|
||||
|
||||
@abstractmethod
|
||||
def _launch_impl(self, plan: PlanT) -> AbstractContextManager[Bottle]:
|
||||
def launch(self, plan: PlanT) -> AbstractContextManager[Bottle]:
|
||||
"""Build/run the bottle and yield a handle; tear down on exit."""
|
||||
|
||||
def provision(self, plan: PlanT, bottle: "Bottle") -> str | None:
|
||||
|
||||
@@ -85,11 +85,8 @@ class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanup
|
||||
stage_dir=stage_dir,
|
||||
)
|
||||
|
||||
def _image_stale_checks(self, plan: DockerBottlePlan) -> None:
|
||||
_launch.stale_checks(plan)
|
||||
|
||||
@contextmanager
|
||||
def _launch_impl(self, plan: DockerBottlePlan) -> Generator[DockerBottle, None, None]:
|
||||
def launch(self, plan: DockerBottlePlan) -> Generator[DockerBottle, None, None]:
|
||||
with _launch.launch(plan, provision=self.provision) as bottle:
|
||||
yield bottle
|
||||
|
||||
|
||||
@@ -34,7 +34,7 @@ from ...egress import (
|
||||
from ...git_gate import GIT_GATE_HOSTNAME
|
||||
from ...log import die, warn
|
||||
from ...supervise import (
|
||||
DB_PATH_IN_CONTAINER,
|
||||
QUEUE_DIR_IN_CONTAINER,
|
||||
SUPERVISE_HOSTNAME,
|
||||
SUPERVISE_PORT,
|
||||
)
|
||||
@@ -163,15 +163,16 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
||||
if sp is not None:
|
||||
env += [
|
||||
f"SUPERVISE_BOTTLE_SLUG={plan.slug}",
|
||||
f"SUPERVISE_DB_PATH={DB_PATH_IN_CONTAINER}",
|
||||
f"SUPERVISE_QUEUE_DIR={QUEUE_DIR_IN_CONTAINER}",
|
||||
f"SUPERVISE_PORT={SUPERVISE_PORT}",
|
||||
]
|
||||
volumes.append({
|
||||
"type": "bind",
|
||||
"source": str(sp.db_path),
|
||||
"target": DB_PATH_IN_CONTAINER,
|
||||
"source": str(sp.queue_dir),
|
||||
"target": QUEUE_DIR_IN_CONTAINER,
|
||||
"read_only": False,
|
||||
})
|
||||
|
||||
internal_aliases = [EGRESS_HOSTNAME]
|
||||
if gp.upstreams:
|
||||
internal_aliases.append(GIT_GATE_HOSTNAME)
|
||||
@@ -180,6 +181,10 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
||||
|
||||
service: dict[str, Any] = {
|
||||
"image": SIDECAR_BUNDLE_IMAGE,
|
||||
"build": {
|
||||
"context": _REPO_DIR,
|
||||
"dockerfile": SIDECAR_BUNDLE_DOCKERFILE,
|
||||
},
|
||||
"container_name": sidecar_bundle_container_name(plan.slug),
|
||||
"networks": {
|
||||
"internal": {"aliases": internal_aliases},
|
||||
@@ -188,11 +193,6 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
||||
"environment": env,
|
||||
"volumes": volumes,
|
||||
}
|
||||
if plan.spec.image_policy != "cached":
|
||||
service["build"] = {
|
||||
"context": _REPO_DIR,
|
||||
"dockerfile": SIDECAR_BUNDLE_DOCKERFILE,
|
||||
}
|
||||
return service
|
||||
|
||||
|
||||
|
||||
@@ -37,12 +37,8 @@ from pathlib import Path
|
||||
from typing import Callable, Generator
|
||||
|
||||
from ...egress import egress_resolve_token_values
|
||||
from ...git_gate import (
|
||||
provision_git_gate_dynamic_keys,
|
||||
revoke_git_gate_provisioned_keys,
|
||||
)
|
||||
from ...image_cache import check_stale
|
||||
from ...log import die, info, warn
|
||||
from ...git_gate import revoke_git_gate_provisioned_keys
|
||||
from ...log import info, warn
|
||||
from . import network as network_mod
|
||||
from . import util as docker_mod
|
||||
from .bottle import DockerBottle
|
||||
@@ -64,7 +60,6 @@ from .compose import (
|
||||
write_compose_file,
|
||||
)
|
||||
from .egress import egress_tls_init
|
||||
from .sidecar_bundle import SIDECAR_BUNDLE_IMAGE
|
||||
|
||||
|
||||
# Where the repo root lives, for `docker build` context. Computed once.
|
||||
@@ -102,26 +97,12 @@ def launch(
|
||||
# Dockerfile. Sidecar images get built lazily by `docker compose
|
||||
# up` via the renderer's `build:` directives.
|
||||
committed = read_committed_image(plan.slug)
|
||||
cached_policy = plan.spec.image_policy == "cached"
|
||||
if committed and docker_mod.image_exists(committed):
|
||||
info(f"using committed image {committed!r}")
|
||||
plan = dataclasses.replace(
|
||||
plan,
|
||||
agent_provision=dataclasses.replace(plan.agent_provision, image=committed),
|
||||
)
|
||||
elif cached_policy:
|
||||
if not docker_mod.image_exists(plan.image):
|
||||
die(
|
||||
f"cached agent image {plan.image!r} not found; "
|
||||
"run without --cached-images to build it"
|
||||
)
|
||||
if not docker_mod.image_exists(SIDECAR_BUNDLE_IMAGE):
|
||||
die(
|
||||
f"cached sidecar image {SIDECAR_BUNDLE_IMAGE!r} not found; "
|
||||
"run without --cached-images to build it"
|
||||
)
|
||||
info(f"using cached agent image {plan.image!r}")
|
||||
info(f"using cached sidecar image {SIDECAR_BUNDLE_IMAGE!r}")
|
||||
else:
|
||||
docker_mod.build_image(
|
||||
plan.image, _REPO_DIR,
|
||||
@@ -137,11 +118,6 @@ def launch(
|
||||
|
||||
git_gate_plan = plan.git_gate_plan
|
||||
if git_gate_plan.upstreams:
|
||||
git_gate_plan = provision_git_gate_dynamic_keys(
|
||||
plan.manifest.bottle,
|
||||
git_gate_plan,
|
||||
git_gate_state_dir(plan.slug),
|
||||
)
|
||||
git_gate_plan = dataclasses.replace(
|
||||
git_gate_plan,
|
||||
internal_network=internal_network,
|
||||
@@ -223,21 +199,3 @@ def launch(
|
||||
yield bottle
|
||||
finally:
|
||||
teardown()
|
||||
|
||||
|
||||
def stale_checks(plan: DockerBottlePlan) -> None:
|
||||
"""Raise StaleImageError if a cached image is older than the configured
|
||||
threshold. Only runs when image_policy is 'cached'. Called by the backend
|
||||
class's _image_stale_checks before _launch_impl starts any resources."""
|
||||
if plan.spec.image_policy != "cached":
|
||||
return
|
||||
committed = read_committed_image(plan.slug)
|
||||
if committed and docker_mod.image_exists(committed):
|
||||
check_stale(f"agent image {committed!r}", docker_mod.image_created_at(committed))
|
||||
return
|
||||
for label, ref in [
|
||||
("agent image", plan.image),
|
||||
("sidecar image", SIDECAR_BUNDLE_IMAGE),
|
||||
]:
|
||||
if docker_mod.image_exists(ref):
|
||||
check_stale(f"{label} {ref!r}", docker_mod.image_created_at(ref))
|
||||
|
||||
@@ -4,7 +4,6 @@ existence, and building images."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timezone
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
@@ -187,45 +186,6 @@ def image_id(ref: str) -> str:
|
||||
return r.stdout.strip()
|
||||
|
||||
|
||||
def image_created_at(ref: str) -> datetime:
|
||||
"""Return Docker's image Created timestamp as an aware UTC datetime."""
|
||||
r = subprocess.run(
|
||||
["docker", "image", "inspect", "--format", "{{.Created}}", ref],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if r.returncode != 0:
|
||||
die(
|
||||
f"docker image inspect for {ref!r} failed: "
|
||||
f"{(r.stderr or '').strip() or '<no stderr>'}"
|
||||
)
|
||||
raw = r.stdout.strip()
|
||||
try:
|
||||
return _parse_docker_timestamp(raw)
|
||||
except ValueError:
|
||||
die(f"docker image inspect for {ref!r} returned invalid Created timestamp: {raw!r}")
|
||||
|
||||
|
||||
def _parse_docker_timestamp(raw: str) -> datetime:
|
||||
text = raw.strip()
|
||||
if text.endswith("Z"):
|
||||
text = text[:-1] + "+00:00"
|
||||
dot = text.find(".")
|
||||
if dot != -1:
|
||||
tz_plus = text.find("+", dot)
|
||||
tz_minus = text.find("-", dot)
|
||||
tz_candidates = [pos for pos in (tz_plus, tz_minus) if pos != -1]
|
||||
if tz_candidates:
|
||||
tz_pos = min(tz_candidates)
|
||||
frac = text[dot + 1:tz_pos]
|
||||
text = text[:dot + 1] + frac[:6].ljust(6, "0") + text[tz_pos:]
|
||||
dt = datetime.fromisoformat(text)
|
||||
if dt.tzinfo is None:
|
||||
dt = dt.replace(tzinfo=timezone.utc)
|
||||
return dt.astimezone(timezone.utc)
|
||||
|
||||
|
||||
def save(ref: str, output: str) -> None:
|
||||
"""`docker save REF -o OUTPUT`. Writes a tarball of the image
|
||||
layers + manifest to the host path. Used by smolmachines
|
||||
|
||||
@@ -67,11 +67,8 @@ class MacosContainerBottleBackend(
|
||||
stage_dir=stage_dir,
|
||||
)
|
||||
|
||||
def _image_stale_checks(self, plan: MacosContainerBottlePlan) -> None:
|
||||
_launch.stale_checks(plan)
|
||||
|
||||
@contextmanager
|
||||
def _launch_impl(
|
||||
def launch(
|
||||
self, plan: MacosContainerBottlePlan
|
||||
) -> Generator[MacosContainerBottle, None, None]:
|
||||
with _launch.launch(plan, provision=self.provision) as bottle:
|
||||
|
||||
@@ -28,13 +28,9 @@ from ...egress import (
|
||||
egress_resolve_token_values,
|
||||
egress_sidecar_env_entries,
|
||||
)
|
||||
from ...git_gate import (
|
||||
provision_git_gate_dynamic_keys,
|
||||
revoke_git_gate_provisioned_keys,
|
||||
)
|
||||
from ...image_cache import check_stale
|
||||
from ...git_gate import revoke_git_gate_provisioned_keys
|
||||
from ...log import die, info, warn
|
||||
from ...supervise import DB_PATH_IN_CONTAINER, SUPERVISE_PORT
|
||||
from ...supervise import QUEUE_DIR_IN_CONTAINER, SUPERVISE_PORT
|
||||
from ...util import expand_tilde
|
||||
from ..docker.egress import EGRESS_CA_IN_CONTAINER, EGRESS_PORT
|
||||
from ..docker.git_gate import (
|
||||
@@ -102,8 +98,6 @@ def launch(
|
||||
egress_network = egress_network_name(plan.slug)
|
||||
_create_networks(internal_network, egress_network, stack)
|
||||
|
||||
plan = _provision_git_gate_keys(plan)
|
||||
|
||||
sidecar_name = sidecar_container_name(plan.slug)
|
||||
container_mod.force_remove_container(sidecar_name)
|
||||
_start_sidecar_bundle(plan, sidecar_name, internal_network, egress_network)
|
||||
@@ -150,53 +144,29 @@ def _mint_certs(plan: MacosContainerBottlePlan) -> MacosContainerBottlePlan:
|
||||
|
||||
|
||||
def _build_images(plan: MacosContainerBottlePlan) -> MacosContainerBottlePlan:
|
||||
cached = plan.spec.image_policy == "cached"
|
||||
container_mod.build_image(
|
||||
SIDECAR_BUNDLE_IMAGE,
|
||||
_REPO_DIR,
|
||||
dockerfile=SIDECAR_BUNDLE_DOCKERFILE,
|
||||
)
|
||||
committed = read_committed_image(plan.slug)
|
||||
if committed and container_mod.image_exists(committed):
|
||||
info(f"using committed image {committed!r}")
|
||||
if not cached:
|
||||
container_mod.build_image(SIDECAR_BUNDLE_IMAGE, _REPO_DIR, dockerfile=SIDECAR_BUNDLE_DOCKERFILE)
|
||||
return dataclasses.replace(
|
||||
plan,
|
||||
agent_provision=dataclasses.replace(plan.agent_provision, image=committed),
|
||||
agent_provision=dataclasses.replace(
|
||||
plan.agent_provision,
|
||||
image=committed,
|
||||
),
|
||||
)
|
||||
if cached:
|
||||
if not container_mod.image_exists(plan.image):
|
||||
die(
|
||||
f"cached agent image {plan.image!r} not found; "
|
||||
"run without --cached-images to build it"
|
||||
)
|
||||
if not container_mod.image_exists(SIDECAR_BUNDLE_IMAGE):
|
||||
die(
|
||||
f"cached sidecar image {SIDECAR_BUNDLE_IMAGE!r} not found; "
|
||||
"run without --cached-images to build it"
|
||||
)
|
||||
info(f"using cached agent image {plan.image!r}")
|
||||
info(f"using cached sidecar image {SIDECAR_BUNDLE_IMAGE!r}")
|
||||
return plan
|
||||
container_mod.build_image(SIDECAR_BUNDLE_IMAGE, _REPO_DIR, dockerfile=SIDECAR_BUNDLE_DOCKERFILE)
|
||||
container_mod.build_image(plan.image, _REPO_DIR, dockerfile=plan.dockerfile_path)
|
||||
container_mod.build_image(
|
||||
plan.image,
|
||||
_REPO_DIR,
|
||||
dockerfile=plan.dockerfile_path,
|
||||
)
|
||||
return plan
|
||||
|
||||
|
||||
def stale_checks(plan: MacosContainerBottlePlan) -> None:
|
||||
"""Raise StaleImageError if a cached image is older than the configured
|
||||
threshold. Only runs when image_policy is 'cached'. Called by the backend
|
||||
class's _image_stale_checks before _launch_impl starts any resources."""
|
||||
if plan.spec.image_policy != "cached":
|
||||
return
|
||||
committed = read_committed_image(plan.slug)
|
||||
if committed and container_mod.image_exists(committed):
|
||||
check_stale(f"agent image {committed!r}", container_mod.image_created_at(committed))
|
||||
return
|
||||
for label, ref in [
|
||||
("agent image", plan.image),
|
||||
("sidecar image", SIDECAR_BUNDLE_IMAGE),
|
||||
]:
|
||||
if container_mod.image_exists(ref):
|
||||
check_stale(f"{label} {ref!r}", container_mod.image_created_at(ref))
|
||||
|
||||
|
||||
def _create_networks(
|
||||
internal_network: str,
|
||||
egress_network: str,
|
||||
@@ -271,19 +241,6 @@ def _stamp_agent_urls(
|
||||
)
|
||||
|
||||
|
||||
def _provision_git_gate_keys(
|
||||
plan: MacosContainerBottlePlan,
|
||||
) -> MacosContainerBottlePlan:
|
||||
if not plan.git_gate_plan.upstreams:
|
||||
return plan
|
||||
git_gate_plan = provision_git_gate_dynamic_keys(
|
||||
plan.manifest.bottle,
|
||||
plan.git_gate_plan,
|
||||
git_gate_state_dir(plan.slug),
|
||||
)
|
||||
return dataclasses.replace(plan, git_gate_plan=git_gate_plan)
|
||||
|
||||
|
||||
def _stage_git_gate(plan: MacosContainerBottlePlan, sidecar_name: str) -> None:
|
||||
gp = plan.git_gate_plan
|
||||
if not gp.upstreams:
|
||||
@@ -404,7 +361,7 @@ def _sidecar_env_entries(plan: MacosContainerBottlePlan) -> tuple[str, ...]:
|
||||
if plan.supervise_plan is not None:
|
||||
env += [
|
||||
f"SUPERVISE_BOTTLE_SLUG={plan.slug}",
|
||||
f"SUPERVISE_DB_PATH={DB_PATH_IN_CONTAINER}",
|
||||
f"SUPERVISE_QUEUE_DIR={QUEUE_DIR_IN_CONTAINER}",
|
||||
f"SUPERVISE_PORT={SUPERVISE_PORT}",
|
||||
]
|
||||
return tuple(env)
|
||||
@@ -430,15 +387,7 @@ def _sidecar_mounts(
|
||||
|
||||
sp = plan.supervise_plan
|
||||
if sp is not None:
|
||||
# `container run --mount type=bind` only accepts directory
|
||||
# sources (a file source fails with "is not a directory") —
|
||||
# mount db_path's dedicated parent dir instead of the file
|
||||
# itself, same as the CA/routes mounts above.
|
||||
mounts.append((
|
||||
str(sp.db_path.parent),
|
||||
str(Path(DB_PATH_IN_CONTAINER).parent),
|
||||
False,
|
||||
))
|
||||
mounts.append((str(sp.queue_dir), QUEUE_DIR_IN_CONTAINER, False))
|
||||
|
||||
return tuple(mounts)
|
||||
|
||||
|
||||
@@ -10,7 +10,6 @@ import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
import time
|
||||
from datetime import datetime, timezone
|
||||
from typing import Iterable
|
||||
|
||||
from ...log import die, info
|
||||
@@ -459,39 +458,6 @@ def image_id(ref: str) -> str:
|
||||
raise AssertionError("unreachable")
|
||||
|
||||
|
||||
def image_created_at(ref: str) -> datetime:
|
||||
"""Return the image creation timestamp as an aware UTC datetime.
|
||||
|
||||
Parses the `created` field from `container image inspect` JSON output."""
|
||||
result = subprocess.run(
|
||||
[_CONTAINER, "image", "inspect", ref],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
die(
|
||||
f"container image inspect for {ref!r} failed: "
|
||||
f"{(result.stderr or '').strip() or '<no stderr>'}"
|
||||
)
|
||||
try:
|
||||
data = json.loads(result.stdout or "{}")
|
||||
except json.JSONDecodeError as exc:
|
||||
die(f"container image inspect for {ref!r} returned malformed JSON: {exc}")
|
||||
if isinstance(data, list) and data:
|
||||
data = data[0]
|
||||
if isinstance(data, dict):
|
||||
value = data.get("created") or data.get("Created")
|
||||
if isinstance(value, str) and value:
|
||||
try:
|
||||
ts = value.rstrip("Z")
|
||||
return datetime.fromisoformat(ts).replace(tzinfo=timezone.utc)
|
||||
except ValueError:
|
||||
pass
|
||||
die(f"container image inspect for {ref!r} did not include a creation timestamp")
|
||||
raise AssertionError("unreachable")
|
||||
|
||||
|
||||
def save(ref: str, output: str) -> None:
|
||||
subprocess.run([_CONTAINER, "image", "save", ref, "-o", output], check=True)
|
||||
|
||||
|
||||
@@ -77,11 +77,8 @@ class SmolmachinesBottleBackend(
|
||||
stage_dir=stage_dir,
|
||||
)
|
||||
|
||||
def _image_stale_checks(self, plan: SmolmachinesBottlePlan) -> None:
|
||||
_launch.stale_checks(plan)
|
||||
|
||||
@contextmanager
|
||||
def _launch_impl(
|
||||
def launch(
|
||||
self, plan: SmolmachinesBottlePlan
|
||||
) -> Generator[SmolmachinesBottle, None, None]:
|
||||
with _launch.launch(plan, provision=self.provision) as bottle:
|
||||
|
||||
@@ -47,24 +47,10 @@ _HOME_FOR = {
|
||||
"root": "/root",
|
||||
}
|
||||
|
||||
_DEFAULT_PATH_FOR = {
|
||||
# Committed smolmachine snapshots are rebuilt from a rootfs tarball and
|
||||
# lose Docker image ENV metadata. Restore the provider CLI path here so
|
||||
# resumed Codex bottles can still find the per-user install.
|
||||
"node": (
|
||||
"/home/node/.local/bin:"
|
||||
"/home/node/.codex/packages/standalone/current/bin:"
|
||||
"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
|
||||
),
|
||||
"root": "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
|
||||
}
|
||||
|
||||
|
||||
def _env_assignments_for(user: str, env: Mapping[str, str]) -> list[str]:
|
||||
home = _HOME_FOR.get(user, f"/home/{user}")
|
||||
out = [f"HOME={home}", f"USER={user}"]
|
||||
if "PATH" not in env:
|
||||
out.append(f"PATH={_DEFAULT_PATH_FOR.get(user, _DEFAULT_PATH_FOR['root'])}")
|
||||
for k, v in env.items():
|
||||
out.append(f"{k}={v}")
|
||||
return out
|
||||
|
||||
@@ -1,51 +1,20 @@
|
||||
"""Egress apply for the smolmachines backend.
|
||||
|
||||
The smolmachines sidecar bundle runs as a sidecar smolVM. Route-file
|
||||
inspection and reload signalling therefore go through ``smolvm machine
|
||||
exec`` instead of Docker.
|
||||
The smolmachines sidecar bundle runs as a host-side Docker container,
|
||||
so egress signalling is identical to the docker backend.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from ...egress import EGRESS_ROUTES_IN_CONTAINER
|
||||
from ...log import warn
|
||||
from ..egress_apply import EgressApplicator, EgressApplyError
|
||||
from . import sidecar_bundle as _bundle
|
||||
from . import smolvm as _smolvm
|
||||
|
||||
|
||||
def fetch_current_routes(slug: str) -> str:
|
||||
machine = _bundle.bundle_machine_name(slug)
|
||||
result = _smolvm.machine_exec(machine, ["cat", EGRESS_ROUTES_IN_CONTAINER])
|
||||
if result.returncode != 0:
|
||||
raise EgressApplyError(
|
||||
f"could not read routes.yaml from {machine}: "
|
||||
f"{(result.stderr or '').strip() or 'sidecar VM not running?'}"
|
||||
)
|
||||
return result.stdout
|
||||
|
||||
|
||||
class SmolmachinesEgressApplicator(EgressApplicator):
|
||||
def _signal_bundle_reload(self, slug: str) -> None:
|
||||
machine = _bundle.bundle_machine_name(slug)
|
||||
result = _smolvm.machine_exec(machine, ["sh", "-c", "kill -HUP 1"])
|
||||
if result.returncode != 0:
|
||||
last_error = (result.stderr or "").strip() or (result.stdout or "").strip()
|
||||
warn(
|
||||
f"egress: routes updated on disk for {slug}, but bundle reload failed: "
|
||||
f"{last_error or 'smolvm exec failed'}"
|
||||
)
|
||||
raise EgressApplyError(
|
||||
f"could not reload egress bundle {machine}: "
|
||||
f"{last_error or 'smolvm exec failed'}"
|
||||
)
|
||||
|
||||
|
||||
applicator = SmolmachinesEgressApplicator()
|
||||
|
||||
from ..docker.egress_apply import ( # noqa: F401
|
||||
DockerEgressApplicator,
|
||||
EgressApplyError,
|
||||
applicator,
|
||||
fetch_current_routes,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"SmolmachinesEgressApplicator",
|
||||
"DockerEgressApplicator",
|
||||
"EgressApplyError",
|
||||
"applicator",
|
||||
"fetch_current_routes",
|
||||
|
||||
@@ -31,7 +31,6 @@ from . import sidecar_bundle as _bundle
|
||||
# matching the bundle container name pattern. We use the prefix
|
||||
# both as a filter and to strip back to the slug.
|
||||
_VM_NAME_PREFIX = "bot-bottle-"
|
||||
_SIDECAR_VM_PREFIX = _bundle.bundle_machine_name("")
|
||||
|
||||
|
||||
def enumerate_active() -> list[ActiveAgent]:
|
||||
@@ -55,11 +54,7 @@ def enumerate_active() -> list[ActiveAgent]:
|
||||
for m in machines:
|
||||
name = m.get("name") or ""
|
||||
state = m.get("state") or ""
|
||||
if (
|
||||
state != "running"
|
||||
or not name.startswith(_VM_NAME_PREFIX)
|
||||
or name.startswith(_SIDECAR_VM_PREFIX)
|
||||
):
|
||||
if state != "running" or not name.startswith(_VM_NAME_PREFIX):
|
||||
continue
|
||||
slug = name[len(_VM_NAME_PREFIX):]
|
||||
metadata = read_metadata(slug)
|
||||
|
||||
@@ -27,7 +27,7 @@ from ...egress import (
|
||||
egress_resolve_token_values,
|
||||
egress_sidecar_env_entries,
|
||||
)
|
||||
from ...supervise import DB_PATH_IN_CONTAINER, SUPERVISE_PORT
|
||||
from ...supervise import QUEUE_DIR_IN_CONTAINER, SUPERVISE_PORT
|
||||
from ...util import expand_tilde
|
||||
from ..docker import util as docker_mod
|
||||
from ..docker.egress import (
|
||||
@@ -41,19 +41,14 @@ from ..docker.git_gate import (
|
||||
GIT_GATE_ENTRYPOINT_IN_CONTAINER,
|
||||
GIT_GATE_HOOK_IN_CONTAINER,
|
||||
)
|
||||
from ...git_gate import (
|
||||
provision_git_gate_dynamic_keys,
|
||||
revoke_git_gate_provisioned_keys,
|
||||
)
|
||||
from ...image_cache import check_stale_path
|
||||
from ...log import die, info, warn
|
||||
from ...git_gate import revoke_git_gate_provisioned_keys
|
||||
from ...log import info, warn
|
||||
from ...bottle_state import (
|
||||
egress_state_dir,
|
||||
git_gate_state_dir,
|
||||
read_committed_image,
|
||||
)
|
||||
from . import loopback_alias as _loopback
|
||||
from . import port_forward as _forward
|
||||
from . import sidecar_bundle as _bundle
|
||||
from . import smolvm as _smolvm
|
||||
from .bottle import SmolmachinesBottle
|
||||
@@ -93,12 +88,12 @@ def launch(
|
||||
try:
|
||||
loopback_ip, network = _allocate_resources(plan, stack)
|
||||
plan = _mint_certs(plan)
|
||||
proxy_host = loopback_ip
|
||||
plan = _start_bundle(plan, network, proxy_host, stack)
|
||||
plan = _start_bundle(plan, network, loopback_ip, stack)
|
||||
plan = _discover_urls(plan, loopback_ip)
|
||||
|
||||
agent_from_path = _agent_from_path(plan)
|
||||
|
||||
_launch_vm(plan, agent_from_path, proxy_host, stack)
|
||||
_launch_vm(plan, agent_from_path, loopback_ip, stack)
|
||||
_init_vm(plan)
|
||||
|
||||
bottle = SmolmachinesBottle(
|
||||
@@ -144,17 +139,17 @@ def _allocate_resources(
|
||||
plan: SmolmachinesBottlePlan,
|
||||
stack: ExitStack,
|
||||
) -> tuple[str, str]:
|
||||
"""Reserve a per-bottle host address.
|
||||
"""Reserve a loopback alias and create the per-bottle docker bridge.
|
||||
|
||||
The per-bottle address scopes TSI's allowlist to this bottle's
|
||||
forwarder-published ports so the agent can't reach other bottles'
|
||||
or host services. The returned network name remains in the bundle
|
||||
spec for compatibility with older helper tests; no Docker sidecar
|
||||
container is launched."""
|
||||
del stack
|
||||
macOS only routes 127.0.0.1 by default; the per-bottle alias
|
||||
scopes TSI's allowlist to this bottle's published ports so the
|
||||
agent can't reach other bottles' or host services' ports on
|
||||
loopback. No-op on Linux."""
|
||||
_loopback.ensure_pool()
|
||||
loopback_ip = _loopback.allocate(plan.slug)
|
||||
network = _bundle.bundle_network_name(plan.slug)
|
||||
_bundle.create_bundle_network(network, plan.bundle_subnet, plan.bundle_gateway)
|
||||
stack.callback(_bundle.remove_bundle_network, network)
|
||||
return loopback_ip, network
|
||||
|
||||
|
||||
@@ -174,89 +169,54 @@ def _mint_certs(plan: SmolmachinesBottlePlan) -> SmolmachinesBottlePlan:
|
||||
def _start_bundle(
|
||||
plan: SmolmachinesBottlePlan,
|
||||
network: str,
|
||||
proxy_host: str,
|
||||
loopback_ip: str,
|
||||
stack: ExitStack,
|
||||
) -> SmolmachinesBottlePlan:
|
||||
"""Build the BundleLaunchSpec, start the sidecar VM, wrap its raw
|
||||
smolVM-published loopback ports with per-bottle forwarders, stamp
|
||||
agent URLs from those forwarder ports, and register teardown."""
|
||||
plan = _provision_git_gate_keys(plan)
|
||||
bundle_spec = _bundle_launch_spec(plan, network, proxy_host)
|
||||
"""Build the BundleLaunchSpec, resolve token env, start the
|
||||
sidecar bundle container, and register teardown."""
|
||||
bundle_spec = _bundle_launch_spec(plan, network, loopback_ip)
|
||||
token_env = _resolve_token_env(plan, dict(os.environ))
|
||||
if _image_policy(plan) == "cached":
|
||||
artifact = _cached_smolmachine(bundle_spec.image, label="sidecar")
|
||||
else:
|
||||
artifact = _ensure_smolmachine(
|
||||
bundle_spec.image,
|
||||
dockerfile=_bundle.SIDECAR_BUNDLE_DOCKERFILE,
|
||||
)
|
||||
launch = _bundle.start_bundle_vm(
|
||||
bundle_spec,
|
||||
from_path=artifact,
|
||||
host_env={**os.environ, **token_env},
|
||||
)
|
||||
stack.callback(_bundle.stop_bundle_vm, plan.slug)
|
||||
|
||||
forward_specs = tuple(
|
||||
_forward.ForwardSpec(
|
||||
label=_label_for_port(container_port),
|
||||
listen_host=proxy_host,
|
||||
listen_port=0,
|
||||
target_host="127.0.0.1",
|
||||
target_port=host_port,
|
||||
)
|
||||
for container_port, host_port in launch.raw_ports.items()
|
||||
)
|
||||
handle = _forward.start_forwarder(forward_specs)
|
||||
stack.callback(_forward.stop_forwarder, handle)
|
||||
published_ports = {
|
||||
_port_for_label(spec.label): spec.listen_port
|
||||
for spec in handle.forwards
|
||||
}
|
||||
return _discover_urls(plan, proxy_host, published_ports)
|
||||
|
||||
|
||||
def _provision_git_gate_keys(
|
||||
plan: SmolmachinesBottlePlan,
|
||||
) -> SmolmachinesBottlePlan:
|
||||
if not plan.git_gate_plan.upstreams:
|
||||
return plan
|
||||
git_gate_plan = provision_git_gate_dynamic_keys(
|
||||
plan.manifest.bottle,
|
||||
plan.git_gate_plan,
|
||||
git_gate_state_dir(plan.slug),
|
||||
)
|
||||
return dataclasses.replace(plan, git_gate_plan=git_gate_plan)
|
||||
_bundle.ensure_bundle_image(bundle_spec.image)
|
||||
_bundle.start_bundle(bundle_spec, env={**os.environ, **token_env})
|
||||
stack.callback(_bundle.stop_bundle, plan.slug)
|
||||
return plan
|
||||
|
||||
|
||||
def _discover_urls(
|
||||
plan: SmolmachinesBottlePlan,
|
||||
proxy_host: str,
|
||||
published_ports: dict[int, int],
|
||||
loopback_ip: str,
|
||||
) -> SmolmachinesBottlePlan:
|
||||
"""Stamp URLs + guest_env from per-bottle forwarder ports.
|
||||
"""Discover host-side ports for published container ports and
|
||||
return the plan with URLs + guest_env stamped in.
|
||||
|
||||
`proxy_host` is the host IP that both TSI's allowlist and the
|
||||
forwarder listeners are keyed to. The raw smolVM-published ports
|
||||
are intentionally not advertised to the agent.
|
||||
Docker container IPs (192.168.x.x in the daemon's bridge)
|
||||
aren't reachable from the smolvm guest on macOS — TSI uses
|
||||
macOS networking, and macOS sees the daemon's bridge via the
|
||||
published-port loopback forward only.
|
||||
|
||||
NO_PROXY includes `proxy_host` so supervise + git-gate URLs
|
||||
bypass HTTPS_PROXY."""
|
||||
agent_facing_host_port = published_ports[_EGRESS_PORT]
|
||||
agent_proxy_url = f"http://{proxy_host}:{agent_facing_host_port}"
|
||||
NO_PROXY includes the per-bottle loopback alias so the
|
||||
supervise + git-gate URLs bypass HTTPS_PROXY."""
|
||||
agent_facing_host_port = _bundle.bundle_host_port(
|
||||
plan.slug, _EGRESS_PORT, host_ip=loopback_ip,
|
||||
)
|
||||
agent_proxy_url = f"http://{loopback_ip}:{agent_facing_host_port}"
|
||||
|
||||
agent_git_gate_host = ""
|
||||
if plan.git_gate_plan.upstreams:
|
||||
git_gate_host_port = published_ports[_GIT_HTTP_PORT]
|
||||
agent_git_gate_host = f"{proxy_host}:{git_gate_host_port}"
|
||||
git_gate_host_port = _bundle.bundle_host_port(
|
||||
plan.slug, _GIT_HTTP_PORT, host_ip=loopback_ip,
|
||||
)
|
||||
agent_git_gate_host = f"{loopback_ip}:{git_gate_host_port}"
|
||||
|
||||
agent_supervise_url = ""
|
||||
if plan.supervise_plan is not None:
|
||||
supervise_host_port = published_ports[_SUPERVISE_PORT]
|
||||
agent_supervise_url = f"http://{proxy_host}:{supervise_host_port}/"
|
||||
supervise_host_port = _bundle.bundle_host_port(
|
||||
plan.slug, _SUPERVISE_PORT, host_ip=loopback_ip,
|
||||
)
|
||||
agent_supervise_url = f"http://{loopback_ip}:{supervise_host_port}/"
|
||||
|
||||
existing_no_proxy = plan.guest_env.get("NO_PROXY", "localhost,127.0.0.1")
|
||||
no_proxy = f"{existing_no_proxy},{proxy_host}"
|
||||
no_proxy = f"{existing_no_proxy},{loopback_ip}"
|
||||
guest_env = {
|
||||
**plan.guest_env,
|
||||
"HTTPS_PROXY": agent_proxy_url,
|
||||
@@ -286,31 +246,27 @@ def _discover_urls(
|
||||
def _launch_vm(
|
||||
plan: SmolmachinesBottlePlan,
|
||||
agent_from_path: Path,
|
||||
proxy_host: str,
|
||||
loopback_ip: str,
|
||||
stack: ExitStack,
|
||||
) -> None:
|
||||
"""Create, patch, and start the smolvm VM; register teardown.
|
||||
|
||||
--allow-cidr is `proxy_host/32` — the per-bottle loopback alias.
|
||||
This ensures the guest can only reach sidecar forwarders published
|
||||
on that IP, not host localhost or another bottle's alias.
|
||||
force_allowlist confirms the allowlist persisted (patching smolvm
|
||||
0.8.0's silent-drop of --allow-cidr when combined with --from) and
|
||||
fails closed if it can't. Smolfile isn't usable here — smolvm 0.8.0
|
||||
makes --from and --smolfile mutually exclusive."""
|
||||
tsi_cidr = f"{proxy_host}/32"
|
||||
--allow-cidr is the per-bottle loopback alias so the guest can
|
||||
only reach this bottle's bundle ports. force_allowlist patches
|
||||
smolvm 0.8.0's silent-drop of --allow-cidr when combined with
|
||||
--from. Smolfile isn't usable here — smolvm 0.8.0 makes --from
|
||||
and --smolfile mutually exclusive."""
|
||||
_smolvm.machine_create(
|
||||
plan.machine_name,
|
||||
from_path=agent_from_path,
|
||||
allow_cidrs=[tsi_cidr],
|
||||
allow_cidrs=[f"{loopback_ip}/32"],
|
||||
env=plan.guest_env,
|
||||
)
|
||||
stack.callback(_smolvm.machine_delete, plan.machine_name)
|
||||
# Confirm the booted VM's TSI allowlist will actually enforce the
|
||||
# /32 before start (smolvm 0.8.0 silently drops `--allow-cidr`
|
||||
# with `--from`, so the persisted state DB is patched if needed).
|
||||
# Fails closed if enforcement can't be confirmed.
|
||||
_loopback.force_allowlist(plan.machine_name, [tsi_cidr])
|
||||
# Workaround smolvm 0.8.0: `--allow-cidr` is silently dropped
|
||||
# when combined with `--from`. Patch the persisted state DB
|
||||
# before start so the booted VM's TSI actually enforces.
|
||||
_loopback.force_allowlist(plan.machine_name, [f"{loopback_ip}/32"])
|
||||
_smolvm.machine_start(plan.machine_name)
|
||||
stack.callback(_smolvm.machine_stop, plan.machine_name)
|
||||
|
||||
@@ -319,9 +275,7 @@ def _init_vm(plan: SmolmachinesBottlePlan) -> None:
|
||||
"""Repair filesystem ownership and wait for exec channel readiness.
|
||||
|
||||
Ownership repair: smolvm's pack process remaps files to the host
|
||||
invoker's uid (e.g. 501 on macOS, 1000 on Linux). The chowns use
|
||||
names not numbers so they're correct on either. /home/node must
|
||||
be node:node so
|
||||
invoker's uid (501 on macOS). /home/node must be node:node so
|
||||
Claude Code can write ~/.claude.json; /tmp + /var/tmp need root
|
||||
mode 1777 so non-root processes can create per-uid scratch dirs.
|
||||
All folded into one sh -c to avoid back-to-back exec calls
|
||||
@@ -344,30 +298,8 @@ def _init_vm(plan: SmolmachinesBottlePlan) -> None:
|
||||
_smolvm.wait_exec_ready(plan.machine_name)
|
||||
|
||||
|
||||
def _label_for_port(port: int) -> str:
|
||||
if port == _EGRESS_PORT:
|
||||
return "egress"
|
||||
if port == _GIT_HTTP_PORT:
|
||||
return "git-http"
|
||||
if port == _SUPERVISE_PORT:
|
||||
return "supervise"
|
||||
return f"port-{port}"
|
||||
|
||||
|
||||
def _port_for_label(label: str) -> int:
|
||||
if label == "egress":
|
||||
return _EGRESS_PORT
|
||||
if label == "git-http":
|
||||
return _GIT_HTTP_PORT
|
||||
if label == "supervise":
|
||||
return _SUPERVISE_PORT
|
||||
if label.startswith("port-"):
|
||||
return int(label.removeprefix("port-"))
|
||||
raise ValueError(f"unknown sidecar forward label: {label}")
|
||||
|
||||
|
||||
def _bundle_launch_spec(
|
||||
plan: SmolmachinesBottlePlan, network: str, proxy_host: str,
|
||||
plan: SmolmachinesBottlePlan, network: str, loopback_ip: str,
|
||||
) -> _bundle.BundleLaunchSpec:
|
||||
"""Build a BundleLaunchSpec from the resolved inner Plans.
|
||||
|
||||
@@ -420,15 +352,14 @@ def _bundle_launch_spec(
|
||||
daemons.append("supervise")
|
||||
env += [
|
||||
f"SUPERVISE_BOTTLE_SLUG={plan.slug}",
|
||||
f"SUPERVISE_DB_PATH={DB_PATH_IN_CONTAINER}",
|
||||
f"SUPERVISE_QUEUE_DIR={QUEUE_DIR_IN_CONTAINER}",
|
||||
f"SUPERVISE_PORT={SUPERVISE_PORT}",
|
||||
]
|
||||
volumes.append((str(sp.db_path), DB_PATH_IN_CONTAINER, False))
|
||||
volumes.append((str(sp.queue_dir), QUEUE_DIR_IN_CONTAINER, False))
|
||||
|
||||
# Container ports the agent reaches from the smolvm guest —
|
||||
# published on `proxy_host` so the TSI allowlist and the docker
|
||||
# port-forward bindings point at the same IP. Egress is always
|
||||
# the agent's HTTP/HTTPS proxy.
|
||||
# published on host loopback so the guest can dial via TSI +
|
||||
# macOS networking. Egress is always the agent's HTTP/HTTPS proxy.
|
||||
ports_to_publish: list[int] = [_EGRESS_PORT]
|
||||
if gp.upstreams:
|
||||
ports_to_publish.append(_GIT_HTTP_PORT)
|
||||
@@ -445,7 +376,7 @@ def _bundle_launch_spec(
|
||||
environment=tuple(env),
|
||||
volumes=tuple(volumes),
|
||||
ports_to_publish=tuple(ports_to_publish),
|
||||
publish_host_ip=proxy_host,
|
||||
publish_host_ip=loopback_ip,
|
||||
)
|
||||
|
||||
|
||||
@@ -473,9 +404,6 @@ def _agent_from_path(plan: SmolmachinesBottlePlan) -> Path:
|
||||
info(f"using committed smolmachine {str(committed_path)!r}")
|
||||
return committed_path
|
||||
|
||||
if _image_policy(plan) == "cached":
|
||||
return _cached_smolmachine(plan.agent_image, label="agent")
|
||||
|
||||
# Build the agent image and pack it into a `.smolmachine`
|
||||
# artifact (or hit the per-Dockerfile-digest cache). Runs here,
|
||||
# not in prepare, so the docker-build output doesn't garble the
|
||||
@@ -486,60 +414,6 @@ def _agent_from_path(plan: SmolmachinesBottlePlan) -> Path:
|
||||
)
|
||||
|
||||
|
||||
def stale_checks(plan: SmolmachinesBottlePlan) -> None:
|
||||
"""Raise StaleImageError if a cached smolmachine artifact is older than the
|
||||
configured threshold. Only runs when image_policy is 'cached'. Checks the
|
||||
committed agent artifact first, then the cache-keyed agent and sidecar
|
||||
artifacts. Called by the backend class's _image_stale_checks before any
|
||||
resources are allocated."""
|
||||
if _image_policy(plan) != "cached":
|
||||
return
|
||||
committed = read_committed_image(plan.slug)
|
||||
if committed:
|
||||
committed_path = Path(committed)
|
||||
if committed_path.is_file():
|
||||
check_stale_path("agent smolmachine artifact", committed_path)
|
||||
elif docker_mod.image_exists(plan.agent_image):
|
||||
digest = docker_mod.image_id(plan.agent_image).split(":", 1)[-1][:16]
|
||||
artifact = _SMOLMACHINE_CACHE_DIR / f"{digest}.smolmachine.smolmachine"
|
||||
if artifact.is_file():
|
||||
check_stale_path("agent smolmachine artifact", artifact)
|
||||
sidecar_image = _bundle.SIDECAR_BUNDLE_IMAGE
|
||||
if docker_mod.image_exists(sidecar_image):
|
||||
digest = docker_mod.image_id(sidecar_image).split(":", 1)[-1][:16]
|
||||
sidecar_artifact = _SMOLMACHINE_CACHE_DIR / f"{digest}.smolmachine.smolmachine"
|
||||
if sidecar_artifact.is_file():
|
||||
check_stale_path("sidecar smolmachine artifact", sidecar_artifact)
|
||||
|
||||
|
||||
def _image_policy(plan: object) -> str:
|
||||
spec = getattr(plan, "spec", None)
|
||||
return str(getattr(spec, "image_policy", "fresh"))
|
||||
|
||||
|
||||
def _cached_smolmachine(image_ref: str, *, label: str) -> Path:
|
||||
"""Return the cached smolmachine artifact for the current local image.
|
||||
|
||||
This is intentionally buildless: it inspects the existing local Docker
|
||||
image ID and looks for the artifact keyed by that ID. If either side is
|
||||
missing, the caller must use the fresh path to build/pack it.
|
||||
"""
|
||||
if not docker_mod.image_exists(image_ref):
|
||||
die(
|
||||
f"cached {label} image {image_ref!r} not found; "
|
||||
"run without --cached-images to build it"
|
||||
)
|
||||
digest = docker_mod.image_id(image_ref).split(":", 1)[-1][:16]
|
||||
sidecar = _SMOLMACHINE_CACHE_DIR / f"{digest}.smolmachine.smolmachine"
|
||||
if not sidecar.is_file():
|
||||
die(
|
||||
f"cached {label} smolmachine artifact for {image_ref!r} not found; "
|
||||
"run without --cached-images to build it"
|
||||
)
|
||||
info(f"using cached {label} smolmachine {str(sidecar)!r}")
|
||||
return sidecar
|
||||
|
||||
|
||||
def _ensure_smolmachine(image_ref: str, *, dockerfile: str = "") -> Path:
|
||||
"""Build the agent docker image and convert it into a
|
||||
`.smolmachine` artifact, caching the result under
|
||||
@@ -576,12 +450,6 @@ def _ensure_smolmachine(image_ref: str, *, dockerfile: str = "") -> Path:
|
||||
return sidecar
|
||||
tarball = _SMOLMACHINE_CACHE_DIR / f"{digest}.image.tar"
|
||||
docker_mod.save(image_ref, str(tarball))
|
||||
# On Linux, `docker save -o` writes the tarball with owner-only
|
||||
# permissions (mode 600). The crane push container runs as UID
|
||||
# 65532 (distroless nonroot) and can't read it through a bind
|
||||
# mount unless world-read is set. The tarball is temporary and
|
||||
# lives in ~/.cache, so 644 is safe.
|
||||
tarball.chmod(0o644)
|
||||
try:
|
||||
with ephemeral_registry() as handle:
|
||||
push_ref = f"{handle.push_endpoint}/bot-bottle:{digest}"
|
||||
|
||||
@@ -33,13 +33,10 @@ sudo-add the missing pool on first use per boot — the aliases
|
||||
persist on `lo0` until reboot, so subsequent launches don't
|
||||
prompt.
|
||||
|
||||
On Linux the whole `127.0.0.0/8` is already routed to `lo`, so
|
||||
docker can publish a bundle's ports directly on `127.0.0.<N>`
|
||||
with no `ifconfig`/sudo step. `ensure_pool` is therefore a no-op
|
||||
on Linux, but per-bottle alias *allocation* and the TSI allowlist
|
||||
DB patch run on both platforms — the isolation property is
|
||||
identical, it's just cheaper to set up on Linux. The state-DB
|
||||
path differs per platform (see `_smolvm_db_path`).
|
||||
Linux native daemons share the host's network namespace; the
|
||||
whole `127.0.0.0/8` is reachable by default and aliases are
|
||||
unnecessary. The pool logic detects native-Linux and skips sudo
|
||||
entirely; the DB patch is also gated on macOS.
|
||||
|
||||
Allocation is coordinated by inspecting running bundle
|
||||
containers' published host IPs — each bottle's bundle owns the
|
||||
@@ -50,7 +47,6 @@ from __future__ import annotations
|
||||
|
||||
import fcntl
|
||||
import json
|
||||
import os
|
||||
import platform
|
||||
import re
|
||||
import sqlite3
|
||||
@@ -61,34 +57,20 @@ from typing import Iterable
|
||||
from ...log import die, info
|
||||
|
||||
|
||||
def _smolvm_db_path() -> Path:
|
||||
"""smolvm's persistent VM state — a SQLite DB whose `vms` table
|
||||
holds one JSON BLOB per machine. macOS stores it under
|
||||
`Application Support`; Linux follows the XDG base-dir spec
|
||||
(`$XDG_DATA_HOME`, default `~/.local/share`).
|
||||
|
||||
NOTE: the Linux location is inferred from smolvm's documented
|
||||
`~/.local/share` install layout and must be confirmed against a
|
||||
real Linux smolvm install. If it's wrong, `force_allowlist`'s
|
||||
fail-closed check turns it into a clear launch-time error rather
|
||||
than a silent escape."""
|
||||
if platform.system() == "Darwin":
|
||||
return (
|
||||
Path.home()
|
||||
/ "Library"
|
||||
/ "Application Support"
|
||||
/ "smolvm"
|
||||
/ "server"
|
||||
/ "smolvm.db"
|
||||
)
|
||||
xdg_data = os.environ.get("XDG_DATA_HOME")
|
||||
base = Path(xdg_data) if xdg_data else Path.home() / ".local" / "share"
|
||||
return base / "smolvm" / "server" / "smolvm.db"
|
||||
|
||||
|
||||
# Resolved once at import: the host platform doesn't change within a
|
||||
# process. Tests patch this attribute directly.
|
||||
_SMOLVM_DB_PATH = _smolvm_db_path()
|
||||
# smolvm's persistent VM state on macOS — a SQLite DB whose `vms`
|
||||
# table holds one JSON BLOB per machine. The Linux path is
|
||||
# different, but smolmachines is macOS-only in v1 (PRD 0023) so
|
||||
# we hard-code this. If the file moves under us we'll see a
|
||||
# clear FileNotFoundError; not worth defensive cross-platform
|
||||
# detection until the backend actually needs Linux.
|
||||
_SMOLVM_DB_PATH = (
|
||||
Path.home()
|
||||
/ "Library"
|
||||
/ "Application Support"
|
||||
/ "smolvm"
|
||||
/ "server"
|
||||
/ "smolvm.db"
|
||||
)
|
||||
|
||||
|
||||
# Sixteen aliases by default. Tunable for hosts that want more
|
||||
@@ -149,74 +131,51 @@ def ensure_pool() -> None:
|
||||
|
||||
|
||||
def force_allowlist(machine_name: str, allowed_cidrs: list[str]) -> None:
|
||||
"""Ensure the machine's persisted TSI allowlist equals
|
||||
`allowed_cidrs`, failing **closed** if that can't be confirmed.
|
||||
"""Patch smolvm's persistent VM-state DB to set the machine's
|
||||
`allowed_cidrs` to the given list. Workaround for smolvm
|
||||
0.8.0's silent-drop of `--allow-cidr` when used with `--from`.
|
||||
|
||||
Runs on both macOS and Linux. It exists because smolvm 0.8.0
|
||||
silently drops `--allow-cidr` when combined with `--from`, so
|
||||
the allowlist has to be written into smolvm's persistent state
|
||||
DB before `machine start`. Rather than assume the flag was
|
||||
dropped, we read the persisted row and only patch when it
|
||||
doesn't already match — so a newer smolvm that honors the flag
|
||||
is left untouched.
|
||||
Must run AFTER `smolvm machine create` (the row has to
|
||||
exist) and BEFORE `smolvm machine start` (smolvm reads the
|
||||
row on start; in-flight VMs don't pick up changes). Once
|
||||
smolvm honors the CLI flag upstream this whole function is
|
||||
redundant — flag-respecting create + remove this call from
|
||||
launch.
|
||||
|
||||
Must run AFTER `smolvm machine create` (the row has to exist)
|
||||
and BEFORE `smolvm machine start` (smolvm reads the row on
|
||||
start; in-flight VMs don't pick up changes).
|
||||
|
||||
Fail-closed: if the state DB is missing, the row is missing, or
|
||||
the allowlist still doesn't match after patching, we `die()`
|
||||
rather than boot a VM whose egress confinement we can't verify
|
||||
— an unconfirmed allowlist is a sandbox-escape risk (the agent
|
||||
VM could reach all of host loopback)."""
|
||||
want = list(allowed_cidrs)
|
||||
No-op on non-macOS — the DB path differs and the Linux
|
||||
smolmachines code path isn't exercised in v1."""
|
||||
if not _is_macos():
|
||||
return
|
||||
if not _SMOLVM_DB_PATH.is_file():
|
||||
die(
|
||||
f"smolvm state DB not found at {_SMOLVM_DB_PATH}; cannot "
|
||||
f"confirm the TSI allowlist is enforced. Refusing to launch "
|
||||
f"(fail-closed). Check `smolvm --version` and the DB "
|
||||
f"location for your platform."
|
||||
f"smolvm state DB not found at {_SMOLVM_DB_PATH}. "
|
||||
f"smolvm 0.8.0 expected? `smolvm --version` to check."
|
||||
)
|
||||
con = sqlite3.connect(str(_SMOLVM_DB_PATH))
|
||||
try:
|
||||
cfg = _read_machine_cfg(con, machine_name)
|
||||
if cfg.get("allowed_cidrs") != want:
|
||||
cfg["allowed_cidrs"] = want
|
||||
# Write as BLOB (the column type smolvm uses) — passing a
|
||||
# plain str makes sqlite store it as Text and smolvm then
|
||||
# fails to read it.
|
||||
con.execute(
|
||||
"UPDATE vms SET data = ? WHERE name = ?",
|
||||
(sqlite3.Binary(json.dumps(cfg).encode()), machine_name),
|
||||
)
|
||||
con.commit()
|
||||
cfg = _read_machine_cfg(con, machine_name)
|
||||
if cfg.get("allowed_cidrs") != want:
|
||||
cur = con.cursor()
|
||||
row = cur.execute(
|
||||
"SELECT data FROM vms WHERE name = ?", (machine_name,),
|
||||
).fetchone()
|
||||
if row is None:
|
||||
die(
|
||||
f"could not enforce TSI allowlist {want!r} for machine "
|
||||
f"{machine_name!r} (persisted value is "
|
||||
f"{cfg.get('allowed_cidrs')!r}). Refusing to launch "
|
||||
f"(fail-closed)."
|
||||
f"smolvm DB has no row for machine {machine_name!r} — "
|
||||
f"machine_create must run before force_allowlist."
|
||||
)
|
||||
cfg = json.loads(row[0])
|
||||
cfg["allowed_cidrs"] = list(allowed_cidrs)
|
||||
# Write as BLOB (the column type smolvm uses) — passing a
|
||||
# plain str makes sqlite store it as Text and smolvm then
|
||||
# fails to read it.
|
||||
cur.execute(
|
||||
"UPDATE vms SET data = ? WHERE name = ?",
|
||||
(sqlite3.Binary(json.dumps(cfg).encode()), machine_name),
|
||||
)
|
||||
con.commit()
|
||||
finally:
|
||||
con.close()
|
||||
|
||||
|
||||
def _read_machine_cfg(con: sqlite3.Connection, machine_name: str) -> dict[str, object]:
|
||||
"""Read + JSON-decode a machine's `data` BLOB from the smolvm
|
||||
state DB. Dies (fail-closed) if the row is missing — the caller
|
||||
can't confirm enforcement without it."""
|
||||
row = con.execute(
|
||||
"SELECT data FROM vms WHERE name = ?", (machine_name,),
|
||||
).fetchone()
|
||||
if row is None:
|
||||
die(
|
||||
f"smolvm DB has no row for machine {machine_name!r} — "
|
||||
f"machine_create must run before force_allowlist."
|
||||
)
|
||||
return json.loads(row[0])
|
||||
|
||||
|
||||
def allocate(_slug: str) -> str:
|
||||
"""Pick the lowest-numbered alias from the pool not already
|
||||
in use by a running smolmachines bundle. Bails when the pool
|
||||
@@ -225,17 +184,16 @@ def allocate(_slug: str) -> str:
|
||||
used (no on-disk reservation, allocation is purely
|
||||
docker-state-driven).
|
||||
|
||||
Runs on both platforms: the allocation logic (docker-state
|
||||
inspection + the file lock) is platform-independent. macOS
|
||||
needs `ensure_pool` to have aliased the addresses on `lo0`
|
||||
first; on Linux all of `127.0.0.0/8` is already loopback, so
|
||||
docker can publish on the chosen `127.0.0.<N>` with no setup.
|
||||
Per-bottle scoping (so the agent can't reach other bottles' or
|
||||
host services' loopback ports) therefore holds on both.
|
||||
On non-macOS the whole `127.0.0.0/8` is loopback by default;
|
||||
`127.0.0.1` is fine to share and we skip the alias dance.
|
||||
This still returns a deterministic address so launch.py's
|
||||
callers don't have to branch on platform.
|
||||
|
||||
An exclusive file lock serialises concurrent calls so two
|
||||
simultaneous launches don't read the same docker state and
|
||||
claim the same alias."""
|
||||
if not _is_macos():
|
||||
return "127.0.0.1"
|
||||
_ALLOC_LOCK_PATH.parent.mkdir(parents=True, exist_ok=True)
|
||||
with open(_ALLOC_LOCK_PATH, "w", encoding="utf-8") as lf:
|
||||
fcntl.flock(lf, fcntl.LOCK_EX)
|
||||
|
||||
@@ -1,245 +0,0 @@
|
||||
"""Per-bottle TCP forwarder for smolmachines sidecar VMs.
|
||||
|
||||
smolVM currently publishes guest ports on host loopback as
|
||||
``HOST:GUEST`` port pairs, without an address-scoped bind. The agent
|
||||
VM's TSI allowlist is IP-only, so bot-bottle exposes a second, stricter
|
||||
surface: listeners bound only to the bottle's allocated host address,
|
||||
forwarding to the raw smolVM-published loopback ports.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import selectors
|
||||
import signal
|
||||
import socket
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
from dataclasses import asdict, dataclass
|
||||
from typing import Any, Iterable, Sequence
|
||||
|
||||
from ...log import die, warn
|
||||
|
||||
|
||||
_BUFFER_SIZE = 64 * 1024
|
||||
_LOOPBACK_TARGETS = {"127.0.0.1", "::1"}
|
||||
_FORBIDDEN_LISTEN_HOSTS = {"", "0.0.0.0", "::", "127.0.0.1", "::1"}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ForwardSpec:
|
||||
label: str
|
||||
listen_host: str
|
||||
listen_port: int
|
||||
target_host: str
|
||||
target_port: int
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ForwarderHandle:
|
||||
process: Any
|
||||
forwards: tuple[ForwardSpec, ...]
|
||||
|
||||
|
||||
def start_forwarder(specs: Sequence[ForwardSpec]) -> ForwarderHandle:
|
||||
"""Start one helper process for a bottle and wait until all
|
||||
listeners are bound. ``listen_port`` may be 0; the returned specs
|
||||
contain the kernel-assigned ports printed by the helper."""
|
||||
if not specs:
|
||||
return ForwarderHandle(process=_CompletedProcess(), forwards=())
|
||||
_validate_specs(specs)
|
||||
argv = [
|
||||
sys.executable,
|
||||
"-c",
|
||||
"from bot_bottle.backend.smolmachines.port_forward import main; "
|
||||
"raise SystemExit(main())",
|
||||
"--spec-json",
|
||||
json.dumps([asdict(s) for s in specs], separators=(",", ":")),
|
||||
]
|
||||
proc = subprocess.Popen(
|
||||
argv,
|
||||
stdin=subprocess.DEVNULL,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=None,
|
||||
text=True,
|
||||
env=os.environ,
|
||||
)
|
||||
assert proc.stdout is not None
|
||||
line = proc.stdout.readline()
|
||||
proc.stdout.close()
|
||||
if not line:
|
||||
proc.terminate()
|
||||
proc.wait(timeout=5)
|
||||
die("smolmachines forwarder failed before reporting readiness")
|
||||
try:
|
||||
payload = json.loads(line)
|
||||
bound = tuple(ForwardSpec(**item) for item in payload["forwards"])
|
||||
except (KeyError, TypeError, ValueError, json.JSONDecodeError) as exc:
|
||||
proc.terminate()
|
||||
proc.wait(timeout=5)
|
||||
die(f"smolmachines forwarder returned invalid readiness payload: {exc}")
|
||||
return ForwarderHandle(process=proc, forwards=bound)
|
||||
|
||||
|
||||
def stop_forwarder(handle: ForwarderHandle) -> None:
|
||||
proc = handle.process
|
||||
if proc.poll() is not None:
|
||||
return
|
||||
proc.terminate()
|
||||
try:
|
||||
proc.wait(timeout=5)
|
||||
except subprocess.TimeoutExpired:
|
||||
proc.kill()
|
||||
proc.wait(timeout=5)
|
||||
|
||||
|
||||
def _validate_specs(specs: Iterable[ForwardSpec]) -> None:
|
||||
for spec in specs:
|
||||
if spec.listen_host in _FORBIDDEN_LISTEN_HOSTS:
|
||||
die(f"refusing unsafe smolmachines forwarder bind: {spec.listen_host!r}")
|
||||
if spec.target_host not in _LOOPBACK_TARGETS:
|
||||
die(f"refusing non-loopback smolmachines forwarder target: {spec.target_host!r}")
|
||||
if not 0 <= spec.listen_port <= 65535:
|
||||
die(f"invalid smolmachines forwarder listen port: {spec.listen_port}")
|
||||
if not 1 <= spec.target_port <= 65535:
|
||||
die(f"invalid smolmachines forwarder target port: {spec.target_port}")
|
||||
|
||||
|
||||
class _CompletedProcess:
|
||||
"""Popen-like no-op used when a bottle has no forwards."""
|
||||
|
||||
def poll(self) -> int:
|
||||
return 0
|
||||
|
||||
def terminate(self) -> None:
|
||||
return None
|
||||
|
||||
def kill(self) -> None:
|
||||
return None
|
||||
|
||||
def wait(self, timeout: float | None = None) -> int:
|
||||
del timeout
|
||||
return 0
|
||||
|
||||
|
||||
class _Forwarder:
|
||||
def __init__(self, specs: Sequence[ForwardSpec]):
|
||||
_validate_specs(specs)
|
||||
self.specs = tuple(specs)
|
||||
self.selector = selectors.DefaultSelector()
|
||||
self.stop_event = threading.Event()
|
||||
self.listeners: list[tuple[socket.socket, ForwardSpec]] = []
|
||||
self.bound: list[ForwardSpec] = []
|
||||
|
||||
def start(self) -> None:
|
||||
for spec in self.specs:
|
||||
listener = socket.create_server(
|
||||
(spec.listen_host, spec.listen_port),
|
||||
reuse_port=False,
|
||||
backlog=64,
|
||||
)
|
||||
listener.setblocking(False)
|
||||
host, port = listener.getsockname()[:2]
|
||||
bound = ForwardSpec(
|
||||
label=spec.label,
|
||||
listen_host=str(host),
|
||||
listen_port=int(port),
|
||||
target_host=spec.target_host,
|
||||
target_port=spec.target_port,
|
||||
)
|
||||
self.listeners.append((listener, bound))
|
||||
self.bound.append(bound)
|
||||
self.selector.register(listener, selectors.EVENT_READ, bound)
|
||||
|
||||
def serve(self) -> None:
|
||||
while not self.stop_event.is_set():
|
||||
try:
|
||||
events = self.selector.select(timeout=0.25)
|
||||
except OSError:
|
||||
return
|
||||
for key, _ in events:
|
||||
listener = key.fileobj
|
||||
if not isinstance(listener, socket.socket):
|
||||
continue
|
||||
spec = key.data
|
||||
try:
|
||||
client, _ = listener.accept()
|
||||
except OSError:
|
||||
continue
|
||||
threading.Thread(
|
||||
target=self._handle_client,
|
||||
args=(client, spec),
|
||||
daemon=True,
|
||||
).start()
|
||||
|
||||
def stop(self) -> None:
|
||||
self.stop_event.set()
|
||||
for listener, _ in self.listeners:
|
||||
try:
|
||||
self.selector.unregister(listener)
|
||||
except Exception: # noqa: BLE001 - best effort shutdown
|
||||
pass
|
||||
listener.close()
|
||||
self.selector.close()
|
||||
|
||||
def _handle_client(self, client: socket.socket, spec: ForwardSpec) -> None:
|
||||
with client:
|
||||
try:
|
||||
target = socket.create_connection((spec.target_host, spec.target_port))
|
||||
except OSError as exc:
|
||||
warn(f"smolmachines forwarder {spec.label}: target connect failed: {exc}")
|
||||
return
|
||||
with target:
|
||||
left = threading.Thread(target=_pipe, args=(client, target), daemon=True)
|
||||
right = threading.Thread(target=_pipe, args=(target, client), daemon=True)
|
||||
left.start()
|
||||
right.start()
|
||||
left.join()
|
||||
right.join()
|
||||
|
||||
|
||||
def _pipe(src: socket.socket, dst: socket.socket) -> None:
|
||||
while True:
|
||||
try:
|
||||
data = src.recv(_BUFFER_SIZE)
|
||||
except OSError:
|
||||
break
|
||||
if not data:
|
||||
break
|
||||
try:
|
||||
dst.sendall(data)
|
||||
except OSError:
|
||||
break
|
||||
try:
|
||||
dst.shutdown(socket.SHUT_WR)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def main(argv: Sequence[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--spec-json", required=True)
|
||||
ns = parser.parse_args(argv)
|
||||
specs = tuple(ForwardSpec(**item) for item in json.loads(ns.spec_json))
|
||||
forwarder = _Forwarder(specs)
|
||||
|
||||
def request_stop(signum: int, _frame: object) -> None:
|
||||
del signum
|
||||
forwarder.stop()
|
||||
|
||||
signal.signal(signal.SIGTERM, request_stop)
|
||||
signal.signal(signal.SIGINT, request_stop)
|
||||
forwarder.start()
|
||||
print(json.dumps({"forwards": [asdict(s) for s in forwarder.bound]}), flush=True)
|
||||
try:
|
||||
forwarder.serve()
|
||||
finally:
|
||||
forwarder.stop()
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -23,8 +23,6 @@ Plans (EgressPlan, …) lands in chunk 2d."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import socket
|
||||
import subprocess
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
@@ -36,7 +34,6 @@ from ..docker.sidecar_bundle import (
|
||||
SIDECAR_BUNDLE_DOCKERFILE,
|
||||
SIDECAR_BUNDLE_IMAGE,
|
||||
)
|
||||
from . import smolvm as _smolvm
|
||||
|
||||
|
||||
_REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent)
|
||||
@@ -57,11 +54,6 @@ def bundle_container_name(slug: str) -> str:
|
||||
return f"bot-bottle-sidecars-{slug}"
|
||||
|
||||
|
||||
def bundle_machine_name(slug: str) -> str:
|
||||
"""Sidecar smolVM machine name."""
|
||||
return bundle_container_name(slug)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class BundleLaunchSpec:
|
||||
"""Everything `start_bundle` needs to bring up one bundle
|
||||
@@ -100,11 +92,6 @@ class BundleLaunchSpec:
|
||||
publish_host_ip: str = "127.0.0.1"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class BundleVmLaunch:
|
||||
raw_ports: dict[int, int]
|
||||
|
||||
|
||||
def ensure_bundle_image(image: str = SIDECAR_BUNDLE_IMAGE) -> None:
|
||||
"""Build the sidecar bundle image before `docker run`.
|
||||
|
||||
@@ -120,70 +107,6 @@ def ensure_bundle_image(image: str = SIDECAR_BUNDLE_IMAGE) -> None:
|
||||
)
|
||||
|
||||
|
||||
def allocate_raw_host_ports(container_ports: Sequence[int]) -> dict[int, int]:
|
||||
"""Reserve candidate host loopback ports for smolVM `-p HOST:GUEST`.
|
||||
|
||||
The sockets are closed before smolVM binds them, so this remains
|
||||
best-effort. smolVM failing to bind a selected port is fatal in
|
||||
the caller's launch path."""
|
||||
out: dict[int, int] = {}
|
||||
for container_port in container_ports:
|
||||
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
|
||||
sock.bind(("127.0.0.1", 0))
|
||||
out[container_port] = int(sock.getsockname()[1])
|
||||
return out
|
||||
|
||||
|
||||
def start_bundle_vm(
|
||||
spec: BundleLaunchSpec,
|
||||
*,
|
||||
from_path: Path,
|
||||
host_env: dict[str, str] | None = None,
|
||||
) -> BundleVmLaunch:
|
||||
"""Create and start the sidecar bundle as a smolVM.
|
||||
|
||||
smolVM's own published ports are raw host-loopback ports. The
|
||||
launch flow wraps them with per-bottle address-bound forwarders
|
||||
before exposing anything to the agent VM."""
|
||||
raw_ports = allocate_raw_host_ports(spec.ports_to_publish)
|
||||
effective_host_env = host_env if host_env is not None else os.environ
|
||||
env: dict[str, str] = {"BOT_BOTTLE_SIDECAR_DAEMONS": spec.daemons_csv}
|
||||
for entry in spec.environment:
|
||||
name, sep, value = entry.partition("=")
|
||||
if sep:
|
||||
env[name] = value
|
||||
elif entry in effective_host_env:
|
||||
env[entry] = effective_host_env[entry]
|
||||
name = bundle_machine_name(spec.slug)
|
||||
_smolvm.machine_create(
|
||||
name,
|
||||
from_path=from_path,
|
||||
net=True,
|
||||
env=env,
|
||||
volumes=spec.volumes,
|
||||
ports=tuple(
|
||||
(host_port, guest_port)
|
||||
for guest_port, host_port in raw_ports.items()
|
||||
),
|
||||
)
|
||||
_smolvm.machine_start(name)
|
||||
_smolvm.wait_exec_ready(name)
|
||||
return BundleVmLaunch(raw_ports=raw_ports)
|
||||
|
||||
|
||||
def stop_bundle_vm(slug: str) -> None:
|
||||
"""Best-effort sidecar VM teardown."""
|
||||
name = bundle_machine_name(slug)
|
||||
try:
|
||||
_smolvm.machine_stop(name)
|
||||
except _smolvm.SmolvmError as exc:
|
||||
warn(f"smolvm machine stop {name} failed: {exc}")
|
||||
try:
|
||||
_smolvm.machine_delete(name)
|
||||
except _smolvm.SmolvmError as exc:
|
||||
warn(f"smolvm machine delete {name} failed: {exc}")
|
||||
|
||||
|
||||
def create_bundle_network(network_name: str, subnet: str, gateway: str) -> None:
|
||||
"""`docker network create` with an explicit subnet + gateway
|
||||
so the bundle's `--ip` lands on the address the Smolfile's
|
||||
|
||||
@@ -113,15 +113,13 @@ def machine_create(
|
||||
*,
|
||||
image: str | None = None,
|
||||
from_path: Path | None = None,
|
||||
net: bool = False,
|
||||
allow_cidrs: Sequence[str] = (),
|
||||
env: Mapping[str, str] | None = None,
|
||||
ports: Sequence[tuple[int, int]] = (),
|
||||
volumes: Sequence[tuple[str, str, bool]] = (),
|
||||
) -> None:
|
||||
"""`smolvm machine create --name NAME [--image IMG | --from PATH]
|
||||
[--allow-cidr CIDR ...] [-e K=V ...]`. NAME is passed as
|
||||
`--name` (smolvm 1.4.7+; earlier versions took it positionally).
|
||||
"""`smolvm machine create NAME [--image IMG | --from PATH]
|
||||
[--allow-cidr CIDR ...] [-e K=V ...]`. NAME is positional
|
||||
(the CLI's exception to the `--name` pattern other
|
||||
subcommands use).
|
||||
|
||||
`image` (registry ref like `alpine:latest`) and `from_path`
|
||||
(a `.smolmachine` artifact) are mutually exclusive — one or
|
||||
@@ -134,27 +132,25 @@ def machine_create(
|
||||
no-pull-at-start property. The flag form gives the same
|
||||
result without the Smolfile complication.
|
||||
|
||||
`--net` is sent explicitly when `net=True` or `allow_cidrs` is
|
||||
non-empty. `--allow-cidr` implies `--net` per the CLI help, but
|
||||
sending `--net` explicitly is harmless and ensures the guest has
|
||||
network access even if that implication changes across versions."""
|
||||
args: list[str] = ["machine", "create", "--name", name]
|
||||
`--net` is sent explicitly when `allow_cidrs` is non-empty.
|
||||
smolvm 0.8.0's docs say `--allow-cidr` implies `--net`, but
|
||||
empirically the implication only fires when no `--from` is
|
||||
set — `--from PATH --allow-cidr X/32` silently produces a
|
||||
machine with `network: false` and no routes in the guest, so
|
||||
the agent can't reach the bundle's pinned IP."""
|
||||
args: list[str] = ["machine", "create"]
|
||||
if image is not None:
|
||||
args += ["--image", image]
|
||||
if from_path is not None:
|
||||
args += ["--from", str(from_path)]
|
||||
if net or allow_cidrs:
|
||||
if allow_cidrs:
|
||||
args.append("--net")
|
||||
for cidr in allow_cidrs:
|
||||
args += ["--allow-cidr", cidr]
|
||||
for host_port, guest_port in ports:
|
||||
args += ["-p", f"{host_port}:{guest_port}"]
|
||||
for host_path, guest_path, read_only in volumes:
|
||||
suffix = ":ro" if read_only else ""
|
||||
args += ["-v", f"{host_path}:{guest_path}{suffix}"]
|
||||
if env:
|
||||
for k, v in env.items():
|
||||
args += ["-e", f"{k}={v}"]
|
||||
args.append(name)
|
||||
_smolvm(*args)
|
||||
|
||||
|
||||
@@ -186,9 +182,10 @@ def machine_stop(name: str) -> None:
|
||||
|
||||
|
||||
def machine_delete(name: str) -> None:
|
||||
"""`smolvm machine delete --name NAME -f`. `-f` skips the
|
||||
interactive confirmation — required for non-interactive teardown."""
|
||||
_smolvm("machine", "delete", "--name", name, "-f")
|
||||
"""`smolvm machine delete -f NAME`. NAME is positional. `-f`
|
||||
skips the interactive confirmation — required for
|
||||
non-interactive teardown."""
|
||||
_smolvm("machine", "delete", "-f", name)
|
||||
|
||||
|
||||
def machine_exec(
|
||||
|
||||
@@ -5,58 +5,26 @@ unit-tested without importing the docker subprocess paths."""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import os
|
||||
import platform
|
||||
import shutil
|
||||
|
||||
from ...log import die
|
||||
|
||||
# libkrun's Linux backend drives the guest through KVM, so the host
|
||||
# must expose `/dev/kvm` and the invoking user must be able to open
|
||||
# it. macOS uses Hypervisor.framework and needs no device node.
|
||||
_KVM_DEVICE = "/dev/kvm"
|
||||
|
||||
|
||||
def smolmachines_preflight() -> None:
|
||||
"""Ensure the host can run the smolmachines backend before the
|
||||
launch flow starts. Called from `_resolve_plan`; surfaces a
|
||||
clear, actionable error instead of a cryptic `smolvm` failure
|
||||
deep in launch.
|
||||
|
||||
Checks `smolvm` is on PATH (both platforms) and, on Linux,
|
||||
that `/dev/kvm` exists and is accessible. `gvproxy` is no
|
||||
longer required — see the PRD's design pivot section."""
|
||||
if shutil.which("smolvm") is None:
|
||||
die(
|
||||
"BOT_BOTTLE_BACKEND=smolmachines requires `smolvm` on "
|
||||
"PATH. Install with: "
|
||||
"curl -sSL https://smolmachines.com/install.sh | sh. "
|
||||
"To use the legacy Docker backend instead, set "
|
||||
"BOT_BOTTLE_BACKEND=docker or pass --backend=docker."
|
||||
)
|
||||
if platform.system() == "Linux":
|
||||
_preflight_kvm()
|
||||
|
||||
|
||||
def _preflight_kvm() -> None:
|
||||
"""Linux-only: libkrun needs `/dev/kvm`. Distinguish 'KVM not
|
||||
enabled' from 'no permission' so the operator knows which to
|
||||
fix."""
|
||||
if not os.path.exists(_KVM_DEVICE):
|
||||
die(
|
||||
f"BOT_BOTTLE_BACKEND=smolmachines needs {_KVM_DEVICE} on "
|
||||
"Linux but it is missing. Enable KVM: load the kvm-intel "
|
||||
"or kvm-amd kernel module (and confirm virtualization is "
|
||||
"enabled in BIOS/firmware). To use the legacy Docker "
|
||||
"backend instead, set BOT_BOTTLE_BACKEND=docker."
|
||||
)
|
||||
if not os.access(_KVM_DEVICE, os.R_OK | os.W_OK):
|
||||
die(
|
||||
f"{_KVM_DEVICE} exists but is not readable/writable by the "
|
||||
"current user. Add your user to the `kvm` group "
|
||||
"(`sudo usermod -aG kvm \"$USER\"`) and re-login, or run "
|
||||
"with access to the device."
|
||||
)
|
||||
"""Ensure `smolvm` is on PATH before the launch flow runs.
|
||||
Called from `_resolve_plan`; gives the operator a clear
|
||||
install pointer rather than a cryptic FileNotFoundError
|
||||
later. `gvproxy` is no longer required — see the PRD's design
|
||||
pivot section."""
|
||||
if shutil.which("smolvm") is not None:
|
||||
return
|
||||
die(
|
||||
"BOT_BOTTLE_BACKEND=smolmachines requires `smolvm` on "
|
||||
"PATH. Install with: "
|
||||
"curl -sSL https://smolmachines.com/install.sh | sh. "
|
||||
"To use the legacy Docker backend instead, set "
|
||||
"BOT_BOTTLE_BACKEND=docker or pass --backend=docker."
|
||||
)
|
||||
|
||||
|
||||
def smolmachines_bundle_subnet(slug: str) -> tuple[str, str, str]:
|
||||
|
||||
@@ -284,8 +284,9 @@ def git_gate_state_dir(identity: str) -> Path:
|
||||
|
||||
def supervise_state_dir(identity: str) -> Path:
|
||||
"""State subdir reserved for supervise sidecar bind-mount sources.
|
||||
Runtime queue/audit rows live in the host-level bot-bottle SQLite
|
||||
database, so they survive state-dir cleanup."""
|
||||
The queue dir is intentionally NOT under here — it lives at
|
||||
~/.bot-bottle/queue/<slug>/ alongside the audit logs, so it
|
||||
survives state-dir cleanup."""
|
||||
return bottle_state_dir(identity) / _SUPERVISE_SUBDIR
|
||||
|
||||
|
||||
|
||||
@@ -7,10 +7,8 @@ from __future__ import annotations
|
||||
|
||||
import sys
|
||||
|
||||
from ..errors import MissingEnvVarError
|
||||
from ..log import Die, die, error
|
||||
from ..manifest import ManifestError
|
||||
from ..store_manager import StoreManager
|
||||
from ._common import PROG
|
||||
from . import list as _list_mod
|
||||
from .cleanup import cmd_cleanup
|
||||
@@ -76,25 +74,11 @@ def main(argv: list[str] | None = None) -> int:
|
||||
if handler is None:
|
||||
usage()
|
||||
die(f"unknown command: {command}")
|
||||
mgr = StoreManager.instance()
|
||||
if not mgr.is_migrated():
|
||||
sys.stderr.write("bot-bottle: database schema is out of date\n")
|
||||
sys.stderr.write("Migrate now? [y/N] ")
|
||||
sys.stderr.flush()
|
||||
try:
|
||||
answer = sys.stdin.readline().strip().lower()
|
||||
except EOFError:
|
||||
answer = ""
|
||||
if answer != "y":
|
||||
error("migration required — re-run and confirm to migrate")
|
||||
return 1
|
||||
mgr.migrate()
|
||||
try:
|
||||
return handler(rest) or 0
|
||||
except MissingEnvVarError as e:
|
||||
error(str(e))
|
||||
return 1
|
||||
except ManifestError as e:
|
||||
# Manifest/config problems surface as a catchable exception;
|
||||
# print the reason and exit non-zero (same UX die() used to give).
|
||||
error(str(e))
|
||||
return 1
|
||||
except Die as e:
|
||||
|
||||
+26
-202
@@ -2,11 +2,6 @@
|
||||
interactive claude-code session. The container is torn down when the
|
||||
session ends.
|
||||
|
||||
`--headless` selects a non-interactive launch (agent/bottles/label from
|
||||
flags, no TUI selectors, no y/N prompt) for orchestrators,
|
||||
CI, and webhook dispatch. The agent still execs on the inherited
|
||||
stdio/PTY, so an orchestrator that allocates the PTY drives the session.
|
||||
|
||||
The launch core is shared with `cli.py resume <identity>` through
|
||||
the private orchestrator `_launch_bottle`.
|
||||
"""
|
||||
@@ -21,7 +16,7 @@ import tempfile
|
||||
from pathlib import Path
|
||||
from typing import Callable
|
||||
|
||||
from ..agent_provider import get_provider, runtime_for
|
||||
from ..agent_provider import runtime_for
|
||||
from ..backend import (
|
||||
Bottle,
|
||||
BottleSpec,
|
||||
@@ -36,8 +31,7 @@ from ..bottle_state import (
|
||||
is_preserved,
|
||||
mark_preserved,
|
||||
)
|
||||
from ..image_cache import StaleImageError
|
||||
from ..log import info, die
|
||||
from ..log import info
|
||||
from ..manifest import Manifest, ManifestIndex
|
||||
from ._common import PROG, USER_CWD, read_tty_line
|
||||
from . import tui
|
||||
@@ -56,47 +50,6 @@ def cmd_start(argv: list[str]) -> int:
|
||||
"or host auto-selection). Overrides the env var when set."
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--headless",
|
||||
action="store_true",
|
||||
help=(
|
||||
"non-interactive launch: take agent/bottles/label from flags, "
|
||||
"skip all prompts. For orchestrators, CI, and webhooks."
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--cached-images",
|
||||
action="store_true",
|
||||
help=(
|
||||
"quickstart with existing local agent and sidecar images; "
|
||||
"only valid with --headless"
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--bottle",
|
||||
action="append",
|
||||
default=None,
|
||||
metavar="NAME",
|
||||
help=(
|
||||
"bottle to compose, repeatable (order = merge order). In "
|
||||
"--headless, defaults to the agent's own bottle when omitted."
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--label",
|
||||
default=None,
|
||||
help="bottle label / terminal title (--headless default: agent name)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--color",
|
||||
default=None,
|
||||
help="bottle color, one of the 16 ANSI color names (--headless default: none)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--prompt",
|
||||
default=None,
|
||||
help="initial task prompt delivered to the agent (required with --headless)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"name",
|
||||
nargs="?",
|
||||
@@ -104,28 +57,13 @@ def cmd_start(argv: list[str]) -> int:
|
||||
help="agent name defined in bot-bottle.json (omit to pick interactively)",
|
||||
)
|
||||
args = parser.parse_args(argv)
|
||||
if args.cached_images and not args.headless:
|
||||
die("--cached-images is only supported with --headless")
|
||||
|
||||
dry_run = args.dry_run or os.environ.get("BOT_BOTTLE_DRY_RUN") == "1"
|
||||
|
||||
manifest = ManifestIndex.resolve(USER_CWD)
|
||||
backend_name: str | None = args.backend
|
||||
|
||||
if args.headless:
|
||||
return _start_headless(
|
||||
manifest, args, dry_run=dry_run, backend_name=backend_name
|
||||
)
|
||||
|
||||
agent_name: str | None = args.name
|
||||
if agent_name is None:
|
||||
if not manifest.all_agent_names:
|
||||
print(
|
||||
"bot-bottle: no agents defined. "
|
||||
"Add an agent to ~/.bot-bottle/agents/ or ./bot-bottle/agents/ to get started.",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 1
|
||||
agent_name = tui.filter_select(
|
||||
manifest.all_agent_names,
|
||||
title="Select agent",
|
||||
@@ -133,6 +71,8 @@ def cmd_start(argv: list[str]) -> int:
|
||||
if agent_name is None:
|
||||
return 0
|
||||
|
||||
backend_name: str | None = args.backend
|
||||
|
||||
# Bottle multiselect: always show after agent selection so operators
|
||||
# can compose bottles at launch time without editing agent manifests.
|
||||
available_bottles = manifest.all_bottle_names
|
||||
@@ -153,10 +93,6 @@ def cmd_start(argv: list[str]) -> int:
|
||||
label, color = tui.name_color_modal(default_label=agent_name)
|
||||
label, color = _resolve_unique_label(label, color)
|
||||
|
||||
image_policy = _select_image_policy()
|
||||
if image_policy is None:
|
||||
return 0
|
||||
|
||||
spec = BottleSpec(
|
||||
manifest=manifest,
|
||||
agent_name=agent_name,
|
||||
@@ -165,7 +101,6 @@ def cmd_start(argv: list[str]) -> int:
|
||||
label=label,
|
||||
color=color,
|
||||
bottle_names=bottle_names,
|
||||
image_policy=image_policy,
|
||||
)
|
||||
return _launch_bottle(
|
||||
spec,
|
||||
@@ -174,85 +109,6 @@ def cmd_start(argv: list[str]) -> int:
|
||||
)
|
||||
|
||||
|
||||
# --- Headless launch -----------------------------------------------------
|
||||
|
||||
|
||||
def _start_headless(
|
||||
manifest: ManifestIndex,
|
||||
args: argparse.Namespace,
|
||||
*,
|
||||
dry_run: bool,
|
||||
backend_name: str | None,
|
||||
) -> int:
|
||||
"""Non-interactive launch path for orchestrators / CI / webhooks.
|
||||
|
||||
Resolves agent, bottles, label, and color from flags + manifest
|
||||
defaults instead of the TUI selectors, and auto-confirms the
|
||||
preflight. Otherwise runs the same launch core as the interactive
|
||||
path, so the agent still execs on the inherited stdio/PTY — an
|
||||
orchestrator allocates that PTY and relays it to its
|
||||
desktop/mobile clients."""
|
||||
agent_name = args.name
|
||||
if not agent_name:
|
||||
die("--headless requires an agent name: ./cli.py start <agent> --headless")
|
||||
manifest.require_agent(agent_name) # raises ManifestError if unknown
|
||||
|
||||
prompt = args.prompt
|
||||
if not prompt:
|
||||
die(
|
||||
"--headless requires --prompt: "
|
||||
"./cli.py start <agent> --headless --prompt 'Do the thing'"
|
||||
)
|
||||
|
||||
if args.bottle:
|
||||
bottle_names: tuple[str, ...] = tuple(args.bottle)
|
||||
else:
|
||||
default_bottle = _peek_agent_bottle(manifest, agent_name)
|
||||
if not default_bottle:
|
||||
die(
|
||||
f"--headless: agent '{agent_name}' has no default bottle; "
|
||||
f"pass one or more --bottle NAME"
|
||||
)
|
||||
bottle_names = (default_bottle,)
|
||||
|
||||
label = _uniquify_label_headless(args.label or agent_name)
|
||||
|
||||
spec = BottleSpec(
|
||||
manifest=manifest,
|
||||
agent_name=agent_name,
|
||||
copy_cwd=args.cwd,
|
||||
user_cwd=USER_CWD,
|
||||
label=label,
|
||||
color=args.color or "",
|
||||
bottle_names=bottle_names,
|
||||
headless=True,
|
||||
image_policy="cached" if args.cached_images else "fresh",
|
||||
)
|
||||
return _launch_bottle(
|
||||
spec,
|
||||
dry_run=dry_run,
|
||||
backend_name=backend_name,
|
||||
assume_yes=True,
|
||||
headless_prompt_text=prompt,
|
||||
)
|
||||
|
||||
|
||||
def _uniquify_label_headless(label: str) -> str:
|
||||
"""Non-interactive analog of `_resolve_unique_label`: if the label's
|
||||
slug collides with a running bottle, append -2, -3, … until free,
|
||||
logging the chosen label. Orchestrators fire-and-forget many bottles,
|
||||
so silently picking a free name beats erroring on every collision."""
|
||||
active_slugs = {a.slug for a in enumerate_active_agents()}
|
||||
if docker_mod.slugify(label) not in active_slugs:
|
||||
return label
|
||||
n = 2
|
||||
while docker_mod.slugify(f"{label}-{n}") in active_slugs:
|
||||
n += 1
|
||||
chosen = f"{label}-{n}"
|
||||
info(f"label '{label}' already in use; using '{chosen}'")
|
||||
return chosen
|
||||
|
||||
|
||||
# --- Launch helpers ------------------------------------------------------
|
||||
|
||||
|
||||
@@ -260,7 +116,7 @@ def prepare_with_preflight(
|
||||
spec: BottleSpec,
|
||||
*,
|
||||
stage_dir: Path,
|
||||
render_preflight: Callable[[DockerBottlePlan, str], None],
|
||||
render_preflight: Callable[[DockerBottlePlan], None],
|
||||
prompt_yes: Callable[[], bool],
|
||||
dry_run: bool = False,
|
||||
backend_name: str | None = None,
|
||||
@@ -281,7 +137,7 @@ def prepare_with_preflight(
|
||||
plan = backend.prepare(spec, stage_dir=stage_dir)
|
||||
identity = _identity_from_plan(plan)
|
||||
|
||||
render_preflight(plan, backend.name)
|
||||
render_preflight(plan)
|
||||
|
||||
if dry_run:
|
||||
info("dry-run requested; not starting container.")
|
||||
@@ -407,17 +263,9 @@ def _text_prompt_yes() -> bool:
|
||||
return reply in ("y", "Y", "yes", "YES")
|
||||
|
||||
|
||||
def _select_image_policy() -> str | None:
|
||||
return tui.filter_select(
|
||||
["fresh", "cached"],
|
||||
title="Select image startup mode",
|
||||
)
|
||||
|
||||
|
||||
def _text_render_preflight():
|
||||
def _render(plan: DockerBottlePlan, backend_name: str) -> None:
|
||||
def _render(plan: DockerBottlePlan) -> None:
|
||||
print(file=sys.stderr)
|
||||
print(f"backend: {backend_name}", file=sys.stderr)
|
||||
print(_manifest_to_yaml(plan.manifest), file=sys.stderr)
|
||||
return _render
|
||||
|
||||
@@ -528,19 +376,10 @@ def _launch_bottle(
|
||||
*,
|
||||
dry_run: bool,
|
||||
backend_name: str | None = None,
|
||||
assume_yes: bool = False,
|
||||
headless_prompt_text: str = "",
|
||||
) -> int:
|
||||
"""Shared launch core for `start` and `resume`. Builds the plan,
|
||||
prints / dry-runs / prompts as appropriate, brings the bottle up,
|
||||
attaches claude, and prints the resume hint on session end.
|
||||
|
||||
`assume_yes` skips the interactive y/N confirmation (headless /
|
||||
orchestrator launches), where there is no human at the prompt.
|
||||
|
||||
`headless_prompt_text` is passed to the provider's `headless_prompt`
|
||||
method and the resulting args are appended to startup_args so the
|
||||
agent receives the initial task without interactive input."""
|
||||
attaches claude, and prints the resume hint on session end."""
|
||||
stage_dir = Path(tempfile.mkdtemp(prefix="bot-bottle-stage."))
|
||||
identity = ""
|
||||
try:
|
||||
@@ -548,7 +387,7 @@ def _launch_bottle(
|
||||
spec,
|
||||
stage_dir=stage_dir,
|
||||
render_preflight=_text_render_preflight(),
|
||||
prompt_yes=(lambda: True) if assume_yes else _text_prompt_yes,
|
||||
prompt_yes=_text_prompt_yes,
|
||||
dry_run=dry_run,
|
||||
backend_name=backend_name,
|
||||
)
|
||||
@@ -556,38 +395,23 @@ def _launch_bottle(
|
||||
return 0
|
||||
|
||||
backend = get_bottle_backend(backend_name)
|
||||
skip_stale = False
|
||||
while True:
|
||||
try:
|
||||
with backend.launch(plan, skip_stale=skip_stale) as bottle:
|
||||
agent_provider_template = getattr(plan, "agent_provider_template", "claude")
|
||||
extra_args: tuple[str, ...] = ()
|
||||
if headless_prompt_text:
|
||||
extra_args = tuple(
|
||||
get_provider(agent_provider_template).headless_prompt(
|
||||
headless_prompt_text
|
||||
)
|
||||
)
|
||||
exit_code = attach_agent(
|
||||
bottle,
|
||||
agent_provider_template=agent_provider_template,
|
||||
startup_args=plan.agent_provision.startup_args + extra_args,
|
||||
)
|
||||
info(
|
||||
f"session ended (exit {exit_code}); "
|
||||
f"container {bottle.name} will be removed"
|
||||
)
|
||||
if agent_provider_template == "claude":
|
||||
capture_claude_session_state(identity, exit_code)
|
||||
break
|
||||
except StaleImageError as exc:
|
||||
if assume_yes:
|
||||
die(str(exc))
|
||||
sys.stderr.write(f"bot-bottle: {exc}\nLaunch anyway? [y/N] ")
|
||||
sys.stderr.flush()
|
||||
if read_tty_line() not in ("y", "Y", "yes", "YES"):
|
||||
break
|
||||
skip_stale = True
|
||||
with backend.launch(plan) as bottle:
|
||||
agent_provider_template = getattr(plan, "agent_provider_template", "claude")
|
||||
exit_code = attach_agent(
|
||||
bottle,
|
||||
agent_provider_template=agent_provider_template,
|
||||
startup_args=plan.agent_provision.startup_args,
|
||||
)
|
||||
info(
|
||||
f"session ended (exit {exit_code}); "
|
||||
f"container {bottle.name} will be removed"
|
||||
)
|
||||
# While the container is still alive: always snapshot the
|
||||
# transcript and — if the agent exited non-zero — mark
|
||||
# the state for preservation. This picks up crashes /
|
||||
# Ctrl-Cs / OOM kills before cleanup removes the state dir.
|
||||
if agent_provider_template == "claude":
|
||||
capture_claude_session_state(identity, exit_code)
|
||||
return 0
|
||||
finally:
|
||||
# PRD 0018 chunk 2: prepare now writes the bottle's bind-mount
|
||||
|
||||
@@ -45,7 +45,7 @@ from ..supervise import (
|
||||
TOOL_EGRESS_BLOCK,
|
||||
TOOL_GITLEAKS_ALLOW,
|
||||
TOOL_EGRESS_TOKEN_ALLOW,
|
||||
list_all_pending_proposals,
|
||||
list_pending_proposals,
|
||||
render_diff,
|
||||
write_audit_entry,
|
||||
write_response,
|
||||
@@ -63,9 +63,10 @@ _REPORT_ONLY_TOOLS: tuple[str, ...] = (TOOL_GITLEAKS_ALLOW, TOOL_EGRESS_TOKEN_AL
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class QueuedProposal:
|
||||
"""A pending proposal from the supervise queue."""
|
||||
"""A pending proposal plus the queue dir it was found in."""
|
||||
|
||||
proposal: Proposal
|
||||
queue_dir: Path
|
||||
|
||||
|
||||
# Errors any remediation engine may raise. Caught by the TUI key
|
||||
@@ -85,11 +86,16 @@ def apply_routes_change(slug: str, content: str) -> tuple[str, str]:
|
||||
|
||||
|
||||
def discover_pending() -> list[QueuedProposal]:
|
||||
"""Collect pending proposals across bottles."""
|
||||
out = [
|
||||
QueuedProposal(proposal=proposal)
|
||||
for proposal in list_all_pending_proposals()
|
||||
]
|
||||
"""Walk ~/.bot-bottle/queue/* and collect pending proposals."""
|
||||
queue_root = _supervise.bot_bottle_root() / "queue"
|
||||
if not queue_root.is_dir():
|
||||
return []
|
||||
out: list[QueuedProposal] = []
|
||||
for slug_dir in sorted(queue_root.iterdir()):
|
||||
if not slug_dir.is_dir():
|
||||
continue
|
||||
for proposal in list_pending_proposals(slug_dir):
|
||||
out.append(QueuedProposal(proposal=proposal, queue_dir=slug_dir))
|
||||
out.sort(key=lambda q: q.proposal.arrival_timestamp)
|
||||
return out
|
||||
|
||||
@@ -112,6 +118,7 @@ def _detail_lines(
|
||||
(f"tool: {p.tool}", 0),
|
||||
(f"id: {p.id}", 0),
|
||||
(f"arrived: {p.arrival_timestamp}", 0),
|
||||
(f"queue: {qp.queue_dir}", 0),
|
||||
("", 0),
|
||||
("justification:", 0),
|
||||
]
|
||||
@@ -158,7 +165,7 @@ def approve(
|
||||
notes=notes,
|
||||
final_file=final_file,
|
||||
)
|
||||
write_response(qp.proposal.bottle_slug, response)
|
||||
write_response(qp.queue_dir, response)
|
||||
_write_audit(
|
||||
qp, action=status, notes=notes,
|
||||
diff_before=diff_before, diff_after=diff_after,
|
||||
@@ -172,7 +179,7 @@ def reject(qp: QueuedProposal, *, reason: str) -> None:
|
||||
notes=reason,
|
||||
final_file=None,
|
||||
)
|
||||
write_response(qp.proposal.bottle_slug, response)
|
||||
write_response(qp.queue_dir, response)
|
||||
_write_audit(qp, action=STATUS_REJECTED, notes=reason, diff_before="", diff_after="")
|
||||
|
||||
|
||||
|
||||
@@ -1,71 +0,0 @@
|
||||
"""SQLite-backed bot-bottle configuration store."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
from .db_store import DbStore
|
||||
from .migrations import TableMigrations
|
||||
from .supervise_types import host_db_path
|
||||
except ImportError:
|
||||
from db_store import DbStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
|
||||
from migrations import TableMigrations # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
|
||||
from supervise_types import host_db_path # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
|
||||
|
||||
|
||||
DEFAULT_CACHED_IMAGE_STALE_WARNING_DAYS = 1
|
||||
|
||||
|
||||
class ConfigStore(DbStore):
|
||||
"""SQLite configuration for host-side bot-bottle settings."""
|
||||
|
||||
def __init__(self, db_path: Path | None = None) -> None:
|
||||
migrations = TableMigrations("config_store", [
|
||||
# v1 — host-side bot-bottle settings
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS bot_bottle_config (
|
||||
id INTEGER PRIMARY KEY CHECK (id = 1),
|
||||
cached_image_stale_warning_days INTEGER NOT NULL DEFAULT 1
|
||||
)
|
||||
""",
|
||||
])
|
||||
super().__init__(db_path or host_db_path(), migrations)
|
||||
|
||||
def cached_image_stale_warning_days(self) -> int:
|
||||
if not self.db_path.is_file():
|
||||
return DEFAULT_CACHED_IMAGE_STALE_WARNING_DAYS
|
||||
with self._connect() as conn:
|
||||
row = conn.execute(
|
||||
"""
|
||||
SELECT cached_image_stale_warning_days
|
||||
FROM bot_bottle_config
|
||||
WHERE id = 1
|
||||
""",
|
||||
).fetchone()
|
||||
if row is None:
|
||||
return DEFAULT_CACHED_IMAGE_STALE_WARNING_DAYS
|
||||
try:
|
||||
return int(row["cached_image_stale_warning_days"])
|
||||
except (TypeError, ValueError):
|
||||
return DEFAULT_CACHED_IMAGE_STALE_WARNING_DAYS
|
||||
|
||||
def set_cached_image_stale_warning_days(self, days: int) -> Path:
|
||||
with self._connect() as conn:
|
||||
conn.execute(
|
||||
"""
|
||||
INSERT INTO bot_bottle_config (id, cached_image_stale_warning_days)
|
||||
VALUES (1, ?)
|
||||
ON CONFLICT(id) DO UPDATE SET
|
||||
cached_image_stale_warning_days = excluded.cached_image_stale_warning_days
|
||||
""",
|
||||
(days,),
|
||||
)
|
||||
self._chmod()
|
||||
return self.db_path
|
||||
|
||||
|
||||
__all__ = [
|
||||
"DEFAULT_CACHED_IMAGE_STALE_WARNING_DAYS",
|
||||
"ConfigStore",
|
||||
]
|
||||
@@ -21,7 +21,7 @@ FROM node:22-slim
|
||||
# to it) works against egress's bumped TLS without the agent needing
|
||||
# local DNS.
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends git ca-certificates curl ripgrep iproute2 dnsutils \
|
||||
&& apt-get install -y --no-install-recommends git ca-certificates curl ripgrep \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# App-specific deps. Python isn't required by claude-code itself
|
||||
|
||||
@@ -313,9 +313,6 @@ class ClaudeAgentProvider(AgentProvider):
|
||||
f"claude mcp add --scope user --transport http supervise {supervise_url}"
|
||||
)
|
||||
|
||||
def headless_prompt(self, prompt: str) -> list[str]:
|
||||
return ["-p", prompt]
|
||||
|
||||
|
||||
def _exec(bottle: "Bottle", script: str, error: str) -> None:
|
||||
result = bottle.exec(script, user="root")
|
||||
|
||||
@@ -34,12 +34,6 @@ if TYPE_CHECKING:
|
||||
|
||||
|
||||
_SUPERVISE_MCP_NAME = "supervise"
|
||||
_CODEX_CLI = "/home/node/.codex/packages/standalone/current/bin/codex"
|
||||
_CODEX_CLI_PATH = (
|
||||
"/home/node/.local/bin:"
|
||||
"/home/node/.codex/packages/standalone/current/bin:"
|
||||
"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
|
||||
)
|
||||
|
||||
|
||||
def _skills_dir(guest_home: str) -> str:
|
||||
@@ -56,7 +50,7 @@ def _prompt_path(guest_home: str) -> str:
|
||||
|
||||
_RUNTIME = AgentProviderRuntime(
|
||||
template="codex",
|
||||
command=_CODEX_CLI,
|
||||
command="codex",
|
||||
image="bot-bottle-codex:latest",
|
||||
prompt_mode="read_prompt_file",
|
||||
bypass_args=("--dangerously-bypass-approvals-and-sandbox",),
|
||||
@@ -151,8 +145,7 @@ class CodexAgentProvider(AgentProvider):
|
||||
"env",
|
||||
f"HOME={guest_home}",
|
||||
f"CODEX_HOME={auth_dir}",
|
||||
f"PATH={_CODEX_CLI_PATH}",
|
||||
_CODEX_CLI, "login", "status",
|
||||
"codex", "login", "status",
|
||||
), (
|
||||
"codex host credentials: dummy auth was copied into the "
|
||||
"guest, but Codex did not accept it"
|
||||
@@ -274,7 +267,7 @@ class CodexAgentProvider(AgentProvider):
|
||||
return
|
||||
info(f"registering supervise MCP server in agent codex config → {supervise_url}")
|
||||
r = bottle.exec(
|
||||
f"{shlex.quote(_CODEX_CLI)} mcp add {_SUPERVISE_MCP_NAME} --url "
|
||||
f"codex mcp add {_SUPERVISE_MCP_NAME} --url "
|
||||
f"{shlex.quote(supervise_url)}",
|
||||
user="node",
|
||||
)
|
||||
@@ -286,9 +279,6 @@ class CodexAgentProvider(AgentProvider):
|
||||
f"codex mcp add supervise --url {shlex.quote(supervise_url)}"
|
||||
)
|
||||
|
||||
def headless_prompt(self, prompt: str) -> list[str]:
|
||||
return [prompt]
|
||||
|
||||
|
||||
def _exec(bottle: "Bottle", script: str, error: str) -> None:
|
||||
result = bottle.exec(script, user="root")
|
||||
|
||||
@@ -315,9 +315,6 @@ class PiAgentProvider(AgentProvider):
|
||||
) -> None:
|
||||
del plan, bottle, supervise_url
|
||||
|
||||
def headless_prompt(self, prompt: str) -> list[str]:
|
||||
return ["-p", prompt]
|
||||
|
||||
|
||||
def _exec(bottle: "Bottle", script: str, error: str) -> None:
|
||||
result = bottle.exec(script, user="root")
|
||||
|
||||
@@ -1,59 +0,0 @@
|
||||
"""Shared SQLite-backed store base class for bot-bottle (PRD 0013)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sqlite3
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
from .migrations import TableMigrations
|
||||
except ImportError:
|
||||
from migrations import TableMigrations # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
|
||||
|
||||
|
||||
class DbVersionError(Exception):
|
||||
"""Raised when the on-disk schema is behind the current migration list."""
|
||||
|
||||
|
||||
class DbStore:
|
||||
"""Base for SQLite-backed stores. Subclasses resolve db_path then call super().__init__."""
|
||||
|
||||
def __init__(self, db_path: Path, migrations: TableMigrations) -> None:
|
||||
self.db_path = db_path
|
||||
self._migrations = migrations
|
||||
self.db_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
def _connect(self) -> sqlite3.Connection:
|
||||
conn = sqlite3.connect(self.db_path)
|
||||
conn.row_factory = sqlite3.Row
|
||||
return conn
|
||||
|
||||
def is_migrated(self) -> bool:
|
||||
"""Return True if the DB is fully up-to-date, False if migration is needed."""
|
||||
if not self.db_path.exists():
|
||||
return False
|
||||
try:
|
||||
with self._connect() as conn:
|
||||
row = conn.execute(
|
||||
"SELECT version FROM schema_versions WHERE module = ?",
|
||||
(self._migrations.schema_key,),
|
||||
).fetchone()
|
||||
except sqlite3.OperationalError:
|
||||
return False
|
||||
version = row[0] if row else 0
|
||||
return version == len(self._migrations.migrations)
|
||||
|
||||
def migrate(self) -> None:
|
||||
"""Apply any pending migrations and set permissions on the DB file."""
|
||||
with self._connect() as conn:
|
||||
self._migrations.apply(conn)
|
||||
self._chmod()
|
||||
|
||||
def _chmod(self) -> None:
|
||||
try:
|
||||
self.db_path.chmod(0o600)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
__all__ = ["DbStore", "DbVersionError"]
|
||||
@@ -23,7 +23,6 @@ from .egress_addon_core import (
|
||||
PathMatch as CorePathMatch,
|
||||
Route,
|
||||
)
|
||||
from .errors import MissingEnvVarError
|
||||
from .log import die
|
||||
|
||||
if TYPE_CHECKING:
|
||||
@@ -355,18 +354,16 @@ def egress_resolve_token_values(
|
||||
for token_env, token_ref in token_env_map.items():
|
||||
value = host_env.get(token_ref)
|
||||
if value is None:
|
||||
raise MissingEnvVarError(
|
||||
token_ref,
|
||||
die(
|
||||
f"egress: host env var '{token_ref}' is unset. Set it "
|
||||
f"before launching, or remove the corresponding auth block "
|
||||
f"from bottle.egress.routes.",
|
||||
f"from bottle.egress.routes."
|
||||
)
|
||||
if not value:
|
||||
raise MissingEnvVarError(
|
||||
token_ref,
|
||||
die(
|
||||
f"egress: host env var '{token_ref}' is empty. The "
|
||||
f"egress will not inject an empty token; set it to "
|
||||
f"the real value or remove the route's auth block.",
|
||||
f"the real value or remove the route's auth block."
|
||||
)
|
||||
out[token_env] = value
|
||||
return out
|
||||
|
||||
@@ -79,13 +79,14 @@ class EgressAddon:
|
||||
# only — a restart re-prompts. Mutated only from the asyncio loop that
|
||||
# runs the addon hooks, so no lock is needed.
|
||||
self.safe_tokens: set[str] = set()
|
||||
self._supervise_queue_dir = os.environ.get("SUPERVISE_QUEUE_DIR", "").strip()
|
||||
self._supervise_slug = os.environ.get("SUPERVISE_BOTTLE_SLUG", "").strip()
|
||||
self._token_allow_timeout = _token_allow_timeout_from_env(os.environ)
|
||||
self._reload(initial=True)
|
||||
self._install_sighup()
|
||||
|
||||
def _supervise_available(self) -> bool:
|
||||
return bool(self._supervise_slug)
|
||||
return bool(self._supervise_queue_dir and self._supervise_slug)
|
||||
|
||||
def _reload(self, *, initial: bool = False) -> None:
|
||||
try:
|
||||
@@ -392,8 +393,9 @@ class EgressAddon:
|
||||
justification=_TOKEN_ALLOW_JUSTIFICATION,
|
||||
current_file_hash=_sv.sha256_hex(payload),
|
||||
)
|
||||
queue_dir = Path(self._supervise_queue_dir)
|
||||
try:
|
||||
_sv.write_proposal(proposal)
|
||||
_sv.write_proposal(queue_dir, proposal)
|
||||
except OSError as e:
|
||||
sys.stderr.write(
|
||||
f"egress: could not queue token-allow proposal: {e}; "
|
||||
@@ -409,8 +411,8 @@ class EgressAddon:
|
||||
**self._req_ctx(flow),
|
||||
}) + "\n")
|
||||
|
||||
response = await self._await_token_response(proposal.id)
|
||||
_sv.archive_proposal(self._supervise_slug, proposal.id)
|
||||
response = await self._await_token_response(queue_dir, proposal.id)
|
||||
_sv.archive_proposal(queue_dir, proposal.id)
|
||||
|
||||
if response is not None and response.status in (
|
||||
_sv.STATUS_APPROVED, _sv.STATUS_MODIFIED,
|
||||
@@ -437,15 +439,16 @@ class EgressAddon:
|
||||
|
||||
async def _await_token_response(
|
||||
self,
|
||||
queue_dir: Path,
|
||||
proposal_id: str,
|
||||
) -> "_sv.Response | None":
|
||||
"""Poll the DB for the operator's response without blocking the
|
||||
"""Poll the queue dir for the operator's response without blocking the
|
||||
proxy event loop. Returns the Response, or None on timeout."""
|
||||
loop = asyncio.get_running_loop()
|
||||
deadline = loop.time() + self._token_allow_timeout
|
||||
while True:
|
||||
try:
|
||||
return _sv.read_response(self._supervise_slug, proposal_id)
|
||||
return _sv.read_response(queue_dir, proposal_id)
|
||||
except (OSError, ValueError, KeyError):
|
||||
# Not written yet, or a partial/malformed write — retry until
|
||||
# the deadline, then fail closed.
|
||||
|
||||
+2
-4
@@ -35,7 +35,6 @@ import re
|
||||
import sys
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
from .errors import MissingEnvVarError
|
||||
from .log import die
|
||||
from .manifest import Manifest
|
||||
|
||||
@@ -137,10 +136,9 @@ def resolve_env(manifest: Manifest) -> ResolvedEnv:
|
||||
host_var = env_entry_interpolated_from(raw)
|
||||
host_value = os.environ.get(host_var, "")
|
||||
if not host_value:
|
||||
raise MissingEnvVarError(
|
||||
host_var,
|
||||
die(
|
||||
f"env entry {name} is interpolated from ${host_var}, "
|
||||
f"but ${host_var} is unset or empty in the host environment.",
|
||||
f"but ${host_var} is unset or empty in the host environment."
|
||||
)
|
||||
forwarded[name] = host_value
|
||||
else: # literal
|
||||
|
||||
@@ -1,18 +0,0 @@
|
||||
"""bot-bottle application-level exception hierarchy.
|
||||
|
||||
Exceptions here are caught by the CLI dispatcher (``cli/__init__.py``)
|
||||
and rendered as ``bot-bottle: error: …`` lines — same UX as ``die()``,
|
||||
but without coupling library code to stderr output."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
|
||||
class MissingEnvVarError(Exception):
|
||||
"""Raised when a required host environment variable is unset or empty.
|
||||
|
||||
``var_name`` is the exact name of the missing variable so callers
|
||||
can display it without re-parsing the message string."""
|
||||
|
||||
def __init__(self, var_name: str, message: str) -> None:
|
||||
self.var_name = var_name
|
||||
super().__init__(message)
|
||||
+11
-7
@@ -30,6 +30,7 @@ backend-specific and lives on concrete subclasses (see
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import dataclasses
|
||||
from abc import ABC
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
@@ -52,7 +53,6 @@ from .git_gate_render import (
|
||||
_gitconfig_validate_value,
|
||||
)
|
||||
from .git_gate_provision import (
|
||||
provision_git_gate_dynamic_keys,
|
||||
revoke_git_gate_provisioned_keys,
|
||||
_provision_dynamic_key,
|
||||
_resolve_identity_file,
|
||||
@@ -82,7 +82,6 @@ class GitGatePlan:
|
||||
|
||||
|
||||
|
||||
|
||||
class GitGate(ABC):
|
||||
"""The per-agent git-gate. Encapsulates the host-side prepare
|
||||
(upstream lift + entrypoint/hook render); the sidecar's
|
||||
@@ -94,14 +93,20 @@ class GitGate(ABC):
|
||||
entrypoint, pre-receive hook, and access-hook scripts (mode
|
||||
600) under `stage_dir`. Pure host-side, no docker subprocess.
|
||||
|
||||
For `gitea` key entries, the returned upstream intentionally
|
||||
has an empty identity file. Backend launch fills that in after
|
||||
the operator confirms the preflight.
|
||||
For `gitea` key entries, also generates and registers
|
||||
a fresh deploy key via the forge API and writes the private key
|
||||
+ key ID to `stage_dir`.
|
||||
|
||||
Returned plan is incomplete: the launch step must fill
|
||||
`internal_network` / `egress_network` via `dataclasses.replace`
|
||||
before passing the plan to `.start`."""
|
||||
upstreams = git_gate_upstreams_for_bottle(bottle)
|
||||
upstreams_list = list(git_gate_upstreams_for_bottle(bottle))
|
||||
for i, entry in enumerate(bottle.git):
|
||||
upstreams_list[i] = dataclasses.replace(
|
||||
upstreams_list[i],
|
||||
identity_file=_resolve_identity_file(entry, slug, stage_dir),
|
||||
)
|
||||
upstreams = tuple(upstreams_list)
|
||||
entrypoint = stage_dir / "git_gate_entrypoint.sh"
|
||||
entrypoint.write_text(git_gate_render_entrypoint(upstreams))
|
||||
entrypoint.chmod(0o600)
|
||||
@@ -157,7 +162,6 @@ __all__ = [
|
||||
"git_gate_render_entrypoint",
|
||||
"git_gate_render_hook",
|
||||
"git_gate_render_access_hook",
|
||||
"provision_git_gate_dynamic_keys",
|
||||
"revoke_git_gate_provisioned_keys",
|
||||
"_gitconfig_validate_value",
|
||||
"_provision_dynamic_key",
|
||||
|
||||
@@ -1,268 +0,0 @@
|
||||
"""Preflight host-key population for git-gate upstreams (issue #333).
|
||||
|
||||
When a git-gate repo entry lacks a `host_key`, this module either:
|
||||
- headless: dies with a clear config error.
|
||||
- interactive: fetches the key via ssh-keyscan, prompts the operator to
|
||||
confirm, and optionally persists it to the bottle config file on disk.
|
||||
|
||||
Public entry point: `preflight_host_keys(manifest, headless=..., home_md=...)`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import dataclasses
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import cast
|
||||
|
||||
from .log import die, info
|
||||
from .manifest import Manifest
|
||||
from .yaml_subset import YamlSubsetError, parse_frontmatter, serialize_yaml_subset
|
||||
|
||||
|
||||
# Preferred key types, most secure first.
|
||||
_KEY_TYPE_PREFERENCE = (
|
||||
"ssh-ed25519",
|
||||
"ecdsa-sha2-nistp256",
|
||||
"ecdsa-sha2-nistp384",
|
||||
"ecdsa-sha2-nistp521",
|
||||
"ssh-rsa",
|
||||
)
|
||||
|
||||
|
||||
def fetch_host_key(host: str, port: str) -> str:
|
||||
"""Return an SSH public key for `host`:`port` via ssh-keyscan.
|
||||
|
||||
Returns the key in `<type> <base64-data>` format (the host prefix is
|
||||
stripped so the result can be stored in `host_key` and later formatted
|
||||
into a known_hosts line by `git_gate_known_hosts_line`).
|
||||
|
||||
Prefers ed25519 > ecdsa > rsa; falls back to the first key type
|
||||
returned if none of the preferred types are present.
|
||||
|
||||
Raises `RuntimeError` on subprocess failure, timeout, or no result.
|
||||
Uses only the Python stdlib (subprocess)."""
|
||||
args = ["ssh-keyscan"]
|
||||
if port and port != "22":
|
||||
args += ["-p", port]
|
||||
args.append(host)
|
||||
try:
|
||||
result = subprocess.run(
|
||||
args, capture_output=True, text=True, timeout=15, check=False,
|
||||
)
|
||||
except OSError as e:
|
||||
raise RuntimeError(
|
||||
f"ssh-keyscan: could not launch for {host}:{port}: {e}"
|
||||
) from e
|
||||
except subprocess.TimeoutExpired as e:
|
||||
raise RuntimeError(f"ssh-keyscan timed out for {host}:{port}") from e
|
||||
|
||||
# known_hosts format: "[host]:port type data" or "host type data"
|
||||
# Strip the host/port prefix; collect "type -> type data" by type.
|
||||
found: dict[str, str] = {}
|
||||
for line in result.stdout.splitlines():
|
||||
line = line.strip()
|
||||
if not line or line.startswith("#"):
|
||||
continue
|
||||
parts = line.split(None, 2)
|
||||
if len(parts) == 3 and parts[1] not in found:
|
||||
found[parts[1]] = f"{parts[1]} {parts[2]}"
|
||||
|
||||
for preferred in _KEY_TYPE_PREFERENCE:
|
||||
if preferred in found:
|
||||
return found[preferred]
|
||||
if found:
|
||||
return next(iter(found.values()))
|
||||
|
||||
raise RuntimeError(
|
||||
f"ssh-keyscan returned no host key for {host}:{port}."
|
||||
+ (f" stderr: {result.stderr.strip()!r}" if result.stderr.strip() else "")
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Frontmatter editing
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def add_host_key_to_frontmatter(file_text: str, repo_name: str, host_key: str) -> str:
|
||||
"""Return an updated copy of `file_text` with `host_key` set on the
|
||||
named repo entry in the YAML frontmatter.
|
||||
|
||||
Parses the frontmatter into a dict, sets the key, and re-serializes.
|
||||
Returns the original text unchanged when: the file has no frontmatter,
|
||||
the git-gate.repos.<repo_name> entry is absent or already has a
|
||||
host_key, or the frontmatter cannot be parsed."""
|
||||
try:
|
||||
fm, body = parse_frontmatter(file_text)
|
||||
except YamlSubsetError:
|
||||
return file_text
|
||||
|
||||
git_gate = fm.get("git-gate")
|
||||
if not isinstance(git_gate, dict):
|
||||
return file_text
|
||||
repos = git_gate.get("repos")
|
||||
if not isinstance(repos, dict):
|
||||
return file_text
|
||||
repo = repos.get(repo_name)
|
||||
if not isinstance(repo, dict):
|
||||
return file_text
|
||||
if repo.get("host_key"):
|
||||
return file_text
|
||||
|
||||
cast(dict[str, object], repo)["host_key"] = host_key
|
||||
return f"---\n{serialize_yaml_subset(fm)}---\n{body}"
|
||||
|
||||
|
||||
def find_repo_bottle_file(bottles_dir: Path, repo_name: str) -> Path | None:
|
||||
"""Return the first `bottles_dir/*.md` that declares `repo_name` without
|
||||
a `host_key`, without modifying anything. Returns None if not found."""
|
||||
if not bottles_dir.is_dir():
|
||||
return None
|
||||
for path in sorted(bottles_dir.glob("*.md")):
|
||||
try:
|
||||
text = path.read_text(encoding="utf-8")
|
||||
fm, _ = parse_frontmatter(text)
|
||||
except (OSError, UnicodeDecodeError, YamlSubsetError):
|
||||
continue
|
||||
git_gate = fm.get("git-gate")
|
||||
if not isinstance(git_gate, dict):
|
||||
continue
|
||||
repos = git_gate.get("repos")
|
||||
if not isinstance(repos, dict):
|
||||
continue
|
||||
repo = repos.get(repo_name)
|
||||
if not isinstance(repo, dict) or repo.get("host_key"):
|
||||
continue
|
||||
return path
|
||||
return None
|
||||
|
||||
|
||||
def find_and_update_bottle_file(
|
||||
bottles_dir: Path, repo_name: str, host_key: str,
|
||||
) -> bool:
|
||||
"""Write `host_key` into the bottle file returned by `find_repo_bottle_file`.
|
||||
|
||||
Returns True on success, False when no suitable file is found or the
|
||||
write fails."""
|
||||
path = find_repo_bottle_file(bottles_dir, repo_name)
|
||||
if path is None:
|
||||
return False
|
||||
try:
|
||||
text = path.read_text(encoding="utf-8")
|
||||
except (OSError, UnicodeDecodeError):
|
||||
return False
|
||||
updated = add_host_key_to_frontmatter(text, repo_name, host_key)
|
||||
if updated == text:
|
||||
return False
|
||||
path.write_text(updated, encoding="utf-8")
|
||||
info(f"wrote host_key for {repo_name!r} to {path}")
|
||||
return True
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Interactive prompt helper
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def prompt_tty(message: str) -> str:
|
||||
"""Write `message` to stderr and read a line from /dev/tty (or stdin)."""
|
||||
sys.stderr.write(message)
|
||||
sys.stderr.flush()
|
||||
try:
|
||||
with open("/dev/tty", "r", encoding="utf-8") as tty:
|
||||
return tty.readline().rstrip("\n")
|
||||
except OSError:
|
||||
return sys.stdin.readline().rstrip("\n")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Public API
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def preflight_host_keys(
|
||||
manifest: Manifest,
|
||||
*,
|
||||
headless: bool,
|
||||
home_md: Path | None,
|
||||
) -> Manifest:
|
||||
"""Ensure every git-gate repo entry has a `host_key` configured.
|
||||
|
||||
For entries whose `KnownHostKey` is empty:
|
||||
- headless: calls `die()` with a clear message naming the repos.
|
||||
- interactive: fetches the key via ssh-keyscan, shows it to the
|
||||
operator, and requests confirmation. If accepted, optionally
|
||||
persists it to the bottle config file on disk; the key is always
|
||||
applied in memory for this launch regardless of the persistence
|
||||
choice. Aborted confirmation calls `die()`.
|
||||
|
||||
Returns a (possibly updated) Manifest. If all entries already have
|
||||
host keys the original manifest is returned unchanged."""
|
||||
bottle = manifest.bottle
|
||||
missing = [e for e in bottle.git if not e.KnownHostKey]
|
||||
if not missing:
|
||||
return manifest
|
||||
|
||||
if headless:
|
||||
names = ", ".join(repr(e.Name) for e in missing)
|
||||
die(
|
||||
f"git-gate: no host_key configured for repo(s) {names}. "
|
||||
f"Add host_key to each bottle git-gate.repos entry, or run "
|
||||
f"interactively once to have it fetched and saved automatically."
|
||||
)
|
||||
|
||||
bottles_dir = (home_md / "bottles") if home_md is not None else None
|
||||
updated_entries = list(bottle.git)
|
||||
|
||||
for entry in missing:
|
||||
host = entry.UpstreamHost
|
||||
port = entry.UpstreamPort
|
||||
label = f"git-gate.repos[{entry.Name!r}]"
|
||||
|
||||
info(f"{label}: no host_key configured; fetching from {host}:{port}")
|
||||
try:
|
||||
key = fetch_host_key(host, port)
|
||||
except RuntimeError as e:
|
||||
die(f"git-gate: {label}: {e}")
|
||||
|
||||
sys.stderr.write(f"\ngit-gate: host key for {label}:\n {key}\n\n")
|
||||
confirm = prompt_tty("Is this host key correct? [y/N] ")
|
||||
if confirm.strip().lower() not in ("y", "yes"):
|
||||
die(f"git-gate: {label}: host key not confirmed; aborting launch")
|
||||
|
||||
if bottles_dir is not None:
|
||||
target_file = find_repo_bottle_file(bottles_dir, entry.Name)
|
||||
if target_file is not None:
|
||||
save = prompt_tty(
|
||||
f"Save host_key for {entry.Name!r} to {target_file}? [y/N] "
|
||||
)
|
||||
if save.strip().lower() in ("y", "yes"):
|
||||
ok = find_and_update_bottle_file(bottles_dir, entry.Name, key)
|
||||
if not ok:
|
||||
sys.stderr.write(
|
||||
f"git-gate: {label}: could not write to {target_file}; "
|
||||
f"host_key kept in memory for this session only\n"
|
||||
)
|
||||
else:
|
||||
sys.stderr.write(
|
||||
f"git-gate: {label}: no bottle config file found for "
|
||||
f"{entry.Name!r}; host_key kept in memory for this session only\n"
|
||||
)
|
||||
|
||||
idx = next(i for i, e in enumerate(updated_entries) if e.Name == entry.Name)
|
||||
updated_entries[idx] = dataclasses.replace(entry, KnownHostKey=key)
|
||||
|
||||
updated_bottle = dataclasses.replace(bottle, git=tuple(updated_entries))
|
||||
return dataclasses.replace(manifest, bottle=updated_bottle)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"fetch_host_key",
|
||||
"preflight_host_keys",
|
||||
"add_host_key_to_frontmatter",
|
||||
"find_repo_bottle_file",
|
||||
"find_and_update_bottle_file",
|
||||
"prompt_tty",
|
||||
]
|
||||
@@ -9,17 +9,10 @@ imported (`deploy_key_provisioner`) to keep its cost off the host path.
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import dataclasses
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from .errors import MissingEnvVarError
|
||||
from .log import info
|
||||
from .manifest import ManifestBottle, ManifestGitEntry
|
||||
from .git_gate_render import GitGateUpstream
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from .git_gate import GitGatePlan
|
||||
|
||||
def _provision_dynamic_key(
|
||||
entry: ManifestGitEntry,
|
||||
@@ -35,10 +28,9 @@ def _provision_dynamic_key(
|
||||
pk = entry.Key
|
||||
token = os.environ.get(pk.forge_token_env)
|
||||
if token is None:
|
||||
raise MissingEnvVarError(
|
||||
pk.forge_token_env,
|
||||
raise RuntimeError(
|
||||
f"git-gate.repos[{entry.Name!r}] key.forge_token_env"
|
||||
f" = {pk.forge_token_env!r}: env var is not set",
|
||||
f" = {pk.forge_token_env!r}: env var is not set"
|
||||
)
|
||||
api_url = pk.api_url or f"https://{entry.UpstreamHost}"
|
||||
provisioner = get_provisioner(pk.provider, token, api_url)
|
||||
@@ -80,11 +72,10 @@ def revoke_git_gate_provisioned_keys(bottle: ManifestBottle, stage_dir: Path) ->
|
||||
key_id = id_file.read_text().strip()
|
||||
token = os.environ.get(pk.forge_token_env)
|
||||
if token is None:
|
||||
raise MissingEnvVarError(
|
||||
pk.forge_token_env,
|
||||
raise RuntimeError(
|
||||
f"git-gate.repos[{entry.Name!r}] key.forge_token_env"
|
||||
f" = {pk.forge_token_env!r}: env var is not set;"
|
||||
f" cannot revoke deploy key {key_id}",
|
||||
f" cannot revoke deploy key {key_id}"
|
||||
)
|
||||
api_url = pk.api_url or f"https://{entry.UpstreamHost}"
|
||||
provisioner = get_provisioner(pk.provider, token, api_url)
|
||||
@@ -104,45 +95,8 @@ def _resolve_identity_file(entry: ManifestGitEntry, slug: str, stage_dir: Path)
|
||||
return entry.IdentityFile
|
||||
|
||||
|
||||
def provision_git_gate_dynamic_keys(
|
||||
bottle: ManifestBottle,
|
||||
plan: "GitGatePlan",
|
||||
stage_dir: Path,
|
||||
) -> "GitGatePlan":
|
||||
"""Provision dynamic git-gate keys and return an updated plan.
|
||||
|
||||
This runs during backend launch, after the operator confirms the
|
||||
preflight. Plan preparation intentionally stays side-effect-light:
|
||||
dry-runs and aborted launches must not create remote deploy keys.
|
||||
"""
|
||||
if not plan.upstreams:
|
||||
return plan
|
||||
|
||||
upstreams_by_name: dict[str, GitGateUpstream] = {
|
||||
upstream.name: upstream for upstream in plan.upstreams
|
||||
}
|
||||
updated: list[GitGateUpstream] = []
|
||||
for entry in bottle.git:
|
||||
upstream = upstreams_by_name.get(entry.Name)
|
||||
if upstream is None:
|
||||
continue
|
||||
if entry.Key.provider == "gitea":
|
||||
identity_file = _provision_dynamic_key(entry, plan.slug, stage_dir)
|
||||
upstream = dataclasses.replace(upstream, identity_file=identity_file)
|
||||
updated.append(upstream)
|
||||
|
||||
if len(updated) != len(plan.upstreams):
|
||||
updated_names = {u.name for u in updated}
|
||||
for upstream in plan.upstreams:
|
||||
if upstream.name not in updated_names:
|
||||
updated.append(upstream)
|
||||
|
||||
return dataclasses.replace(plan, upstreams=tuple(updated))
|
||||
|
||||
|
||||
__all__ = [
|
||||
"revoke_git_gate_provisioned_keys",
|
||||
"provision_git_gate_dynamic_keys",
|
||||
"_provision_dynamic_key",
|
||||
"_resolve_identity_file",
|
||||
]
|
||||
|
||||
@@ -228,22 +228,19 @@ supervise_gitleaks_allow() {
|
||||
fi
|
||||
|
||||
proposal_id=$(
|
||||
PYTHONPATH="/app${PYTHONPATH:+:$PYTHONPATH}" GITLEAKS_ALLOW_REF="$ref" python3 - "$report_file" <<'PY'
|
||||
GITLEAKS_ALLOW_REF="$ref" python3 - "$report_file" <<'PY'
|
||||
import datetime
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
import supervise as _sv
|
||||
except ImportError:
|
||||
from bot_bottle import supervise as _sv
|
||||
|
||||
report_path = Path(sys.argv[1])
|
||||
queue_dir = os.environ.get("SUPERVISE_QUEUE_DIR", "")
|
||||
slug = os.environ.get("SUPERVISE_BOTTLE_SLUG", "")
|
||||
if not slug:
|
||||
if not queue_dir or not slug:
|
||||
sys.exit(2)
|
||||
|
||||
try:
|
||||
@@ -280,19 +277,31 @@ for i, finding in enumerate(raw, 1):
|
||||
])
|
||||
|
||||
payload = "\n".join(lines).rstrip() + "\n"
|
||||
proposal = _sv.Proposal.new(
|
||||
bottle_slug=slug,
|
||||
tool=_sv.TOOL_GITLEAKS_ALLOW,
|
||||
proposed_file=payload,
|
||||
justification=(
|
||||
proposal_id = str(uuid.uuid4())
|
||||
proposal = {
|
||||
"id": proposal_id,
|
||||
"bottle_slug": slug,
|
||||
"tool": "gitleaks-allow",
|
||||
"proposed_file": payload,
|
||||
"justification": (
|
||||
"git-gate found gitleaks findings hidden by # gitleaks:allow; "
|
||||
"approve only for dummy test fixtures or confirmed false positives"
|
||||
),
|
||||
current_file_hash=hashlib.sha256(payload.encode("utf-8")).hexdigest(),
|
||||
now=datetime.datetime.now(datetime.timezone.utc),
|
||||
)
|
||||
_sv.write_proposal(proposal)
|
||||
print(proposal.id)
|
||||
"arrival_timestamp": datetime.datetime.now(
|
||||
datetime.timezone.utc
|
||||
).isoformat(),
|
||||
"current_file_hash": hashlib.sha256(payload.encode("utf-8")).hexdigest(),
|
||||
}
|
||||
queue = Path(queue_dir)
|
||||
queue.mkdir(parents=True, exist_ok=True)
|
||||
path = queue / f"{proposal_id}.proposal.json"
|
||||
tmp = path.with_suffix(path.suffix + ".tmp")
|
||||
with tmp.open("w", encoding="utf-8") as f:
|
||||
json.dump(proposal, f, indent=2)
|
||||
f.write("\n")
|
||||
os.chmod(tmp, 0o600)
|
||||
os.replace(tmp, path)
|
||||
print(proposal_id)
|
||||
PY
|
||||
)
|
||||
rc=$?
|
||||
@@ -305,7 +314,8 @@ PY
|
||||
return 1
|
||||
fi
|
||||
|
||||
slug=${SUPERVISE_BOTTLE_SLUG:-}
|
||||
queue_dir=${SUPERVISE_QUEUE_DIR:-}
|
||||
response_file="$queue_dir/${proposal_id}.response.json"
|
||||
timeout=${SUPERVISE_GITLEAKS_ALLOW_TIMEOUT_SECONDS:-300}
|
||||
case "$timeout" in
|
||||
''|*[!0-9]*)
|
||||
@@ -317,41 +327,26 @@ PY
|
||||
echo "git-gate: approve with './cli.py supervise' to continue this push" >&2
|
||||
waited=0
|
||||
while [ "$waited" -lt "$timeout" ]; do
|
||||
status=$(PYTHONPATH="/app${PYTHONPATH:+:$PYTHONPATH}" python3 - "$slug" "$proposal_id" <<'PY'
|
||||
if [ -f "$response_file" ]; then
|
||||
status=$(python3 - "$response_file" <<'PY'
|
||||
import json
|
||||
import sys
|
||||
|
||||
try:
|
||||
import supervise as _sv
|
||||
except ImportError:
|
||||
from bot_bottle import supervise as _sv
|
||||
|
||||
slug = sys.argv[1]
|
||||
try:
|
||||
response = _sv.read_response(slug, sys.argv[2])
|
||||
except FileNotFoundError:
|
||||
sys.exit(2)
|
||||
print(response.status)
|
||||
with open(sys.argv[1], encoding="utf-8") as f:
|
||||
raw = json.load(f)
|
||||
except (OSError, json.JSONDecodeError):
|
||||
sys.exit(1)
|
||||
status = raw.get("status")
|
||||
if not isinstance(status, str):
|
||||
sys.exit(1)
|
||||
print(status)
|
||||
PY
|
||||
)
|
||||
rc=$?
|
||||
if [ "$rc" -eq 2 ]; then
|
||||
status=""
|
||||
elif [ "$rc" -ne 0 ]; then
|
||||
status="invalid"
|
||||
fi
|
||||
if [ -n "$status" ]; then
|
||||
) || status=""
|
||||
case "$status" in
|
||||
approved|modified)
|
||||
PYTHONPATH="/app${PYTHONPATH:+:$PYTHONPATH}" python3 - "$slug" "$proposal_id" <<'PY' || true
|
||||
import sys
|
||||
|
||||
try:
|
||||
import supervise as _sv
|
||||
except ImportError:
|
||||
from bot_bottle import supervise as _sv
|
||||
|
||||
_sv.archive_proposal(sys.argv[1], sys.argv[2])
|
||||
PY
|
||||
mkdir -p "$queue_dir/processed"
|
||||
mv -f "$queue_dir/${proposal_id}.proposal.json" "$queue_dir/processed/" 2>/dev/null || true
|
||||
mv -f "$queue_dir/${proposal_id}.response.json" "$queue_dir/processed/" 2>/dev/null || true
|
||||
echo "git-gate: supervisor approved # gitleaks:allow for $ref" >&2
|
||||
return 0
|
||||
;;
|
||||
@@ -504,3 +499,4 @@ if ! git -C "$repo_dir" rev-parse --verify HEAD >/dev/null 2>&1; then
|
||||
fi
|
||||
exit 0
|
||||
"""
|
||||
|
||||
|
||||
@@ -16,16 +16,11 @@ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from pathlib import Path
|
||||
from urllib.parse import urlsplit
|
||||
|
||||
from .git_gate import GIT_GATE_TIMEOUT_SECS
|
||||
|
||||
|
||||
DEFAULT_PORT = 9420
|
||||
|
||||
# Mirrors git_gate_render.GIT_GATE_TIMEOUT_SECS. Duplicated rather than
|
||||
# imported: this module ships as a flat top-level sibling in the sidecar
|
||||
# bundle image (see Dockerfile.sidecars), not as part of the bot_bottle
|
||||
# package, so `bot_bottle.git_gate` and its dependency chain aren't
|
||||
# available at runtime.
|
||||
GIT_GATE_TIMEOUT_SECS = 15
|
||||
|
||||
# Bound memory use while still allowing ordinary git push packfiles.
|
||||
MAX_BODY_BYTES = 100 * 1024 * 1024
|
||||
|
||||
|
||||
@@ -1,42 +0,0 @@
|
||||
"""Shared helpers for cached-image quickstart stale checks."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
from .config_store import ConfigStore
|
||||
except ImportError:
|
||||
from config_store import ConfigStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
|
||||
|
||||
|
||||
class StaleImageError(Exception):
|
||||
"""Raised when a cached image or artifact exceeds the configured staleness
|
||||
threshold. Callers can catch this to prompt interactively; headless paths
|
||||
let it propagate as a fatal error."""
|
||||
|
||||
|
||||
def check_stale(label: str, created_at: datetime) -> None:
|
||||
"""Raise StaleImageError if `created_at` is older than the configured
|
||||
stale-warning threshold. Negative threshold disables the check."""
|
||||
threshold_days = ConfigStore().cached_image_stale_warning_days()
|
||||
if threshold_days < 0:
|
||||
return
|
||||
now = datetime.now(timezone.utc)
|
||||
created = created_at.astimezone(timezone.utc)
|
||||
age = now - created
|
||||
if age.total_seconds() <= threshold_days * 86400:
|
||||
return
|
||||
raise StaleImageError(
|
||||
f"cached {label} is {age.days} day(s) old; "
|
||||
"quickstart does not verify it matches the current Dockerfile/context"
|
||||
)
|
||||
|
||||
|
||||
def check_stale_path(label: str, path: Path) -> None:
|
||||
"""Raise StaleImageError if `path`'s mtime exceeds the staleness threshold."""
|
||||
check_stale(label, datetime.fromtimestamp(path.stat().st_mtime, tz=timezone.utc))
|
||||
|
||||
|
||||
__all__ = ["StaleImageError", "check_stale", "check_stale_path"]
|
||||
@@ -1,37 +0,0 @@
|
||||
"""SQLite migration runner for bot-bottle stores."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sqlite3
|
||||
|
||||
|
||||
class TableMigrations:
|
||||
"""Runs a sequential list of DDL migrations tracked by schema_key in schema_versions."""
|
||||
|
||||
def __init__(self, schema_key: str, migrations: list[str]) -> None:
|
||||
self.schema_key = schema_key
|
||||
self.migrations = migrations
|
||||
|
||||
def apply(self, conn: sqlite3.Connection) -> None:
|
||||
conn.execute(
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS schema_versions (
|
||||
module TEXT PRIMARY KEY,
|
||||
version INTEGER NOT NULL DEFAULT 0
|
||||
)
|
||||
"""
|
||||
)
|
||||
row = conn.execute(
|
||||
"SELECT version FROM schema_versions WHERE module = ?",
|
||||
(self.schema_key,),
|
||||
).fetchone()
|
||||
version = row[0] if row else 0
|
||||
for i, sql in enumerate(self.migrations[version:], start=version + 1):
|
||||
conn.execute(sql)
|
||||
conn.execute(
|
||||
"INSERT OR REPLACE INTO schema_versions (module, version) VALUES (?, ?)",
|
||||
(self.schema_key, i),
|
||||
)
|
||||
|
||||
|
||||
__all__ = ["TableMigrations"]
|
||||
@@ -1,215 +0,0 @@
|
||||
"""SQLite-backed queue store for supervise proposals and responses (PRD 0013)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sqlite3
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
from .supervise_types import Proposal, Response, host_db_path
|
||||
from .db_store import DbStore
|
||||
from .migrations import TableMigrations
|
||||
except ImportError:
|
||||
from supervise_types import Proposal, Response, host_db_path # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
|
||||
from db_store import DbStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
|
||||
from migrations import TableMigrations # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
|
||||
|
||||
|
||||
class QueueStore(DbStore):
|
||||
"""SQLite-backed persistent store for supervise proposals and responses."""
|
||||
|
||||
def __init__(self, queue_key: str, db_path: Path | None = None) -> None:
|
||||
self.queue_key = queue_key
|
||||
if db_path is not None:
|
||||
resolved = db_path
|
||||
else:
|
||||
# In the sidecar container SUPERVISE_DB_PATH points at the
|
||||
# bind-mounted host DB. On the host this env var is never set,
|
||||
# so we always fall through to host_db_path().
|
||||
env_path = os.environ.get("SUPERVISE_DB_PATH", "").strip()
|
||||
resolved = Path(env_path) if env_path else host_db_path()
|
||||
# One entry per schema version: migrations[0] brings a fresh DB to
|
||||
# version 1, [1] to version 2, etc. Add new entries at the end; never
|
||||
# edit existing ones.
|
||||
migrations = TableMigrations("queue_store", [
|
||||
# v1 — proposals table
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS supervise_proposals (
|
||||
queue_key TEXT NOT NULL,
|
||||
id TEXT NOT NULL,
|
||||
bottle_slug TEXT NOT NULL,
|
||||
tool TEXT NOT NULL,
|
||||
proposed_file TEXT NOT NULL,
|
||||
justification TEXT NOT NULL,
|
||||
arrival_timestamp TEXT NOT NULL,
|
||||
current_file_hash TEXT NOT NULL,
|
||||
archived INTEGER NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (queue_key, id)
|
||||
)
|
||||
""",
|
||||
# v2 — responses table
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS supervise_responses (
|
||||
queue_key TEXT NOT NULL,
|
||||
proposal_id TEXT NOT NULL,
|
||||
status TEXT NOT NULL,
|
||||
notes TEXT NOT NULL,
|
||||
final_file TEXT,
|
||||
archived INTEGER NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (queue_key, proposal_id)
|
||||
)
|
||||
""",
|
||||
])
|
||||
super().__init__(resolved, migrations)
|
||||
|
||||
def write_proposal(self, proposal: Proposal) -> Path:
|
||||
with self._connect() as conn:
|
||||
conn.execute(
|
||||
"""
|
||||
INSERT OR REPLACE INTO supervise_proposals (
|
||||
queue_key, id, bottle_slug, tool, proposed_file, justification,
|
||||
arrival_timestamp, current_file_hash, archived
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, 0)
|
||||
""",
|
||||
(
|
||||
self.queue_key,
|
||||
proposal.id,
|
||||
proposal.bottle_slug,
|
||||
proposal.tool,
|
||||
proposal.proposed_file,
|
||||
proposal.justification,
|
||||
proposal.arrival_timestamp,
|
||||
proposal.current_file_hash,
|
||||
),
|
||||
)
|
||||
self._chmod()
|
||||
return self.db_path
|
||||
|
||||
def read_proposal(self, proposal_id: str) -> Proposal:
|
||||
with self._connect() as conn:
|
||||
row = conn.execute(
|
||||
"""
|
||||
SELECT * FROM supervise_proposals
|
||||
WHERE queue_key = ? AND id = ? AND archived = 0
|
||||
""",
|
||||
(self.queue_key, proposal_id),
|
||||
).fetchone()
|
||||
if row is None:
|
||||
raise FileNotFoundError(proposal_id)
|
||||
return self._row_to_proposal(row)
|
||||
|
||||
def list_pending_proposals(self) -> list[Proposal]:
|
||||
if not self.db_path.is_file():
|
||||
return []
|
||||
with self._connect() as conn:
|
||||
rows = conn.execute(
|
||||
"""
|
||||
SELECT p.* FROM supervise_proposals p
|
||||
WHERE p.archived = 0
|
||||
AND p.queue_key = ?
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM supervise_responses r
|
||||
WHERE r.queue_key = p.queue_key
|
||||
AND r.proposal_id = p.id
|
||||
AND r.archived = 0
|
||||
)
|
||||
ORDER BY p.arrival_timestamp, p.id
|
||||
""",
|
||||
(self.queue_key,),
|
||||
).fetchall()
|
||||
return [self._row_to_proposal(row) for row in rows]
|
||||
|
||||
def list_all_pending_proposals(self) -> list[Proposal]:
|
||||
if not self.db_path.is_file():
|
||||
return []
|
||||
with self._connect() as conn:
|
||||
rows = conn.execute(
|
||||
"""
|
||||
SELECT p.* FROM supervise_proposals p
|
||||
WHERE p.archived = 0
|
||||
AND NOT EXISTS (
|
||||
SELECT 1 FROM supervise_responses r
|
||||
WHERE r.queue_key = p.queue_key
|
||||
AND r.proposal_id = p.id
|
||||
AND r.archived = 0
|
||||
)
|
||||
ORDER BY p.arrival_timestamp, p.id
|
||||
"""
|
||||
).fetchall()
|
||||
return [self._row_to_proposal(row) for row in rows]
|
||||
|
||||
def write_response(self, response: Response) -> Path:
|
||||
with self._connect() as conn:
|
||||
conn.execute(
|
||||
"""
|
||||
INSERT OR REPLACE INTO supervise_responses (
|
||||
queue_key, proposal_id, status, notes, final_file, archived
|
||||
) VALUES (?, ?, ?, ?, ?, 0)
|
||||
""",
|
||||
(
|
||||
self.queue_key,
|
||||
response.proposal_id,
|
||||
response.status,
|
||||
response.notes,
|
||||
response.final_file,
|
||||
),
|
||||
)
|
||||
self._chmod()
|
||||
return self.db_path
|
||||
|
||||
def read_response(self, proposal_id: str) -> Response:
|
||||
with self._connect() as conn:
|
||||
row = conn.execute(
|
||||
"""
|
||||
SELECT * FROM supervise_responses
|
||||
WHERE queue_key = ? AND proposal_id = ? AND archived = 0
|
||||
""",
|
||||
(self.queue_key, proposal_id),
|
||||
).fetchone()
|
||||
if row is None:
|
||||
raise FileNotFoundError(proposal_id)
|
||||
return self._row_to_response(row)
|
||||
|
||||
def archive_proposal(self, proposal_id: str) -> None:
|
||||
if not self.db_path.is_file():
|
||||
return
|
||||
with self._connect() as conn:
|
||||
conn.execute(
|
||||
"""
|
||||
UPDATE supervise_proposals SET archived = 1
|
||||
WHERE queue_key = ? AND id = ?
|
||||
""",
|
||||
(self.queue_key, proposal_id),
|
||||
)
|
||||
conn.execute(
|
||||
"""
|
||||
UPDATE supervise_responses SET archived = 1
|
||||
WHERE queue_key = ? AND proposal_id = ?
|
||||
""",
|
||||
(self.queue_key, proposal_id),
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _row_to_proposal(row: sqlite3.Row) -> Proposal:
|
||||
return Proposal(
|
||||
id=row["id"],
|
||||
bottle_slug=row["bottle_slug"],
|
||||
tool=row["tool"],
|
||||
proposed_file=row["proposed_file"],
|
||||
justification=row["justification"],
|
||||
arrival_timestamp=row["arrival_timestamp"],
|
||||
current_file_hash=row["current_file_hash"],
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _row_to_response(row: sqlite3.Row) -> Response:
|
||||
return Response(
|
||||
proposal_id=row["proposal_id"],
|
||||
status=row["status"],
|
||||
notes=row["notes"],
|
||||
final_file=row["final_file"],
|
||||
)
|
||||
|
||||
|
||||
__all__ = ["QueueStore"]
|
||||
@@ -1,64 +0,0 @@
|
||||
"""Singleton manager for all bot-bottle SQLite stores (PRD 0013)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
from .audit_store import AuditStore
|
||||
from .config_store import ConfigStore
|
||||
from .queue_store import QueueStore
|
||||
except ImportError:
|
||||
from audit_store import AuditStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
|
||||
from config_store import ConfigStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
|
||||
from queue_store import QueueStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
|
||||
|
||||
_instance: StoreManager | None = None
|
||||
|
||||
|
||||
class StoreManager:
|
||||
"""Owns db_path and delegates migrate/is_migrated across all stores.
|
||||
|
||||
Use instance() for normal access. Call reset(db_path) in tests to swap
|
||||
the singleton to a temp path, then reset() with no args to restore the
|
||||
default."""
|
||||
|
||||
def __init__(self, db_path: Path | None = None) -> None:
|
||||
if db_path is None:
|
||||
# Lazy import to avoid a circular dependency: supervise imports
|
||||
# StoreManager at module level; StoreManager must not import
|
||||
# supervise at module level in return.
|
||||
try:
|
||||
from .supervise import host_db_path
|
||||
except ImportError:
|
||||
from supervise import host_db_path # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
|
||||
db_path = host_db_path()
|
||||
self.db_path = db_path
|
||||
|
||||
@classmethod
|
||||
def instance(cls) -> StoreManager:
|
||||
global _instance
|
||||
if _instance is None:
|
||||
_instance = cls()
|
||||
return _instance
|
||||
|
||||
@classmethod
|
||||
def reset(cls, db_path: Path | None = None) -> None:
|
||||
"""Replace the singleton. Pass db_path for test isolation; omit to restore default."""
|
||||
global _instance
|
||||
_instance = cls(db_path)
|
||||
|
||||
def is_migrated(self) -> bool:
|
||||
return (
|
||||
QueueStore("", self.db_path).is_migrated()
|
||||
and AuditStore(self.db_path).is_migrated()
|
||||
and ConfigStore(self.db_path).is_migrated()
|
||||
)
|
||||
|
||||
def migrate(self) -> None:
|
||||
QueueStore("", self.db_path).migrate()
|
||||
AuditStore(self.db_path).migrate()
|
||||
ConfigStore(self.db_path).migrate()
|
||||
|
||||
|
||||
__all__ = ["StoreManager"]
|
||||
+351
-114
@@ -9,14 +9,15 @@ calls when it needs an operator-reviewed egress change:
|
||||
|
||||
Each tool call: the agent passes the full proposed file plus a
|
||||
justification text. The sidecar validates the proposal syntactically,
|
||||
writes it to the host SQLite queue table, and holds the tool-call
|
||||
writes it to the host's per-bottle queue dir, and holds the tool-call
|
||||
connection open. The operator's supervise TUI
|
||||
(bot_bottle.cli.supervise) sees the proposal, accepts
|
||||
approve / modify / reject, and writes a response row. The sidecar sees
|
||||
the response and returns `{status, notes}` to the agent.
|
||||
approve / modify / reject, and writes a response file alongside the
|
||||
proposal. The sidecar sees the response and returns `{status, notes}`
|
||||
to the agent.
|
||||
|
||||
This module defines the host-side library: dataclasses for the queue
|
||||
record shapes, queue read/write helpers, the audit log writer, and the
|
||||
file shapes, queue read/write helpers, the audit log writer, and the
|
||||
diff renderer. The in-container sidecar lives in
|
||||
bot_bottle/supervise_server.py; the supervise daemon's container
|
||||
lifecycle is owned by the sidecar bundle (PRD 0024).
|
||||
@@ -30,56 +31,37 @@ remediation engines that wire real config changes land in PRDs 0014,
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import dataclasses
|
||||
import difflib
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import time
|
||||
import uuid
|
||||
from abc import ABC
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
from .supervise_types import (
|
||||
ACTION_OPERATOR_EDIT,
|
||||
AuditEntry,
|
||||
HOST_DB_FILENAME,
|
||||
Proposal,
|
||||
Response,
|
||||
STATUSES,
|
||||
STATUS_APPROVED,
|
||||
STATUS_MODIFIED,
|
||||
STATUS_REJECTED,
|
||||
TOOLS,
|
||||
TOOL_EGRESS_ALLOW,
|
||||
TOOL_EGRESS_BLOCK,
|
||||
TOOL_EGRESS_TOKEN_ALLOW,
|
||||
TOOL_GITLEAKS_ALLOW,
|
||||
TOOL_LIST_EGRESS_ROUTES,
|
||||
bot_bottle_root,
|
||||
)
|
||||
except ImportError:
|
||||
from supervise_types import ( # type: ignore[import-not-found,no-redef] # pylint: disable=import-error,no-name-in-module
|
||||
ACTION_OPERATOR_EDIT,
|
||||
AuditEntry,
|
||||
HOST_DB_FILENAME,
|
||||
Proposal,
|
||||
Response,
|
||||
STATUSES,
|
||||
STATUS_APPROVED,
|
||||
STATUS_MODIFIED,
|
||||
STATUS_REJECTED,
|
||||
TOOLS,
|
||||
TOOL_EGRESS_ALLOW,
|
||||
TOOL_EGRESS_BLOCK,
|
||||
TOOL_EGRESS_TOKEN_ALLOW,
|
||||
TOOL_GITLEAKS_ALLOW,
|
||||
TOOL_LIST_EGRESS_ROUTES,
|
||||
bot_bottle_root,
|
||||
)
|
||||
|
||||
|
||||
SUPERVISE_HOSTNAME = "supervise"
|
||||
SUPERVISE_PORT = 9100
|
||||
|
||||
TOOL_EGRESS_BLOCK = "egress-block"
|
||||
TOOL_EGRESS_ALLOW = "egress-allow"
|
||||
TOOL_GITLEAKS_ALLOW = "gitleaks-allow"
|
||||
# Written directly by the egress addon (not an agent-facing MCP tool) when an
|
||||
# outbound DLP token block is routed to the operator for override (PRD 0062).
|
||||
TOOL_EGRESS_TOKEN_ALLOW = "egress-token-allow"
|
||||
TOOL_LIST_EGRESS_ROUTES = "list-egress-routes"
|
||||
TOOLS: tuple[str, ...] = (
|
||||
TOOL_EGRESS_ALLOW,
|
||||
TOOL_EGRESS_BLOCK,
|
||||
TOOL_GITLEAKS_ALLOW,
|
||||
TOOL_EGRESS_TOKEN_ALLOW,
|
||||
TOOL_LIST_EGRESS_ROUTES,
|
||||
)
|
||||
|
||||
# The supervise sidecar uses these to query egress's
|
||||
# introspection endpoint for the `list-egress-routes` MCP
|
||||
# tool. The hostname + port match egress's docker network
|
||||
@@ -95,77 +77,236 @@ COMPONENT_FOR_TOOL: dict[str, str] = {
|
||||
TOOL_EGRESS_BLOCK: "egress",
|
||||
}
|
||||
|
||||
DB_PATH_IN_CONTAINER = "/run/supervise/bot-bottle.db"
|
||||
STATUS_APPROVED = "approved"
|
||||
STATUS_MODIFIED = "modified"
|
||||
STATUS_REJECTED = "rejected"
|
||||
STATUSES: tuple[str, ...] = (STATUS_APPROVED, STATUS_MODIFIED, STATUS_REJECTED)
|
||||
|
||||
# Operator-initiated audit entries (no tool call). PRD 0014's
|
||||
# `routes edit <bottle>` verb writes entries with this action.
|
||||
ACTION_OPERATOR_EDIT = "operator-edit"
|
||||
|
||||
QUEUE_DIR_IN_CONTAINER = "/run/supervise/queue"
|
||||
DEFAULT_POLL_INTERVAL_SEC = 0.5
|
||||
|
||||
|
||||
# --- Paths -----------------------------------------------------------------
|
||||
|
||||
|
||||
def bot_bottle_root() -> Path:
|
||||
return Path.home() / ".bot-bottle"
|
||||
|
||||
|
||||
def queue_dir_for_slug(slug: str) -> Path:
|
||||
return bot_bottle_root() / "queue" / slug
|
||||
|
||||
|
||||
def audit_dir() -> Path:
|
||||
return bot_bottle_root() / "audit"
|
||||
|
||||
|
||||
def host_db_path() -> Path:
|
||||
# Calls bot_bottle_root() through this module's globals so that patches
|
||||
# on supervise.bot_bottle_root propagate to callers going through
|
||||
# supervise.host_db_path().
|
||||
#
|
||||
# Kept in its own "db" subdirectory (see supervise_types.host_db_path
|
||||
# for why) — must stay in sync with that copy.
|
||||
return bot_bottle_root() / "db" / HOST_DB_FILENAME
|
||||
|
||||
|
||||
def audit_log_path(component: str, slug: str) -> Path:
|
||||
return audit_dir() / f"{component}-{slug}.log"
|
||||
|
||||
|
||||
try:
|
||||
from .queue_store import QueueStore
|
||||
from .audit_store import AuditStore
|
||||
from .store_manager import StoreManager
|
||||
except ImportError:
|
||||
# Sidecar bundle: files are flat-copied under /app, not a package.
|
||||
from queue_store import QueueStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
|
||||
from audit_store import AuditStore # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
|
||||
from store_manager import StoreManager # type: ignore[import-not-found] # pylint: disable=import-error,no-name-in-module
|
||||
# --- Dataclasses -----------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Proposal:
|
||||
"""One pending tool-call from the agent. The sidecar writes one
|
||||
of these to the queue dir on a tool call; the operator's TUI
|
||||
reads them; the sidecar polls for a matching Response."""
|
||||
|
||||
id: str
|
||||
bottle_slug: str
|
||||
tool: str
|
||||
proposed_file: str
|
||||
justification: str
|
||||
arrival_timestamp: str
|
||||
current_file_hash: str
|
||||
|
||||
@classmethod
|
||||
def new(
|
||||
cls,
|
||||
*,
|
||||
bottle_slug: str,
|
||||
tool: str,
|
||||
proposed_file: str,
|
||||
justification: str,
|
||||
current_file_hash: str,
|
||||
now: datetime | None = None,
|
||||
) -> "Proposal":
|
||||
ts = (now or datetime.now(timezone.utc)).isoformat()
|
||||
return cls(
|
||||
id=str(uuid.uuid4()),
|
||||
bottle_slug=bottle_slug,
|
||||
tool=tool,
|
||||
proposed_file=proposed_file,
|
||||
justification=justification,
|
||||
arrival_timestamp=ts,
|
||||
current_file_hash=current_file_hash,
|
||||
)
|
||||
|
||||
def to_dict(self) -> dict[str, object]:
|
||||
return dataclasses.asdict(self)
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, raw: dict[str, object]) -> "Proposal":
|
||||
tool = _require_str(raw, "tool")
|
||||
if tool not in TOOLS:
|
||||
raise ValueError(f"tool must be one of {TOOLS}; got {tool!r}")
|
||||
return cls(
|
||||
id=_require_str(raw, "id"),
|
||||
bottle_slug=_require_str(raw, "bottle_slug"),
|
||||
tool=tool,
|
||||
proposed_file=_require_str(raw, "proposed_file"),
|
||||
justification=_require_str(raw, "justification"),
|
||||
arrival_timestamp=_require_str(raw, "arrival_timestamp"),
|
||||
current_file_hash=_require_str(raw, "current_file_hash"),
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Response:
|
||||
"""The operator's decision on a proposal. The TUI writes one of
|
||||
these to the queue dir; the sidecar reads it and returns the
|
||||
`{status, notes}` pair to the agent's tool call.
|
||||
|
||||
`final_file` carries the file content the supervisor will
|
||||
actually apply: for `approved`, equal to the proposal's
|
||||
`proposed_file`; for `modified`, the operator's edited version
|
||||
(the audit diff is current → final_file, not current →
|
||||
proposed_file); for `rejected`, None."""
|
||||
|
||||
proposal_id: str
|
||||
status: str
|
||||
notes: str
|
||||
final_file: str | None = None
|
||||
|
||||
def to_dict(self) -> dict[str, object]:
|
||||
return dataclasses.asdict(self)
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, raw: dict[str, object]) -> "Response":
|
||||
status = _require_str(raw, "status")
|
||||
if status not in STATUSES:
|
||||
raise ValueError(
|
||||
f"response status must be one of {STATUSES}; got {status!r}"
|
||||
)
|
||||
final = raw.get("final_file")
|
||||
if final is not None and not isinstance(final, str):
|
||||
raise ValueError(
|
||||
f"final_file must be a string or null; got {type(final).__name__}"
|
||||
)
|
||||
return cls(
|
||||
proposal_id=_require_str(raw, "proposal_id"),
|
||||
status=status,
|
||||
notes=_require_str(raw, "notes"),
|
||||
final_file=final,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AuditEntry:
|
||||
"""One row of the per-bottle audit log. JSON-Lines, append-only."""
|
||||
|
||||
timestamp: str
|
||||
bottle_slug: str
|
||||
component: str
|
||||
operator_action: str
|
||||
operator_notes: str
|
||||
justification: str
|
||||
diff: str
|
||||
|
||||
def to_dict(self) -> dict[str, object]:
|
||||
return dataclasses.asdict(self)
|
||||
|
||||
|
||||
# --- Queue I/O -------------------------------------------------------------
|
||||
|
||||
|
||||
def write_proposal(proposal: Proposal) -> Path:
|
||||
"""Persist `proposal` in the queue database, mode 0o600.
|
||||
def _proposal_filename(proposal_id: str) -> str:
|
||||
return f"{proposal_id}.proposal.json"
|
||||
|
||||
|
||||
def _response_filename(proposal_id: str) -> str:
|
||||
return f"{proposal_id}.response.json"
|
||||
|
||||
|
||||
def _id_from_proposal_filename(path: Path) -> str | None:
|
||||
name = path.name
|
||||
if not name.endswith(".proposal.json"):
|
||||
return None
|
||||
return name[: -len(".proposal.json")]
|
||||
|
||||
|
||||
def write_proposal(queue_dir: Path, proposal: Proposal) -> Path:
|
||||
"""Persist `proposal` as JSON in the queue dir, mode 0o600.
|
||||
Directory is created if missing."""
|
||||
return QueueStore(proposal.bottle_slug).write_proposal(proposal)
|
||||
queue_dir.mkdir(parents=True, exist_ok=True)
|
||||
path = queue_dir / _proposal_filename(proposal.id)
|
||||
payload = json.dumps(proposal.to_dict(), indent=2) + "\n"
|
||||
_atomic_write(path, payload, mode=0o600)
|
||||
return path
|
||||
|
||||
|
||||
def read_proposal(bottle_slug: str, proposal_id: str) -> Proposal:
|
||||
return QueueStore(bottle_slug).read_proposal(proposal_id)
|
||||
def read_proposal(queue_dir: Path, proposal_id: str) -> Proposal:
|
||||
path = queue_dir / _proposal_filename(proposal_id)
|
||||
with path.open() as f:
|
||||
raw = json.load(f)
|
||||
if not isinstance(raw, dict):
|
||||
raise ValueError(f"{path}: top-level must be an object")
|
||||
return Proposal.from_dict(raw)
|
||||
|
||||
|
||||
def list_pending_proposals(bottle_slug: str) -> list[Proposal]:
|
||||
"""All proposals for `bottle_slug` that do not yet have a matching
|
||||
response. Sorted by `arrival_timestamp` so the operator
|
||||
def list_pending_proposals(queue_dir: Path) -> list[Proposal]:
|
||||
"""All proposals in `queue_dir` that do not yet have a matching
|
||||
response file. Sorted by `arrival_timestamp` so the operator
|
||||
sees the queue FIFO."""
|
||||
return QueueStore(bottle_slug).list_pending_proposals()
|
||||
if not queue_dir.is_dir():
|
||||
return []
|
||||
out: list[Proposal] = []
|
||||
for path in sorted(queue_dir.glob("*.proposal.json")):
|
||||
proposal_id = _id_from_proposal_filename(path)
|
||||
if proposal_id is None:
|
||||
continue
|
||||
if (queue_dir / _response_filename(proposal_id)).exists():
|
||||
continue
|
||||
try:
|
||||
with path.open() as f:
|
||||
raw = json.load(f)
|
||||
except (OSError, json.JSONDecodeError):
|
||||
continue
|
||||
if not isinstance(raw, dict):
|
||||
continue
|
||||
try:
|
||||
out.append(Proposal.from_dict(raw))
|
||||
except (KeyError, ValueError):
|
||||
continue
|
||||
out.sort(key=lambda p: p.arrival_timestamp)
|
||||
return out
|
||||
|
||||
|
||||
def list_all_pending_proposals() -> list[Proposal]:
|
||||
"""All pending proposals across bottles, sorted FIFO."""
|
||||
return QueueStore("").list_all_pending_proposals()
|
||||
def write_response(queue_dir: Path, response: Response) -> Path:
|
||||
queue_dir.mkdir(parents=True, exist_ok=True)
|
||||
path = queue_dir / _response_filename(response.proposal_id)
|
||||
payload = json.dumps(response.to_dict(), indent=2) + "\n"
|
||||
_atomic_write(path, payload, mode=0o600)
|
||||
return path
|
||||
|
||||
|
||||
def write_response(bottle_slug: str, response: Response) -> Path:
|
||||
return QueueStore(bottle_slug).write_response(response)
|
||||
|
||||
|
||||
def read_response(bottle_slug: str, proposal_id: str) -> Response:
|
||||
return QueueStore(bottle_slug).read_response(proposal_id)
|
||||
def read_response(queue_dir: Path, proposal_id: str) -> Response:
|
||||
path = queue_dir / _response_filename(proposal_id)
|
||||
with path.open() as f:
|
||||
raw = json.load(f)
|
||||
if not isinstance(raw, dict):
|
||||
raise ValueError(f"{path}: top-level must be an object")
|
||||
return Response.from_dict(raw)
|
||||
|
||||
|
||||
def wait_for_response(
|
||||
bottle_slug: str,
|
||||
queue_dir: Path,
|
||||
proposal_id: str,
|
||||
*,
|
||||
poll_interval: float = DEFAULT_POLL_INTERVAL_SEC,
|
||||
@@ -176,35 +317,90 @@ def wait_for_response(
|
||||
which the wait raises TimeoutError. None waits forever — the
|
||||
natural shape, since the operator's response time is unbounded.
|
||||
|
||||
Polls SQLite so the implementation stays portable and stdlib-only."""
|
||||
store = QueueStore(bottle_slug)
|
||||
Polls the filesystem so the implementation stays portable and
|
||||
stdlib-only."""
|
||||
path = queue_dir / _response_filename(proposal_id)
|
||||
while True:
|
||||
try:
|
||||
return store.read_response(proposal_id)
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
if path.exists():
|
||||
try:
|
||||
with path.open() as f:
|
||||
raw = json.load(f)
|
||||
except (OSError, json.JSONDecodeError):
|
||||
raw = None
|
||||
if isinstance(raw, dict):
|
||||
try:
|
||||
return Response.from_dict(raw)
|
||||
except (KeyError, ValueError):
|
||||
pass
|
||||
if deadline is not None and time.monotonic() >= deadline:
|
||||
raise TimeoutError(f"no response for proposal {proposal_id!r}")
|
||||
time.sleep(poll_interval)
|
||||
|
||||
|
||||
def archive_proposal(bottle_slug: str, proposal_id: str) -> None:
|
||||
"""Mark both proposal and response rows processed.
|
||||
Idempotent — missing rows are silently skipped."""
|
||||
QueueStore(bottle_slug).archive_proposal(proposal_id)
|
||||
def archive_proposal(queue_dir: Path, proposal_id: str) -> None:
|
||||
"""Move both proposal and response files to `<queue_dir>/processed/`.
|
||||
Idempotent — missing files are silently skipped."""
|
||||
processed = queue_dir / "processed"
|
||||
processed.mkdir(parents=True, exist_ok=True)
|
||||
for name in (_proposal_filename(proposal_id), _response_filename(proposal_id)):
|
||||
src = queue_dir / name
|
||||
if src.exists():
|
||||
src.rename(processed / name)
|
||||
|
||||
|
||||
# --- Audit log -------------------------------------------------------------
|
||||
|
||||
|
||||
def write_audit_entry(entry: AuditEntry) -> Path:
|
||||
"""Append `entry` to the host supervise audit table."""
|
||||
return AuditStore().write_audit_entry(entry)
|
||||
"""Append `entry` as one JSON-Lines record to the per-bottle
|
||||
audit log. Acquires an advisory exclusive lock so concurrent
|
||||
writers don't interleave bytes."""
|
||||
path = audit_log_path(entry.component, entry.bottle_slug)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
line = json.dumps(entry.to_dict(), sort_keys=False) + "\n"
|
||||
fd = os.open(path, os.O_WRONLY | os.O_APPEND | os.O_CREAT, 0o600)
|
||||
try:
|
||||
_try_flock(fd)
|
||||
try:
|
||||
os.write(fd, line.encode("utf-8"))
|
||||
finally:
|
||||
_try_funlock(fd)
|
||||
finally:
|
||||
os.close(fd)
|
||||
return path
|
||||
|
||||
|
||||
def read_audit_entries(component: str, slug: str) -> list[AuditEntry]:
|
||||
"""Load all audit entries for the given component+slug."""
|
||||
return AuditStore().read_audit_entries(component, slug)
|
||||
"""Load all audit entries for the given component+slug. Empty
|
||||
list if the log doesn't exist."""
|
||||
path = audit_log_path(component, slug)
|
||||
if not path.is_file():
|
||||
return []
|
||||
out: list[AuditEntry] = []
|
||||
with path.open() as f:
|
||||
for raw_line in f:
|
||||
raw_line = raw_line.strip()
|
||||
if not raw_line:
|
||||
continue
|
||||
try:
|
||||
raw = json.loads(raw_line)
|
||||
except json.JSONDecodeError:
|
||||
continue
|
||||
if not isinstance(raw, dict):
|
||||
continue
|
||||
try:
|
||||
out.append(AuditEntry(
|
||||
timestamp=_require_str(raw, "timestamp"),
|
||||
bottle_slug=_require_str(raw, "bottle_slug"),
|
||||
component=_require_str(raw, "component"),
|
||||
operator_action=_require_str(raw, "operator_action"),
|
||||
operator_notes=_require_str(raw, "operator_notes"),
|
||||
justification=_require_str(raw, "justification"),
|
||||
diff=_require_str(raw, "diff"),
|
||||
))
|
||||
except ValueError:
|
||||
continue
|
||||
return out
|
||||
|
||||
|
||||
# --- Diff rendering --------------------------------------------------------
|
||||
@@ -237,47 +433,89 @@ def sha256_hex(content: str) -> str:
|
||||
class SupervisePlan:
|
||||
"""Output of Supervise.prepare; consumed by .start.
|
||||
|
||||
`db_path` is the host database bind-mounted into the sidecar at
|
||||
/run/supervise/bot-bottle.db. `internal_network` is empty at
|
||||
prepare time; the backend's launch step fills it via
|
||||
dataclasses.replace before calling .start."""
|
||||
`queue_dir` is the host directory bind-mounted into the sidecar
|
||||
at /run/supervise/queue. `internal_network` is empty at prepare
|
||||
time; the backend's launch step fills it via dataclasses.replace
|
||||
before calling .start."""
|
||||
|
||||
slug: str
|
||||
db_path: Path
|
||||
queue_dir: Path
|
||||
internal_network: str = ""
|
||||
|
||||
|
||||
class Supervise(ABC):
|
||||
"""Per-bottle supervise sidecar. Encapsulates host-side database
|
||||
staging; the sidecar's start/stop lifecycle is backend-specific."""
|
||||
"""Per-bottle supervise sidecar. Encapsulates the host-side
|
||||
prepare (queue dir staging); the sidecar's start/stop lifecycle
|
||||
is backend-specific."""
|
||||
|
||||
def prepare(
|
||||
self,
|
||||
slug: str,
|
||||
stage_dir: Path,
|
||||
) -> SupervisePlan:
|
||||
"""Stage the host database. Returns the plan; `internal_network`
|
||||
must be set by the launch step before .start runs."""
|
||||
"""Stage the per-bottle queue dir on the host. Returns the
|
||||
plan; `internal_network` must be set by the launch step before
|
||||
.start runs."""
|
||||
del stage_dir
|
||||
mgr = StoreManager.instance()
|
||||
mgr.migrate()
|
||||
queue_dir = queue_dir_for_slug(slug)
|
||||
queue_dir.mkdir(parents=True, exist_ok=True)
|
||||
return SupervisePlan(
|
||||
slug=slug,
|
||||
db_path=mgr.db_path,
|
||||
queue_dir=queue_dir,
|
||||
)
|
||||
|
||||
# --- Helpers ---------------------------------------------------------------
|
||||
|
||||
|
||||
def _require_str(raw: dict[str, object], key: str) -> str:
|
||||
value = raw.get(key)
|
||||
if not isinstance(value, str):
|
||||
raise ValueError(f"missing or non-string field {key!r}")
|
||||
return value
|
||||
|
||||
|
||||
def _atomic_write(path: Path, content: str, *, mode: int) -> None:
|
||||
"""Atomic: write to a sibling tmp file, fsync, rename."""
|
||||
tmp = path.with_suffix(path.suffix + ".tmp")
|
||||
fd = os.open(tmp, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, mode)
|
||||
try:
|
||||
os.write(fd, content.encode("utf-8"))
|
||||
os.fsync(fd)
|
||||
finally:
|
||||
os.close(fd)
|
||||
os.replace(tmp, path)
|
||||
|
||||
|
||||
try:
|
||||
import fcntl as _fcntl
|
||||
|
||||
def _try_flock(fd: int) -> None: # type: ignore[reportRedeclaration]
|
||||
try:
|
||||
_fcntl.flock(fd, _fcntl.LOCK_EX)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
def _try_funlock(fd: int) -> None: # type: ignore[reportRedeclaration]
|
||||
try:
|
||||
_fcntl.flock(fd, _fcntl.LOCK_UN)
|
||||
except OSError:
|
||||
pass
|
||||
except ImportError: # pragma: no cover — Windows path
|
||||
def _try_flock(fd: int) -> None: # noqa: F841 — Windows fallback
|
||||
return None
|
||||
|
||||
def _try_funlock(fd: int) -> None: # noqa: F841 — Windows fallback
|
||||
return None
|
||||
|
||||
|
||||
__all__ = [
|
||||
"ACTION_OPERATOR_EDIT",
|
||||
"AuditEntry",
|
||||
"AuditStore",
|
||||
"COMPONENT_FOR_TOOL",
|
||||
"DEFAULT_POLL_INTERVAL_SEC",
|
||||
"DB_PATH_IN_CONTAINER",
|
||||
"Proposal",
|
||||
"QueueStore",
|
||||
"QUEUE_DIR_IN_CONTAINER",
|
||||
"Response",
|
||||
"StoreManager",
|
||||
"STATUSES",
|
||||
"STATUS_APPROVED",
|
||||
"STATUS_MODIFIED",
|
||||
@@ -298,9 +536,8 @@ __all__ = [
|
||||
"audit_dir",
|
||||
"audit_log_path",
|
||||
"bot_bottle_root",
|
||||
"host_db_path",
|
||||
"list_pending_proposals",
|
||||
"list_all_pending_proposals",
|
||||
"queue_dir_for_slug",
|
||||
"read_audit_entries",
|
||||
"read_proposal",
|
||||
"read_response",
|
||||
|
||||
@@ -7,13 +7,14 @@ config changes when stuck. The tools are `egress-allow`,
|
||||
Each queued tool call:
|
||||
|
||||
1. Validates the proposed file syntactically.
|
||||
2. Writes a Proposal to the host SQLite database.
|
||||
3. Blocks polling for a matching Response row.
|
||||
2. Writes a Proposal to /run/supervise/queue/ (bind-mounted from
|
||||
the host's ~/.bot-bottle/queue/<slug>/).
|
||||
3. Blocks polling for a matching Response file.
|
||||
4. Returns the operator's `{status, notes}` to the agent.
|
||||
|
||||
The bottle slug arrives via SUPERVISE_BOTTLE_SLUG env (stamped at
|
||||
container creation by the backend's start step). SUPERVISE_DB_PATH
|
||||
points at the bind-mounted host database.
|
||||
container creation by the backend's start step). The queue dir comes
|
||||
from SUPERVISE_QUEUE_DIR (default `/run/supervise/queue`).
|
||||
|
||||
Speaks MCP over HTTP+JSON-RPC. Methods handled:
|
||||
|
||||
@@ -41,6 +42,7 @@ import typing
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
# Same-directory imports inside the bundle container; these files are
|
||||
@@ -275,6 +277,7 @@ def validate_proposed_file(tool: str, content: str) -> None:
|
||||
@dataclass(frozen=True)
|
||||
class ServerConfig:
|
||||
bottle_slug: str
|
||||
queue_dir: Path
|
||||
response_timeout_seconds: float = DEFAULT_RESPONSE_TIMEOUT_SECONDS
|
||||
|
||||
|
||||
@@ -373,7 +376,7 @@ def handle_tools_call(
|
||||
current_file_hash=_sv.sha256_hex(proposed_file),
|
||||
)
|
||||
try:
|
||||
_sv.write_proposal(proposal)
|
||||
_sv.write_proposal(config.queue_dir, proposal)
|
||||
except OSError as e:
|
||||
raise _RpcInternalError(f"failed to write proposal to queue: {e}") from e
|
||||
sys.stderr.write(
|
||||
@@ -384,7 +387,7 @@ def handle_tools_call(
|
||||
deadline = time.monotonic() + config.response_timeout_seconds
|
||||
try:
|
||||
response = _sv.wait_for_response(
|
||||
config.bottle_slug,
|
||||
config.queue_dir,
|
||||
proposal.id,
|
||||
poll_interval=MIN_RESPONSE_POLL_INTERVAL_SECONDS,
|
||||
deadline=deadline,
|
||||
@@ -396,7 +399,7 @@ def handle_tools_call(
|
||||
"isError": False,
|
||||
}
|
||||
try:
|
||||
_sv.archive_proposal(config.bottle_slug, proposal.id)
|
||||
_sv.archive_proposal(config.queue_dir, proposal.id)
|
||||
except OSError as e:
|
||||
raise _RpcInternalError(f"failed to archive proposal: {e}") from e
|
||||
|
||||
@@ -536,7 +539,7 @@ class MCPHandler(http.server.BaseHTTPRequestHandler):
|
||||
class MCPServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
|
||||
allow_reuse_address = True
|
||||
daemon_threads = True
|
||||
config: ServerConfig = ServerConfig(bottle_slug="")
|
||||
config: ServerConfig = ServerConfig(bottle_slug="", queue_dir=Path())
|
||||
|
||||
|
||||
# --- Entry point -----------------------------------------------------------
|
||||
@@ -545,18 +548,21 @@ class MCPServer(socketserver.ThreadingMixIn, http.server.HTTPServer):
|
||||
def serve(
|
||||
*,
|
||||
bottle_slug: str,
|
||||
queue_dir: Path,
|
||||
port: int = _sv.SUPERVISE_PORT,
|
||||
bind: str = "0.0.0.0",
|
||||
response_timeout_seconds: float = DEFAULT_RESPONSE_TIMEOUT_SECONDS,
|
||||
) -> typing.NoReturn:
|
||||
queue_dir.mkdir(parents=True, exist_ok=True)
|
||||
server = MCPServer((bind, port), MCPHandler)
|
||||
server.config = ServerConfig(
|
||||
bottle_slug=bottle_slug,
|
||||
queue_dir=queue_dir,
|
||||
response_timeout_seconds=response_timeout_seconds,
|
||||
)
|
||||
sys.stderr.write(
|
||||
f"supervise listening on {bind}:{port}; "
|
||||
f"slug={bottle_slug!r}; "
|
||||
f"slug={bottle_slug!r}; queue={queue_dir}; "
|
||||
f"tools: {', '.join(t['name'] for t in TOOL_DEFINITIONS)}\n" # type: ignore[arg-type]
|
||||
)
|
||||
sys.stderr.flush()
|
||||
@@ -575,6 +581,7 @@ def main(argv: list[str]) -> int:
|
||||
if not bottle_slug:
|
||||
sys.stderr.write("supervise: SUPERVISE_BOTTLE_SLUG env is unset\n")
|
||||
return 2
|
||||
queue_dir = Path(os.environ.get("SUPERVISE_QUEUE_DIR", _sv.QUEUE_DIR_IN_CONTAINER))
|
||||
port = int(os.environ.get("SUPERVISE_PORT", str(_sv.SUPERVISE_PORT)))
|
||||
bind = os.environ.get("SUPERVISE_BIND", "0.0.0.0")
|
||||
try:
|
||||
@@ -584,6 +591,7 @@ def main(argv: list[str]) -> int:
|
||||
return 2
|
||||
serve(
|
||||
bottle_slug=bottle_slug,
|
||||
queue_dir=queue_dir,
|
||||
port=port,
|
||||
bind=bind,
|
||||
response_timeout_seconds=response_timeout_seconds,
|
||||
|
||||
@@ -1,189 +0,0 @@
|
||||
"""Shared types and path helpers for supervise stores (PRD 0013).
|
||||
|
||||
Extracted from supervise.py so queue_store and audit_store can import
|
||||
Proposal, Response, AuditEntry, and host_db_path without creating a
|
||||
circular import (supervise imports from queue_store/audit_store and
|
||||
vice-versa).
|
||||
|
||||
Patching bot_bottle_root on this module propagates through host_db_path
|
||||
because host_db_path looks up bot_bottle_root via sys.modules[__name__]
|
||||
at call time rather than capturing it at import time.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import dataclasses
|
||||
import sys
|
||||
import uuid
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
HOST_DB_FILENAME = "bot-bottle.db"
|
||||
|
||||
TOOL_EGRESS_BLOCK = "egress-block"
|
||||
TOOL_EGRESS_ALLOW = "egress-allow"
|
||||
TOOL_GITLEAKS_ALLOW = "gitleaks-allow"
|
||||
TOOL_EGRESS_TOKEN_ALLOW = "egress-token-allow"
|
||||
TOOL_LIST_EGRESS_ROUTES = "list-egress-routes"
|
||||
TOOLS: tuple[str, ...] = (
|
||||
TOOL_EGRESS_ALLOW,
|
||||
TOOL_EGRESS_BLOCK,
|
||||
TOOL_GITLEAKS_ALLOW,
|
||||
TOOL_EGRESS_TOKEN_ALLOW,
|
||||
TOOL_LIST_EGRESS_ROUTES,
|
||||
)
|
||||
|
||||
STATUS_APPROVED = "approved"
|
||||
STATUS_MODIFIED = "modified"
|
||||
STATUS_REJECTED = "rejected"
|
||||
STATUSES: tuple[str, ...] = (STATUS_APPROVED, STATUS_MODIFIED, STATUS_REJECTED)
|
||||
|
||||
ACTION_OPERATOR_EDIT = "operator-edit"
|
||||
|
||||
|
||||
def bot_bottle_root() -> Path:
|
||||
return Path.home() / ".bot-bottle"
|
||||
|
||||
|
||||
def host_db_path() -> Path:
|
||||
# Look up bot_bottle_root through this module's live namespace so that
|
||||
# monkey-patches on supervise_types.bot_bottle_root take effect.
|
||||
#
|
||||
# Lives in its own "db" subdirectory, not directly under
|
||||
# bot_bottle_root(), so backends that can only bind-mount
|
||||
# directories (macos_container's `container run --mount`, which
|
||||
# rejects file sources with "is not a directory") can share this
|
||||
# one file with a sidecar without also exposing bot_bottle_root()'s
|
||||
# other contents (git-gate keys, per-bottle state, etc.).
|
||||
return sys.modules[__name__].bot_bottle_root() / "db" / HOST_DB_FILENAME
|
||||
|
||||
|
||||
def _require_str(raw: dict[str, object], key: str) -> str:
|
||||
value = raw.get(key)
|
||||
if not isinstance(value, str):
|
||||
raise ValueError(f"missing or non-string field {key!r}")
|
||||
return value
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Proposal:
|
||||
"""One pending tool-call from the agent."""
|
||||
|
||||
id: str
|
||||
bottle_slug: str
|
||||
tool: str
|
||||
proposed_file: str
|
||||
justification: str
|
||||
arrival_timestamp: str
|
||||
current_file_hash: str
|
||||
|
||||
@classmethod
|
||||
def new(
|
||||
cls,
|
||||
*,
|
||||
bottle_slug: str,
|
||||
tool: str,
|
||||
proposed_file: str,
|
||||
justification: str,
|
||||
current_file_hash: str,
|
||||
now: datetime | None = None,
|
||||
) -> "Proposal":
|
||||
ts = (now or datetime.now(timezone.utc)).isoformat()
|
||||
return cls(
|
||||
id=str(uuid.uuid4()),
|
||||
bottle_slug=bottle_slug,
|
||||
tool=tool,
|
||||
proposed_file=proposed_file,
|
||||
justification=justification,
|
||||
arrival_timestamp=ts,
|
||||
current_file_hash=current_file_hash,
|
||||
)
|
||||
|
||||
def to_dict(self) -> dict[str, object]:
|
||||
return dataclasses.asdict(self)
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, raw: dict[str, object]) -> "Proposal":
|
||||
tool = _require_str(raw, "tool")
|
||||
if tool not in TOOLS:
|
||||
raise ValueError(f"tool must be one of {TOOLS}; got {tool!r}")
|
||||
return cls(
|
||||
id=_require_str(raw, "id"),
|
||||
bottle_slug=_require_str(raw, "bottle_slug"),
|
||||
tool=tool,
|
||||
proposed_file=_require_str(raw, "proposed_file"),
|
||||
justification=_require_str(raw, "justification"),
|
||||
arrival_timestamp=_require_str(raw, "arrival_timestamp"),
|
||||
current_file_hash=_require_str(raw, "current_file_hash"),
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Response:
|
||||
"""The operator's decision on a proposal."""
|
||||
|
||||
proposal_id: str
|
||||
status: str
|
||||
notes: str
|
||||
final_file: str | None = None
|
||||
|
||||
def to_dict(self) -> dict[str, object]:
|
||||
return dataclasses.asdict(self)
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, raw: dict[str, object]) -> "Response":
|
||||
status = _require_str(raw, "status")
|
||||
if status not in STATUSES:
|
||||
raise ValueError(
|
||||
f"response status must be one of {STATUSES}; got {status!r}"
|
||||
)
|
||||
final = raw.get("final_file")
|
||||
if final is not None and not isinstance(final, str):
|
||||
raise ValueError(
|
||||
f"final_file must be a string or null; got {type(final).__name__}"
|
||||
)
|
||||
return cls(
|
||||
proposal_id=_require_str(raw, "proposal_id"),
|
||||
status=status,
|
||||
notes=_require_str(raw, "notes"),
|
||||
final_file=final,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AuditEntry:
|
||||
"""One row of the per-bottle audit log."""
|
||||
|
||||
timestamp: str
|
||||
bottle_slug: str
|
||||
component: str
|
||||
operator_action: str
|
||||
operator_notes: str
|
||||
justification: str
|
||||
diff: str
|
||||
|
||||
def to_dict(self) -> dict[str, object]:
|
||||
return dataclasses.asdict(self)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"ACTION_OPERATOR_EDIT",
|
||||
"AuditEntry",
|
||||
"HOST_DB_FILENAME",
|
||||
"Proposal",
|
||||
"Response",
|
||||
"STATUSES",
|
||||
"STATUS_APPROVED",
|
||||
"STATUS_MODIFIED",
|
||||
"STATUS_REJECTED",
|
||||
"TOOLS",
|
||||
"TOOL_EGRESS_ALLOW",
|
||||
"TOOL_EGRESS_BLOCK",
|
||||
"TOOL_EGRESS_TOKEN_ALLOW",
|
||||
"TOOL_GITLEAKS_ALLOW",
|
||||
"TOOL_LIST_EGRESS_ROUTES",
|
||||
"bot_bottle_root",
|
||||
"host_db_path",
|
||||
]
|
||||
@@ -21,11 +21,6 @@ Public API:
|
||||
For a Markdown file with YAML frontmatter delimited by `---`
|
||||
lines. Returns (frontmatter_dict, body_text).
|
||||
|
||||
serialize_yaml_subset(data) -> str
|
||||
Serialize a dict (as produced by parse_yaml_subset) back to
|
||||
block-style YAML text. The result ends with a newline and
|
||||
can be parsed back by parse_yaml_subset.
|
||||
|
||||
What we accept (block-style):
|
||||
|
||||
key: value # mapping entry, value is inline
|
||||
@@ -581,105 +576,3 @@ def parse_frontmatter(text: str) -> tuple[dict[str, object], str]:
|
||||
fm = parse_yaml_subset(fm_text)
|
||||
body = text[body_start:]
|
||||
return fm, body
|
||||
|
||||
|
||||
# --- Serializer -------------------------------------------------------------
|
||||
|
||||
|
||||
def _needs_quoting(s: str) -> bool:
|
||||
"""True when the string must be single-quoted to survive a round-trip."""
|
||||
if not s:
|
||||
return True
|
||||
if s in ("true", "false", "null", "~") or s in _RESERVED_BOOL_LIKE:
|
||||
return True
|
||||
if (
|
||||
_INT_RX.match(s)
|
||||
or _DATE_RX.match(s)
|
||||
or _OCTAL_RX.match(s)
|
||||
or _HEX_RX.match(s)
|
||||
or _FLOAT_RX.match(s)
|
||||
):
|
||||
return True
|
||||
# Characters that have special meaning at the start of a YAML value
|
||||
if s[0] in ('"', "'", "[", "{", "!", "&", "*", "#", "|", ">", "%", "@", "`"):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _yaml_scalar(v: object) -> str:
|
||||
"""Serialize a scalar Python value to its YAML text form."""
|
||||
if v is None:
|
||||
return "null"
|
||||
if isinstance(v, bool):
|
||||
return "true" if v else "false"
|
||||
if isinstance(v, int):
|
||||
return str(v)
|
||||
s = str(v)
|
||||
if _needs_quoting(s):
|
||||
return "'" + s.replace("'", "''") + "'"
|
||||
return s
|
||||
|
||||
|
||||
def _serialize_node(node: object, indent: int) -> list[str]:
|
||||
"""Return lines (without trailing newlines) for `node` at `indent`.
|
||||
|
||||
Called only for non-empty dicts and lists (the caller guards with
|
||||
`isinstance(val, (dict, list)) and val`), plus scalars at the leaf."""
|
||||
prefix = " " * indent
|
||||
if isinstance(node, dict):
|
||||
out: list[str] = []
|
||||
for key, val in node.items():
|
||||
if isinstance(val, (dict, list)) and val:
|
||||
out.append(f"{prefix}{key}:")
|
||||
out.extend(_serialize_node(val, indent + 2))
|
||||
else:
|
||||
scalar = (
|
||||
"{}" if isinstance(val, dict)
|
||||
else "[]" if isinstance(val, list)
|
||||
else _yaml_scalar(val)
|
||||
)
|
||||
out.append(f"{prefix}{key}: {scalar}")
|
||||
return out
|
||||
if isinstance(node, list):
|
||||
out = []
|
||||
for item in node:
|
||||
if isinstance(item, dict) and item:
|
||||
entries = list(item.items())
|
||||
first_key, first_val = entries[0]
|
||||
if isinstance(first_val, (dict, list)) and first_val:
|
||||
out.append(f"{prefix}- {first_key}:")
|
||||
out.extend(_serialize_node(first_val, indent + 4))
|
||||
else:
|
||||
scalar = (
|
||||
"{}" if isinstance(first_val, dict)
|
||||
else "[]" if isinstance(first_val, list)
|
||||
else _yaml_scalar(first_val)
|
||||
)
|
||||
out.append(f"{prefix}- {first_key}: {scalar}")
|
||||
cont = prefix + " "
|
||||
for key, val in entries[1:]:
|
||||
if isinstance(val, (dict, list)) and val:
|
||||
out.append(f"{cont}{key}:")
|
||||
out.extend(_serialize_node(val, indent + 4))
|
||||
else:
|
||||
scalar = (
|
||||
"{}" if isinstance(val, dict)
|
||||
else "[]" if isinstance(val, list)
|
||||
else _yaml_scalar(val)
|
||||
)
|
||||
out.append(f"{cont}{key}: {scalar}")
|
||||
else:
|
||||
out.append(f"{prefix}- {_yaml_scalar(item)}")
|
||||
return out
|
||||
return [_yaml_scalar(node)] # pragma: no cover
|
||||
|
||||
|
||||
def serialize_yaml_subset(data: dict[str, object]) -> str:
|
||||
"""Serialize `data` (as produced by parse_yaml_subset) to YAML text.
|
||||
|
||||
Produces block-style output with 2-space indentation. The result ends
|
||||
with a newline and can be parsed back by parse_yaml_subset. Keys are
|
||||
emitted in iteration order (insertion order in Python 3.7+)."""
|
||||
if not data:
|
||||
return ""
|
||||
return "\n".join(_serialize_node(data, 0)) + "\n"
|
||||
|
||||
@@ -1,135 +0,0 @@
|
||||
# PRD 0067: SQLite local storage
|
||||
|
||||
- **Status:** Active
|
||||
- **Author:** codex
|
||||
- **Created:** 2026-07-01
|
||||
- **Issue:** #319
|
||||
|
||||
## Summary
|
||||
|
||||
Add a small stdlib SQLite storage layer for bot-bottle host runtime state,
|
||||
starting with the supervise queue and audit log. This replaces scattered JSON
|
||||
queue files and JSONL audit logs with structured tables while preserving the
|
||||
existing public supervise helper functions and sidecar queue mount contract.
|
||||
|
||||
## Problem
|
||||
|
||||
Bot-bottle currently stores supervise proposals and responses as individual JSON
|
||||
files under `~/.bot-bottle/queue/<slug>/`, and audit entries as JSONL files
|
||||
under `~/.bot-bottle/audit/`. That worked for the original interactive TUI, but
|
||||
new forge-native orchestration needs durable, queryable local state for queues,
|
||||
audit trails, watchdogs, and lifecycle records. PR #318 started introducing
|
||||
SQLite-shaped boilerplate for forge state; the storage foundation should live in
|
||||
its own PR so forge work can build on the shared runtime store instead of adding
|
||||
one-off persistence.
|
||||
|
||||
## Goals / Success Criteria
|
||||
|
||||
1. Supervise proposals and responses are persisted through SQLite.
|
||||
2. Audit entries are persisted through SQLite.
|
||||
3. Supervise queue helpers use the bottle slug / queue key instead of a queue
|
||||
directory path.
|
||||
4. The sidecar receives the host database mount across docker, smolmachines,
|
||||
and macOS-container backends.
|
||||
5. The implementation stays stdlib-only.
|
||||
6. Schema migrations use a `PRAGMA user_version` runner — no third-party deps.
|
||||
7. Unit tests cover queue round-trips, pending discovery, response waits,
|
||||
archive semantics, audit round-trips, and path creation.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Migrating old JSON queue files or JSONL audit logs.
|
||||
- Adding forge orchestration state tables.
|
||||
- Adding egress metering or budget tables.
|
||||
- Changing the supervise TUI workflow or remediation behavior.
|
||||
- Introducing a third-party ORM or migration library.
|
||||
|
||||
## Design
|
||||
|
||||
### Database locations
|
||||
|
||||
Queue and audit state use the host-level local database:
|
||||
|
||||
```text
|
||||
~/.bot-bottle/bot-bottle.db
|
||||
```
|
||||
|
||||
The supervise sidecar receives that database as a writable bind mount at
|
||||
`/run/supervise/bot-bottle.db` and gets the path through `SUPERVISE_DB_PATH`.
|
||||
No per-slug queue directory is mounted into the sidecar. This creates the shared
|
||||
host database that later forge/native lifecycle work can extend in separate
|
||||
PRDs.
|
||||
|
||||
### Tables
|
||||
|
||||
`supervise_proposals` lives in the host database:
|
||||
|
||||
```sql
|
||||
CREATE TABLE supervise_proposals (
|
||||
queue_key TEXT NOT NULL,
|
||||
id TEXT NOT NULL,
|
||||
bottle_slug TEXT NOT NULL,
|
||||
tool TEXT NOT NULL,
|
||||
proposed_file TEXT NOT NULL,
|
||||
justification TEXT NOT NULL,
|
||||
arrival_timestamp TEXT NOT NULL,
|
||||
current_file_hash TEXT NOT NULL,
|
||||
archived INTEGER NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (queue_key, id)
|
||||
);
|
||||
```
|
||||
|
||||
`supervise_responses` lives in the host database:
|
||||
|
||||
```sql
|
||||
CREATE TABLE supervise_responses (
|
||||
queue_key TEXT NOT NULL,
|
||||
proposal_id TEXT NOT NULL,
|
||||
status TEXT NOT NULL,
|
||||
notes TEXT NOT NULL,
|
||||
final_file TEXT,
|
||||
archived INTEGER NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (queue_key, proposal_id)
|
||||
);
|
||||
```
|
||||
|
||||
`supervise_audit_entries` lives in the host database:
|
||||
|
||||
```sql
|
||||
CREATE TABLE supervise_audit_entries (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
timestamp TEXT NOT NULL,
|
||||
bottle_slug TEXT NOT NULL,
|
||||
component TEXT NOT NULL,
|
||||
operator_action TEXT NOT NULL,
|
||||
operator_notes TEXT NOT NULL,
|
||||
justification TEXT NOT NULL,
|
||||
diff TEXT NOT NULL
|
||||
);
|
||||
```
|
||||
|
||||
### Compatibility
|
||||
|
||||
The queue helpers take a bottle slug / queue key and perform equivalent
|
||||
operations against `~/.bot-bottle/bot-bottle.db`:
|
||||
|
||||
- `list_pending_proposals` returns non-archived proposals without a non-archived
|
||||
response, sorted by arrival time.
|
||||
- `archive_proposal` marks matching proposal/response rows archived instead of
|
||||
moving files into `processed/`.
|
||||
- `wait_for_response` keeps the current polling behavior but polls SQLite.
|
||||
|
||||
The old audit path helpers (`audit_dir`, `audit_log_path`) stay available for
|
||||
compatibility. `audit_log_path` no longer describes the active storage location;
|
||||
callers should use `read_audit_entries`.
|
||||
|
||||
## Implementation chunks
|
||||
|
||||
1. Add SQLite store helpers for supervise queue and audit state.
|
||||
2. Rewire `bot_bottle.supervise` queue/audit functions to the store.
|
||||
3. Update supervise CLI discovery tests and queue/audit unit tests.
|
||||
4. Run unit tests, pyright, and pylint for touched modules.
|
||||
|
||||
## Open questions
|
||||
|
||||
None.
|
||||
@@ -1,227 +0,0 @@
|
||||
# PRD 0068: smolmachines backend on Linux
|
||||
|
||||
- **Status:** Active
|
||||
- **Author:** Claude
|
||||
- **Created:** 2026-06-25
|
||||
- **Issue:** #283
|
||||
|
||||
## Summary
|
||||
|
||||
Make the `smolmachines` backend (PRD 0023) runnable on Linux, not
|
||||
just macOS. `smolvm` already supports Linux via KVM (`/dev/kvm`);
|
||||
the gap is entirely in bot-bottle's host-side glue, which hard-codes
|
||||
macOS assumptions in three places:
|
||||
|
||||
1. **Preflight** only checks that `smolvm` is on `PATH` — it never
|
||||
checks the Linux KVM prerequisite, so a misconfigured host fails
|
||||
deep in the launch flow with an opaque `smolvm` error.
|
||||
2. **The TSI allowlist enforcement** (`force_allowlist`) — the
|
||||
security property that confines the agent VM to its sidecar
|
||||
bundle's `/32` — **no-ops on Linux today, failing _open_**. The
|
||||
smolvm state-DB path it patches is hard-coded to macOS's
|
||||
`~/Library/Application Support/...`.
|
||||
3. **Per-bottle loopback scoping** (`allocate`) returns the shared
|
||||
`127.0.0.1` on Linux, which would let the agent VM reach every
|
||||
service on host loopback — a downgrade from the per-bottle alias
|
||||
isolation macOS gets.
|
||||
|
||||
This PRD closes all three so a bottle launched with
|
||||
`BOT_BOTTLE_BACKEND=smolmachines` on Linux gets the same isolation
|
||||
guarantee it gets on macOS, and documents the Linux/NixOS host
|
||||
setup. The primary validation target is NixOS, but the changes are
|
||||
distro-agnostic.
|
||||
|
||||
## Problem
|
||||
|
||||
The smolmachines backend runs each bottle's agent inside a libkrun
|
||||
microVM via `smolvm`, with egress confined by TSI's `--allow-cidr`
|
||||
allowlist set to a single `/32` — the sidecar bundle's loopback
|
||||
address. Everything else (host loopback, LAN, internet) is denied at
|
||||
the VMM layer. That security property is the entire reason the
|
||||
backend exists.
|
||||
|
||||
libkrun runs on Hypervisor.framework (macOS) **and** KVM (Linux), and
|
||||
`smolvm` ships Linux x86_64 / aarch64 builds that require `/dev/kvm`.
|
||||
So the microVM layer already works on Linux. What does not work is
|
||||
bot-bottle's host integration, which PRD 0023 explicitly scoped to
|
||||
macOS-only for v1. Three concrete blockers:
|
||||
|
||||
- **No KVM preflight.** On a Linux host without `/dev/kvm` (kernel
|
||||
module not loaded) or without access to it (user not in the `kvm`
|
||||
group), the failure surfaces as a cryptic `smolvm` non-zero exit
|
||||
mid-launch instead of an actionable message.
|
||||
|
||||
- **TSI enforcement fails open on Linux.** `force_allowlist`
|
||||
early-returns on non-macOS. It exists because `smolvm` 0.8.0
|
||||
silently drops `--allow-cidr` when combined with `--from`, so the
|
||||
allowlist has to be patched into smolvm's persisted state DB before
|
||||
`machine start`. On Linux that patch never runs **and** the DB path
|
||||
is the macOS path, so the booted VM's TSI allowlist is whatever
|
||||
smolvm defaulted to — potentially all of `127.0.0.0/8`. That is the
|
||||
exact sandbox-escape the backend is supposed to prevent.
|
||||
|
||||
- **No per-bottle loopback isolation on Linux.** `allocate` returns
|
||||
`127.0.0.1` on Linux. Even with a correct allowlist, `127.0.0.1/32`
|
||||
is shared by every service on host loopback, so the agent could
|
||||
reach other bottles' published ports and host services. On macOS
|
||||
this is solved with per-bottle `127.0.0.16..31` aliases added via
|
||||
`sudo ifconfig lo0 alias`. On Linux the whole `127.0.0.0/8` is
|
||||
already routed to `lo`, so docker can publish to `127.0.0.<N>`
|
||||
with **no `ifconfig`/sudo step at all** — the isolation is actually
|
||||
cheaper to achieve than on macOS.
|
||||
|
||||
## Goals / Success Criteria
|
||||
|
||||
- `BOT_BOTTLE_BACKEND=smolmachines ./cli.py start <agent>` launches,
|
||||
runs, and tears down a bottle on a Linux host with `/dev/kvm`.
|
||||
- The TSI allowlist is enforced on Linux: PRD 0022's
|
||||
`tests/integration/test_sandbox_escape.py` passes against
|
||||
`BOT_BOTTLE_BACKEND=smolmachines` on Linux (the acceptance gate).
|
||||
- Each Linux bottle is scoped to its own `127.0.0.<N>/32`, matching
|
||||
the macOS per-bottle isolation property.
|
||||
- A clear, actionable preflight error when `/dev/kvm` is missing or
|
||||
inaccessible, with remediation (load `kvm-intel`/`kvm-amd`, join the
|
||||
`kvm` group).
|
||||
- **Fail-closed:** if bot-bottle cannot positively confirm the TSI
|
||||
allowlist was persisted for a machine (DB missing, row missing,
|
||||
patch didn't take), it `die()`s before `machine start` rather than
|
||||
booting a VM with an unverified allowlist.
|
||||
- macOS behavior is unchanged.
|
||||
- README documents Linux + NixOS host setup.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Rootless / non-KVM fallbacks (e.g. software emulation). Linux
|
||||
smolmachines requires `/dev/kvm`, full stop.
|
||||
- Removing Docker as a host dependency — the sidecar bundle and
|
||||
image-build pipeline still use Docker on Linux, same as macOS.
|
||||
- Auto-installing `smolvm` or configuring KVM on the operator's
|
||||
behalf. Preflight reports; the operator remediates.
|
||||
- Nested-virtualization tuning for running the runner itself inside a
|
||||
VM (documented as a caveat, not solved here).
|
||||
|
||||
## Design
|
||||
|
||||
### Platform detection
|
||||
|
||||
Reuse the existing `platform.system()` check already in
|
||||
`loopback_alias.py` (`_is_macos()`). "Linux" is "not macOS" for every
|
||||
branch below; no new third-platform path.
|
||||
|
||||
### Preflight: KVM gate (`util.smolmachines_preflight`)
|
||||
|
||||
After the existing `smolvm`-on-`PATH` check, add a Linux-only gate:
|
||||
|
||||
- `/dev/kvm` must exist → else `die()` with "enable KVM
|
||||
(`kvm-intel`/`kvm-amd` kernel module)".
|
||||
- `/dev/kvm` must be readable + writable by the current user
|
||||
(`os.access(..., R_OK | W_OK)`) → else `die()` with "add your user
|
||||
to the `kvm` group (and re-login)".
|
||||
|
||||
macOS is unaffected (Hypervisor.framework needs no device node).
|
||||
|
||||
### smolvm state-DB path (platform-aware)
|
||||
|
||||
`loopback_alias._SMOLVM_DB_PATH` becomes platform-derived:
|
||||
|
||||
- macOS: `~/Library/Application Support/smolvm/server/smolvm.db`
|
||||
(unchanged).
|
||||
- Linux: `$XDG_DATA_HOME/smolvm/server/smolvm.db`, defaulting to
|
||||
`~/.local/share/smolvm/server/smolvm.db`.
|
||||
|
||||
> **Verification note:** the Linux DB location is inferred from
|
||||
> smolvm's documented `~/.local/share` install layout and the XDG
|
||||
> base-dir spec. It must be confirmed on a real Linux smolvm install;
|
||||
> if smolvm uses a different path or schema, the fail-closed check
|
||||
> below turns that into a clear `die()` at launch rather than a silent
|
||||
> escape.
|
||||
|
||||
### TSI enforcement: cross-platform + fail-closed (`force_allowlist`)
|
||||
|
||||
Rework `force_allowlist(machine_name, allowed_cidrs)` to run on
|
||||
**both** platforms and to fail closed:
|
||||
|
||||
1. Resolve the state DB; if the file is missing, `die()` (cannot
|
||||
confirm enforcement → refuse to launch).
|
||||
2. Read the machine's persisted row; if the row is missing, `die()`.
|
||||
3. If the row's `allowed_cidrs` already equals the requested list
|
||||
(e.g. a newer `smolvm` that honors `--allow-cidr` at create), do
|
||||
nothing — no write.
|
||||
4. Otherwise patch `allowed_cidrs` (the existing BLOB-encoded write)
|
||||
and re-read.
|
||||
5. If, after the patch, `allowed_cidrs` still does not equal the
|
||||
requested list, `die()`.
|
||||
|
||||
This is robust across smolvm versions: it works whether `--allow-cidr`
|
||||
is silently dropped (0.8.0) or honored (newer), and it never boots a
|
||||
VM whose persisted allowlist it could not confirm. It is a strict
|
||||
improvement on macOS too (today's code writes unconditionally and
|
||||
never verifies).
|
||||
|
||||
> The persisted-row check confirms our write took, not that smolvm's
|
||||
> runtime TSI enforces it. The runtime guarantee is covered by the
|
||||
> sandbox-escape acceptance test; the persisted check is the cheap
|
||||
> fail-closed guard at launch.
|
||||
|
||||
### Per-bottle loopback scoping on Linux (`allocate`)
|
||||
|
||||
`allocate` runs the same docker-state-driven allocation on Linux as on
|
||||
macOS (`_allocate_locked`, the file lock, and `_aliases_in_use` via
|
||||
`docker inspect` are all already cross-platform). The only macOS-only
|
||||
step, `ensure_pool` (the `sudo ifconfig lo0 alias` dance), stays
|
||||
macOS-only: on Linux `127.0.0.0/8` is already loopback, so docker can
|
||||
publish bundle ports directly on `127.0.0.<N>` with no setup.
|
||||
|
||||
Net effect: Linux bottles get per-bottle `127.0.0.16..31/32` scoping
|
||||
identical to macOS, without sudo.
|
||||
|
||||
### Launch flow
|
||||
|
||||
`launch.py` needs no structural change — `_allocate_resources` already
|
||||
calls `ensure_pool()` (now a Linux no-op) then `allocate()` (now
|
||||
per-bottle on Linux), and `_launch_vm` already calls
|
||||
`force_allowlist()` (now active on Linux). Only the macOS-specific
|
||||
docstrings are updated to describe the cross-platform behavior.
|
||||
|
||||
## Implementation chunks
|
||||
|
||||
1. **Preflight KVM gate** — `util.smolmachines_preflight` +
|
||||
unit tests for the missing-device and no-access branches.
|
||||
2. **Platform-aware DB path + fail-closed `force_allowlist`** —
|
||||
`loopback_alias.py`; update/extend `TestForceAllowlist`.
|
||||
3. **Cross-platform `allocate`** — drop the Linux early-return; update
|
||||
`TestAllocate` / `TestAllocateLock` for the new Linux behavior.
|
||||
4. **Docstring + comment cleanup** in `launch.py` and module headers.
|
||||
5. **Docs** — README requirements + a Linux/NixOS host-setup section.
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
- **Unit (CI, any OS):** the suite mocks `platform.system()` /
|
||||
`subprocess` and patches `_SMOLVM_DB_PATH`, so the new Linux
|
||||
branches are testable on the macOS/Linux CI runner without `smolvm`
|
||||
or KVM. Covers: KVM preflight branches, fail-closed `force_allowlist`
|
||||
(DB missing, row missing, patch-doesn't-take), per-bottle Linux
|
||||
allocation + locking, platform-derived DB path.
|
||||
- **Integration (Linux host with KVM — the acceptance gate):**
|
||||
`tests/integration/test_sandbox_escape.py` against
|
||||
`BOT_BOTTLE_BACKEND=smolmachines`. This cannot run on the macOS dev
|
||||
box and must be executed on NixOS before merge.
|
||||
|
||||
## Open questions / verification pending
|
||||
|
||||
- **Confirm the Linux smolvm state-DB path and schema** on a real
|
||||
install (the `~/.local/share/...` inference above).
|
||||
- **Confirm whether the current smolvm Linux build still drops
|
||||
`--allow-cidr` with `--from`** (the 0.8.0 bug). The fail-closed
|
||||
design handles either answer, but knowing lets us drop the DB patch
|
||||
if upstream fixed it.
|
||||
- **Confirm docker publishing to `127.0.0.<N>` on Linux** behaves as
|
||||
expected end-to-end with TSI (high confidence; standard loopback
|
||||
behavior, but unverified on the target host).
|
||||
|
||||
## References
|
||||
|
||||
- PRD 0023 — smolmachines bottle backend (macOS v1).
|
||||
- PRD 0022 — `test_sandbox_escape.py` acceptance gate.
|
||||
- PRD 0024 — sidecar bundle image.
|
||||
- smolvm: https://github.com/smol-machines/smolvm
|
||||
@@ -1,115 +0,0 @@
|
||||
# PRD prd-new: smolmachines sidecar VM
|
||||
|
||||
- **Status:** Active
|
||||
- **Author:** codex
|
||||
- **Created:** 2026-07-09
|
||||
- **Issue:** #332
|
||||
|
||||
## Summary
|
||||
|
||||
Run the smolmachines backend's trusted sidecar bundle as its own smolVM instead
|
||||
of a Docker container. A bottle then consists of an agent VM plus a sidecar VM,
|
||||
with the agent VM's TSI allowlist limited to the per-bottle agent-facing
|
||||
sidecar surface.
|
||||
|
||||
## Problem
|
||||
|
||||
The smolmachines backend currently runs the agent in smolVM but keeps
|
||||
egress/git-gate/supervise in a Docker sidecar bundle. That hybrid launch path
|
||||
keeps Docker in the trusted runtime path and leaves smolmachines with a
|
||||
different isolation boundary than its agent VM design suggests.
|
||||
|
||||
The existing Docker sidecar path also uses Docker port publishing to bind only
|
||||
agent-facing services to the per-bottle host address. TSI is IP-only, so the
|
||||
smolVM replacement must preserve that scoping: publishing sidecar services on
|
||||
generic host localhost would let the agent reach unrelated host services or
|
||||
other bottle sidecars if those services share the allowed address.
|
||||
|
||||
## Goals / Success Criteria
|
||||
|
||||
1. `BOT_BOTTLE_BACKEND=smolmachines ./cli.py start <agent>` runs with both the
|
||||
agent and sidecar as smolVMs.
|
||||
2. The smolmachines launch path no longer uses `docker run` for the sidecar
|
||||
bundle.
|
||||
3. Agent-facing egress, git-gate, and supervise endpoints are exposed only
|
||||
through the bottle's own published sidecar surface.
|
||||
4. Internal sidecar-only ports are not reachable from the agent VM.
|
||||
5. The agent VM TSI allowlist remains fail-closed and limited to the per-bottle
|
||||
address.
|
||||
6. Sidecar config, secrets, and state are delivered to the sidecar VM without
|
||||
exposing provider or forge credentials to the agent VM.
|
||||
7. Teardown removes both VMs and any host-side published port state.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Rewriting the egress, git-gate, or supervise daemons.
|
||||
- Changing the Docker backend's sidecar bundle behavior.
|
||||
- Replacing the existing agent image build and pack pipeline except where shared
|
||||
helper extraction is needed for sidecar image packing.
|
||||
- Weakening the current smolmachines TSI allowlist checks.
|
||||
|
||||
## Design
|
||||
|
||||
The sidecar VM continues to use the existing sidecar bundle image and
|
||||
`/app/sidecar_init.py` supervisor. The launch flow builds the sidecar image,
|
||||
packs it into a `.smolmachine` artifact, creates a per-bottle sidecar VM from
|
||||
that artifact, passes the same daemon-selection environment the Docker bundle
|
||||
uses today, mounts or copies the same daemon-private config/state inputs, starts
|
||||
the sidecar VM, wraps the sidecar VM's raw smolVM-published ports with a local
|
||||
per-bottle forwarder, then starts the agent VM with
|
||||
`--allow-cidr <per-bottle-address>/32`.
|
||||
|
||||
smolVM currently exposes guest ports as `HOST:GUEST` port pairs bound on host
|
||||
loopback. It does not expose an address-scoped bind like
|
||||
`127.0.0.16:<port>:9099`. Because TSI is IP-only, bot-bottle must not advertise
|
||||
those raw loopback ports directly to the agent. Instead, bot-bottle runs one
|
||||
small stdlib Python TCP forwarder process per bottle:
|
||||
|
||||
```text
|
||||
agent VM
|
||||
-> TSI allows only <bottle-address>/32
|
||||
-> bot-bottle forwarder bound to <bottle-address>:<random-port>
|
||||
-> raw smolVM-published sidecar port on 127.0.0.1:<raw-port>
|
||||
-> sidecar VM service
|
||||
```
|
||||
|
||||
Agent-facing URLs are stamped from the forwarder ports, never the raw smolVM
|
||||
ports:
|
||||
|
||||
- `HTTP_PROXY` / `HTTPS_PROXY` point at the published egress proxy port.
|
||||
- `GIT_GATE_URL` points at the published git HTTP port when git-gate is enabled.
|
||||
- `MCP_SUPERVISE_URL` points at the published supervise port when supervise is
|
||||
enabled.
|
||||
|
||||
Internal sidecar-only services stay bound inside the sidecar VM and are not
|
||||
published.
|
||||
|
||||
### Forwarder constraints
|
||||
|
||||
- Reject wildcard listener binds (`0.0.0.0`, `::`) and generic localhost
|
||||
listener binds (`127.0.0.1`, `::1`) for agent-facing forwards.
|
||||
- Accept only fixed loopback targets selected by launch; the client cannot
|
||||
choose or influence the forwarding target.
|
||||
- Forward bytes transparently without parsing HTTP, CONNECT, TLS, Git, or MCP.
|
||||
- Log lifecycle and endpoint metadata only; never log payload bytes.
|
||||
- Create forwards only for egress, git HTTP when enabled, and supervise when
|
||||
enabled.
|
||||
- Fail closed if any listener cannot bind exactly the requested per-bottle
|
||||
address.
|
||||
|
||||
## Implementation chunks
|
||||
|
||||
1. Extract the existing image-to-smolmachine pack helper so agent and sidecar
|
||||
artifacts share the same cache and registry path.
|
||||
2. Add a strict stdlib host TCP forwarder for per-bottle address-bound
|
||||
forwarding to raw smolVM-published sidecar ports.
|
||||
3. Add a sidecar VM launch spec and lifecycle helpers: pack, create, start,
|
||||
publish/discover ports, stop, delete.
|
||||
4. Switch smolmachines launch from Docker sidecar bundle lifecycle to sidecar VM
|
||||
plus forwarder lifecycle.
|
||||
5. Update egress apply / reload paths for the sidecar VM supervisor.
|
||||
6. Add unit tests for argv shape, URL stamping, teardown, and fail-closed
|
||||
behavior.
|
||||
7. Add or update integration coverage for agent-to-sidecar reachability, host
|
||||
localhost denial, other-bottle alias denial, internal port denial, and
|
||||
teardown cleanup.
|
||||
@@ -0,0 +1,490 @@
|
||||
# Monetization & competitive positioning
|
||||
|
||||
Where, if anywhere, bot-bottle has a paid wedge — given a 2026
|
||||
competitive field that has largely commoditized "sandbox a coding
|
||||
agent." Folds together the agent-provider-agnostic framing, the Fly
|
||||
remote-backend idea, the supervisor/egress-audit play, and the
|
||||
solo-dev/Linux brand instinct, then asks the only question that
|
||||
matters: is there a viable path to revenue that the competition does
|
||||
not already foreclose?
|
||||
|
||||
Companion to
|
||||
[`agent-sandbox-landscape.md`](agent-sandbox-landscape.md) (the
|
||||
isolation-tech survey),
|
||||
[`built-in-supervisor-design.md`](built-in-supervisor-design.md) (the
|
||||
supervise surface this would extend), and
|
||||
[`secret-minimization-over-dlp.md`](secret-minimization-over-dlp.md)
|
||||
(why custody, not detection, is the real moat).
|
||||
|
||||
Market data current as of June 2026.
|
||||
|
||||
## Summary
|
||||
|
||||
**Verdict: a path exists, but it is narrow, and it is not the path the
|
||||
project is currently shaped for.** Every individual property bot-bottle
|
||||
leans on — isolation, BYO-image, egress filtering, OSS, self-hosting —
|
||||
is matched by some competitor, and several are now *free* from the agent
|
||||
vendors themselves. There is exactly one defensible position left: the
|
||||
**bundle** that no single competitor occupies —
|
||||
|
||||
> uniform egress audit + secret custody + policy, across *heterogeneous
|
||||
> coding agents you don't trust*, on your infra or a managed pool.
|
||||
|
||||
Monetization is viable **only** if the product is sold as cross-vendor
|
||||
**fleet governance + egress audit for teams**, not as solo-dev agent
|
||||
safety (which the labs give away free). The solo-dev/Linux/anti-corporate
|
||||
energy is real and worth using — but as a *distribution and trust*
|
||||
engine that drives bottom-up adoption into teams, never as the revenue
|
||||
positioning itself. Get those two wires crossed and the business dies:
|
||||
you'd be courting the lowest-willingness-to-pay audience on earth while
|
||||
repelling the only buyer who pays.
|
||||
|
||||
Net: **viable, conditional, and unforgiving of positioning error.** Do
|
||||
Phase 1 (self-hostable egress-audit dashboard) regardless — it's
|
||||
low-risk and it's the demo that makes everything else legible. Gate the
|
||||
go/no-go on whether 5–10 teams confirm they'd pay for cross-vendor
|
||||
egress audit *before* building the hosted tier.
|
||||
|
||||
## The two axes of "agnostic"
|
||||
|
||||
bot-bottle differentiates on two orthogonal axes, and conflating them
|
||||
muddies the pitch:
|
||||
|
||||
1. **Agent-provider agnostic** — run Claude Code, Codex, Aider, a local
|
||||
model, behind one control layer. Already real in the code
|
||||
(`agent_provider.py`, Claude/Codex templates, BYO Dockerfile). This
|
||||
is the axis the labs *structurally cannot* match — Anthropic only
|
||||
runs Claude, OpenAI only their models. Durable.
|
||||
2. **Compute backend** — local (docker / Apple Container / smolmachines)
|
||||
today; a remote **Fly** backend would add a managed pool. This is the
|
||||
axis that makes "fleet" literal for orgs and opens metered billing.
|
||||
Fly is a strong first remote backend because it also subsumes remote
|
||||
spin-up (Machines API) and the tunnel problem (6PN/WireGuard) — but
|
||||
"provider-agnostic compute" should be *earned* after backend #2, not
|
||||
designed up front (premature generalization trap).
|
||||
|
||||
## Competitive field, by capability
|
||||
|
||||
The field doesn't have one competitor; it has a different set on each
|
||||
capability bot-bottle touches. Five dimensions:
|
||||
|
||||
| Capability | Who has it | bot-bottle's standing |
|
||||
| :-- | :-- | :-- |
|
||||
| **Isolation / sandbox** | Anthropic & OpenAI **native, free**; OSS devcontainer wrappers; E2B/Modal/Daytona/Northflank | Commoditized. Not a wedge. |
|
||||
| **Arbitrary BYO Docker image** | Sandbox PaaS (E2B/Modal/Daytona/Northflank) yes; **managed agents: ~none** (Codex = fixed `codex-universal` + setup scripts; Copilot "not supported"; Devin/Jules constrained) | Wedge **vs. managed agents** (structural: it's their infra). Table stakes vs. PaaS. |
|
||||
| **Egress audit + alerts** | LLM-observability tools (Braintrust/Langfuse/Phoenix/Helicone/Datadog) — but on *model calls*, wrong layer. Network-egress security (DeepInspect, AI gateways) — right layer, but decoupled from the agent, not cross-vendor. Sandbox PaaS = gateway/filter, not an audit surface. | **~Nobody in bot-bottle's exact shape** (per-agent egress, tied to the sandbox, with DLP context, cross-vendor). This is the wedge. |
|
||||
| **OSS / self-hosting** | Managed agents: ~none. Sandbox PaaS: ~half (E2B OSS+self-host; Northflank BYOC; Modal closed; **Daytona leaving OSS**). Devcontainer wrappers: ~all. Observability: several. | Real wedge **vs. managed agents only**. Table stakes vs. PaaS, zero differentiation vs. wrappers. |
|
||||
| **Cross-vendor uniformity** | Nobody — the labs won't, PaaS is agent-neutral infra not agent-aware control, wrappers are single-tool | Wedge. The connective tissue of the whole position. |
|
||||
|
||||
The pattern: **isolation and OSS/self-host are commodity; BYO-image and
|
||||
cross-vendor are wedges only against the managed agents; egress-audit in
|
||||
the integrated form is the one thing genuinely unoccupied.**
|
||||
|
||||
## Where bot-bottle is alone vs. where it's table stakes
|
||||
|
||||
- **Alone (the moat):** egress audit + secret custody + policy, *tied to
|
||||
the agent sandbox*, *with DLP context* (which secret, which host,
|
||||
which agent/task), *uniform across vendors*. No competitor bundles
|
||||
these. An enterprise *could* bolt DeepInspect-style egress monitoring
|
||||
onto a sandbox, so the defensibility is the **integration and
|
||||
per-agent context**, not "we can see egress."
|
||||
- **Table stakes (do not lead with these):** "we sandbox agents" (free
|
||||
from the labs), "we're open source" (E2B is; the wrapper crowd all
|
||||
is), "we self-host" (Northflank BYOC, E2B, every wrapper).
|
||||
|
||||
## The two existential competitive facts
|
||||
|
||||
1. **The agent vendors ship good-enough sandboxing for free.** Claude
|
||||
Code now has Seatbelt/bubblewrap + a network proxy natively; Codex
|
||||
has its own sandbox + approvals. This compresses the *single-vendor,
|
||||
single-dev* market to ~zero willingness-to-pay. It is *why* the
|
||||
product must be cross-vendor fleet governance, not local agent
|
||||
safety.
|
||||
2. **Northflank is converging from the infra side.** It already ships
|
||||
dedicated egress gateways + proxy-based secret injection + BYOC.
|
||||
It is the nearest thing to bot-bottle's differentiator as a managed
|
||||
platform — but infra-first and agent-neutral, not agent-aware,
|
||||
cross-vendor, or audit-first. Watch it.
|
||||
|
||||
## Monetization path (sequenced)
|
||||
|
||||
Open-core: **give away the sandbox, charge for the control plane.**
|
||||
|
||||
- **Phase 0 — validate (1–2 wks, parallel).** Ask 5–10 teams running 2+
|
||||
agents: would you pay for one egress-audit + policy plane across
|
||||
Claude *and* Codex? Gate the rest on a yes.
|
||||
- **Phase 1 — the wedge (self-hostable, OSS).** Multi-bottle egress
|
||||
dashboard + web approval queue + exportable audit log, built over the
|
||||
existing `supervise_server.py` JSON-RPC and the egress event levels
|
||||
(`LOG_BLOCKS` / `LOG_FULL`). Low risk, half-built, and the 30-second
|
||||
demo that sells everything. The compliance hook (75% of enterprises
|
||||
rank auditability #1) lives here.
|
||||
- **Phase 2 — the paywall (hosted team tier).** Multi-tenant supervisor:
|
||||
SSO/RBAC, audit retention, alerting, **centralized policy push**
|
||||
(define egress allowlist + DLP once, enforce across all agents —
|
||||
the moat made concrete). Gate on team/compliance features, *never* on
|
||||
the core security.
|
||||
- **Phase 3 — Fly remote backend.** Managed agent pool → "fleet" becomes
|
||||
literal; metered (agent-hours) billing; subsumes remote spin-up +
|
||||
tunnel.
|
||||
- **Phase 4 — deepen.** Second agent provider done deeply (lean
|
||||
open-source/open-weight for rug-pull resistance); egress anomaly
|
||||
detection (the DLP stream becomes a product); SOC2/audit-export for
|
||||
larger buyers.
|
||||
|
||||
**Do not build first:** the p2p mobile app (least monetizable, 6PN
|
||||
gives the tunnel free), a generic multi-cloud abstraction (premature),
|
||||
or the hosted SaaS before Phase 0.
|
||||
|
||||
## Brand vs. revenue: the solo-dev / Linux instinct
|
||||
|
||||
The instinct to court Linux/hacker/solo-dev users and stay "not too
|
||||
corporate" is **right for distribution, dangerous as strategy.**
|
||||
|
||||
- **Right:** it's how OSS infra gets discovered and trusted (HN, stars,
|
||||
word-of-mouth, security-circle vouching); authenticity is a real moat
|
||||
vs. the corporate players *because the architecture sincerely embodies
|
||||
it* (local-first, `$HOME` trust boundary, no phone-home); and it fits
|
||||
the founder.
|
||||
- **Dangerous:** that audience is the lowest-WTP cohort that exists
|
||||
(self-hosts the free thing, forks rather than pays), and "not too
|
||||
corporate" reads to a VP of Eng as "not enterprise-ready." Building an
|
||||
anti-SaaS brand and then shipping a paid tier invites the sell-out /
|
||||
rug-pull backlash — which **Daytona just triggered** going closed.
|
||||
|
||||
**Resolution — be Tailscale, not a manifesto.** Use the developer-first,
|
||||
respects-you energy as the *funnel*; sell *through* the solo advocate,
|
||||
bottom-up, into the team that pays. Two guardrails:
|
||||
|
||||
1. "Anti-corporate" must not mean "anti-team-features." SSO/RBAC/audit
|
||||
retention *are* the monetization; build them in a developer-respecting
|
||||
way (Tailscale has SSO and is still beloved). Tone is the brand; team
|
||||
features are the product.
|
||||
2. Set the open-core social contract publicly **on day one** — core
|
||||
sandbox open and self-hostable forever; hosted control plane is how
|
||||
the lights stay on. The communities that don't revolt are the ones
|
||||
told the deal upfront.
|
||||
|
||||
Concrete: the README frames the Docker/**Linux** backend as "legacy."
|
||||
If courting the Linux crowd, make the Linux path (Docker+gVisor,
|
||||
libkrun/smolmachines) first-class in the docs, not the fallback.
|
||||
|
||||
## Individuals, mobile, and the Pi-ecosystem reality check
|
||||
|
||||
"Individual devs won't pay" (above) is too blunt and needs refining.
|
||||
The accurate claim: individuals won't pay for **safety-as-insurance**
|
||||
(abstract risk reduction the labs give away free), but they *do* pay for
|
||||
**capability/convenience felt daily** — Claude Pro, Cursor, Tailscale
|
||||
Personal. "Drive my self-hosted agent from my phone" is capability, not
|
||||
insurance, so it has a real (low-priced, high-churn) WTP profile. The
|
||||
self-hoster/Linux crowd specifically pays for **sovereignty/control**,
|
||||
just not for enterprise insurance. So an individual "sovereign remote
|
||||
agent access" tier is *not* unreasonable in principle.
|
||||
|
||||
**But the market has already run that experiment, in public, for free.**
|
||||
The Pi ecosystem (pi.dev) has commoditized every convenience layer an
|
||||
individual product would charge for:
|
||||
|
||||
| Capability | Already free/OSS | bot-bottle differentiates? |
|
||||
| :-- | :-- | :-- |
|
||||
| Remote control from mobile | remote-pi, Paseo, TelePi | ❌ commoditized |
|
||||
| Multi-agent orchestration from mobile | Paseo, pi-agent-dashboard | ❌ commoditized |
|
||||
| **Launch** new agents from mobile | Paseo (`paseo run`) | ❌ commoditized |
|
||||
| Launch into a **sandboxed, egress-audited** env | nobody | ✅ the moat |
|
||||
|
||||
Paseo (`getpaseo/paseo`, on the App Store) does the full thing an
|
||||
individual remote-control tier would charge for — launch *and* attach
|
||||
agents on a laptop/VM/dev-server, driven from mobile over an E2E relay —
|
||||
free and open source. It *orchestrates* agents; it does **not** sandbox them, run
|
||||
an egress chokepoint, DLP-scan, or audit. None of the Pi-ecosystem tools
|
||||
do. So the residue, yet again, is **isolation + governance**, not
|
||||
remote/launch convenience.
|
||||
|
||||
Two takeaways:
|
||||
1. **Don't compete on orchestration/launch/remote UX** — it's a solved,
|
||||
free, fast-moving, App-Store-shipping space around Pi. You won't win
|
||||
it and it isn't the moat.
|
||||
2. **Be the safe runtime orchestrators launch *into*.** Launch-from-mobile
|
||||
is table stakes; *launch-into-a-sealed-egress-audited-bottle* is the
|
||||
differentiator. bot-bottle is the sandbox an orchestrator like Paseo
|
||||
would target, or that you wrap thin orchestration around — never the
|
||||
orchestrator itself.
|
||||
|
||||
Capability layers commoditize fast: every individual/mobile angle
|
||||
probed in this analysis collapsed back to the same cross-vendor +
|
||||
sandbox + egress-audit + custody bundle. Mobile remote belongs as a
|
||||
*funnel delighter* on top of the team product, not a standalone paid
|
||||
line.
|
||||
|
||||
## Forge-native orchestration as the delivery vehicle
|
||||
|
||||
The strongest concrete *product shape* for the moat is not a bespoke
|
||||
dashboard and not a Paseo competitor — it is **the git forge as the
|
||||
orchestrator, with bot-bottle as the safe runtime it launches into.**
|
||||
The forge already provides, for free, everything an orchestrator would
|
||||
otherwise have to build: identity (agent/bot users, signed commits),
|
||||
state (issues, labels, PRs/MRs, comments), triggers (webhooks, CI,
|
||||
comment commands), review (diffs, approvals, status checks), audit
|
||||
(commits/comments/reviews), and permissions (repo access, protected
|
||||
branches, token scopes). bot-bottle supplies the one thing the forge
|
||||
doesn't: **least-privilege, secret-isolated, audited execution of
|
||||
untrusted agents.** Same moat (custody + audit + policy), better
|
||||
vehicle — and it lands the product where teams already live, so it
|
||||
avoids building an agent dashboard before one is needed.
|
||||
|
||||
The flow is essentially free to assemble:
|
||||
|
||||
```
|
||||
issue/PR/MR event → webhook → policy/router → assign agent user +
|
||||
branch/worktree → run agent in an isolated bottle (no ambient secrets)
|
||||
→ commit as agent identity → open PR/MR → CI + human review + merge
|
||||
```
|
||||
|
||||
**Crowding (why this is less saturated than it looks):**
|
||||
|
||||
| Layer | How crowded |
|
||||
| :-- | :-- |
|
||||
| Generic multi-agent orchestrators (worktree/TUI/dashboard) | very — 50–100+ |
|
||||
| Forge-native issue/PR/MR orchestration | moderate — ~10–30 serious |
|
||||
| Self-hostable, least-privilege, audited, forge-portable | **single digits** |
|
||||
|
||||
The deeper you go toward *untrusted-agent safety + auditability +
|
||||
self-hostable + forge-portable*, the emptier it gets.
|
||||
|
||||
**The GitHub/GitLab first-party trap → lead Gitea + sovereignty.**
|
||||
GitHub (Agentic Workflows, Copilot coding agent) and GitLab (Duo Agent
|
||||
Platform) are the forge *vendors* building native issue-to-PR agent
|
||||
orchestration with native identity/permissions/audit. On their turf you
|
||||
lose the integration-depth battle the same way single-vendor agent
|
||||
safety loses to Anthropic/OpenAI — the same "incumbent ships it free,
|
||||
deeper" dynamic, one layer up. So the durable opening is **Gitea +
|
||||
self-hosted** (no first-party agent platform exists — the open Gitea
|
||||
feature request for an AI code agent confirms the vacuum) plus
|
||||
**cross-forge *untrusted-agent* safety**, which no forge vendor will
|
||||
build because they want you running *their* agent, not arbitrary ones
|
||||
under uniform least-privilege across competitors' forges. Cross-vendor
|
||||
neutrality, applied to forges.
|
||||
|
||||
**Buyer reconciliation.** The least-crowded opening (self-hosted Gitea)
|
||||
overlaps the lowest-WTP crowd (indie self-hosters), while the paying
|
||||
teams sit on GitHub/GitLab where first-party competition is fiercest.
|
||||
The intersection that resolves it: **orgs running self-hosted forges for
|
||||
sovereignty/compliance reasons** (regulated, air-gapped, security-
|
||||
conscious, on-prem). They have budget, they run self-hosted GitLab/Gitea,
|
||||
*and* shipping code to a cloud agent vendor is a non-starter — so "run
|
||||
untrusted agents sandboxed, least-privilege, fully audited, inside our
|
||||
forge, on our infra" is a procurement checkbox, not a nicety. That is
|
||||
where "least-crowded" finally meets "has money."
|
||||
|
||||
**Separate moat-hard-parts from cost-hard-parts.** The orchestration
|
||||
"hard parts" are two different things, and conflating them oversells the
|
||||
fit:
|
||||
|
||||
| Moat (your differentiated strength) | Undifferentiated cost (everyone faces) |
|
||||
| :-- | :-- |
|
||||
| permission isolation | idempotency / dedupe / run ledger |
|
||||
| secret handling under malicious prompts | concurrency, locks, cancellation |
|
||||
| run provenance | queueing / scheduling / cleanup |
|
||||
| policy language | merge-conflict handling (~27% agent-PR conflict rate) |
|
||||
|
||||
The right column is generic distributed-systems plumbing that wins you
|
||||
nothing and that merge-conflict resolution especially is a *different
|
||||
competency* from sandbox/custody. Keep it thin in the MVP; do not build a
|
||||
policy DSL + durable ledger + conflict resolver before one org pays.
|
||||
|
||||
**The killer feature: run provenance on every agent PR.** A check/comment
|
||||
answering — which agent, which model, which prompt, which base commit,
|
||||
which policy, which tools, which network egress, which test results —
|
||||
attached at the moment a human reviews. It renders the (invisible)
|
||||
custody + egress-audit work as a PR artifact the buyer sees at the exact
|
||||
trust-decision point. No forge vendor's first-party agent will show you
|
||||
"here is everything the untrusted agent could reach." Build this first.
|
||||
|
||||
**MVP** (`@bot-bottle fix this`): create an isolated worktree/bottle →
|
||||
check out the issue branch → run the selected harness as a named agent
|
||||
user → deny ambient secrets by default → record prompt/model/tools/policy
|
||||
→ commit with bot identity → open PR/MR → attach the run-provenance
|
||||
footer (log + tests + permission/egress summary) → require human merge.
|
||||
The security model *is* the product. This rides the headless launch
|
||||
primitive directly: webhook → `start --headless` into an isolated bottle
|
||||
→ commit as agent identity → PR with provenance.
|
||||
|
||||
Open-core line, refined in the next section: the trigger *convention*
|
||||
(label/assignee) stays open so anyone can adopt it, but the
|
||||
**orchestrator that receives webhooks and governs lifecycle is the paid
|
||||
control plane**; the runtime — and a signed-provenance emission API —
|
||||
stay free.
|
||||
|
||||
## The open/paid boundary, refined: orchestrator as the paid control plane
|
||||
|
||||
The forge-native shape sharpens the open-core line past the rough
|
||||
"trigger free, execution paid" cut above. Working it through four
|
||||
constraints — value capture, provenance integrity, the sovereignty
|
||||
buyer, and what the forge *structurally cannot do* — yields a precise
|
||||
boundary.
|
||||
|
||||
**The orchestrator is the control plane, and the control plane is the
|
||||
paid product.** With the forge supplying identity / state / triggers /
|
||||
review, bot-bottle's orchestrator (`bot-bottle-orchestrator`, already
|
||||
specced as a separate binary in the forge-native PRD) is where webhooks
|
||||
land and bottle lifecycle + governance live. That binary can stay
|
||||
**closed/private from day one** without breaking the open-core contract:
|
||||
the runtime stays OSS; the control plane is how the lights stay on. This
|
||||
is "give away the sandbox, charge for the control plane" made literal —
|
||||
the orchestrator *is* the control plane.
|
||||
|
||||
**Charge for the moat, not the webhook.** Holding webhooks and managing
|
||||
bottle lifecycle is commodity — the forge vendors build it first-party,
|
||||
and it's the "undifferentiated cost" column above (idempotency, queueing,
|
||||
dispatch). If the pitch is "we catch the webhook," they out-build it
|
||||
free. The paid value is the two things the forge *cannot* do:
|
||||
|
||||
1. **See inside the run** — which model / prompt / policy / tools / egress
|
||||
produced the diff, whether a secret nearly left. Runtime-level data
|
||||
only the bottle holds.
|
||||
2. **Aggregate and enforce across runs** — retain / search / export every
|
||||
run across every repo; push one egress/DLP/capability policy
|
||||
fleet-wide and detect drift.
|
||||
|
||||
The explainable heuristic: **anything legible within a single run on a
|
||||
single node is free; anything requiring cross-run aggregation, central
|
||||
enforcement, or identity/fleet management is paid.** That is also the
|
||||
individual-vs-team line — individuals live in single runs, teams need the
|
||||
aggregate.
|
||||
|
||||
**Provenance: emit free (signed), sell the product.** The forge is the
|
||||
wrong system of record for provenance — a markdown footer is mutable by
|
||||
any maintainer, unsigned, per-PR, with no aggregation, so a maintainer
|
||||
could simply edit it. The authoritative record therefore lives in the
|
||||
(paid) control plane. The *runtime* emits **signed** provenance through a
|
||||
**free API** — tamper-evident offline (edit it and the signature breaks;
|
||||
verify with no server), so on-prem teams can route it into their own
|
||||
SIEM. What's paid is the *product* over that stream: retention, search,
|
||||
cross-run, export, policy. Whether a copy also lands in the PR footer is
|
||||
an optional, off-by-default marketing dial — one consumer of the free
|
||||
API, not a free provenance surface, and never the audit record. The
|
||||
mutability "bug" becomes a paid feature: the control plane flags *"PR
|
||||
footer edited / doesn't match the signed run."* (Prometheus model:
|
||||
`/metrics` is free to scrape; managed retention + dashboards are the
|
||||
business.)
|
||||
|
||||
**On-prem priority: self-hosted runners over self-hosted provenance.**
|
||||
The sovereignty buyer's *hard structural constraint* is where the agent
|
||||
**executes** against private code, secrets, and network — that's the
|
||||
runner, and it cannot leave the perimeter. Audit metadata is softer; many
|
||||
regulated orgs ship logs to SaaS while keeping the workload inside. So:
|
||||
|
||||
- Self-hosted **runner** = baseline, always, for that buyer.
|
||||
- Self-hosted **provenance store** = premium tier of the strictest subset
|
||||
(air-gapped, hard data-residency) — and largely covered by the free
|
||||
emission API → their own SIEM, so it may never need to be a product you
|
||||
build.
|
||||
- Precision so you don't trip your own free tier: a single self-hosted
|
||||
runner *is the OSS runtime on their box* — free. What's paid is the
|
||||
**fleet control plane**: enrolling/managing many runners, central
|
||||
policy push, dispatch/identity/quota, health/scaling. You don't sell
|
||||
"a runner," you sell **running a governed fleet**.
|
||||
|
||||
**Resulting tiers:**
|
||||
|
||||
| Layer | What it is | Open/Paid | Deployment |
|
||||
| :-- | :-- | :-- | :-- |
|
||||
| **Runtime** | isolation + ephemeral bottles, cred-proxy, supervise, `start --headless`, signed-provenance emission API | Free / OSS | Always self-host |
|
||||
| **Single runner** | the OSS runtime on a box | Free / OSS | Self-host |
|
||||
| **Control plane** | cross-run audit retention/search/export, central policy push, SSO/RBAC dispatch, fleet management of runners, alerting | **Paid** | Hosted *or* self-host-licensed — same code |
|
||||
| **Capacity** | managed Fly runner pool, metered (agent-hours) | **Paid add-on** | Hosted only |
|
||||
|
||||
Fly stays a **capacity/convenience line, not the moat** — it monetizes
|
||||
even solo hackers (capability, not insurance), but a managed runner pool
|
||||
is reselling compute against Fly/E2B/Northflank on price. It's a bundle
|
||||
attached to the governance, never the thing defended. Self-host is *not*
|
||||
a separate product: on-prem buyers get the same closed control plane,
|
||||
licensed, pointed at their own runners.
|
||||
|
||||
## Risks to the thesis
|
||||
|
||||
- **Lab encroachment.** If Anthropic/OpenAI add cross-agent governance
|
||||
or open their managed egress logs, the wedge narrows. Mitigate by
|
||||
going deep on cross-vendor + custody + audit *now*, while they're
|
||||
single-vendor.
|
||||
- **Rug-pull dependency.** You run the labs' agents; they can restrict
|
||||
their agent to their own sandbox via ToS/tech. Hedge toward
|
||||
open-source/open-weight agents for durability.
|
||||
- **Northflank (or E2B) ships agent-aware audit.** Plausible from the
|
||||
infra side. Your defense is agent-awareness + the supervise approval
|
||||
loop + cross-vendor, not raw egress visibility.
|
||||
- **WTP may simply not be there.** The honest failure mode: teams like
|
||||
the audit but won't pay because "we already sandbox in CI." Phase 0
|
||||
exists to find this out cheaply before building Phase 2/3.
|
||||
- **Forge-vendor encroachment (forge-native path).** GitHub Agentic
|
||||
Workflows / Copilot and GitLab Duo are first-party and deepening.
|
||||
Defense: aim at self-hosted Gitea + sovereignty buyers where no
|
||||
first-party agent platform exists, and at cross-forge untrusted-agent
|
||||
neutrality the vendors won't build. Don't fight them GitHub-native.
|
||||
- **Orchestration-reliability scope creep.** The forge-native build
|
||||
drags in idempotency, queueing, concurrency, and merge-conflict
|
||||
handling — undifferentiated plumbing that isn't the moat. Keep it thin
|
||||
until a paying org forces it.
|
||||
|
||||
## Recommendation
|
||||
|
||||
Build Phase 1 now — it's low-risk, half-built, and the proof artifact.
|
||||
Run Phase 0 in parallel. Treat a clear yes from 5–10 teams as the
|
||||
green light for the hosted tier; treat a soft maybe as a signal to stay
|
||||
an excellent OSS tool with a tip-jar/support model rather than a
|
||||
venture-shaped SaaS. The technology is not the risk — the codebase is
|
||||
exemplary and the architecture already supports the pivot. The risk is
|
||||
**positioning discipline**: sell cross-vendor fleet governance to teams,
|
||||
use the indie brand as the funnel, and never let the anti-corporate
|
||||
aesthetic veto the features that pay.
|
||||
|
||||
## Sources
|
||||
|
||||
- Anthropic — Claude Code sandboxing:
|
||||
https://www.anthropic.com/engineering/claude-code-sandboxing
|
||||
- OpenAI Codex — cloud environments:
|
||||
https://developers.openai.com/codex/cloud/environments ;
|
||||
custom-image feature request:
|
||||
https://community.openai.com/t/feature-request-custom-docker-images/1265333
|
||||
- GitHub Copilot — custom container image (not supported), discussion
|
||||
#194105: https://github.com/orgs/community/discussions/194105
|
||||
- DeepInspect — AI egress monitoring:
|
||||
https://www.deepinspect.ai/blog/ai-egress-monitoring
|
||||
- Braintrust — AI agent observability/alerting:
|
||||
https://www.braintrust.dev/articles/best-ai-agent-observability-tools-2026
|
||||
- E2B (OSS, Apache-2.0): https://github.com/e2b-dev/e2b ;
|
||||
infra/self-host: https://github.com/e2b-dev/infra
|
||||
- Daytona going closed source:
|
||||
https://www.daytona.io/dotfiles/updates/daytona-is-going-closed-source
|
||||
- Northflank — BYOC / egress gateways:
|
||||
https://northflank.com/blog/what-is-byoc-in-cloud-computing ;
|
||||
https://northflank.com/blog/self-hostable-alternatives-to-e2b-for-ai-agents
|
||||
- Modal Sandboxes: https://modal.com/products/sandboxes
|
||||
- AI agent orchestration / enterprise governance (75% cite
|
||||
auditability):
|
||||
https://viston.tech/ai-agent-orchestration-in-2026-moving-from-pilots-to-enterprise-wide-execution/
|
||||
- Pi harness (provider-agnostic CLI): https://pi.dev/packages/remote-pi ;
|
||||
https://github.com/earendil-works/pi
|
||||
- Paseo (launch + attach agents from desktop/mobile, OSS):
|
||||
https://github.com/getpaseo/paseo ;
|
||||
https://apps.apple.com/us/app/paseo-remote-coding-agents/id6758887924
|
||||
- pi-agent-dashboard (mobile-first remote control via mDNS/zrok):
|
||||
https://github.com/BlackBeltTechnology/pi-agent-dashboard
|
||||
- TelePi (Telegram remote control for Pi):
|
||||
https://futurelab.studio/blog/telepi-telegram-remote-control-for-pi/
|
||||
- Forge-native landscape (provided via conversation, not independently
|
||||
re-verified):
|
||||
- awesome-agent-orchestrators (50+ generic orchestrators):
|
||||
https://github.com/andyrewlee/awesome-agent-orchestrators
|
||||
- GitHub Agentic Workflows (first-party repo automation):
|
||||
https://github.blog/ai-and-ml/automate-repository-tasks-with-github-agentic-workflows/
|
||||
- GitLab Duo Agent Platform GA:
|
||||
https://ir.gitlab.com/news/news-details/2026/GitLab-Announces-the-General-Availability-of-GitLab-Duo-Agent-Platform/default.aspx
|
||||
- ai-review (cross-forge review incl. Gitea):
|
||||
https://github.com/Nikita-Filonov/ai-review
|
||||
- Gitea feature request — AI code agent (the vacuum):
|
||||
https://github.com/go-gitea/gitea/issues/34527
|
||||
- Phoenix — safe GitHub issue resolution (label-based webhook state
|
||||
machine): https://arxiv.org/abs/2606.20243
|
||||
- AgenticFlict — ~27% merge-conflict rate in agent PRs:
|
||||
https://arxiv.org/abs/2604.03551
|
||||
@@ -3,5 +3,5 @@
|
||||
# These tools are used for code quality checks in CI/CD.
|
||||
|
||||
pylint>=3.0.0
|
||||
pyright>=1.1.411
|
||||
pyright>=1.1.300
|
||||
coverage>=7.0.0
|
||||
|
||||
@@ -81,10 +81,10 @@ class TestSandboxEscape(unittest.TestCase):
|
||||
# those are missing rather than die-ing inside backend.prepare.
|
||||
backend_name = os.environ.get("BOT_BOTTLE_BACKEND", "docker")
|
||||
if backend_name == "smolmachines":
|
||||
if sys.platform not in ("darwin", "linux"):
|
||||
if sys.platform != "darwin":
|
||||
raise unittest.SkipTest(
|
||||
f"BOT_BOTTLE_BACKEND=smolmachines is not supported "
|
||||
f"on {sys.platform} (macOS and Linux only)"
|
||||
"BOT_BOTTLE_BACKEND=smolmachines is macOS-only in "
|
||||
"v1 (libkrun TSI)"
|
||||
)
|
||||
if shutil.which("smolvm") is None:
|
||||
raise unittest.SkipTest(
|
||||
|
||||
@@ -21,7 +21,7 @@ security properties the design pivot was about:
|
||||
bind-address mitigation is what closes TSI's port-granularity
|
||||
gap.
|
||||
|
||||
Gated on macOS/Linux + smolvm + docker + not GITEA_ACTIONS — the
|
||||
Gated on macOS + smolvm + docker + not GITEA_ACTIONS — the
|
||||
runner can't host libkrun-backed VMs."""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -65,8 +65,8 @@ def _minimal_manifest() -> ManifestIndex:
|
||||
|
||||
@skip_unless_docker()
|
||||
@unittest.skipUnless(
|
||||
platform.system() in ("Darwin", "Linux"),
|
||||
"smolvm requires macOS or Linux",
|
||||
platform.system() == "Darwin",
|
||||
"smolvm is macOS-only for v1; Linux+KVM path is a future PRD",
|
||||
)
|
||||
@unittest.skipUnless(
|
||||
_smolvm_available(),
|
||||
@@ -141,15 +141,7 @@ class TestSmolmachinesLaunch(unittest.TestCase):
|
||||
proxies = [line.strip() for line in r.stdout.splitlines()]
|
||||
self.assertEqual(2, len(proxies), proxies)
|
||||
self.assertEqual(proxies[0], proxies[1], proxies)
|
||||
# macOS: proxy binds to the per-bottle loopback alias (127.x.x.x) so
|
||||
# TSI can intercept guest connections to it. Linux: the guest kernel
|
||||
# routes 127.0.0.0/8 to its own loopback (TSI never sees those), so
|
||||
# the proxy instead binds to the per-bottle bridge gateway (192.168.x.1)
|
||||
# which routes via eth0 and is intercepted by TSI normally.
|
||||
self.assertRegex(
|
||||
proxies[0], r"^http://\d+\.\d+\.\d+\.\d+:\d+$",
|
||||
"expected proxy URL to be an http://IP:port address",
|
||||
)
|
||||
self.assertTrue(proxies[0].startswith("http://127."), proxies[0])
|
||||
|
||||
r = self.bottle.exec(
|
||||
"curl -fsS --max-time 20 https://example.com >/dev/null && echo OK"
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
exercised against the real binary.
|
||||
|
||||
The full machine-lifecycle round trip (create → start → exec →
|
||||
delete) is gated behind macOS/Linux platform check and lives
|
||||
delete) is gated behind macOS + Darwin platform check and lives
|
||||
in chunk 2d's smoke. This file just verifies `is_available()`
|
||||
correctly reports presence and `_smolvm()` can run a no-op
|
||||
subcommand without errors — enough to flag wrapper drift if
|
||||
@@ -23,8 +23,8 @@ from bot_bottle.backend.smolmachines.smolvm import is_available
|
||||
"skipped under act_runner: smolvm not installed on the runner",
|
||||
)
|
||||
@unittest.skipUnless(
|
||||
platform.system() in ("Darwin", "Linux"),
|
||||
"smolvm requires macOS or Linux",
|
||||
platform.system() == "Darwin",
|
||||
"smolvm is macOS-only for v1; Linux+KVM path is a future PRD",
|
||||
)
|
||||
@unittest.skipUnless(
|
||||
is_available(),
|
||||
|
||||
@@ -35,10 +35,7 @@ class TestAgentProviderRuntime(unittest.TestCase):
|
||||
)
|
||||
config = Path(tmp, "codex-config.toml").read_text()
|
||||
self.assertEqual("codex", plan.template)
|
||||
self.assertEqual(
|
||||
"/home/node/.codex/packages/standalone/current/bin/codex",
|
||||
plan.command,
|
||||
)
|
||||
self.assertEqual("codex", plan.command)
|
||||
self.assertEqual("read_prompt_file", plan.prompt_mode)
|
||||
self.assertEqual("/tmp/Dockerfile.codex", plan.dockerfile)
|
||||
self.assertEqual(
|
||||
|
||||
@@ -12,19 +12,11 @@ from unittest.mock import patch
|
||||
|
||||
import bot_bottle.cli as climod
|
||||
from bot_bottle.cli import main
|
||||
from bot_bottle.db_store import DbStore
|
||||
from bot_bottle.log import Die
|
||||
from bot_bottle.manifest import ManifestError
|
||||
from bot_bottle.store_manager import StoreManager
|
||||
|
||||
|
||||
class TestMainDispatch(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
patcher = patch.object(DbStore, "is_migrated", return_value=True)
|
||||
self._mock_check = patcher.start()
|
||||
self.addCleanup(patcher.stop)
|
||||
self.addCleanup(StoreManager.reset)
|
||||
|
||||
def test_no_args_prints_usage_returns_2(self) -> None:
|
||||
with patch("sys.stderr", io.StringIO()):
|
||||
self.assertEqual(2, main([]))
|
||||
@@ -64,15 +56,6 @@ class TestMainDispatch(unittest.TestCase):
|
||||
main(["x", "a", "b"])
|
||||
self.assertEqual([["a", "b"]], seen)
|
||||
|
||||
def test_missing_env_var_error_maps_to_1(self) -> None:
|
||||
from bot_bottle.errors import MissingEnvVarError
|
||||
|
||||
def boom(_rest: list[str]) -> int:
|
||||
raise MissingEnvVarError("MY_VAR", "MY_VAR is not set")
|
||||
|
||||
with patch.dict(climod.COMMANDS, {"x": boom}), patch("sys.stderr", io.StringIO()):
|
||||
self.assertEqual(1, main(["x"]))
|
||||
|
||||
def test_manifest_error_maps_to_1(self) -> None:
|
||||
def boom(_rest: list[str]) -> int:
|
||||
raise ManifestError("bad manifest")
|
||||
|
||||
@@ -1,251 +0,0 @@
|
||||
"""Unit: `cli.py start --headless` non-interactive launch path.
|
||||
|
||||
Headless is the keystone for orchestrators, CI, and webhook
|
||||
dispatch: agent/bottles/label come from flags + manifest defaults, no
|
||||
TUI selectors fire, and the preflight y/N is auto-confirmed
|
||||
(`assume_yes=True`). All actual launch work is stubbed so no container
|
||||
is created.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
import os
|
||||
import unittest
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import bot_bottle.cli.start as start_mod
|
||||
import bot_bottle.cli.tui as tui_mod
|
||||
from bot_bottle.backend import ActiveAgent
|
||||
from bot_bottle.log import Die
|
||||
from bot_bottle.manifest import ManifestError
|
||||
|
||||
|
||||
def _make_manifest(
|
||||
agent_names: list[str],
|
||||
bottle_names: list[str] | None = None,
|
||||
agent_bottle: str = "",
|
||||
):
|
||||
manifest = MagicMock()
|
||||
manifest.agents = {name: MagicMock(bottle=agent_bottle) for name in agent_names}
|
||||
manifest.all_agent_names = sorted(agent_names)
|
||||
manifest.all_bottle_names = sorted(bottle_names or [])
|
||||
manifest.home_md = None # eager mode so _peek_agent_bottle uses agents dict
|
||||
manifest.require_agent = MagicMock(return_value=None)
|
||||
return manifest
|
||||
|
||||
|
||||
def _active_agent(slug: str) -> ActiveAgent:
|
||||
return ActiveAgent(
|
||||
backend_name="docker",
|
||||
slug=slug,
|
||||
agent_name="demo",
|
||||
started_at="2026-01-01T00:00:00+00:00",
|
||||
services=(),
|
||||
)
|
||||
|
||||
|
||||
class TestCmdStartHeadless(unittest.TestCase):
|
||||
"""Drive `cmd_start --headless` with launch + TUI stubbed out."""
|
||||
|
||||
def setUp(self):
|
||||
self._manifest = _make_manifest(
|
||||
["researcher", "implementer"], ["claude", "dev"], agent_bottle="claude"
|
||||
)
|
||||
patch(
|
||||
"bot_bottle.cli.start.ManifestIndex.resolve",
|
||||
return_value=self._manifest,
|
||||
).start()
|
||||
self._launch_mock = patch(
|
||||
"bot_bottle.cli.start._launch_bottle", return_value=0
|
||||
).start()
|
||||
# No bottles running by default → no label collision.
|
||||
patch(
|
||||
"bot_bottle.cli.start.enumerate_active_agents", return_value=[]
|
||||
).start()
|
||||
# If any TUI picker fires in headless mode, that's a bug.
|
||||
self._agent_picker = patch.object(tui_mod, "filter_select").start()
|
||||
self._bottle_picker = patch.object(tui_mod, "filter_multiselect").start()
|
||||
self._modal = patch.object(tui_mod, "name_color_modal").start()
|
||||
patch.dict(os.environ, {}, clear=False).start()
|
||||
os.environ.pop("BOT_BOTTLE_BACKEND", None)
|
||||
self.addCleanup(patch.stopall)
|
||||
|
||||
def _spec(self):
|
||||
self._launch_mock.assert_called_once()
|
||||
return self._launch_mock.call_args[0][0]
|
||||
|
||||
# -- no TUI in headless --------------------------------------------
|
||||
|
||||
def test_headless_fires_no_pickers(self):
|
||||
rc = start_mod.cmd_start(
|
||||
["--headless", "researcher", "--bottle", "claude", "--prompt", "Do it"]
|
||||
)
|
||||
self.assertEqual(0, rc)
|
||||
self._agent_picker.assert_not_called()
|
||||
self._bottle_picker.assert_not_called()
|
||||
self._modal.assert_not_called()
|
||||
|
||||
def test_headless_assume_yes_forwarded(self):
|
||||
start_mod.cmd_start(
|
||||
["--headless", "researcher", "--bottle", "claude", "--prompt", "Do it"]
|
||||
)
|
||||
self.assertTrue(self._launch_mock.call_args[1]["assume_yes"])
|
||||
|
||||
# -- prompt --------------------------------------------------------
|
||||
|
||||
def test_headless_without_prompt_dies(self):
|
||||
with self.assertRaises(Die):
|
||||
start_mod.cmd_start(["--headless", "researcher", "--bottle", "claude"])
|
||||
self._launch_mock.assert_not_called()
|
||||
|
||||
def test_headless_prompt_forwarded_to_launch(self):
|
||||
start_mod.cmd_start(
|
||||
["--headless", "researcher", "--bottle", "claude",
|
||||
"--prompt", "Implement issue #42"]
|
||||
)
|
||||
self.assertEqual(
|
||||
"Implement issue #42",
|
||||
self._launch_mock.call_args[1]["headless_prompt_text"],
|
||||
)
|
||||
|
||||
# -- bottle resolution ---------------------------------------------
|
||||
|
||||
def test_explicit_bottles_forwarded_in_order(self):
|
||||
start_mod.cmd_start(
|
||||
["--headless", "researcher", "--bottle", "dev", "--bottle", "claude",
|
||||
"--prompt", "Do it"]
|
||||
)
|
||||
self.assertEqual(("dev", "claude"), self._spec().bottle_names)
|
||||
|
||||
def test_omitted_bottle_falls_back_to_agent_default(self):
|
||||
start_mod.cmd_start(["--headless", "implementer", "--prompt", "Do it"])
|
||||
self.assertEqual(("claude",), self._spec().bottle_names)
|
||||
|
||||
def test_no_bottle_and_no_default_dies(self):
|
||||
manifest = _make_manifest(["researcher"], ["claude"], agent_bottle="")
|
||||
with patch(
|
||||
"bot_bottle.cli.start.ManifestIndex.resolve", return_value=manifest
|
||||
):
|
||||
with self.assertRaises(Die):
|
||||
start_mod.cmd_start(
|
||||
["--headless", "researcher", "--prompt", "Do it"]
|
||||
)
|
||||
self._launch_mock.assert_not_called()
|
||||
|
||||
# -- agent resolution ----------------------------------------------
|
||||
|
||||
def test_missing_agent_name_dies(self):
|
||||
with self.assertRaises(Die):
|
||||
start_mod.cmd_start(["--headless"])
|
||||
self._launch_mock.assert_not_called()
|
||||
|
||||
def test_unknown_agent_raises_manifest_error(self):
|
||||
self._manifest.require_agent.side_effect = ManifestError("agent 'x' not defined")
|
||||
with self.assertRaises(ManifestError):
|
||||
start_mod.cmd_start(
|
||||
["--headless", "x", "--bottle", "claude", "--prompt", "Do it"]
|
||||
)
|
||||
self._launch_mock.assert_not_called()
|
||||
|
||||
# -- label / color -------------------------------------------------
|
||||
|
||||
def test_label_defaults_to_agent_name(self):
|
||||
start_mod.cmd_start(
|
||||
["--headless", "researcher", "--bottle", "claude", "--prompt", "Do it"]
|
||||
)
|
||||
self.assertEqual("researcher", self._spec().label)
|
||||
|
||||
def test_explicit_label_and_color_forwarded(self):
|
||||
start_mod.cmd_start(
|
||||
["--headless", "researcher", "--bottle", "claude",
|
||||
"--label", "nightly", "--color", "green", "--prompt", "Do it"]
|
||||
)
|
||||
spec = self._spec()
|
||||
self.assertEqual("nightly", spec.label)
|
||||
self.assertEqual("green", spec.color)
|
||||
|
||||
def test_label_collision_uniquifies(self):
|
||||
with patch(
|
||||
"bot_bottle.cli.start.enumerate_active_agents",
|
||||
return_value=[_active_agent("researcher")],
|
||||
):
|
||||
start_mod.cmd_start(
|
||||
["--headless", "researcher", "--bottle", "claude", "--prompt", "Do it"]
|
||||
)
|
||||
self.assertEqual("researcher-2", self._spec().label)
|
||||
|
||||
# -- backend wiring ------------------------------------------------
|
||||
|
||||
def test_backend_flag_forwarded(self):
|
||||
start_mod.cmd_start(
|
||||
["--headless", "--backend=docker", "researcher", "--bottle", "claude",
|
||||
"--prompt", "Do it"]
|
||||
)
|
||||
self.assertEqual("docker", self._launch_mock.call_args[1]["backend_name"])
|
||||
|
||||
def test_cached_images_sets_cached_policy(self):
|
||||
start_mod.cmd_start(
|
||||
["--headless", "--cached-images", "researcher", "--bottle", "claude",
|
||||
"--prompt", "Do it"]
|
||||
)
|
||||
self.assertEqual("cached", self._spec().image_policy)
|
||||
|
||||
def test_cached_images_requires_headless(self):
|
||||
with self.assertRaises(Die):
|
||||
start_mod.cmd_start(["--cached-images", "researcher"])
|
||||
self._launch_mock.assert_not_called()
|
||||
|
||||
|
||||
class TestPrepareWithPreflight(unittest.TestCase):
|
||||
"""prepare_with_preflight calls render_preflight with the plan and backend name."""
|
||||
|
||||
def test_render_preflight_called(self):
|
||||
from pathlib import Path
|
||||
mock_backend = MagicMock()
|
||||
mock_plan = MagicMock()
|
||||
mock_backend.prepare.return_value = mock_plan
|
||||
mock_backend.name = "test-backend"
|
||||
render = MagicMock()
|
||||
|
||||
with patch("bot_bottle.cli.start.get_bottle_backend", return_value=mock_backend), \
|
||||
patch("bot_bottle.cli.start._identity_from_plan", return_value="id"), \
|
||||
patch("bot_bottle.cli.start.info"):
|
||||
start_mod.prepare_with_preflight(
|
||||
MagicMock(),
|
||||
stage_dir=Path("/tmp"),
|
||||
render_preflight=render,
|
||||
prompt_yes=MagicMock(),
|
||||
dry_run=True,
|
||||
)
|
||||
|
||||
render.assert_called_once_with(mock_plan, "test-backend")
|
||||
|
||||
|
||||
class TestNoAgentsDefined(unittest.TestCase):
|
||||
"""cmd_start prints an error and returns 1 when the manifest has no agents."""
|
||||
|
||||
def test_no_agents_defined_returns_1(self):
|
||||
manifest = _make_manifest([], [])
|
||||
with patch(
|
||||
"bot_bottle.cli.start.ManifestIndex.resolve", return_value=manifest
|
||||
), patch("sys.stderr", io.StringIO()) as err:
|
||||
rc = start_mod.cmd_start([])
|
||||
self.assertEqual(1, rc)
|
||||
self.assertIn("no agents defined", err.getvalue())
|
||||
|
||||
|
||||
class TestTextRenderPreflight(unittest.TestCase):
|
||||
"""_text_render_preflight returns a renderer that prints the backend name."""
|
||||
|
||||
def test_backend_name_in_output(self):
|
||||
render = start_mod._text_render_preflight()
|
||||
plan = MagicMock()
|
||||
with patch("bot_bottle.cli.start._manifest_to_yaml", return_value=""), \
|
||||
patch("sys.stderr", io.StringIO()) as err:
|
||||
render(plan, "my-backend")
|
||||
self.assertIn("my-backend", err.getvalue())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -57,12 +57,6 @@ class TestCmdStartSelector(unittest.TestCase):
|
||||
self._bottle_picker_mock = self._bottle_picker_patch.start()
|
||||
self._bottle_picker_mock.return_value = ["claude"] # default: one bottle selected
|
||||
|
||||
self._image_policy_patch = patch(
|
||||
"bot_bottle.cli.start._select_image_policy",
|
||||
return_value="fresh",
|
||||
)
|
||||
self._image_policy_patch.start()
|
||||
|
||||
self._env_patch = patch.dict(os.environ, {}, clear=False)
|
||||
self._env_patch.start()
|
||||
os.environ.pop("BOT_BOTTLE_BACKEND", None)
|
||||
@@ -72,7 +66,6 @@ class TestCmdStartSelector(unittest.TestCase):
|
||||
self._launch_patch.stop()
|
||||
self._agent_picker_patch.stop()
|
||||
self._bottle_picker_patch.stop()
|
||||
self._image_policy_patch.stop()
|
||||
self._env_patch.stop()
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
@@ -131,19 +124,6 @@ class TestCmdStartSelector(unittest.TestCase):
|
||||
spec = self._launch_mock.call_args[0][0]
|
||||
self.assertEqual(("claude", "dev"), spec.bottle_names)
|
||||
|
||||
def test_image_policy_forwarded_to_spec(self):
|
||||
with patch("bot_bottle.cli.start._select_image_policy", return_value="cached"):
|
||||
start_mod.cmd_start(["researcher"])
|
||||
self._launch_mock.assert_called_once()
|
||||
spec = self._launch_mock.call_args[0][0]
|
||||
self.assertEqual("cached", spec.image_policy)
|
||||
|
||||
def test_image_policy_cancel_returns_0(self):
|
||||
with patch("bot_bottle.cli.start._select_image_policy", return_value=None):
|
||||
rc = start_mod.cmd_start(["researcher"])
|
||||
self.assertEqual(0, rc)
|
||||
self._launch_mock.assert_not_called()
|
||||
|
||||
def test_empty_bottle_selection_forwarded(self):
|
||||
self._bottle_picker_mock.return_value = []
|
||||
start_mod.cmd_start(["researcher"])
|
||||
@@ -235,7 +215,6 @@ class TestCmdStartLabelCollision(unittest.TestCase):
|
||||
).start()
|
||||
# Stub the bottle picker to always return a selection.
|
||||
patch.object(tui_mod, "filter_multiselect", return_value=["claude"]).start()
|
||||
patch("bot_bottle.cli.start._select_image_policy", return_value="fresh").start()
|
||||
self.addCleanup(patch.stopall)
|
||||
|
||||
def test_no_collision_proceeds_without_reprompt(self):
|
||||
|
||||
@@ -1,164 +0,0 @@
|
||||
"""Unit: _launch_bottle StaleImageError handling.
|
||||
|
||||
Exercises the while-True / try-except loop around backend.launch:
|
||||
- headless mode → die on stale
|
||||
- interactive mode, user declines → stop without launching
|
||||
- interactive mode, user confirms → retry with skip_stale=True
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
import tempfile
|
||||
import unittest
|
||||
from types import SimpleNamespace
|
||||
from typing import Any, cast
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from bot_bottle.image_cache import StaleImageError
|
||||
from bot_bottle.log import Die
|
||||
|
||||
|
||||
def _fake_plan() -> Any:
|
||||
provision = SimpleNamespace(startup_args=())
|
||||
return cast(Any, SimpleNamespace(
|
||||
agent_provision=provision,
|
||||
agent_provider_template="claude",
|
||||
slug="dev-abc",
|
||||
))
|
||||
|
||||
|
||||
def _stale_cm() -> MagicMock:
|
||||
"""Return a context-manager mock whose __enter__ raises StaleImageError."""
|
||||
cm = MagicMock()
|
||||
cm.__enter__ = MagicMock(side_effect=StaleImageError("image is 5 day(s) old"))
|
||||
cm.__exit__ = MagicMock(return_value=False)
|
||||
return cm
|
||||
|
||||
|
||||
def _ok_cm(bottle: Any) -> MagicMock:
|
||||
"""Return a context-manager mock that yields `bottle`."""
|
||||
cm = MagicMock()
|
||||
cm.__enter__ = MagicMock(return_value=bottle)
|
||||
cm.__exit__ = MagicMock(return_value=False)
|
||||
return cm
|
||||
|
||||
|
||||
class TestLaunchBottleStaleHandling(unittest.TestCase):
|
||||
def setUp(self) -> None:
|
||||
self._tmp = tempfile.mkdtemp(prefix="cli-stale-test.")
|
||||
|
||||
def _spec(self) -> Any:
|
||||
from bot_bottle.backend import BottleSpec
|
||||
from bot_bottle.manifest import ManifestIndex
|
||||
idx = ManifestIndex.from_json_obj({
|
||||
"bottles": {"dev": {}},
|
||||
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
|
||||
})
|
||||
return BottleSpec(
|
||||
manifest=idx,
|
||||
agent_name="demo",
|
||||
copy_cwd=False,
|
||||
user_cwd=self._tmp,
|
||||
identity="dev-abc",
|
||||
)
|
||||
|
||||
def _run_launch(self, **patch_kwargs: Any) -> int:
|
||||
import bot_bottle.cli.start as start_mod
|
||||
spec = self._spec()
|
||||
with patch.object(start_mod, "prepare_with_preflight",
|
||||
return_value=(_fake_plan(), "dev-abc")), \
|
||||
patch.object(start_mod, "settle_state"), \
|
||||
patch.object(start_mod, "info"):
|
||||
return start_mod._launch_bottle(
|
||||
spec,
|
||||
dry_run=False,
|
||||
backend_name="docker",
|
||||
**patch_kwargs,
|
||||
)
|
||||
|
||||
def test_headless_stale_calls_die(self) -> None:
|
||||
"""In headless mode (assume_yes=True), a StaleImageError must call die()."""
|
||||
import bot_bottle.cli.start as start_mod
|
||||
|
||||
backend_mock = MagicMock()
|
||||
backend_mock.launch.return_value = _stale_cm()
|
||||
|
||||
with patch.object(start_mod, "get_bottle_backend", return_value=backend_mock), \
|
||||
patch.object(start_mod, "die", side_effect=Die()):
|
||||
with self.assertRaises(Die):
|
||||
self._run_launch(assume_yes=True)
|
||||
|
||||
def test_interactive_user_declines_stops_loop(self) -> None:
|
||||
"""Interactive user answering 'n' → loop exits without a second launch."""
|
||||
import bot_bottle.cli.start as start_mod
|
||||
|
||||
backend_mock = MagicMock()
|
||||
backend_mock.launch.return_value = _stale_cm()
|
||||
|
||||
with patch.object(start_mod, "get_bottle_backend", return_value=backend_mock), \
|
||||
patch.object(start_mod, "read_tty_line", return_value="n"), \
|
||||
patch("sys.stderr", new_callable=io.StringIO):
|
||||
rc = self._run_launch(assume_yes=False)
|
||||
|
||||
self.assertEqual(0, rc)
|
||||
# launch was called once (stale), then the user declined → no second call.
|
||||
backend_mock.launch.assert_called_once()
|
||||
|
||||
def test_interactive_user_confirms_retries_with_skip_stale(self) -> None:
|
||||
"""Interactive user answering 'y' → second launch with skip_stale=True."""
|
||||
import bot_bottle.cli.start as start_mod
|
||||
|
||||
bottle_mock = MagicMock()
|
||||
bottle_mock.name = "dev-abc"
|
||||
|
||||
def launch_side_effect(plan: Any, *, skip_stale: bool = False) -> Any:
|
||||
if not skip_stale:
|
||||
return _stale_cm()
|
||||
return _ok_cm(bottle_mock)
|
||||
|
||||
backend_mock = MagicMock()
|
||||
backend_mock.launch.side_effect = launch_side_effect
|
||||
|
||||
with patch.object(start_mod, "get_bottle_backend", return_value=backend_mock), \
|
||||
patch.object(start_mod, "read_tty_line", return_value="y"), \
|
||||
patch.object(start_mod, "attach_agent", return_value=0), \
|
||||
patch.object(start_mod, "capture_claude_session_state"), \
|
||||
patch("sys.stderr", new_callable=io.StringIO):
|
||||
rc = self._run_launch(assume_yes=False)
|
||||
|
||||
self.assertEqual(0, rc)
|
||||
# First call: skip_stale=False → stale. Second call: skip_stale=True → ok.
|
||||
self.assertEqual(2, backend_mock.launch.call_count)
|
||||
calls = backend_mock.launch.call_args_list
|
||||
self.assertFalse(calls[0].kwargs.get("skip_stale", False))
|
||||
self.assertTrue(calls[1].kwargs.get("skip_stale", False))
|
||||
|
||||
def test_interactive_yes_uppercase_also_accepted(self) -> None:
|
||||
"""'Y' or 'YES' should also be accepted as confirmation."""
|
||||
import bot_bottle.cli.start as start_mod
|
||||
|
||||
bottle_mock = MagicMock()
|
||||
bottle_mock.name = "dev-abc"
|
||||
|
||||
def launch_side_effect(plan: Any, *, skip_stale: bool = False) -> Any:
|
||||
if not skip_stale:
|
||||
return _stale_cm()
|
||||
return _ok_cm(bottle_mock)
|
||||
|
||||
backend_mock = MagicMock()
|
||||
backend_mock.launch.side_effect = launch_side_effect
|
||||
|
||||
with patch.object(start_mod, "get_bottle_backend", return_value=backend_mock), \
|
||||
patch.object(start_mod, "read_tty_line", return_value="YES"), \
|
||||
patch.object(start_mod, "attach_agent", return_value=0), \
|
||||
patch.object(start_mod, "capture_claude_session_state"), \
|
||||
patch("sys.stderr", new_callable=io.StringIO):
|
||||
rc = self._run_launch(assume_yes=False)
|
||||
|
||||
self.assertEqual(0, rc)
|
||||
self.assertEqual(2, backend_mock.launch.call_count)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -107,7 +107,7 @@ def _egress_plan(
|
||||
def _supervise_plan() -> SupervisePlan:
|
||||
return SupervisePlan(
|
||||
slug=SLUG,
|
||||
db_path=STATE / "bot-bottle.db",
|
||||
queue_dir=STATE / "supervise" / "queue",
|
||||
internal_network=f"bot-bottle-net-{SLUG}",
|
||||
)
|
||||
|
||||
@@ -118,7 +118,6 @@ def _plan(
|
||||
with_egress: bool = False,
|
||||
supervise: bool = False,
|
||||
canary: bool = False,
|
||||
image_policy: str = "fresh",
|
||||
) -> DockerBottlePlan:
|
||||
"""Build a fully-resolved DockerBottlePlan. Toggles cover the
|
||||
matrix the renderer's conditional-service logic branches on."""
|
||||
@@ -149,7 +148,6 @@ def _plan(
|
||||
agent_name="demo",
|
||||
copy_cwd=False,
|
||||
user_cwd="/tmp/x",
|
||||
image_policy=image_policy,
|
||||
)
|
||||
return DockerBottlePlan(
|
||||
spec=spec,
|
||||
@@ -267,8 +265,7 @@ class TestAgentAlwaysPresent(unittest.TestCase):
|
||||
def test_agent_depends_only_on_sidecars(self):
|
||||
# Bundle shape: the init supervisor owns intra-bundle daemon
|
||||
# ordering, so the agent waits on the bundle container alone.
|
||||
cases: list[dict[str, Any]] = [{}, {"with_git": True, "with_egress": True, "supervise": True}]
|
||||
for kwargs in cases:
|
||||
for kwargs in [{}, {"with_git": True, "with_egress": True, "supervise": True}]:
|
||||
with self.subTest(**kwargs):
|
||||
s = bottle_plan_to_compose(_plan(**kwargs))["services"]["agent"]
|
||||
self.assertEqual(["sidecars"], s["depends_on"])
|
||||
@@ -303,11 +300,6 @@ class TestSidecarBundleShape(unittest.TestCase):
|
||||
self.assertEqual("bot-bottle-sidecars:latest", sc["image"])
|
||||
self.assertEqual("Dockerfile.sidecars", sc["build"]["dockerfile"])
|
||||
|
||||
def test_cached_policy_omits_bundle_build(self):
|
||||
sc = self._render(image_policy="cached")["services"]["sidecars"]
|
||||
self.assertEqual("bot-bottle-sidecars:latest", sc["image"])
|
||||
self.assertNotIn("build", sc)
|
||||
|
||||
def test_bundle_container_name_uses_sidecars_prefix(self):
|
||||
sc = self._render()["services"]["sidecars"]
|
||||
self.assertEqual(f"bot-bottle-sidecars-{SLUG}", sc["container_name"])
|
||||
@@ -400,7 +392,7 @@ class TestSidecarBundleShape(unittest.TestCase):
|
||||
sc = self._render(supervise=True)["services"]["sidecars"]
|
||||
env_strings = sc["environment"]
|
||||
self.assertIn(f"SUPERVISE_BOTTLE_SLUG={SLUG}", env_strings)
|
||||
self.assertIn("SUPERVISE_DB_PATH=/run/supervise/bot-bottle.db", env_strings)
|
||||
self.assertTrue(any(e.startswith("SUPERVISE_QUEUE_DIR=") for e in env_strings))
|
||||
self.assertTrue(any(e.startswith("SUPERVISE_PORT=") for e in env_strings))
|
||||
|
||||
def test_volumes_always_includes_egress_ca(self):
|
||||
@@ -416,7 +408,8 @@ class TestSidecarBundleShape(unittest.TestCase):
|
||||
self.assertIn("/etc/egress", targets)
|
||||
self.assertIn("/git-gate-entrypoint.sh", targets)
|
||||
self.assertIn("/git-gate/creds/upstream-known_hosts", targets)
|
||||
self.assertIn("/run/supervise/bot-bottle.db", targets)
|
||||
self.assertTrue(any("supervise/queue" in t or t.startswith("/run/supervise")
|
||||
for t in targets))
|
||||
|
||||
def test_extra_hosts_omitted_for_git_upstreams(self):
|
||||
sc = self._render(with_git=True)["services"]["sidecars"]
|
||||
|
||||
@@ -1,86 +0,0 @@
|
||||
"""Unit tests for the host-side configuration store."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sqlite3
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
from bot_bottle.config_store import (
|
||||
DEFAULT_CACHED_IMAGE_STALE_WARNING_DAYS,
|
||||
ConfigStore,
|
||||
)
|
||||
from bot_bottle.store_manager import StoreManager
|
||||
|
||||
|
||||
class TestConfigStore(unittest.TestCase):
|
||||
def test_cached_image_warning_days_defaults_to_one(self) -> None:
|
||||
with tempfile.TemporaryDirectory(prefix="config-store.") as tmp:
|
||||
store = ConfigStore(Path(tmp) / "bot-bottle.db")
|
||||
store.migrate()
|
||||
self.assertEqual(
|
||||
DEFAULT_CACHED_IMAGE_STALE_WARNING_DAYS,
|
||||
store.cached_image_stale_warning_days(),
|
||||
)
|
||||
|
||||
def test_cached_image_warning_days_reads_value(self) -> None:
|
||||
with tempfile.TemporaryDirectory(prefix="config-store.") as tmp:
|
||||
store = ConfigStore(Path(tmp) / "bot-bottle.db")
|
||||
store.migrate()
|
||||
store.set_cached_image_stale_warning_days(7)
|
||||
self.assertEqual(7, store.cached_image_stale_warning_days())
|
||||
|
||||
def test_config_schema_uses_explicit_settings_columns(self) -> None:
|
||||
with tempfile.TemporaryDirectory(prefix="config-store.") as tmp:
|
||||
store = ConfigStore(Path(tmp) / "bot-bottle.db")
|
||||
store.migrate()
|
||||
with sqlite3.connect(store.db_path) as conn:
|
||||
conn.row_factory = sqlite3.Row
|
||||
columns = [
|
||||
row["name"]
|
||||
for row in conn.execute("PRAGMA table_info(bot_bottle_config)")
|
||||
]
|
||||
self.assertEqual([
|
||||
"id",
|
||||
"cached_image_stale_warning_days",
|
||||
], columns)
|
||||
|
||||
def test_store_manager_includes_config_store(self) -> None:
|
||||
with tempfile.TemporaryDirectory(prefix="config-store.") as tmp:
|
||||
db = Path(tmp) / "bot-bottle.db"
|
||||
manager = StoreManager(db)
|
||||
self.assertFalse(manager.is_migrated())
|
||||
manager.migrate()
|
||||
self.assertTrue(manager.is_migrated())
|
||||
|
||||
def test_cached_image_warning_days_returns_default_when_db_missing(self) -> None:
|
||||
# When the db file doesn't exist yet (parent exists, file doesn't),
|
||||
# the store returns the default without touching the file.
|
||||
with tempfile.TemporaryDirectory(prefix="config-store.") as tmp:
|
||||
store = ConfigStore(Path(tmp) / "missing.db")
|
||||
self.assertEqual(
|
||||
DEFAULT_CACHED_IMAGE_STALE_WARNING_DAYS,
|
||||
store.cached_image_stale_warning_days(),
|
||||
)
|
||||
|
||||
def test_cached_image_warning_days_returns_default_on_null_value(self) -> None:
|
||||
# If the row exists but the value is NULL (or not castable to int),
|
||||
# the store falls back to the default.
|
||||
with tempfile.TemporaryDirectory(prefix="config-store.") as tmp:
|
||||
db_path = Path(tmp) / "bot-bottle.db"
|
||||
store = ConfigStore(db_path)
|
||||
store.migrate()
|
||||
# Write a NULL value directly.
|
||||
with sqlite3.connect(db_path) as conn:
|
||||
conn.execute(
|
||||
"UPDATE bot_bottle_config SET cached_image_stale_warning_days = NULL WHERE id = 1"
|
||||
)
|
||||
self.assertEqual(
|
||||
DEFAULT_CACHED_IMAGE_STALE_WARNING_DAYS,
|
||||
store.cached_image_stale_warning_days(),
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -74,7 +74,7 @@ def _plan(
|
||||
if supervise:
|
||||
supervise_plan = SupervisePlan(
|
||||
slug="demo-abc12",
|
||||
db_path=Path("/tmp/bot-bottle.db"),
|
||||
queue_dir=Path("/tmp/queue"),
|
||||
)
|
||||
return DockerBottlePlan(
|
||||
spec=spec,
|
||||
@@ -343,14 +343,5 @@ class TestClaudeSuperviseMcp(unittest.TestCase):
|
||||
)
|
||||
|
||||
|
||||
class TestClaudeHeadlessPrompt(unittest.TestCase):
|
||||
def test_returns_p_flag_and_prompt(self):
|
||||
self.assertEqual(["-p", "Do the task"], ClaudeAgentProvider().headless_prompt("Do the task"))
|
||||
|
||||
def test_preserves_prompt_text_verbatim(self):
|
||||
text = "Fix issue #42: the widget breaks on empty input"
|
||||
self.assertEqual(["-p", text], ClaudeAgentProvider().headless_prompt(text))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -77,7 +77,7 @@ def _plan(
|
||||
if supervise:
|
||||
supervise_plan = SupervisePlan(
|
||||
slug="demo-abc12",
|
||||
db_path=Path("/tmp/bot-bottle.db"),
|
||||
queue_dir=Path("/tmp/queue"),
|
||||
)
|
||||
return DockerBottlePlan(
|
||||
spec=spec,
|
||||
@@ -261,36 +261,6 @@ class TestCodexProvision(unittest.TestCase):
|
||||
self.assertTrue(any("find" in s and "-delete" in s for s in scripts))
|
||||
self.assertTrue(any("runuser" in s and "codex login status" in s for s in scripts))
|
||||
|
||||
def test_forwarded_credentials_verify_sets_codex_path(self):
|
||||
with tempfile.TemporaryDirectory(prefix="bb-codex-auth.") as tmp:
|
||||
state_dir = Path(tmp)
|
||||
prompt_file = state_dir / "prompt.txt"
|
||||
prompt_file.write_text("")
|
||||
with patch(
|
||||
"bot_bottle.contrib.codex.agent_provider.codex_host_access_token",
|
||||
return_value="token",
|
||||
), patch(
|
||||
"bot_bottle.contrib.codex.agent_provider.write_codex_dummy_auth_file",
|
||||
):
|
||||
provision = CodexAgentProvider().provision_plan(
|
||||
dockerfile="",
|
||||
state_dir=state_dir,
|
||||
instance_name="bot-bottle-demo-abc12",
|
||||
prompt_file=prompt_file,
|
||||
forward_host_credentials=True,
|
||||
host_env={},
|
||||
)
|
||||
|
||||
verify_argv = provision.verify[0].argv
|
||||
self.assertTrue(any(
|
||||
item.startswith("PATH=/home/node/.local/bin:")
|
||||
for item in verify_argv
|
||||
))
|
||||
self.assertIn(
|
||||
"/home/node/.codex/packages/standalone/current/bin/codex",
|
||||
verify_argv,
|
||||
)
|
||||
|
||||
def test_dies_when_dir_creation_fails(self):
|
||||
provision = AgentProvisionPlan(
|
||||
template="codex", command="codex",
|
||||
@@ -331,8 +301,7 @@ class TestCodexSuperviseMcp(unittest.TestCase):
|
||||
script = bottle.exec.call_args.args[0]
|
||||
self.assertEqual("node", bottle.exec.call_args.kwargs.get("user"))
|
||||
self.assertEqual(
|
||||
"/home/node/.codex/packages/standalone/current/bin/codex "
|
||||
f"mcp add supervise --url {_URL}",
|
||||
f"codex mcp add supervise --url {_URL}",
|
||||
script,
|
||||
)
|
||||
|
||||
@@ -345,14 +314,5 @@ class TestCodexSuperviseMcp(unittest.TestCase):
|
||||
)
|
||||
|
||||
|
||||
class TestCodexHeadlessPrompt(unittest.TestCase):
|
||||
def test_returns_prompt_as_positional_arg(self):
|
||||
self.assertEqual(["Do the task"], CodexAgentProvider().headless_prompt("Do the task"))
|
||||
|
||||
def test_preserves_prompt_text_verbatim(self):
|
||||
text = "Fix issue #42: the widget breaks on empty input"
|
||||
self.assertEqual([text], CodexAgentProvider().headless_prompt(text))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -223,14 +223,5 @@ class TestPiDockerfile(unittest.TestCase):
|
||||
self.assertIn("chmod 1777 /tmp /var/tmp", dockerfile)
|
||||
|
||||
|
||||
class TestPiHeadlessPrompt(unittest.TestCase):
|
||||
def test_returns_p_flag_and_prompt(self):
|
||||
self.assertEqual(["-p", "Do the task"], PiAgentProvider().headless_prompt("Do the task"))
|
||||
|
||||
def test_preserves_prompt_text_verbatim(self):
|
||||
text = "Fix issue #42: the widget breaks on empty input"
|
||||
self.assertEqual(["-p", text], PiAgentProvider().headless_prompt(text))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -3,7 +3,6 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import dataclasses
|
||||
import io
|
||||
import tempfile
|
||||
import unittest
|
||||
@@ -17,7 +16,6 @@ from bot_bottle.backend.docker import launch as launch_mod
|
||||
from bot_bottle.backend.docker.bottle_plan import DockerBottlePlan
|
||||
from bot_bottle.egress import EgressPlan
|
||||
from bot_bottle.git_gate import GitGatePlan
|
||||
from bot_bottle.log import Die
|
||||
from bot_bottle.manifest import ManifestIndex
|
||||
|
||||
|
||||
@@ -182,24 +180,6 @@ class TestLaunchCommittedImage(unittest.TestCase):
|
||||
built = self._run_launch(plan, committed_tag=None)
|
||||
self.assertEqual([_DEFAULT_IMAGE], built)
|
||||
|
||||
def test_cached_images_skip_build_when_present(self) -> None:
|
||||
base = _plan(self._tmp)
|
||||
plan = dataclasses.replace(
|
||||
base,
|
||||
spec=dataclasses.replace(base.spec, image_policy="cached"),
|
||||
)
|
||||
built = self._run_launch(plan, committed_tag=None, image_present=True)
|
||||
self.assertEqual([], built)
|
||||
|
||||
def test_cached_images_die_when_agent_missing(self) -> None:
|
||||
base = _plan(self._tmp)
|
||||
plan = dataclasses.replace(
|
||||
base,
|
||||
spec=dataclasses.replace(base.spec, image_policy="cached"),
|
||||
)
|
||||
with self.assertRaises(Die):
|
||||
self._run_launch(plan, committed_tag=None, image_present=False)
|
||||
|
||||
def test_falls_back_to_build_when_committed_image_missing_from_daemon(self) -> None:
|
||||
plan = _plan(self._tmp)
|
||||
built = self._run_launch(
|
||||
|
||||
@@ -38,7 +38,6 @@ class _Provider(AgentProvider):
|
||||
def provision_prompt(self, plan, bottle): ... # type: ignore[override]
|
||||
def provision(self, plan, bottle): ... # type: ignore[override]
|
||||
def provision_supervise_mcp(self, plan, bottle, supervise_url): ... # type: ignore[override]
|
||||
def headless_prompt(self, prompt): return [] # type: ignore[override]
|
||||
|
||||
|
||||
_PROVIDER = _Provider()
|
||||
|
||||
@@ -9,7 +9,6 @@ from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
import unittest
|
||||
from datetime import timezone
|
||||
from unittest.mock import patch
|
||||
|
||||
from bot_bottle.backend.docker import util as docker_mod
|
||||
@@ -55,51 +54,6 @@ class TestImageId(unittest.TestCase):
|
||||
self.assertIn("missing:tag", die.call_args.args[0])
|
||||
|
||||
|
||||
class TestImageCreatedAt(unittest.TestCase):
|
||||
def test_parses_docker_timestamp_with_nanoseconds(self):
|
||||
with patch.object(
|
||||
docker_mod.subprocess, "run",
|
||||
return_value=_ok(stdout="2026-07-06T15:33:47.123456789Z\n"),
|
||||
) as run:
|
||||
created = docker_mod.image_created_at("bot-bottle-claude:latest")
|
||||
self.assertEqual(2026, created.year)
|
||||
self.assertEqual(123456, created.microsecond)
|
||||
self.assertEqual(timezone.utc, created.tzinfo)
|
||||
self.assertEqual(
|
||||
["docker", "image", "inspect", "--format", "{{.Created}}", "bot-bottle-claude:latest"],
|
||||
run.call_args.args[0],
|
||||
)
|
||||
|
||||
def test_dies_on_inspect_failure(self):
|
||||
with patch.object(
|
||||
docker_mod.subprocess, "run", return_value=_fail("No such image"),
|
||||
), patch.object(
|
||||
docker_mod, "die", side_effect=SystemExit("die"),
|
||||
) as die:
|
||||
with self.assertRaises(SystemExit):
|
||||
docker_mod.image_created_at("missing:tag")
|
||||
die.assert_called_once()
|
||||
self.assertIn("missing:tag", die.call_args.args[0])
|
||||
|
||||
def test_dies_on_invalid_timestamp(self):
|
||||
with patch.object(
|
||||
docker_mod.subprocess, "run",
|
||||
return_value=_ok(stdout="not-a-timestamp\n"),
|
||||
), patch.object(
|
||||
docker_mod, "die", side_effect=SystemExit("die"),
|
||||
) as die:
|
||||
with self.assertRaises(SystemExit):
|
||||
docker_mod.image_created_at("some:tag")
|
||||
die.assert_called_once()
|
||||
self.assertIn("some:tag", die.call_args.args[0])
|
||||
|
||||
def test_parse_docker_timestamp_no_tzinfo_defaults_to_utc(self):
|
||||
# A bare datetime with no tz offset should be treated as UTC.
|
||||
dt = docker_mod._parse_docker_timestamp("2024-05-01T10:00:00.000000")
|
||||
self.assertIsNotNone(dt.tzinfo)
|
||||
self.assertEqual(timezone.utc, dt.tzinfo)
|
||||
|
||||
|
||||
class TestSave(unittest.TestCase):
|
||||
def test_save_runs_docker_save(self):
|
||||
with patch.object(
|
||||
|
||||
@@ -19,7 +19,7 @@ from bot_bottle.egress import (
|
||||
egress_sidecar_env_entries,
|
||||
egress_token_env_map,
|
||||
)
|
||||
from bot_bottle.errors import MissingEnvVarError
|
||||
from bot_bottle.log import Die
|
||||
from bot_bottle.manifest import ManifestIndex
|
||||
from bot_bottle.yaml_subset import parse_yaml_subset
|
||||
|
||||
@@ -499,14 +499,14 @@ class TestResolveTokenValues(unittest.TestCase):
|
||||
self.assertEqual({"EGRESS_TOKEN_0": "the-value"}, out)
|
||||
|
||||
def test_missing_token_ref_dies(self):
|
||||
with self.assertRaises(MissingEnvVarError):
|
||||
with self.assertRaises(Die):
|
||||
egress_resolve_token_values(
|
||||
{"EGRESS_TOKEN_0": "GH_PAT"},
|
||||
{},
|
||||
)
|
||||
|
||||
def test_empty_token_ref_dies(self):
|
||||
with self.assertRaises(MissingEnvVarError):
|
||||
with self.assertRaises(Die):
|
||||
egress_resolve_token_values(
|
||||
{"EGRESS_TOKEN_0": "GH_PAT"},
|
||||
{"GH_PAT": ""},
|
||||
|
||||
@@ -47,6 +47,7 @@ def _addon() -> EgressAddon:
|
||||
a: EgressAddon = EgressAddon.__new__(EgressAddon)
|
||||
a.config = Config(routes=(), log=LOG_FULL)
|
||||
a.safe_tokens = set()
|
||||
a._supervise_queue_dir = ""
|
||||
a._supervise_slug = ""
|
||||
a._token_allow_timeout = 300.0
|
||||
return a
|
||||
|
||||
@@ -212,6 +212,7 @@ def _addon(config: Config) -> EgressAddon:
|
||||
a: EgressAddon = EgressAddon.__new__(EgressAddon)
|
||||
a.config = config
|
||||
a.safe_tokens = set()
|
||||
a._supervise_queue_dir = ""
|
||||
a._supervise_slug = ""
|
||||
a._token_allow_timeout = 300.0
|
||||
a.routes_path = "/nonexistent/routes.yaml"
|
||||
@@ -385,10 +386,10 @@ def _fake_sv(response_status: str | None) -> types.SimpleNamespace:
|
||||
def _sha256_hex(_payload: Any) -> str:
|
||||
return "hash"
|
||||
|
||||
def _noop(*_args: Any) -> None:
|
||||
def _noop(_a: Any, _b: Any) -> None:
|
||||
return None
|
||||
|
||||
def _read_response(_slug: Any, _pid: Any) -> Any:
|
||||
def _read_response(_qd: Any, _pid: Any) -> Any:
|
||||
if response_status is None:
|
||||
raise OSError("not written yet") # forces poll -> timeout
|
||||
return types.SimpleNamespace(status=response_status)
|
||||
@@ -408,6 +409,7 @@ def _fake_sv(response_status: str | None) -> types.SimpleNamespace:
|
||||
class TestSuperviseBranch(unittest.TestCase):
|
||||
def _supervised_addon(self) -> EgressAddon:
|
||||
addon = _addon(Config(routes=(Route(host="api.example.com"),)))
|
||||
addon._supervise_queue_dir = "/tmp/egress-queue"
|
||||
addon._supervise_slug = "test-bottle"
|
||||
addon._token_allow_timeout = 0.05
|
||||
return addon
|
||||
@@ -630,13 +632,14 @@ class TestRedactSurfaces(unittest.TestCase):
|
||||
class TestSuperviseWriteFailure(unittest.TestCase):
|
||||
def test_write_proposal_oserror_blocks(self) -> None:
|
||||
addon = _addon(Config(routes=(Route(host="api.example.com"),)))
|
||||
addon._supervise_queue_dir = "/tmp/egress-queue"
|
||||
addon._supervise_slug = "test-bottle"
|
||||
addon._token_allow_timeout = 0.05
|
||||
flow = _Flow(_Request(host="api.example.com", method="POST", body=f"k={_OPENAI_KEY}"))
|
||||
|
||||
fake = _fake_sv("approved")
|
||||
|
||||
def _raise(_p: Any) -> None:
|
||||
def _raise(_qd: Any, _p: Any) -> None:
|
||||
raise OSError("disk full")
|
||||
|
||||
fake.write_proposal = _raise
|
||||
|
||||
@@ -14,12 +14,10 @@ from bot_bottle.git_gate import (
|
||||
git_gate_render_access_hook,
|
||||
git_gate_render_entrypoint,
|
||||
git_gate_render_hook,
|
||||
provision_git_gate_dynamic_keys,
|
||||
revoke_git_gate_provisioned_keys,
|
||||
_resolve_identity_file,
|
||||
git_gate_upstreams_for_bottle,
|
||||
)
|
||||
from bot_bottle.errors import MissingEnvVarError
|
||||
from bot_bottle.manifest import ManifestIndex
|
||||
from tests.fixtures import fixture_minimal, fixture_with_git
|
||||
|
||||
@@ -211,23 +209,12 @@ class TestHookRender(unittest.TestCase):
|
||||
# the suppressed findings for human approval.
|
||||
self.assertIn("--ignore-gitleaks-allow", hook)
|
||||
self.assertIn("--report-format=json", hook)
|
||||
self.assertIn("tool=_sv.TOOL_GITLEAKS_ALLOW", hook)
|
||||
self.assertIn("_sv.write_proposal", hook)
|
||||
self.assertIn("_sv.read_response", hook)
|
||||
self.assertIn('"tool": "gitleaks-allow"', hook)
|
||||
self.assertIn("SUPERVISE_QUEUE_DIR", hook)
|
||||
self.assertIn("SUPERVISE_BOTTLE_SLUG", hook)
|
||||
self.assertIn("supervisor approved # gitleaks:allow", hook)
|
||||
self.assertIn("supervisor rejected # gitleaks:allow", hook)
|
||||
|
||||
def test_inline_gitleaks_allow_python_imports_work_in_sidecar_layout(self):
|
||||
hook = git_gate_render_hook()
|
||||
# The sidecar image copies supervise.py flat under /app, while
|
||||
# host-side tests import it through the bot_bottle package.
|
||||
# Hooks execute from the bare repo directory, so the embedded
|
||||
# Python must include /app and support both import layouts.
|
||||
self.assertIn('PYTHONPATH="/app${PYTHONPATH:+:$PYTHONPATH}"', hook)
|
||||
self.assertIn("import supervise as _sv", hook)
|
||||
self.assertIn("from bot_bottle import supervise as _sv", hook)
|
||||
|
||||
def test_inline_gitleaks_allow_fails_closed_without_supervisor(self):
|
||||
hook = git_gate_render_hook()
|
||||
self.assertIn(
|
||||
@@ -384,27 +371,6 @@ class TestDynamicKeyProvisioning(unittest.TestCase):
|
||||
self.assertEqual("/tmp/provisioned-key", _resolve_identity_file(entry, "demo", self.stage))
|
||||
mock_provision.assert_called_once()
|
||||
|
||||
def test_prepare_defers_gitea_key_provisioning(self):
|
||||
bottle = self._gitea_manifest().bottles["dev"]
|
||||
with patch("bot_bottle.git_gate_provision._provision_dynamic_key") as mock_provision:
|
||||
plan = _StubGate().prepare(bottle, "demo", self.stage)
|
||||
|
||||
mock_provision.assert_not_called()
|
||||
self.assertEqual("", plan.upstreams[0].identity_file)
|
||||
|
||||
def test_launch_time_helper_provisions_gitea_keys(self):
|
||||
bottle = self._gitea_manifest().bottles["dev"]
|
||||
plan = _StubGate().prepare(bottle, "demo", self.stage)
|
||||
|
||||
with patch(
|
||||
"bot_bottle.git_gate_provision._provision_dynamic_key",
|
||||
return_value="/tmp/provisioned-key",
|
||||
) as mock_provision:
|
||||
updated = provision_git_gate_dynamic_keys(bottle, plan, self.stage)
|
||||
|
||||
mock_provision.assert_called_once_with(bottle.git[0], "demo", self.stage)
|
||||
self.assertEqual("/tmp/provisioned-key", updated.upstreams[0].identity_file)
|
||||
|
||||
def test_revoke_skips_non_gitea_and_missing_id_file(self):
|
||||
revoke_git_gate_provisioned_keys(fixture_with_git().bottles["dev"], self.stage)
|
||||
|
||||
@@ -422,7 +388,7 @@ class TestDynamicKeyProvisioning(unittest.TestCase):
|
||||
def test_revoke_missing_token_raises(self):
|
||||
bottle = self._gitea_manifest().bottles["dev"]
|
||||
(self.stage / "repo-deploy-key-id").write_text("123\n")
|
||||
with patch.dict("os.environ", {}, clear=True), self.assertRaises(MissingEnvVarError) as cm:
|
||||
with patch.dict("os.environ", {}, clear=True), self.assertRaises(RuntimeError) as cm:
|
||||
revoke_git_gate_provisioned_keys(bottle, self.stage)
|
||||
self.assertIn("env var is not set", str(cm.exception))
|
||||
|
||||
|
||||
@@ -1,698 +0,0 @@
|
||||
"""Unit tests for git_gate_host_key (issue #333)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from bot_bottle.git_gate_host_key import (
|
||||
add_host_key_to_frontmatter,
|
||||
find_and_update_bottle_file,
|
||||
find_repo_bottle_file,
|
||||
prompt_tty,
|
||||
fetch_host_key,
|
||||
preflight_host_keys,
|
||||
)
|
||||
from bot_bottle.manifest import Manifest, ManifestIndex
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# fetch_host_key
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestFetchHostKey(unittest.TestCase):
|
||||
def _run_result(self, stdout: str, returncode: int = 0) -> MagicMock:
|
||||
r = MagicMock()
|
||||
r.stdout = stdout
|
||||
r.stderr = ""
|
||||
r.returncode = returncode
|
||||
return r
|
||||
|
||||
def test_returns_type_and_key(self) -> None:
|
||||
stdout = "gitea.example.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAA\n"
|
||||
with patch("subprocess.run", return_value=self._run_result(stdout)):
|
||||
key = fetch_host_key("gitea.example.com", "22")
|
||||
self.assertEqual("ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAA", key)
|
||||
|
||||
def test_non_default_port_passes_p_flag(self) -> None:
|
||||
stdout = "[gitea.example.com]:30009 ssh-ed25519 AAAA\n"
|
||||
with patch("subprocess.run", return_value=self._run_result(stdout)) as mock_run:
|
||||
fetch_host_key("gitea.example.com", "30009")
|
||||
args = mock_run.call_args[0][0]
|
||||
self.assertIn("-p", args)
|
||||
self.assertIn("30009", args)
|
||||
|
||||
def test_default_port_omits_p_flag(self) -> None:
|
||||
stdout = "github.com ssh-ed25519 AAAA\n"
|
||||
with patch("subprocess.run", return_value=self._run_result(stdout)) as mock_run:
|
||||
fetch_host_key("github.com", "22")
|
||||
args = mock_run.call_args[0][0]
|
||||
self.assertNotIn("-p", args)
|
||||
|
||||
def test_skips_comment_lines(self) -> None:
|
||||
stdout = "# comment\ngithub.com ssh-ed25519 AAAA\n"
|
||||
with patch("subprocess.run", return_value=self._run_result(stdout)):
|
||||
key = fetch_host_key("github.com", "22")
|
||||
self.assertEqual("ssh-ed25519 AAAA", key)
|
||||
|
||||
def test_skips_lines_without_three_parts(self) -> None:
|
||||
# A line with only two whitespace-separated tokens is not a valid
|
||||
# known_hosts entry; it should be skipped and the function should
|
||||
# raise rather than return a partial result.
|
||||
stdout = "malformed-line\ngithub.com ssh-ed25519 AAAA\n"
|
||||
with patch("subprocess.run", return_value=self._run_result(stdout)):
|
||||
key = fetch_host_key("github.com", "22")
|
||||
self.assertEqual("ssh-ed25519 AAAA", key)
|
||||
|
||||
def test_raises_on_empty_output(self) -> None:
|
||||
with patch("subprocess.run", return_value=self._run_result("")):
|
||||
with self.assertRaises(RuntimeError) as cm:
|
||||
fetch_host_key("example.com", "22")
|
||||
self.assertIn("no host key", str(cm.exception))
|
||||
|
||||
def test_prefers_ed25519_over_ecdsa(self) -> None:
|
||||
stdout = (
|
||||
"gitea.example.com ecdsa-sha2-nistp256 BBBBB\n"
|
||||
"gitea.example.com ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAA\n"
|
||||
)
|
||||
with patch("subprocess.run", return_value=self._run_result(stdout)):
|
||||
key = fetch_host_key("gitea.example.com", "22")
|
||||
self.assertEqual("ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAA", key)
|
||||
|
||||
def test_falls_back_to_ecdsa_without_ed25519(self) -> None:
|
||||
stdout = "gitea.example.com ecdsa-sha2-nistp256 BBBBB\n"
|
||||
with patch("subprocess.run", return_value=self._run_result(stdout)):
|
||||
key = fetch_host_key("gitea.example.com", "22")
|
||||
self.assertEqual("ecdsa-sha2-nistp256 BBBBB", key)
|
||||
|
||||
def test_falls_back_to_rsa_without_ed25519_or_ecdsa(self) -> None:
|
||||
stdout = "gitea.example.com ssh-rsa AAAAB3NzaC1yc2EAAAA\n"
|
||||
with patch("subprocess.run", return_value=self._run_result(stdout)):
|
||||
key = fetch_host_key("gitea.example.com", "22")
|
||||
self.assertEqual("ssh-rsa AAAAB3NzaC1yc2EAAAA", key)
|
||||
|
||||
def test_falls_back_to_first_for_unknown_type(self) -> None:
|
||||
stdout = "gitea.example.com ssh-unknown CCCCC\n"
|
||||
with patch("subprocess.run", return_value=self._run_result(stdout)):
|
||||
key = fetch_host_key("gitea.example.com", "22")
|
||||
self.assertEqual("ssh-unknown CCCCC", key)
|
||||
|
||||
def test_raises_includes_stderr_when_present(self) -> None:
|
||||
r = self._run_result("")
|
||||
r.stderr = "connection refused"
|
||||
with patch("subprocess.run", return_value=r):
|
||||
with self.assertRaises(RuntimeError) as cm:
|
||||
fetch_host_key("example.com", "22")
|
||||
self.assertIn("connection refused", str(cm.exception))
|
||||
|
||||
def test_raises_on_os_error(self) -> None:
|
||||
with patch("subprocess.run", side_effect=OSError("not found")):
|
||||
with self.assertRaises(RuntimeError) as cm:
|
||||
fetch_host_key("example.com", "22")
|
||||
self.assertIn("could not launch", str(cm.exception))
|
||||
|
||||
def test_raises_on_timeout(self) -> None:
|
||||
import subprocess
|
||||
with patch("subprocess.run", side_effect=subprocess.TimeoutExpired("ssh-keyscan", 15)):
|
||||
with self.assertRaises(RuntimeError) as cm:
|
||||
fetch_host_key("example.com", "22")
|
||||
self.assertIn("timed out", str(cm.exception))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# prompt_tty
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestPromptTty(unittest.TestCase):
|
||||
def test_reads_from_tty_when_available(self) -> None:
|
||||
import io
|
||||
fake_tty = io.StringIO("yes\n")
|
||||
with patch("builtins.open", return_value=fake_tty), patch("sys.stderr"):
|
||||
result = prompt_tty("question: ")
|
||||
self.assertEqual("yes", result)
|
||||
|
||||
def test_falls_back_to_stdin_on_os_error(self) -> None:
|
||||
import io
|
||||
with patch("builtins.open", side_effect=OSError("no tty")), \
|
||||
patch("sys.stdin", io.StringIO("fallback\n")), \
|
||||
patch("sys.stderr"):
|
||||
result = prompt_tty("question: ")
|
||||
self.assertEqual("fallback", result)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# add_host_key_to_frontmatter
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestAddHostKeyToFrontmatter(unittest.TestCase):
|
||||
_FILE = """\
|
||||
---
|
||||
git-gate:
|
||||
repos:
|
||||
myrepo:
|
||||
url: ssh://git@host/org/repo.git
|
||||
key:
|
||||
provider: static
|
||||
path: /dev/null
|
||||
---
|
||||
# Body text here
|
||||
"""
|
||||
|
||||
def test_inserts_host_key_at_repo_level(self) -> None:
|
||||
result = add_host_key_to_frontmatter(self._FILE, "myrepo", "ssh-ed25519 AAAA")
|
||||
self.assertIn("host_key: ssh-ed25519 AAAA", result)
|
||||
# Must be a sibling of `url` and `key`, not nested inside `key`
|
||||
lines = result.splitlines()
|
||||
hk_line = next(l for l in lines if "host_key" in l)
|
||||
url_line = next(l for l in lines if "url:" in l)
|
||||
self.assertEqual(
|
||||
len(hk_line) - len(hk_line.lstrip()),
|
||||
len(url_line) - len(url_line.lstrip()),
|
||||
)
|
||||
|
||||
def test_preserves_body(self) -> None:
|
||||
result = add_host_key_to_frontmatter(self._FILE, "myrepo", "ssh-ed25519 AAAA")
|
||||
self.assertIn("# Body text here", result)
|
||||
|
||||
def test_preserves_other_repos(self) -> None:
|
||||
text = """\
|
||||
---
|
||||
git-gate:
|
||||
repos:
|
||||
first:
|
||||
url: ssh://git@host/org/first.git
|
||||
key:
|
||||
provider: static
|
||||
path: /dev/null
|
||||
second:
|
||||
url: ssh://git@host/org/second.git
|
||||
key:
|
||||
provider: static
|
||||
path: /dev/null
|
||||
---
|
||||
"""
|
||||
result = add_host_key_to_frontmatter(text, "first", "ssh-ed25519 AAAA")
|
||||
self.assertIn("host_key: ssh-ed25519 AAAA", result)
|
||||
self.assertIn("second:", result)
|
||||
|
||||
def test_no_change_when_host_key_already_present(self) -> None:
|
||||
text = self._FILE.replace(
|
||||
" path: /dev/null\n",
|
||||
" path: /dev/null\n host_key: ssh-ed25519 EXISTING\n",
|
||||
)
|
||||
result = add_host_key_to_frontmatter(text, "myrepo", "ssh-ed25519 NEW")
|
||||
self.assertNotIn("NEW", result)
|
||||
|
||||
def test_no_change_when_repo_not_found(self) -> None:
|
||||
result = add_host_key_to_frontmatter(self._FILE, "other", "ssh-ed25519 AAAA")
|
||||
self.assertEqual(self._FILE, result)
|
||||
|
||||
def test_no_change_without_frontmatter(self) -> None:
|
||||
text = "No frontmatter here\n"
|
||||
result = add_host_key_to_frontmatter(text, "myrepo", "ssh-ed25519 AAAA")
|
||||
self.assertEqual(text, result)
|
||||
|
||||
def test_no_change_when_no_closing_delimiter(self) -> None:
|
||||
text = "---\ngit-gate:\n repos:\n myrepo:\n url: x\n"
|
||||
result = add_host_key_to_frontmatter(text, "myrepo", "ssh-ed25519 AAAA")
|
||||
self.assertEqual(text, result)
|
||||
|
||||
def test_no_change_when_git_gate_missing(self) -> None:
|
||||
text = "---\nenv:\n FOO: bar\n---\n"
|
||||
result = add_host_key_to_frontmatter(text, "myrepo", "ssh-ed25519 AAAA")
|
||||
self.assertEqual(text, result)
|
||||
|
||||
def test_no_change_when_repos_missing(self) -> None:
|
||||
text = "---\ngit-gate:\n foo: bar\n---\n"
|
||||
result = add_host_key_to_frontmatter(text, "myrepo", "ssh-ed25519 AAAA")
|
||||
self.assertEqual(text, result)
|
||||
|
||||
def test_no_change_when_repo_entry_not_a_dict(self) -> None:
|
||||
text = "---\ngit-gate:\n repos:\n - item\n---\n"
|
||||
result = add_host_key_to_frontmatter(text, "myrepo", "ssh-ed25519 AAAA")
|
||||
self.assertEqual(text, result)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# find_and_update_bottle_file
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestFindAndUpdateBottleFile(unittest.TestCase):
|
||||
def _write_bottle(self, d: str, name: str, content: str) -> Path:
|
||||
path = Path(d) / f"{name}.md"
|
||||
path.write_text(content, encoding="utf-8")
|
||||
return path
|
||||
|
||||
def test_updates_file_with_matching_repo(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
content = """\
|
||||
---
|
||||
git-gate:
|
||||
repos:
|
||||
myrepo:
|
||||
url: ssh://git@host/org/repo.git
|
||||
key:
|
||||
provider: static
|
||||
path: /dev/null
|
||||
---
|
||||
"""
|
||||
path = self._write_bottle(d, "dev", content)
|
||||
result = find_and_update_bottle_file(Path(d), "myrepo", "ssh-ed25519 AAAA")
|
||||
self.assertTrue(result)
|
||||
updated = path.read_text()
|
||||
self.assertIn("host_key: ssh-ed25519 AAAA", updated)
|
||||
|
||||
def test_returns_false_when_no_matching_file(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
content = """\
|
||||
---
|
||||
git-gate:
|
||||
repos:
|
||||
other:
|
||||
url: ssh://git@host/org/other.git
|
||||
key:
|
||||
provider: static
|
||||
path: /dev/null
|
||||
---
|
||||
"""
|
||||
self._write_bottle(d, "dev", content)
|
||||
result = find_and_update_bottle_file(Path(d), "myrepo", "ssh-ed25519 AAAA")
|
||||
self.assertFalse(result)
|
||||
|
||||
def test_skips_file_that_already_has_host_key(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
content = """\
|
||||
---
|
||||
git-gate:
|
||||
repos:
|
||||
myrepo:
|
||||
url: ssh://git@host/org/repo.git
|
||||
key:
|
||||
provider: static
|
||||
path: /dev/null
|
||||
host_key: ssh-ed25519 EXISTING
|
||||
---
|
||||
"""
|
||||
path = self._write_bottle(d, "dev", content)
|
||||
result = find_and_update_bottle_file(Path(d), "myrepo", "ssh-ed25519 NEW")
|
||||
self.assertFalse(result)
|
||||
self.assertNotIn("NEW", path.read_text())
|
||||
|
||||
def test_returns_false_for_nonexistent_dir(self) -> None:
|
||||
result = find_and_update_bottle_file(
|
||||
Path("/nonexistent/dir"), "myrepo", "ssh-ed25519 AAAA"
|
||||
)
|
||||
self.assertFalse(result)
|
||||
|
||||
def test_skips_file_without_git_gate_section(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
self._write_bottle(d, "plain", "---\nenv:\n FOO: bar\n---\n")
|
||||
result = find_and_update_bottle_file(Path(d), "myrepo", "ssh-ed25519 AAAA")
|
||||
self.assertFalse(result)
|
||||
|
||||
def test_skips_file_with_non_dict_repos(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
self._write_bottle(d, "bad", "---\ngit-gate:\n repos:\n - item\n---\n")
|
||||
result = find_and_update_bottle_file(Path(d), "myrepo", "ssh-ed25519 AAAA")
|
||||
self.assertFalse(result)
|
||||
|
||||
def test_skips_invalid_utf8_file(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
path = Path(d) / "broken.md"
|
||||
path.write_bytes(b"\xff\xfe") # invalid UTF-8
|
||||
result = find_and_update_bottle_file(Path(d), "myrepo", "ssh-ed25519 AAAA")
|
||||
self.assertFalse(result)
|
||||
|
||||
def test_skips_unreadable_file(self) -> None:
|
||||
import os
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
path = Path(d) / "unreadable.md"
|
||||
path.write_text("---\n---\n")
|
||||
os.chmod(path, 0o000)
|
||||
try:
|
||||
result = find_and_update_bottle_file(Path(d), "myrepo", "ssh-ed25519 AAAA")
|
||||
self.assertFalse(result)
|
||||
finally:
|
||||
os.chmod(path, 0o644)
|
||||
|
||||
def test_returns_false_when_file_unreadable_on_write(self) -> None:
|
||||
# find_repo_bottle_file succeeds, but the second read (for writing)
|
||||
# raises OSError — e.g. file removed or permissions changed.
|
||||
content = """\
|
||||
---
|
||||
git-gate:
|
||||
repos:
|
||||
myrepo:
|
||||
url: ssh://git@host/org/repo.git
|
||||
key:
|
||||
provider: static
|
||||
path: /dev/null
|
||||
---
|
||||
"""
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
path = self._write_bottle(d, "dev", content)
|
||||
with patch(
|
||||
"bot_bottle.git_gate_host_key.find_repo_bottle_file",
|
||||
return_value=path,
|
||||
), patch.object(Path, "read_text", side_effect=OSError("Permission denied")):
|
||||
result = find_and_update_bottle_file(Path(d), "myrepo", "ssh-ed25519 AAAA")
|
||||
self.assertFalse(result)
|
||||
|
||||
def test_skips_when_update_produces_no_change(self) -> None:
|
||||
# add_host_key_to_frontmatter returns the original text unchanged
|
||||
# (e.g. because the entry is absent from the parsed structure).
|
||||
# find_and_update_bottle_file must continue rather than write.
|
||||
content = """\
|
||||
---
|
||||
git-gate:
|
||||
repos:
|
||||
myrepo:
|
||||
url: ssh://git@host/org/repo.git
|
||||
key:
|
||||
provider: static
|
||||
path: /dev/null
|
||||
---
|
||||
"""
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
path = self._write_bottle(d, "dev", content)
|
||||
with patch(
|
||||
"bot_bottle.git_gate_host_key.add_host_key_to_frontmatter",
|
||||
return_value=content, # no change
|
||||
):
|
||||
result = find_and_update_bottle_file(Path(d), "myrepo", "ssh-ed25519 AAAA")
|
||||
self.assertFalse(result)
|
||||
self.assertNotIn("host_key", path.read_text())
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# find_repo_bottle_file
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestFindRepoBottleFile(unittest.TestCase):
|
||||
_CONTENT = """\
|
||||
---
|
||||
git-gate:
|
||||
repos:
|
||||
myrepo:
|
||||
url: ssh://git@host/org/repo.git
|
||||
key:
|
||||
provider: static
|
||||
path: /dev/null
|
||||
---
|
||||
"""
|
||||
|
||||
def _write(self, d: str, name: str, content: str) -> Path:
|
||||
path = Path(d) / f"{name}.md"
|
||||
path.write_text(content, encoding="utf-8")
|
||||
return path
|
||||
|
||||
def test_returns_path_when_repo_found(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
path = self._write(d, "dev", self._CONTENT)
|
||||
result = find_repo_bottle_file(Path(d), "myrepo")
|
||||
self.assertEqual(path, result)
|
||||
|
||||
def test_returns_none_when_no_matching_file(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
self._write(d, "dev", self._CONTENT)
|
||||
self.assertIsNone(find_repo_bottle_file(Path(d), "other"))
|
||||
|
||||
def test_returns_none_when_host_key_already_set(self) -> None:
|
||||
content = self._CONTENT.replace(" path: /dev/null\n",
|
||||
" path: /dev/null\n host_key: X\n")
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
self._write(d, "dev", content)
|
||||
self.assertIsNone(find_repo_bottle_file(Path(d), "myrepo"))
|
||||
|
||||
def test_returns_none_for_nonexistent_dir(self) -> None:
|
||||
self.assertIsNone(find_repo_bottle_file(Path("/nonexistent"), "myrepo"))
|
||||
|
||||
def test_skips_unreadable_file(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
(Path(d) / "unreadable.md").write_text(self._CONTENT)
|
||||
with patch.object(Path, "read_text", side_effect=OSError("Permission denied")):
|
||||
self.assertIsNone(find_repo_bottle_file(Path(d), "myrepo"))
|
||||
|
||||
def test_skips_file_with_non_dict_git_gate(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
(Path(d) / "dev.md").write_text("---\ngit-gate:\n - item\n---\n")
|
||||
self.assertIsNone(find_repo_bottle_file(Path(d), "myrepo"))
|
||||
|
||||
def test_skips_file_with_non_dict_repos(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
(Path(d) / "dev.md").write_text("---\ngit-gate:\n repos:\n - item\n---\n")
|
||||
self.assertIsNone(find_repo_bottle_file(Path(d), "myrepo"))
|
||||
|
||||
def test_save_prompt_includes_filename(self) -> None:
|
||||
"""preflight prompt must name the discovered bottle file."""
|
||||
manifest = _manifest_with_git()
|
||||
|
||||
with tempfile.TemporaryDirectory() as home:
|
||||
bottles_dir = Path(home) / "bottles"
|
||||
bottles_dir.mkdir()
|
||||
bottle_file = bottles_dir / "dev.md"
|
||||
bottle_file.write_text(
|
||||
"---\ngit-gate:\n repos:\n myrepo:\n"
|
||||
" url: ssh://git@gitea.example.com:30009/org/myrepo.git\n"
|
||||
" key:\n provider: static\n path: /dev/null\n---\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
mock_prompt = MagicMock(side_effect=["y", "n"]) # confirm key, skip save
|
||||
with patch(
|
||||
"bot_bottle.git_gate_host_key.fetch_host_key",
|
||||
return_value="ssh-ed25519 FETCHED",
|
||||
), patch(
|
||||
"bot_bottle.git_gate_host_key.prompt_tty", mock_prompt,
|
||||
), patch("sys.stderr"):
|
||||
preflight_host_keys(manifest, headless=False, home_md=Path(home))
|
||||
|
||||
save_call = next(
|
||||
c for c in mock_prompt.call_args_list if "Save" in c.args[0]
|
||||
)
|
||||
self.assertIn(str(bottle_file), save_call.args[0])
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# preflight_host_keys
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _manifest_with_git(host_key: str = "") -> Manifest:
|
||||
"""Build a Manifest with one git entry; host_key defaults to empty."""
|
||||
return ManifestIndex.from_json_obj({
|
||||
"bottles": {
|
||||
"dev": {
|
||||
"git-gate": {
|
||||
"repos": {
|
||||
"myrepo": {
|
||||
"url": "ssh://git@gitea.example.com:30009/org/myrepo.git",
|
||||
"key": {"provider": "static", "path": "/dev/null"},
|
||||
**({"host_key": host_key} if host_key else {}),
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
|
||||
}).load_for_agent("demo")
|
||||
|
||||
|
||||
class TestPreflightHostKeys(unittest.TestCase):
|
||||
def test_no_op_when_all_keys_present(self) -> None:
|
||||
manifest = _manifest_with_git("ssh-ed25519 AAAA")
|
||||
result = preflight_host_keys(manifest, headless=False, home_md=None)
|
||||
self.assertIs(result, manifest)
|
||||
|
||||
def test_headless_dies_when_key_missing(self) -> None:
|
||||
manifest = _manifest_with_git()
|
||||
with self.assertRaises(SystemExit):
|
||||
preflight_host_keys(manifest, headless=True, home_md=None)
|
||||
|
||||
def test_interactive_fetches_and_confirms_key(self) -> None:
|
||||
manifest = _manifest_with_git()
|
||||
|
||||
with patch(
|
||||
"bot_bottle.git_gate_host_key.fetch_host_key",
|
||||
return_value="ssh-ed25519 FETCHED",
|
||||
), patch(
|
||||
"bot_bottle.git_gate_host_key.prompt_tty",
|
||||
side_effect=["y", "n"], # confirm key, don't persist
|
||||
), patch("sys.stderr"):
|
||||
result = preflight_host_keys(manifest, headless=False, home_md=None)
|
||||
|
||||
self.assertEqual("ssh-ed25519 FETCHED", result.bottle.git[0].KnownHostKey)
|
||||
|
||||
def test_interactive_dies_when_key_not_confirmed(self) -> None:
|
||||
manifest = _manifest_with_git()
|
||||
|
||||
with patch(
|
||||
"bot_bottle.git_gate_host_key.fetch_host_key",
|
||||
return_value="ssh-ed25519 FETCHED",
|
||||
), patch(
|
||||
"bot_bottle.git_gate_host_key.prompt_tty",
|
||||
return_value="n",
|
||||
), patch("sys.stderr"), self.assertRaises(SystemExit):
|
||||
preflight_host_keys(manifest, headless=False, home_md=None)
|
||||
|
||||
def test_interactive_persists_when_requested(self) -> None:
|
||||
manifest = _manifest_with_git()
|
||||
|
||||
with tempfile.TemporaryDirectory() as home:
|
||||
bottles_dir = Path(home) / "bottles"
|
||||
bottles_dir.mkdir()
|
||||
bottle_file = bottles_dir / "dev.md"
|
||||
bottle_file.write_text(
|
||||
"---\ngit-gate:\n repos:\n myrepo:\n"
|
||||
" url: ssh://git@gitea.example.com:30009/org/myrepo.git\n"
|
||||
" key:\n provider: static\n path: /dev/null\n---\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
with patch(
|
||||
"bot_bottle.git_gate_host_key.fetch_host_key",
|
||||
return_value="ssh-ed25519 FETCHED",
|
||||
), patch(
|
||||
"bot_bottle.git_gate_host_key.prompt_tty",
|
||||
side_effect=["y", "y"], # confirm key, yes persist
|
||||
), patch("sys.stderr"):
|
||||
result = preflight_host_keys(
|
||||
manifest, headless=False, home_md=Path(home),
|
||||
)
|
||||
|
||||
self.assertEqual("ssh-ed25519 FETCHED", result.bottle.git[0].KnownHostKey)
|
||||
written = bottle_file.read_text()
|
||||
self.assertIn("host_key: ssh-ed25519 FETCHED", written)
|
||||
|
||||
def test_interactive_skips_persist_when_declined(self) -> None:
|
||||
manifest = _manifest_with_git()
|
||||
|
||||
with tempfile.TemporaryDirectory() as home:
|
||||
bottles_dir = Path(home) / "bottles"
|
||||
bottles_dir.mkdir()
|
||||
bottle_file = bottles_dir / "dev.md"
|
||||
bottle_file.write_text(
|
||||
"---\ngit-gate:\n repos:\n myrepo:\n"
|
||||
" url: ssh://git@gitea.example.com:30009/org/myrepo.git\n"
|
||||
" key:\n provider: static\n path: /dev/null\n---\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
with patch(
|
||||
"bot_bottle.git_gate_host_key.fetch_host_key",
|
||||
return_value="ssh-ed25519 FETCHED",
|
||||
), patch(
|
||||
"bot_bottle.git_gate_host_key.prompt_tty",
|
||||
side_effect=["y", "n"], # confirm key, no persist
|
||||
), patch("sys.stderr"):
|
||||
result = preflight_host_keys(
|
||||
manifest, headless=False, home_md=Path(home),
|
||||
)
|
||||
|
||||
self.assertEqual("ssh-ed25519 FETCHED", result.bottle.git[0].KnownHostKey)
|
||||
written = bottle_file.read_text()
|
||||
self.assertNotIn("host_key", written)
|
||||
|
||||
def test_interactive_dies_when_fetch_fails(self) -> None:
|
||||
manifest = _manifest_with_git()
|
||||
|
||||
with patch(
|
||||
"bot_bottle.git_gate_host_key.fetch_host_key",
|
||||
side_effect=RuntimeError("connection refused"),
|
||||
), patch("sys.stderr"), self.assertRaises(SystemExit):
|
||||
preflight_host_keys(manifest, headless=False, home_md=None)
|
||||
|
||||
def test_interactive_warns_when_persist_file_not_found(self) -> None:
|
||||
manifest = _manifest_with_git()
|
||||
|
||||
with tempfile.TemporaryDirectory() as home:
|
||||
# bottles/ dir exists but does not contain the repo entry
|
||||
bottles_dir = Path(home) / "bottles"
|
||||
bottles_dir.mkdir()
|
||||
|
||||
import io
|
||||
buf = io.StringIO()
|
||||
with patch(
|
||||
"bot_bottle.git_gate_host_key.fetch_host_key",
|
||||
return_value="ssh-ed25519 FETCHED",
|
||||
), patch(
|
||||
"bot_bottle.git_gate_host_key.prompt_tty",
|
||||
side_effect=["y"], # confirm key only; no save prompt when no file found
|
||||
), patch("sys.stderr", buf):
|
||||
result = preflight_host_keys(
|
||||
manifest, headless=False, home_md=Path(home),
|
||||
)
|
||||
|
||||
self.assertEqual("ssh-ed25519 FETCHED", result.bottle.git[0].KnownHostKey)
|
||||
self.assertIn("kept in memory", buf.getvalue())
|
||||
|
||||
def test_interactive_warns_when_write_fails_after_prompt(self) -> None:
|
||||
# User says yes to save, but find_and_update_bottle_file fails.
|
||||
manifest = _manifest_with_git()
|
||||
|
||||
with tempfile.TemporaryDirectory() as home:
|
||||
bottles_dir = Path(home) / "bottles"
|
||||
bottles_dir.mkdir()
|
||||
bottle_file = bottles_dir / "dev.md"
|
||||
bottle_file.write_text(
|
||||
"---\ngit-gate:\n repos:\n myrepo:\n"
|
||||
" url: ssh://git@gitea.example.com:30009/org/myrepo.git\n"
|
||||
" key:\n provider: static\n path: /dev/null\n---\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
import io
|
||||
buf = io.StringIO()
|
||||
with patch(
|
||||
"bot_bottle.git_gate_host_key.fetch_host_key",
|
||||
return_value="ssh-ed25519 FETCHED",
|
||||
), patch(
|
||||
"bot_bottle.git_gate_host_key.prompt_tty",
|
||||
side_effect=["y", "y"], # confirm key, yes persist
|
||||
), patch(
|
||||
"bot_bottle.git_gate_host_key.find_and_update_bottle_file",
|
||||
return_value=False,
|
||||
), patch("sys.stderr", buf):
|
||||
result = preflight_host_keys(
|
||||
manifest, headless=False, home_md=Path(home),
|
||||
)
|
||||
|
||||
self.assertEqual("ssh-ed25519 FETCHED", result.bottle.git[0].KnownHostKey)
|
||||
self.assertIn("kept in memory", buf.getvalue())
|
||||
|
||||
def test_headless_names_all_missing_repos_in_error(self) -> None:
|
||||
manifest = ManifestIndex.from_json_obj({
|
||||
"bottles": {
|
||||
"dev": {
|
||||
"git-gate": {
|
||||
"repos": {
|
||||
"alpha": {
|
||||
"url": "ssh://git@host/org/alpha.git",
|
||||
"key": {"provider": "static", "path": "/dev/null"},
|
||||
},
|
||||
"beta": {
|
||||
"url": "ssh://git@host/org/beta.git",
|
||||
"key": {"provider": "static", "path": "/dev/null"},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
"agents": {"demo": {"skills": [], "prompt": "", "bottle": "dev"}},
|
||||
}).load_for_agent("demo")
|
||||
|
||||
import io
|
||||
buf = io.StringIO()
|
||||
with patch("sys.stderr", buf), self.assertRaises(SystemExit):
|
||||
preflight_host_keys(manifest, headless=True, home_md=None)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -13,7 +13,6 @@ from pathlib import Path
|
||||
from typing import Any, cast
|
||||
from unittest.mock import patch
|
||||
|
||||
from bot_bottle.errors import MissingEnvVarError
|
||||
from bot_bottle.git_gate import (
|
||||
_gitconfig_validate_value,
|
||||
_provision_dynamic_key,
|
||||
@@ -121,7 +120,7 @@ class TestProvisionDynamicKey(unittest.TestCase):
|
||||
patch.dict("os.environ", {}, clear=False):
|
||||
import os
|
||||
os.environ.pop("GITEA_TOK", None)
|
||||
with self.assertRaises(MissingEnvVarError):
|
||||
with self.assertRaises(RuntimeError):
|
||||
_provision_dynamic_key(_gitea_entry(), "s", Path(d))
|
||||
|
||||
|
||||
@@ -167,7 +166,7 @@ class TestRevokeProvisionedKeys(unittest.TestCase):
|
||||
import os
|
||||
os.environ.pop("GITEA_TOK", None)
|
||||
(Path(d) / "repo-deploy-key-id").write_text("kid123")
|
||||
with self.assertRaises(MissingEnvVarError):
|
||||
with self.assertRaises(RuntimeError):
|
||||
revoke_git_gate_provisioned_keys(_bottle(_gitea_entry()), Path(d))
|
||||
|
||||
|
||||
|
||||
@@ -1,81 +0,0 @@
|
||||
"""Unit: image_cache.py — check_stale / check_stale_path."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import tempfile
|
||||
import unittest
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
from bot_bottle.image_cache import StaleImageError, check_stale, check_stale_path
|
||||
|
||||
|
||||
class TestCheckStale(unittest.TestCase):
|
||||
def _run(self, threshold: int, age_days: float) -> None:
|
||||
created = datetime.now(tz=timezone.utc) - timedelta(days=age_days)
|
||||
with patch("bot_bottle.image_cache.ConfigStore") as cs:
|
||||
cs.return_value.cached_image_stale_warning_days.return_value = threshold
|
||||
check_stale("test image", created)
|
||||
|
||||
def test_negative_threshold_never_raises(self):
|
||||
# Threshold < 0 means the check is disabled — always passes.
|
||||
self._run(threshold=-1, age_days=9999)
|
||||
|
||||
def test_zero_threshold_raises_immediately(self):
|
||||
# threshold=0 means any image is stale the moment it exists.
|
||||
with self.assertRaises(StaleImageError):
|
||||
self._run(threshold=0, age_days=0.1)
|
||||
|
||||
def test_within_threshold_does_not_raise(self):
|
||||
# Age well under threshold — should pass silently.
|
||||
self._run(threshold=7, age_days=2)
|
||||
|
||||
def test_at_threshold_does_not_raise(self):
|
||||
# Exactly at the boundary is fine (<=, not <).
|
||||
self._run(threshold=1, age_days=0.9999)
|
||||
|
||||
def test_exceeds_threshold_raises(self):
|
||||
created = datetime.now(tz=timezone.utc) - timedelta(days=3)
|
||||
with patch("bot_bottle.image_cache.ConfigStore") as cs:
|
||||
cs.return_value.cached_image_stale_warning_days.return_value = 1
|
||||
with self.assertRaises(StaleImageError) as ctx:
|
||||
check_stale("agent image 'bot-bottle:latest'", created)
|
||||
self.assertIn("agent image", str(ctx.exception))
|
||||
self.assertIn("day(s) old", str(ctx.exception))
|
||||
|
||||
def test_naive_datetime_treated_as_utc(self):
|
||||
# check_stale calls .astimezone(utc) on the input; naive datetimes
|
||||
# that would be interpreted as local time should still work.
|
||||
# We can't control the local tz in a unit test, so just ensure
|
||||
# no exception is thrown for a very recent naive datetime.
|
||||
naive_now = datetime(2099, 1, 1) # far future, always "fresh"
|
||||
with patch("bot_bottle.image_cache.ConfigStore") as cs:
|
||||
cs.return_value.cached_image_stale_warning_days.return_value = 1
|
||||
# Should not raise — the image is brand new.
|
||||
check_stale("test image", naive_now)
|
||||
|
||||
|
||||
class TestCheckStalePath(unittest.TestCase):
|
||||
def test_delegates_to_check_stale_with_mtime(self):
|
||||
with tempfile.NamedTemporaryFile() as f:
|
||||
path = Path(f.name)
|
||||
with patch("bot_bottle.image_cache.check_stale") as mock_check:
|
||||
check_stale_path("some artifact", path)
|
||||
mock_check.assert_called_once()
|
||||
label, dt = mock_check.call_args.args
|
||||
self.assertEqual("some artifact", label)
|
||||
self.assertIsInstance(dt, datetime)
|
||||
self.assertIsNotNone(dt.tzinfo)
|
||||
|
||||
def test_raises_stale_for_old_file(self):
|
||||
with tempfile.NamedTemporaryFile() as f:
|
||||
path = Path(f.name)
|
||||
with patch("bot_bottle.image_cache.ConfigStore") as cs:
|
||||
cs.return_value.cached_image_stale_warning_days.return_value = 0
|
||||
with self.assertRaises(StaleImageError):
|
||||
check_stale_path("cached artifact", path)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -30,19 +30,6 @@ class TestMacosContainerBottle(unittest.TestCase):
|
||||
argv,
|
||||
)
|
||||
|
||||
def test_agent_argv_accepts_absolute_provider_command(self):
|
||||
command = "/home/node/.codex/packages/standalone/current/bin/codex"
|
||||
bottle = MacosContainerBottle(
|
||||
"bot-bottle-dev-abc",
|
||||
lambda: None,
|
||||
None,
|
||||
agent_command=command,
|
||||
)
|
||||
with patch.dict(bottle_mod.os.environ, {}, clear=True):
|
||||
argv = bottle.agent_argv(["run"])
|
||||
self.assertIn(command, argv)
|
||||
self.assertNotIn("codex", argv)
|
||||
|
||||
def test_agent_argv_includes_workdir(self):
|
||||
bottle = MacosContainerBottle(
|
||||
"bot-bottle-dev-abc",
|
||||
|
||||
@@ -2,7 +2,6 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import dataclasses
|
||||
import unittest
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
@@ -16,7 +15,6 @@ from bot_bottle.backend.macos_container import launch
|
||||
from bot_bottle.backend.macos_container.bottle_plan import MacosContainerBottlePlan
|
||||
from bot_bottle.egress import EgressPlan
|
||||
from bot_bottle.git_gate import GitGatePlan
|
||||
from bot_bottle.log import Die
|
||||
from bot_bottle.manifest import ManifestIndex
|
||||
|
||||
_MANIFEST = ManifestIndex.from_json_obj({
|
||||
@@ -73,9 +71,7 @@ def _plan(
|
||||
else:
|
||||
git_gate_plan = SimpleNamespace(upstreams=())
|
||||
supervise_plan = (
|
||||
SimpleNamespace(
|
||||
db_path=Path("/state/bot-bottle.db"),
|
||||
)
|
||||
SimpleNamespace(queue_dir=Path("/state/supervise/queue"))
|
||||
if supervise else None
|
||||
)
|
||||
agent_provision = SimpleNamespace(
|
||||
@@ -83,7 +79,7 @@ def _plan(
|
||||
provisioned_env={"CODEX_HOME": "/run/codex-home"},
|
||||
)
|
||||
return cast(MacosContainerBottlePlan, SimpleNamespace(
|
||||
spec=SimpleNamespace(image_policy="fresh"),
|
||||
spec=SimpleNamespace(),
|
||||
manifest=_MANIFEST,
|
||||
stage_dir=stage_dir,
|
||||
slug="dev-abc",
|
||||
@@ -141,7 +137,7 @@ class TestMacosContainerLaunchArgv(unittest.TestCase):
|
||||
argv,
|
||||
)
|
||||
self.assertIn(
|
||||
"type=bind,source=/state,target=/run/supervise",
|
||||
"type=bind,source=/state/supervise/queue,target=/run/supervise/queue",
|
||||
argv,
|
||||
)
|
||||
|
||||
@@ -294,7 +290,7 @@ class TestMacosContainerLaunchArgv(unittest.TestCase):
|
||||
|
||||
def _build_plan(stage_dir: Path) -> MacosContainerBottlePlan:
|
||||
return MacosContainerBottlePlan(
|
||||
spec=cast(BottleSpec, SimpleNamespace(image_policy="fresh")),
|
||||
spec=cast(BottleSpec, SimpleNamespace()),
|
||||
manifest=_MANIFEST,
|
||||
stage_dir=stage_dir,
|
||||
git_gate_plan=cast(GitGatePlan, SimpleNamespace(upstreams=())),
|
||||
@@ -367,82 +363,5 @@ class TestMacosContainerLaunchCommittedImage(unittest.TestCase):
|
||||
self.assertEqual("bot-bottle-agent:latest", calls[1][0])
|
||||
|
||||
|
||||
class TestMacosContainerLaunchCachedImages(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self._tmp = tempfile.TemporaryDirectory()
|
||||
self.stage_dir = Path(self._tmp.name)
|
||||
|
||||
def tearDown(self):
|
||||
self._tmp.cleanup()
|
||||
|
||||
def _cached_plan(self) -> MacosContainerBottlePlan:
|
||||
return dataclasses.replace(
|
||||
_build_plan(self.stage_dir),
|
||||
spec=cast(BottleSpec, SimpleNamespace(image_policy="cached")),
|
||||
)
|
||||
|
||||
def test_cached_mode_dies_when_agent_image_missing(self) -> None:
|
||||
plan = self._cached_plan()
|
||||
|
||||
def image_exists(ref: str) -> bool: # pylint: disable=unused-argument
|
||||
return False # Nothing cached.
|
||||
|
||||
with patch.object(
|
||||
launch, "read_committed_image", return_value="",
|
||||
), patch.object(
|
||||
launch.container_mod, "image_exists", side_effect=image_exists,
|
||||
), patch.object(
|
||||
launch, "die", side_effect=Die(),
|
||||
):
|
||||
with self.assertRaises(Die):
|
||||
launch._build_images(plan)
|
||||
|
||||
def test_cached_mode_dies_when_sidecar_image_missing(self) -> None:
|
||||
plan = self._cached_plan()
|
||||
|
||||
def image_exists(ref: str) -> bool:
|
||||
return ref == plan.image # Agent present; sidecar missing.
|
||||
|
||||
with patch.object(
|
||||
launch, "read_committed_image", return_value="",
|
||||
), patch.object(
|
||||
launch.container_mod, "image_exists", side_effect=image_exists,
|
||||
), patch.object(
|
||||
launch, "die", side_effect=Die(),
|
||||
):
|
||||
with self.assertRaises(Die):
|
||||
launch._build_images(plan)
|
||||
|
||||
def test_cached_mode_both_present_returns_unchanged_plan(self) -> None:
|
||||
plan = self._cached_plan()
|
||||
with patch.object(
|
||||
launch, "read_committed_image", return_value="",
|
||||
), patch.object(
|
||||
launch.container_mod, "image_exists", return_value=True,
|
||||
), patch.object(
|
||||
launch.container_mod, "build_image",
|
||||
) as build, patch.object(launch, "info"):
|
||||
result = launch._build_images(plan)
|
||||
|
||||
build.assert_not_called()
|
||||
self.assertEqual(plan.image, result.image)
|
||||
|
||||
def test_committed_image_plus_cached_skips_sidecar_build(self) -> None:
|
||||
plan = self._cached_plan()
|
||||
with patch.object(
|
||||
launch, "read_committed_image",
|
||||
return_value="bot-bottle-committed-dev-abc:latest",
|
||||
), patch.object(
|
||||
launch.container_mod, "image_exists", return_value=True,
|
||||
), patch.object(
|
||||
launch.container_mod, "build_image",
|
||||
) as build, patch.object(launch, "info"):
|
||||
result = launch._build_images(plan)
|
||||
|
||||
# In cached mode with a committed image, no builds should fire.
|
||||
build.assert_not_called()
|
||||
self.assertEqual("bot-bottle-committed-dev-abc:latest", result.image)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -273,71 +273,5 @@ resolver #2
|
||||
)
|
||||
|
||||
|
||||
class TestMacosContainerImageCreatedAt(unittest.TestCase):
|
||||
def _ok(self, stdout: str) -> "util.subprocess.CompletedProcess": # type: ignore
|
||||
return util.subprocess.CompletedProcess(
|
||||
args=[], returncode=0, stdout=stdout, stderr="",
|
||||
)
|
||||
|
||||
def _fail(self, stderr: str = "no such image") -> "util.subprocess.CompletedProcess": # type: ignore
|
||||
return util.subprocess.CompletedProcess(
|
||||
args=[], returncode=1, stdout="", stderr=stderr,
|
||||
)
|
||||
|
||||
def test_parses_iso_timestamp_from_dict(self):
|
||||
payload = '[{"created": "2025-06-01T12:00:00"}]'
|
||||
with patch.object(util.subprocess, "run", return_value=self._ok(payload)):
|
||||
dt = util.image_created_at("bot-bottle-agent:latest")
|
||||
self.assertEqual(2025, dt.year)
|
||||
self.assertEqual(6, dt.month)
|
||||
self.assertEqual(1, dt.day)
|
||||
|
||||
def test_accepts_list_or_dict_input(self):
|
||||
# Container CLI may return a list; we take the first element.
|
||||
payload = '[{"created": "2024-01-15T08:30:00"}]'
|
||||
with patch.object(util.subprocess, "run", return_value=self._ok(payload)):
|
||||
dt = util.image_created_at("some-image:latest")
|
||||
self.assertEqual(2024, dt.year)
|
||||
|
||||
def test_accepts_uppercase_Created_field(self):
|
||||
payload = '[{"Created": "2024-03-20T10:00:00"}]'
|
||||
with patch.object(util.subprocess, "run", return_value=self._ok(payload)):
|
||||
dt = util.image_created_at("some-image:latest")
|
||||
self.assertEqual(2024, dt.year)
|
||||
self.assertEqual(3, dt.month)
|
||||
|
||||
def test_dies_on_nonzero_returncode(self):
|
||||
with patch.object(util.subprocess, "run", return_value=self._fail("not found")), \
|
||||
patch.object(util, "die", side_effect=SystemExit("die")) as die:
|
||||
with self.assertRaises(SystemExit):
|
||||
util.image_created_at("missing:tag")
|
||||
die.assert_called_once()
|
||||
self.assertIn("missing:tag", die.call_args.args[0])
|
||||
|
||||
def test_dies_on_malformed_json(self):
|
||||
with patch.object(util.subprocess, "run", return_value=self._ok("not-json {")), \
|
||||
patch.object(util, "die", side_effect=SystemExit("die")) as die:
|
||||
with self.assertRaises(SystemExit):
|
||||
util.image_created_at("some:tag")
|
||||
die.assert_called_once()
|
||||
|
||||
def test_dies_when_no_created_field(self):
|
||||
payload = '[{"id": "sha256:abc123"}]'
|
||||
with patch.object(util.subprocess, "run", return_value=self._ok(payload)), \
|
||||
patch.object(util, "die", side_effect=SystemExit("die")) as die:
|
||||
with self.assertRaises(SystemExit):
|
||||
util.image_created_at("some:tag")
|
||||
die.assert_called_once()
|
||||
self.assertIn("creation timestamp", die.call_args.args[0])
|
||||
|
||||
def test_dies_on_invalid_timestamp_format(self):
|
||||
payload = '[{"created": "not-a-date"}]'
|
||||
with patch.object(util.subprocess, "run", return_value=self._ok(payload)), \
|
||||
patch.object(util, "die", side_effect=SystemExit("die")) as die:
|
||||
with self.assertRaises(SystemExit):
|
||||
util.image_created_at("some:tag")
|
||||
die.assert_called_once()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -9,11 +9,7 @@ from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
|
||||
from unittest.mock import patch
|
||||
|
||||
from bot_bottle.env import resolve_env
|
||||
from bot_bottle.errors import MissingEnvVarError
|
||||
from bot_bottle.manifest import Manifest, ManifestBottle, ManifestIndex
|
||||
from bot_bottle.manifest import ManifestBottle, ManifestIndex
|
||||
from bot_bottle.manifest_agent import (
|
||||
ManifestAgent,
|
||||
ManifestAgentProvider,
|
||||
@@ -242,26 +238,5 @@ class TestEagerIndexLookups(unittest.TestCase):
|
||||
self.assertIsNone(m.git_identity_summary())
|
||||
|
||||
|
||||
class TestResolveEnv(unittest.TestCase):
|
||||
"""resolve_env raises MissingEnvVarError when an interpolated entry's
|
||||
host var is unset."""
|
||||
|
||||
def _manifest(self, env_dict: dict[str, str]) -> Manifest:
|
||||
idx = ManifestIndex.from_json_obj({
|
||||
"bottles": {"dev": {"env": env_dict}},
|
||||
"agents": {"demo": {"bottle": "dev"}},
|
||||
})
|
||||
return idx.load_for_agent("demo")
|
||||
|
||||
def test_missing_interpolated_host_var_raises(self):
|
||||
manifest = self._manifest({"API_KEY": "${HOST_API_KEY}"})
|
||||
with patch.dict("os.environ", {}, clear=False):
|
||||
import os
|
||||
os.environ.pop("HOST_API_KEY", None)
|
||||
with self.assertRaises(MissingEnvVarError) as cm:
|
||||
resolve_env(manifest)
|
||||
self.assertIn("HOST_API_KEY", str(cm.exception))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -79,11 +79,6 @@ class TestClaudeArgvWrapped(unittest.TestCase):
|
||||
"--",
|
||||
"runuser", "-u", "node", "--",
|
||||
"env", "HOME=/home/node", "USER=node",
|
||||
(
|
||||
"PATH=/home/node/.local/bin:"
|
||||
"/home/node/.codex/packages/standalone/current/bin:"
|
||||
"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
|
||||
),
|
||||
"claude",
|
||||
],
|
||||
argv,
|
||||
@@ -132,24 +127,16 @@ class TestClaudeArgvWrapped(unittest.TestCase):
|
||||
self.assertIn("HTTPS_PROXY=http://127.0.0.1:1234", argv)
|
||||
self.assertIn("NO_PROXY=localhost", argv)
|
||||
|
||||
def test_guest_env_path_overrides_default_path(self):
|
||||
argv = _unwrap(_bottle(None, PATH="/custom/bin").agent_argv([]))
|
||||
self.assertIn("PATH=/custom/bin", argv)
|
||||
self.assertFalse(any(
|
||||
item.startswith("PATH=/home/node/.local/bin")
|
||||
for item in argv
|
||||
))
|
||||
|
||||
def test_runuser_switch_precedes_claude(self):
|
||||
# The dashboard's `_build_resume_argv_with_fallback` finds
|
||||
# the `claude` token to split exec-framing from the claude
|
||||
# tail. `runuser -u node --` must sit on the prefix side so
|
||||
# the shell wrap inherits the UID switch.
|
||||
argv = _bottle().agent_argv([])
|
||||
runuser_idx = argv.index("runuser")
|
||||
agent_idx = argv.index("claude")
|
||||
self.assertEqual(
|
||||
["runuser", "-u", "node", "--", "env"],
|
||||
argv[runuser_idx:runuser_idx + 5],
|
||||
argv[agent_idx - 7:agent_idx - 2],
|
||||
)
|
||||
|
||||
def test_pi_provider_appends_system_prompt_without_print_mode(self):
|
||||
|
||||
@@ -1,58 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
from bot_bottle.backend.egress_apply import EgressApplyError
|
||||
from bot_bottle.backend.smolmachines import egress_apply
|
||||
from bot_bottle.backend.smolmachines.smolvm import SmolvmRunResult
|
||||
|
||||
|
||||
class TestFetchCurrentRoutes(unittest.TestCase):
|
||||
def test_reads_routes_from_sidecar_vm(self):
|
||||
with patch.object(
|
||||
egress_apply._smolvm,
|
||||
"machine_exec",
|
||||
return_value=SmolvmRunResult(0, "routes", ""),
|
||||
) as exec_:
|
||||
self.assertEqual("routes", egress_apply.fetch_current_routes("dev-abc"))
|
||||
exec_.assert_called_once_with(
|
||||
"bot-bottle-sidecars-dev-abc",
|
||||
["cat", "/etc/egress/routes.yaml"],
|
||||
)
|
||||
|
||||
def test_read_failure_raises_apply_error(self):
|
||||
with patch.object(
|
||||
egress_apply._smolvm,
|
||||
"machine_exec",
|
||||
return_value=SmolvmRunResult(1, "", "nope"),
|
||||
):
|
||||
with self.assertRaises(EgressApplyError):
|
||||
egress_apply.fetch_current_routes("dev-abc")
|
||||
|
||||
|
||||
class TestReload(unittest.TestCase):
|
||||
def test_sends_hup_to_sidecar_init(self):
|
||||
with patch.object(
|
||||
egress_apply._smolvm,
|
||||
"machine_exec",
|
||||
return_value=SmolvmRunResult(0, "", ""),
|
||||
) as exec_:
|
||||
egress_apply.SmolmachinesEgressApplicator()._signal_bundle_reload("dev-abc")
|
||||
exec_.assert_called_once_with(
|
||||
"bot-bottle-sidecars-dev-abc",
|
||||
["sh", "-c", "kill -HUP 1"],
|
||||
)
|
||||
|
||||
def test_reload_failure_raises_apply_error(self):
|
||||
with patch.object(
|
||||
egress_apply._smolvm,
|
||||
"machine_exec",
|
||||
return_value=SmolvmRunResult(1, "", "nope"),
|
||||
):
|
||||
with self.assertRaises(EgressApplyError):
|
||||
egress_apply.SmolmachinesEgressApplicator()._signal_bundle_reload("dev-abc")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,66 +0,0 @@
|
||||
"""Unit: smolmachines active-agent enumeration."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import subprocess
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
from bot_bottle.backend.smolmachines import enumerate as _enumerate
|
||||
|
||||
|
||||
def _completed(
|
||||
stdout: str,
|
||||
returncode: int = 0,
|
||||
) -> subprocess.CompletedProcess: # type: ignore[type-arg]
|
||||
return subprocess.CompletedProcess(
|
||||
args=[],
|
||||
returncode=returncode,
|
||||
stdout=stdout,
|
||||
stderr="",
|
||||
)
|
||||
|
||||
|
||||
class TestEnumerateActive(unittest.TestCase):
|
||||
def test_skips_sidecar_vms(self):
|
||||
machines = [
|
||||
{"name": "bot-bottle-demo-abc12", "state": "running"},
|
||||
{"name": "bot-bottle-sidecars-demo-abc12", "state": "running"},
|
||||
{"name": "bot-bottle-stopped-xyz", "state": "stopped"},
|
||||
{"name": "other", "state": "running"},
|
||||
]
|
||||
|
||||
class _Metadata:
|
||||
agent_name = "demo"
|
||||
started_at = "2026-07-09T00:00:00Z"
|
||||
label = "Demo"
|
||||
color = "green"
|
||||
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.enumerate.subprocess.run",
|
||||
return_value=_completed(json.dumps(machines)),
|
||||
), patch(
|
||||
"bot_bottle.backend.smolmachines.enumerate._query_bundle_services",
|
||||
return_value={"demo-abc12": ("egress",)},
|
||||
), patch(
|
||||
"bot_bottle.backend.smolmachines.enumerate.read_metadata",
|
||||
return_value=_Metadata(),
|
||||
) as read_metadata:
|
||||
active = _enumerate.enumerate_active()
|
||||
|
||||
self.assertEqual(1, len(active))
|
||||
self.assertEqual("demo-abc12", active[0].slug)
|
||||
self.assertEqual(("egress",), active[0].services)
|
||||
read_metadata.assert_called_once_with("demo-abc12")
|
||||
|
||||
def test_invalid_smolvm_output_returns_empty(self):
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.enumerate.subprocess.run",
|
||||
return_value=_completed("not json"),
|
||||
):
|
||||
self.assertEqual([], _enumerate.enumerate_active())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -94,10 +94,6 @@ class TestEnsureSmolmachine(unittest.TestCase):
|
||||
calls.append(name)
|
||||
return _f
|
||||
|
||||
def save_and_record(image_ref: str, path: str) -> None:
|
||||
Path(path).touch()
|
||||
calls.append("save")
|
||||
|
||||
with patch.object(
|
||||
_launch_mod.docker_mod, "build_image",
|
||||
side_effect=record("build"),
|
||||
@@ -106,7 +102,7 @@ class TestEnsureSmolmachine(unittest.TestCase):
|
||||
return_value=f"sha256:{digest}fffffffffffffffff",
|
||||
), patch.object(
|
||||
_launch_mod.docker_mod, "save",
|
||||
side_effect=save_and_record,
|
||||
side_effect=record("save"),
|
||||
) as save, patch.object(
|
||||
_launch_mod, "ephemeral_registry",
|
||||
return_value=_Reg(),
|
||||
@@ -187,54 +183,6 @@ class TestAgentFromPath(unittest.TestCase):
|
||||
dockerfile="/repo/Dockerfile",
|
||||
)
|
||||
|
||||
def test_cached_policy_uses_existing_artifact_without_build(self):
|
||||
with tempfile.TemporaryDirectory(prefix="cached-smolmachine.") as tmp:
|
||||
digest = "abcdef0123456789"
|
||||
artifact = Path(tmp) / f"{digest}.smolmachine.smolmachine"
|
||||
artifact.write_text("")
|
||||
plan = SimpleNamespace(
|
||||
slug="dev-abc12",
|
||||
agent_image="bot-bottle-claude:latest",
|
||||
spec=SimpleNamespace(image_policy="cached"),
|
||||
)
|
||||
with patch.object(
|
||||
_launch_mod, "_SMOLMACHINE_CACHE_DIR", Path(tmp),
|
||||
), patch.object(
|
||||
_launch_mod, "read_committed_image", return_value="",
|
||||
), patch.object(
|
||||
_launch_mod.docker_mod, "image_exists", return_value=True,
|
||||
), patch.object(
|
||||
_launch_mod.docker_mod, "image_id",
|
||||
return_value=f"sha256:{digest}fffffffffffffffff",
|
||||
), patch.object(
|
||||
_launch_mod, "_ensure_smolmachine",
|
||||
) as ensure:
|
||||
result = _launch_mod._agent_from_path(cast(Any, plan))
|
||||
|
||||
self.assertEqual(artifact, result)
|
||||
ensure.assert_not_called()
|
||||
|
||||
def test_cached_policy_dies_when_artifact_missing(self):
|
||||
with tempfile.TemporaryDirectory(prefix="cached-smolmachine.") as tmp:
|
||||
plan = SimpleNamespace(
|
||||
slug="dev-abc12",
|
||||
agent_image="bot-bottle-claude:latest",
|
||||
spec=SimpleNamespace(image_policy="cached"),
|
||||
)
|
||||
with patch.object(
|
||||
_launch_mod, "_SMOLMACHINE_CACHE_DIR", Path(tmp),
|
||||
), patch.object(
|
||||
_launch_mod, "read_committed_image", return_value="",
|
||||
), patch.object(
|
||||
_launch_mod.docker_mod, "image_exists", return_value=True,
|
||||
), patch.object(
|
||||
_launch_mod.docker_mod, "image_id",
|
||||
return_value="sha256:abcdef0123456789fffffffffffffffff",
|
||||
):
|
||||
from bot_bottle.log import Die
|
||||
with self.assertRaises(Die):
|
||||
_launch_mod._agent_from_path(cast(Any, plan))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -8,7 +8,6 @@ inspecting running bundle containers' port bindings."""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import sqlite3
|
||||
import subprocess
|
||||
import tempfile
|
||||
@@ -113,16 +112,9 @@ class TestEnsurePool(unittest.TestCase):
|
||||
|
||||
|
||||
class TestAllocate(unittest.TestCase):
|
||||
def test_per_bottle_alias_on_linux(self):
|
||||
# Linux gets the same per-bottle scoping as macOS (127/8 is
|
||||
# already loopback, so no ifconfig is needed). A fresh host
|
||||
# with no running bundles allocates the first pool entry.
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
lock_path = Path(tmp) / "smolmachines.lock"
|
||||
with patch.object(loopback_alias, "_is_macos", return_value=False), \
|
||||
patch.object(loopback_alias, "_ALLOC_LOCK_PATH", lock_path), \
|
||||
patch.object(loopback_alias, "_aliases_in_use", return_value=set()):
|
||||
self.assertEqual("127.0.0.16", loopback_alias.allocate("demo"))
|
||||
def test_returns_loopback_on_linux(self):
|
||||
with patch.object(loopback_alias, "_is_macos", return_value=False):
|
||||
self.assertEqual("127.0.0.1", loopback_alias.allocate("demo"))
|
||||
|
||||
def test_picks_lowest_unused_on_macos(self):
|
||||
# No bundles running -> first pool entry.
|
||||
@@ -174,25 +166,12 @@ class TestAllocateLock(unittest.TestCase):
|
||||
|
||||
self.assertIn(fcntl_mod.LOCK_EX, flock_calls)
|
||||
|
||||
def test_acquires_exclusive_lock_on_linux(self):
|
||||
# Linux allocates per-bottle too, so it must take the same
|
||||
# lock to serialise concurrent launches.
|
||||
import fcntl as fcntl_mod
|
||||
flock_calls: list[int] = []
|
||||
|
||||
def record_flock(fd, op): # type: ignore
|
||||
flock_calls.append(op)
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
lock_path = Path(tmp) / "smolmachines.lock"
|
||||
with patch.object(loopback_alias, "_is_macos", return_value=False), \
|
||||
patch.object(loopback_alias, "_ALLOC_LOCK_PATH", lock_path), \
|
||||
patch.object(loopback_alias, "_aliases_in_use", return_value=set()), \
|
||||
patch.object(loopback_alias.fcntl, "flock",
|
||||
side_effect=record_flock):
|
||||
loopback_alias.allocate("demo")
|
||||
|
||||
self.assertIn(fcntl_mod.LOCK_EX, flock_calls)
|
||||
def test_no_lock_on_linux(self):
|
||||
# Linux early-returns before touching the lock file.
|
||||
with patch.object(loopback_alias, "_is_macos", return_value=False), \
|
||||
patch.object(loopback_alias.fcntl, "flock") as flock:
|
||||
loopback_alias.allocate("demo")
|
||||
flock.assert_not_called()
|
||||
|
||||
def test_sequential_allocations_with_shared_lock_are_serialised(self):
|
||||
# Two sequential calls share the same lock file. The second
|
||||
@@ -262,12 +241,10 @@ class TestAliasInUseDetection(unittest.TestCase):
|
||||
|
||||
|
||||
class TestForceAllowlist(unittest.TestCase):
|
||||
"""Smolvm 0.8.0 silently drops `--allow-cidr` with `--from`, so
|
||||
`force_allowlist` opens the state DB directly and sets the row's
|
||||
`allowed_cidrs` field — on both macOS and Linux. It is
|
||||
fail-closed: it dies rather than launching a VM whose allowlist
|
||||
it can't confirm. Round-trip tests against a real SQLite DB to
|
||||
lock down the BLOB encoding."""
|
||||
"""Smolvm 0.8.0 silently drops `--allow-cidr` with `--from`,
|
||||
so `force_allowlist` opens the state DB directly and sets
|
||||
the row's `allowed_cidrs` field. Round-trip tests against a
|
||||
real SQLite DB to lock down the BLOB encoding."""
|
||||
|
||||
def setUp(self):
|
||||
self._tmp = tempfile.TemporaryDirectory(prefix="smolvm-db.")
|
||||
@@ -313,67 +290,17 @@ class TestForceAllowlist(unittest.TestCase):
|
||||
self.assertEqual(4, cfg["cpus"])
|
||||
self.assertTrue(cfg["network"])
|
||||
|
||||
def test_patches_on_linux_too(self):
|
||||
# force_allowlist no longer no-ops on Linux — the TSI
|
||||
# allowlist must be enforced there as well.
|
||||
def test_noop_on_linux(self):
|
||||
with patch.object(loopback_alias, "_is_macos", return_value=False), \
|
||||
patch.object(loopback_alias, "_SMOLVM_DB_PATH", self.db):
|
||||
loopback_alias.force_allowlist("demo-vm", ["127.0.0.16/32"])
|
||||
# DB row should be untouched.
|
||||
con = sqlite3.connect(str(self.db))
|
||||
cfg = json.loads(con.execute(
|
||||
"SELECT data FROM vms WHERE name='demo-vm'",
|
||||
).fetchone()[0])
|
||||
con.close()
|
||||
self.assertEqual(["127.0.0.16/32"], cfg["allowed_cidrs"])
|
||||
|
||||
def test_skips_write_when_already_matching(self):
|
||||
# A newer smolvm that honors --allow-cidr at create leaves the
|
||||
# row already correct; force_allowlist must not rewrite it. We
|
||||
# detect a no-write by comparing the raw BLOB byte-for-byte
|
||||
# (a rewrite re-serialises the JSON, changing key order/bytes
|
||||
# is not guaranteed, but mtime/identity isn't observable — so
|
||||
# we assert the stored bytes are exactly what we pre-seeded).
|
||||
seeded = json.dumps({
|
||||
"name": "demo-vm", "cpus": 4, "mem": 8192,
|
||||
"network": True, "allowed_cidrs": ["127.0.0.16/32"],
|
||||
}).encode()
|
||||
con = sqlite3.connect(str(self.db))
|
||||
con.execute(
|
||||
"UPDATE vms SET data=? WHERE name='demo-vm'",
|
||||
(sqlite3.Binary(seeded),),
|
||||
)
|
||||
con.commit()
|
||||
con.close()
|
||||
|
||||
with patch.object(loopback_alias, "_is_macos", return_value=True), \
|
||||
patch.object(loopback_alias, "_SMOLVM_DB_PATH", self.db):
|
||||
loopback_alias.force_allowlist("demo-vm", ["127.0.0.16/32"])
|
||||
|
||||
con = sqlite3.connect(str(self.db))
|
||||
stored = con.execute(
|
||||
"SELECT data FROM vms WHERE name='demo-vm'").fetchone()[0]
|
||||
con.close()
|
||||
self.assertEqual(seeded, bytes(stored))
|
||||
|
||||
def test_dies_when_patch_does_not_take(self):
|
||||
# If the persisted allowlist still doesn't match after the
|
||||
# patch (e.g. wrong schema / smolvm stores it elsewhere),
|
||||
# force_allowlist must fail closed rather than boot the VM.
|
||||
original = loopback_alias._read_machine_cfg
|
||||
|
||||
def stale_cfg(con: sqlite3.Connection, name: str) -> dict[str, object]:
|
||||
# Always report the un-patched row so the post-write
|
||||
# verification never sees the requested cidrs.
|
||||
cfg = original(con, name)
|
||||
cfg["allowed_cidrs"] = None
|
||||
return cfg
|
||||
|
||||
with patch.object(loopback_alias, "_is_macos", return_value=True), \
|
||||
patch.object(loopback_alias, "_SMOLVM_DB_PATH", self.db), \
|
||||
patch.object(loopback_alias, "_read_machine_cfg", side_effect=stale_cfg), \
|
||||
patch.object(loopback_alias, "die", side_effect=SystemExit("die")):
|
||||
with self.assertRaises(SystemExit):
|
||||
loopback_alias.force_allowlist("demo-vm", ["127.0.0.16/32"])
|
||||
self.assertIsNone(cfg["allowed_cidrs"])
|
||||
|
||||
def test_dies_on_missing_db(self):
|
||||
with patch.object(loopback_alias, "_is_macos", return_value=True), \
|
||||
@@ -396,35 +323,5 @@ class TestForceAllowlist(unittest.TestCase):
|
||||
loopback_alias.force_allowlist("not-in-db", ["127.0.0.16/32"])
|
||||
|
||||
|
||||
class TestSmolvmDbPath(unittest.TestCase):
|
||||
"""The smolvm state-DB path is platform-derived: Application
|
||||
Support on macOS, XDG data dir on Linux."""
|
||||
|
||||
def test_macos_path(self):
|
||||
with patch.object(loopback_alias.platform, "system", return_value="Darwin"):
|
||||
p = loopback_alias._smolvm_db_path()
|
||||
self.assertEqual(
|
||||
("Library", "Application Support", "smolvm", "server", "smolvm.db"),
|
||||
p.parts[-5:],
|
||||
)
|
||||
|
||||
def test_linux_default_xdg_path(self):
|
||||
env = {k: v for k, v in os.environ.items() if k != "XDG_DATA_HOME"}
|
||||
with patch.object(loopback_alias.platform, "system", return_value="Linux"), \
|
||||
patch.dict(loopback_alias.os.environ, env, clear=True):
|
||||
p = loopback_alias._smolvm_db_path()
|
||||
self.assertEqual(
|
||||
(".local", "share", "smolvm", "server", "smolvm.db"),
|
||||
p.parts[-5:],
|
||||
)
|
||||
|
||||
def test_linux_respects_xdg_data_home(self):
|
||||
with patch.object(loopback_alias.platform, "system", return_value="Linux"), \
|
||||
patch.dict(loopback_alias.os.environ,
|
||||
{"XDG_DATA_HOME": "/custom/data"}, clear=False):
|
||||
p = loopback_alias._smolvm_db_path()
|
||||
self.assertEqual(Path("/custom/data/smolvm/server/smolvm.db"), p)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -1,325 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import socket
|
||||
import subprocess
|
||||
import threading
|
||||
import unittest
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from bot_bottle.backend.smolmachines.port_forward import (
|
||||
ForwardSpec,
|
||||
ForwarderHandle,
|
||||
_CompletedProcess,
|
||||
_Forwarder,
|
||||
_pipe,
|
||||
main,
|
||||
start_forwarder,
|
||||
stop_forwarder,
|
||||
)
|
||||
|
||||
|
||||
def _echo_server() -> tuple[socket.socket, int]:
|
||||
listener = socket.create_server(("127.0.0.1", 0))
|
||||
port = int(listener.getsockname()[1])
|
||||
|
||||
def run() -> None:
|
||||
with listener:
|
||||
conn, _ = listener.accept()
|
||||
with conn:
|
||||
while True:
|
||||
data = conn.recv(65536)
|
||||
if not data:
|
||||
return
|
||||
conn.sendall(data)
|
||||
|
||||
threading.Thread(target=run, daemon=True).start()
|
||||
return listener, port
|
||||
|
||||
|
||||
class TestForwarderValidation(unittest.TestCase):
|
||||
def test_empty_specs_returns_noop_handle(self):
|
||||
handle = start_forwarder(())
|
||||
|
||||
self.assertEqual((), handle.forwards)
|
||||
self.assertEqual(0, handle.process.poll())
|
||||
stop_forwarder(handle)
|
||||
|
||||
def test_rejects_generic_localhost_listener(self):
|
||||
with self.assertRaises(SystemExit):
|
||||
start_forwarder((
|
||||
ForwardSpec(
|
||||
label="egress",
|
||||
listen_host="127.0.0.1",
|
||||
listen_port=0,
|
||||
target_host="127.0.0.1",
|
||||
target_port=9099,
|
||||
),
|
||||
))
|
||||
|
||||
def test_rejects_wildcard_listener(self):
|
||||
with self.assertRaises(SystemExit):
|
||||
start_forwarder((
|
||||
ForwardSpec(
|
||||
label="egress",
|
||||
listen_host="0.0.0.0",
|
||||
listen_port=0,
|
||||
target_host="127.0.0.1",
|
||||
target_port=9099,
|
||||
),
|
||||
))
|
||||
|
||||
def test_rejects_non_loopback_target(self):
|
||||
with self.assertRaises(SystemExit):
|
||||
start_forwarder((
|
||||
ForwardSpec(
|
||||
label="egress",
|
||||
listen_host="127.0.0.16",
|
||||
listen_port=0,
|
||||
target_host="192.0.2.10",
|
||||
target_port=9099,
|
||||
),
|
||||
))
|
||||
|
||||
def test_rejects_invalid_ports(self):
|
||||
with self.assertRaises(SystemExit):
|
||||
start_forwarder((
|
||||
ForwardSpec("bad-listen", "127.0.0.2", 70000, "127.0.0.1", 9099),
|
||||
))
|
||||
with self.assertRaises(SystemExit):
|
||||
start_forwarder((
|
||||
ForwardSpec("bad-target", "127.0.0.2", 0, "127.0.0.1", 0),
|
||||
))
|
||||
|
||||
def test_completed_process_methods_are_noops(self):
|
||||
proc = _CompletedProcess()
|
||||
|
||||
proc.terminate()
|
||||
proc.kill()
|
||||
|
||||
self.assertEqual(0, proc.poll())
|
||||
self.assertEqual(0, proc.wait(timeout=1))
|
||||
|
||||
|
||||
class _FakeStdout:
|
||||
def __init__(self, line: str):
|
||||
self.line = line
|
||||
self.closed = False
|
||||
|
||||
def readline(self) -> str:
|
||||
return self.line
|
||||
|
||||
def close(self) -> None:
|
||||
self.closed = True
|
||||
|
||||
|
||||
class TestForwarderProcess(unittest.TestCase):
|
||||
def test_start_parses_ready_payload(self):
|
||||
payload = {
|
||||
"forwards": [{
|
||||
"label": "egress",
|
||||
"listen_host": "127.0.0.2",
|
||||
"listen_port": 61000,
|
||||
"target_host": "127.0.0.1",
|
||||
"target_port": 55000,
|
||||
}],
|
||||
}
|
||||
proc = MagicMock()
|
||||
proc.stdout = _FakeStdout(json.dumps(payload) + "\n")
|
||||
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.port_forward.subprocess.Popen",
|
||||
return_value=proc,
|
||||
) as popen:
|
||||
handle = start_forwarder((
|
||||
ForwardSpec("egress", "127.0.0.2", 0, "127.0.0.1", 55000),
|
||||
))
|
||||
|
||||
self.assertIs(proc, handle.process)
|
||||
self.assertEqual(61000, handle.forwards[0].listen_port)
|
||||
self.assertTrue(proc.stdout.closed)
|
||||
argv = popen.call_args.args[0]
|
||||
self.assertIn("--spec-json", argv)
|
||||
|
||||
def test_start_dies_when_helper_exits_before_ready(self):
|
||||
proc = MagicMock()
|
||||
proc.stdout = _FakeStdout("")
|
||||
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.port_forward.subprocess.Popen",
|
||||
return_value=proc,
|
||||
):
|
||||
with self.assertRaises(SystemExit):
|
||||
start_forwarder((
|
||||
ForwardSpec("egress", "127.0.0.2", 0, "127.0.0.1", 55000),
|
||||
))
|
||||
|
||||
proc.terminate.assert_called_once()
|
||||
proc.wait.assert_called_once_with(timeout=5)
|
||||
|
||||
def test_start_dies_on_invalid_ready_payload(self):
|
||||
proc = MagicMock()
|
||||
proc.stdout = _FakeStdout('{"wrong": []}\n')
|
||||
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.port_forward.subprocess.Popen",
|
||||
return_value=proc,
|
||||
):
|
||||
with self.assertRaises(SystemExit):
|
||||
start_forwarder((
|
||||
ForwardSpec("egress", "127.0.0.2", 0, "127.0.0.1", 55000),
|
||||
))
|
||||
|
||||
proc.terminate.assert_called_once()
|
||||
proc.wait.assert_called_once_with(timeout=5)
|
||||
|
||||
def test_stop_kills_process_after_timeout(self):
|
||||
proc = MagicMock()
|
||||
proc.poll.return_value = None
|
||||
proc.wait.side_effect = [subprocess.TimeoutExpired("cmd", 5), 0]
|
||||
|
||||
stop_forwarder(ForwarderHandle(process=proc, forwards=()))
|
||||
|
||||
proc.terminate.assert_called_once()
|
||||
proc.kill.assert_called_once()
|
||||
self.assertEqual(2, proc.wait.call_count)
|
||||
|
||||
|
||||
class TestForwarding(unittest.TestCase):
|
||||
def test_forwards_bytes_both_directions(self):
|
||||
try:
|
||||
probe = socket.create_server(("127.0.0.2", 0))
|
||||
except OSError:
|
||||
self.skipTest("127.0.0.2 is not bindable on this host")
|
||||
else:
|
||||
probe.close()
|
||||
_listener, target_port = _echo_server()
|
||||
handle = start_forwarder((
|
||||
ForwardSpec(
|
||||
label="egress",
|
||||
listen_host="127.0.0.2",
|
||||
listen_port=0,
|
||||
target_host="127.0.0.1",
|
||||
target_port=target_port,
|
||||
),
|
||||
))
|
||||
try:
|
||||
bound = handle.forwards[0]
|
||||
with socket.create_connection((bound.listen_host, bound.listen_port)) as sock:
|
||||
sock.sendall(b"hello")
|
||||
self.assertEqual(b"hello", sock.recv(5))
|
||||
finally:
|
||||
stop_forwarder(handle)
|
||||
|
||||
|
||||
class TestForwarderInProcess(unittest.TestCase):
|
||||
def test_forwarder_serves_and_stops_in_process(self):
|
||||
_listener, target_port = _echo_server()
|
||||
forwarder = _Forwarder((
|
||||
ForwardSpec("egress", "127.0.0.2", 0, "127.0.0.1", target_port),
|
||||
))
|
||||
forwarder.start()
|
||||
thread = threading.Thread(target=forwarder.serve, daemon=True)
|
||||
thread.start()
|
||||
try:
|
||||
bound = forwarder.bound[0]
|
||||
with socket.create_connection((bound.listen_host, bound.listen_port)) as sock:
|
||||
sock.sendall(b"ping")
|
||||
self.assertEqual(b"ping", sock.recv(4))
|
||||
finally:
|
||||
forwarder.stop()
|
||||
thread.join(timeout=2)
|
||||
|
||||
self.assertFalse(thread.is_alive())
|
||||
|
||||
def test_forwarder_logs_target_connect_failure(self):
|
||||
forwarder = _Forwarder((
|
||||
ForwardSpec("egress", "127.0.0.2", 0, "127.0.0.1", 1),
|
||||
))
|
||||
forwarder.start()
|
||||
thread = threading.Thread(target=forwarder.serve, daemon=True)
|
||||
thread.start()
|
||||
try:
|
||||
bound = forwarder.bound[0]
|
||||
with patch("bot_bottle.backend.smolmachines.port_forward.warn") as warn:
|
||||
with socket.create_connection((bound.listen_host, bound.listen_port)):
|
||||
pass
|
||||
for _ in range(100):
|
||||
if warn.called:
|
||||
break
|
||||
threading.Event().wait(0.01)
|
||||
self.assertTrue(warn.called)
|
||||
finally:
|
||||
forwarder.stop()
|
||||
thread.join(timeout=2)
|
||||
|
||||
def test_pipe_stops_when_recv_fails(self):
|
||||
src, dst = socket.socketpair()
|
||||
src.close()
|
||||
try:
|
||||
_pipe(src, dst)
|
||||
finally:
|
||||
dst.close()
|
||||
|
||||
def test_pipe_stops_when_send_fails(self):
|
||||
src, src_peer = socket.socketpair()
|
||||
_dst_peer, dst = socket.socketpair()
|
||||
dst.close()
|
||||
try:
|
||||
src_peer.sendall(b"hello")
|
||||
src_peer.shutdown(socket.SHUT_WR)
|
||||
_pipe(src, dst)
|
||||
finally:
|
||||
src.close()
|
||||
src_peer.close()
|
||||
_dst_peer.close()
|
||||
|
||||
|
||||
class TestMain(unittest.TestCase):
|
||||
def test_main_starts_serves_and_stops_forwarder(self):
|
||||
calls: list[str] = []
|
||||
handlers = []
|
||||
|
||||
class FakeForwarder:
|
||||
def __init__(self, specs): # type: ignore[no-untyped-def]
|
||||
self.specs = specs
|
||||
self.bound = specs
|
||||
|
||||
def start(self) -> None:
|
||||
calls.append("start")
|
||||
|
||||
def serve(self) -> None:
|
||||
calls.append("serve")
|
||||
|
||||
def stop(self) -> None:
|
||||
calls.append("stop")
|
||||
|
||||
def record_signal(_signum, handler): # type: ignore[no-untyped-def]
|
||||
handlers.append(handler)
|
||||
|
||||
spec = ForwardSpec("egress", "127.0.0.2", 0, "127.0.0.1", 9099)
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.port_forward._Forwarder",
|
||||
FakeForwarder,
|
||||
), patch(
|
||||
"bot_bottle.backend.smolmachines.port_forward.signal.signal",
|
||||
side_effect=record_signal,
|
||||
), patch("builtins.print") as print_mock:
|
||||
self.assertEqual(
|
||||
0,
|
||||
main(("--spec-json", json.dumps([{
|
||||
"label": spec.label,
|
||||
"listen_host": spec.listen_host,
|
||||
"listen_port": spec.listen_port,
|
||||
"target_host": spec.target_host,
|
||||
"target_port": spec.target_port,
|
||||
}]))),
|
||||
)
|
||||
handlers[0](15, None)
|
||||
|
||||
self.assertEqual(["start", "serve", "stop", "stop"], calls)
|
||||
self.assertIn("forwards", print_mock.call_args.args[0])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -9,7 +9,6 @@ from __future__ import annotations
|
||||
import subprocess
|
||||
import tempfile
|
||||
import unittest
|
||||
from contextlib import ExitStack
|
||||
from dataclasses import replace
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock, patch
|
||||
@@ -28,7 +27,6 @@ from bot_bottle.backend.smolmachines.bottle_plan import (
|
||||
SmolmachinesBottlePlan,
|
||||
)
|
||||
from bot_bottle.backend.smolmachines import launch as _launch
|
||||
from bot_bottle.backend.smolmachines import port_forward as _forward
|
||||
from bot_bottle.backend.smolmachines.launch import _bundle_launch_spec
|
||||
from bot_bottle.backend.util import AGENT_CA_PATH
|
||||
from bot_bottle.egress import EgressPlan, EgressRoute
|
||||
@@ -51,7 +49,6 @@ class _Provider(AgentProvider):
|
||||
def provision_prompt(self, plan, bottle): ... # type: ignore[override]
|
||||
def provision(self, plan, bottle): ... # type: ignore[override]
|
||||
def provision_supervise_mcp(self, plan, bottle, supervise_url): ... # type: ignore[override]
|
||||
def headless_prompt(self, prompt): return [] # type: ignore[override]
|
||||
|
||||
|
||||
_PROVIDER = _Provider()
|
||||
@@ -132,7 +129,7 @@ def _plan(
|
||||
if supervise:
|
||||
supervise_plan = SupervisePlan(
|
||||
slug="demo-abc12",
|
||||
db_path=Path("/tmp/bot-bottle.db"),
|
||||
queue_dir=Path("/tmp/queue"),
|
||||
)
|
||||
return SmolmachinesBottlePlan(
|
||||
spec=spec,
|
||||
@@ -278,8 +275,8 @@ class TestProvisionCA(unittest.TestCase):
|
||||
str(self.egress_ca),
|
||||
AGENT_CA_PATH,
|
||||
)
|
||||
self.assertEqual(2, bottle.exec.call_count)
|
||||
script = bottle.exec.call_args_list[1].args[0]
|
||||
bottle.exec.assert_called_once()
|
||||
script = bottle.exec.call_args.args[0]
|
||||
self.assertIn("chmod 644", script)
|
||||
self.assertIn("update-ca-certificates", script)
|
||||
self.assertEqual("root", bottle.exec.call_args.kwargs.get("user"))
|
||||
@@ -424,17 +421,14 @@ class TestBundleLaunchSpec(unittest.TestCase):
|
||||
spec.environment,
|
||||
)
|
||||
|
||||
def test_supervise_adds_daemon_volume_and_env(self):
|
||||
from bot_bottle.supervise import DB_PATH_IN_CONTAINER
|
||||
plan = _plan(supervise=True)
|
||||
spec = _bundle_launch_spec(plan, "net", "127.0.0.16")
|
||||
self.assertIn("supervise", spec.daemons_csv)
|
||||
self.assertIn(f"SUPERVISE_DB_PATH={DB_PATH_IN_CONTAINER}", spec.environment)
|
||||
self.assertIn(("/tmp/bot-bottle.db", DB_PATH_IN_CONTAINER, False), spec.volumes)
|
||||
|
||||
def test_canary_env_visible_to_smolvm_guest(self):
|
||||
plan = _plan(canary=True)
|
||||
stamped = _launch._discover_urls(plan, "127.0.0.16", {9099: 65000})
|
||||
with patch.object(
|
||||
_launch._bundle,
|
||||
"bundle_host_port",
|
||||
return_value="65000",
|
||||
):
|
||||
stamped = _launch._discover_urls(plan, "127.0.0.16")
|
||||
|
||||
self.assertEqual(
|
||||
"fake-canary-value",
|
||||
@@ -503,148 +497,5 @@ class TestProvisionGitUser(unittest.TestCase):
|
||||
self.assertIn("bot@example.com", calls[0][0])
|
||||
|
||||
|
||||
class TestLaunchResourceWiring(unittest.TestCase):
|
||||
def test_allocate_resources_uses_loopback_alias_and_bundle_name(self):
|
||||
plan = _plan()
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.launch._loopback.ensure_pool",
|
||||
) as ensure_pool, patch(
|
||||
"bot_bottle.backend.smolmachines.launch._loopback.allocate",
|
||||
return_value="127.0.0.16",
|
||||
) as allocate:
|
||||
loopback_ip, network = _launch._allocate_resources(plan, ExitStack())
|
||||
|
||||
ensure_pool.assert_called_once_with()
|
||||
allocate.assert_called_once_with("demo-abc12")
|
||||
self.assertEqual("127.0.0.16", loopback_ip)
|
||||
self.assertEqual("bot-bottle-bundle-demo-abc12", network)
|
||||
|
||||
def test_start_bundle_wraps_raw_vm_ports_with_forwarder_ports(self):
|
||||
plan = _plan(supervise=True)
|
||||
plan = replace(
|
||||
plan,
|
||||
git_gate_plan=replace(
|
||||
plan.git_gate_plan,
|
||||
upstreams=(GitGateUpstream(
|
||||
name="bot-bottle",
|
||||
upstream_url="ssh://git@host/repo.git",
|
||||
upstream_host="host",
|
||||
upstream_port="22",
|
||||
identity_file="/tmp/key",
|
||||
known_host_key="",
|
||||
),),
|
||||
),
|
||||
)
|
||||
stack = MagicMock(spec=ExitStack)
|
||||
raw_launch = MagicMock(raw_ports={9099: 55001, 9420: 55002, 9100: 55003})
|
||||
handle = _forward.ForwarderHandle(
|
||||
process=MagicMock(),
|
||||
forwards=(
|
||||
_forward.ForwardSpec("egress", "127.0.0.16", 65001, "127.0.0.1", 55001),
|
||||
_forward.ForwardSpec("git-http", "127.0.0.16", 65002, "127.0.0.1", 55002),
|
||||
_forward.ForwardSpec("supervise", "127.0.0.16", 65003, "127.0.0.1", 55003),
|
||||
),
|
||||
)
|
||||
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.launch._provision_git_gate_keys",
|
||||
return_value=plan,
|
||||
), patch(
|
||||
"bot_bottle.backend.smolmachines.launch._ensure_smolmachine",
|
||||
return_value=Path("/cache/sidecar.smolmachine"),
|
||||
) as ensure, patch(
|
||||
"bot_bottle.backend.smolmachines.launch._bundle.start_bundle_vm",
|
||||
return_value=raw_launch,
|
||||
) as start_vm, patch(
|
||||
"bot_bottle.backend.smolmachines.launch._forward.start_forwarder",
|
||||
return_value=handle,
|
||||
) as start_forwarder:
|
||||
stamped = _launch._start_bundle(plan, "net", "127.0.0.16", stack)
|
||||
|
||||
ensure.assert_called_once()
|
||||
start_vm.assert_called_once()
|
||||
self.assertEqual(Path("/cache/sidecar.smolmachine"), start_vm.call_args.kwargs["from_path"])
|
||||
specs = start_forwarder.call_args.args[0]
|
||||
self.assertEqual(
|
||||
(
|
||||
_forward.ForwardSpec("egress", "127.0.0.16", 0, "127.0.0.1", 55001),
|
||||
_forward.ForwardSpec("git-http", "127.0.0.16", 0, "127.0.0.1", 55002),
|
||||
_forward.ForwardSpec("supervise", "127.0.0.16", 0, "127.0.0.1", 55003),
|
||||
),
|
||||
specs,
|
||||
)
|
||||
self.assertEqual("http://127.0.0.16:65001", stamped.agent_proxy_url)
|
||||
self.assertEqual("127.0.0.16:65002", stamped.agent_git_gate_host)
|
||||
self.assertEqual("http://127.0.0.16:65003/", stamped.agent_supervise_url)
|
||||
self.assertEqual(2, stack.callback.call_count)
|
||||
|
||||
def test_init_vm_repairs_ownership_then_waits_for_exec(self):
|
||||
plan = _plan()
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.launch._smolvm.machine_exec",
|
||||
) as machine_exec, patch(
|
||||
"bot_bottle.backend.smolmachines.launch._smolvm.wait_exec_ready",
|
||||
) as wait_exec_ready:
|
||||
_launch._init_vm(plan)
|
||||
|
||||
machine_exec.assert_called_once()
|
||||
argv = machine_exec.call_args.args[1]
|
||||
self.assertEqual(["sh", "-c"], argv[:2])
|
||||
self.assertIn("chown -R node:node /home/node", argv[2])
|
||||
wait_exec_ready.assert_called_once_with(plan.machine_name)
|
||||
|
||||
|
||||
class TestPortLabels(unittest.TestCase):
|
||||
def test_known_and_dynamic_port_labels_round_trip(self):
|
||||
for port, label in (
|
||||
(9099, "egress"),
|
||||
(9420, "git-http"),
|
||||
(9100, "supervise"),
|
||||
(12345, "port-12345"),
|
||||
):
|
||||
self.assertEqual(label, _launch._label_for_port(port))
|
||||
self.assertEqual(port, _launch._port_for_label(label))
|
||||
|
||||
def test_unknown_label_is_rejected(self):
|
||||
with self.assertRaises(ValueError):
|
||||
_launch._port_for_label("sidecar")
|
||||
|
||||
|
||||
class TestDiscoverUrls(unittest.TestCase):
|
||||
"""_discover_urls stamps git-gate host + supervise URL into the plan."""
|
||||
|
||||
def test_git_gate_host_set_when_upstreams_present(self):
|
||||
plan = _plan()
|
||||
plan = replace(
|
||||
plan,
|
||||
git_gate_plan=replace(
|
||||
plan.git_gate_plan,
|
||||
upstreams=(GitGateUpstream(
|
||||
name="bot-bottle",
|
||||
upstream_url="ssh://git@host/repo.git",
|
||||
upstream_host="host",
|
||||
upstream_port="22",
|
||||
identity_file="/tmp/key",
|
||||
known_host_key="",
|
||||
),),
|
||||
),
|
||||
)
|
||||
stamped = _launch._discover_urls(
|
||||
plan,
|
||||
"127.0.0.16",
|
||||
{9099: 65000, 9420: 65001},
|
||||
)
|
||||
self.assertEqual("127.0.0.16:65001", stamped.agent_git_gate_host)
|
||||
|
||||
def test_supervise_url_set_when_supervise_present(self):
|
||||
plan = _plan(supervise=True)
|
||||
stamped = _launch._discover_urls(
|
||||
plan,
|
||||
"127.0.0.16",
|
||||
{9099: 65000, 9100: 55556},
|
||||
)
|
||||
self.assertEqual("http://127.0.0.16:55556/", stamped.agent_supervise_url)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -14,16 +14,13 @@ from unittest.mock import patch
|
||||
|
||||
from bot_bottle.backend.smolmachines.sidecar_bundle import (
|
||||
BundleLaunchSpec,
|
||||
allocate_raw_host_ports,
|
||||
bundle_container_name,
|
||||
bundle_network_name,
|
||||
create_bundle_network,
|
||||
ensure_bundle_image,
|
||||
start_bundle_vm,
|
||||
remove_bundle_network,
|
||||
start_bundle,
|
||||
stop_bundle,
|
||||
stop_bundle_vm,
|
||||
)
|
||||
|
||||
|
||||
@@ -203,66 +200,6 @@ class TestEnsureBundleImage(unittest.TestCase):
|
||||
self.assertEqual("Dockerfile.sidecars", kwargs["dockerfile"])
|
||||
|
||||
|
||||
class TestStartBundleVm(unittest.TestCase):
|
||||
def test_allocate_raw_host_ports_returns_bindable_ports(self):
|
||||
raw_ports = allocate_raw_host_ports((9099, 9420))
|
||||
|
||||
self.assertEqual({9099, 9420}, set(raw_ports))
|
||||
for host_port in raw_ports.values():
|
||||
self.assertIsInstance(host_port, int)
|
||||
self.assertGreater(host_port, 0)
|
||||
self.assertNotEqual(raw_ports[9099], raw_ports[9420])
|
||||
|
||||
def test_creates_sidecar_vm_with_ports_volumes_and_env(self):
|
||||
calls: list[tuple[str, tuple[object, ...], dict[str, object]]] = []
|
||||
|
||||
def record(name): # type: ignore
|
||||
def inner(*args, **kwargs): # type: ignore
|
||||
calls.append((name, args, kwargs))
|
||||
return inner
|
||||
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.sidecar_bundle.allocate_raw_host_ports",
|
||||
return_value={9099: 55001, 9420: 55002},
|
||||
), patch(
|
||||
"bot_bottle.backend.smolmachines.sidecar_bundle._smolvm.machine_create",
|
||||
side_effect=record("create"),
|
||||
), patch(
|
||||
"bot_bottle.backend.smolmachines.sidecar_bundle._smolvm.machine_start",
|
||||
side_effect=record("start"),
|
||||
), patch(
|
||||
"bot_bottle.backend.smolmachines.sidecar_bundle._smolvm.wait_exec_ready",
|
||||
side_effect=record("wait"),
|
||||
):
|
||||
launch = start_bundle_vm(
|
||||
_spec(
|
||||
daemons_csv="egress,git-http",
|
||||
environment=("FOO=bar", "TOKEN"),
|
||||
volumes=(("/host/routes", "/etc/egress", True),),
|
||||
ports_to_publish=(9099, 9420),
|
||||
),
|
||||
from_path=Path("/cache/sidecar.smolmachine"),
|
||||
host_env={"TOKEN": "secret"},
|
||||
)
|
||||
|
||||
self.assertEqual({9099: 55001, 9420: 55002}, launch.raw_ports)
|
||||
create = calls[0]
|
||||
self.assertEqual("create", create[0])
|
||||
self.assertEqual(("bot-bottle-sidecars-demo-abc12",), create[1])
|
||||
self.assertEqual(Path("/cache/sidecar.smolmachine"), create[2]["from_path"])
|
||||
self.assertTrue(create[2]["net"])
|
||||
self.assertEqual(((55001, 9099), (55002, 9420)), create[2]["ports"])
|
||||
self.assertEqual((("/host/routes", "/etc/egress", True),), create[2]["volumes"])
|
||||
self.assertEqual(
|
||||
{
|
||||
"BOT_BOTTLE_SIDECAR_DAEMONS": "egress,git-http",
|
||||
"FOO": "bar",
|
||||
"TOKEN": "secret",
|
||||
},
|
||||
create[2]["env"],
|
||||
)
|
||||
|
||||
|
||||
class TestStopBundle(unittest.TestCase):
|
||||
def _patch_run(self, **kwargs): # type: ignore
|
||||
return patch(
|
||||
@@ -285,38 +222,5 @@ class TestStopBundle(unittest.TestCase):
|
||||
stop_bundle("demo-abc12") # no raise
|
||||
|
||||
|
||||
class TestStopBundleVm(unittest.TestCase):
|
||||
def test_stops_then_deletes_sidecar_vm(self):
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.sidecar_bundle._smolvm.machine_stop",
|
||||
) as stop, patch(
|
||||
"bot_bottle.backend.smolmachines.sidecar_bundle._smolvm.machine_delete",
|
||||
) as delete:
|
||||
stop_bundle_vm("demo-abc12")
|
||||
stop.assert_called_once_with("bot-bottle-sidecars-demo-abc12")
|
||||
delete.assert_called_once_with("bot-bottle-sidecars-demo-abc12")
|
||||
|
||||
def test_stop_and_delete_errors_are_warnings(self):
|
||||
from bot_bottle.backend.smolmachines.smolvm import SmolvmError
|
||||
result = subprocess.CompletedProcess(
|
||||
args=[],
|
||||
returncode=1,
|
||||
stdout="",
|
||||
stderr="failed",
|
||||
)
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.sidecar_bundle._smolvm.machine_stop",
|
||||
side_effect=SmolvmError(["smolvm", "machine", "stop"], result),
|
||||
), patch(
|
||||
"bot_bottle.backend.smolmachines.sidecar_bundle._smolvm.machine_delete",
|
||||
side_effect=SmolvmError(["smolvm", "machine", "delete"], result),
|
||||
), patch(
|
||||
"bot_bottle.backend.smolmachines.sidecar_bundle.warn",
|
||||
) as warn:
|
||||
stop_bundle_vm("demo-abc12")
|
||||
|
||||
self.assertEqual(2, warn.call_count)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -79,7 +79,7 @@ class TestArgvShapes(unittest.TestCase):
|
||||
with self._patch_run() as m:
|
||||
machine_create("agent-xyz")
|
||||
self.assertEqual(
|
||||
["smolvm", "machine", "create", "--name", "agent-xyz"],
|
||||
["smolvm", "machine", "create", "agent-xyz"],
|
||||
m.call_args.args[0],
|
||||
)
|
||||
|
||||
@@ -103,24 +103,7 @@ class TestArgvShapes(unittest.TestCase):
|
||||
self.assertIn("192.168.50.2/32", argv)
|
||||
self.assertIn("-e", argv)
|
||||
self.assertIn("HTTPS_PROXY=http://192.168.50.2:8888", argv)
|
||||
self.assertIn("--name", argv)
|
||||
self.assertIn("agent-xyz", argv)
|
||||
|
||||
def test_machine_create_with_net_ports_and_volumes(self):
|
||||
with self._patch_run() as m:
|
||||
machine_create(
|
||||
"sidecar-xyz",
|
||||
from_path=Path("/stage/sidecar.smolmachine"),
|
||||
net=True,
|
||||
ports=((55000, 9099),),
|
||||
volumes=(("/host/routes", "/etc/egress", True),),
|
||||
)
|
||||
argv = m.call_args.args[0]
|
||||
self.assertIn("--net", argv)
|
||||
self.assertIn("-p", argv)
|
||||
self.assertIn("55000:9099", argv)
|
||||
self.assertIn("-v", argv)
|
||||
self.assertIn("/host/routes:/etc/egress:ro", argv)
|
||||
self.assertEqual("agent-xyz", argv[-1])
|
||||
|
||||
def test_machine_create_omits_net_when_no_allow_cidrs(self):
|
||||
with self._patch_run() as m:
|
||||
@@ -144,13 +127,13 @@ class TestArgvShapes(unittest.TestCase):
|
||||
m.call_args.args[0],
|
||||
)
|
||||
|
||||
def test_machine_delete_uses_name_flag_and_force(self):
|
||||
# delete uses --name flag; -f required so no interactive
|
||||
def test_machine_delete_uses_positional_name_and_force(self):
|
||||
# delete NAME is positional; -f required so no interactive
|
||||
# confirmation blocks teardown.
|
||||
with self._patch_run() as m:
|
||||
machine_delete("agent-xyz")
|
||||
self.assertEqual(
|
||||
["smolvm", "machine", "delete", "--name", "agent-xyz", "-f"],
|
||||
["smolvm", "machine", "delete", "-f", "agent-xyz"],
|
||||
m.call_args.args[0],
|
||||
)
|
||||
|
||||
|
||||
@@ -56,14 +56,9 @@ class TestBundleSubnet(unittest.TestCase):
|
||||
|
||||
class TestPreflight(unittest.TestCase):
|
||||
def test_smolvm_present_returns_none(self):
|
||||
# Pin macOS so the Linux KVM gate doesn't fire on a CI runner
|
||||
# (ubuntu, no /dev/kvm) — this test isolates the PATH check.
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.util.shutil.which",
|
||||
return_value="/usr/local/bin/smolvm",
|
||||
), patch(
|
||||
"bot_bottle.backend.smolmachines.util.platform.system",
|
||||
return_value="Darwin",
|
||||
):
|
||||
self.assertIsNone(smolmachines_preflight())
|
||||
|
||||
@@ -93,63 +88,5 @@ class TestPreflight(unittest.TestCase):
|
||||
self.assertIn("BOT_BOTTLE_BACKEND=docker", msg)
|
||||
|
||||
|
||||
class TestKvmPreflight(unittest.TestCase):
|
||||
"""Linux-only KVM gate: smolvm needs /dev/kvm present and
|
||||
accessible. macOS skips this entirely (Hypervisor.framework)."""
|
||||
|
||||
def _run(self, *, system: str, exists: bool, access: bool) -> None:
|
||||
with patch(
|
||||
"bot_bottle.backend.smolmachines.util.shutil.which",
|
||||
return_value="/usr/bin/smolvm",
|
||||
), patch(
|
||||
"bot_bottle.backend.smolmachines.util.platform.system",
|
||||
return_value=system,
|
||||
), patch(
|
||||
"bot_bottle.backend.smolmachines.util.os.path.exists",
|
||||
return_value=exists,
|
||||
), patch(
|
||||
"bot_bottle.backend.smolmachines.util.os.access",
|
||||
return_value=access,
|
||||
):
|
||||
return smolmachines_preflight()
|
||||
|
||||
def test_macos_skips_kvm_check(self):
|
||||
# Even with /dev/kvm absent, macOS must not run the gate.
|
||||
self.assertIsNone(self._run(system="Darwin", exists=False, access=False))
|
||||
|
||||
def test_linux_ok_returns_none(self):
|
||||
self.assertIsNone(self._run(system="Linux", exists=True, access=True))
|
||||
|
||||
def test_linux_missing_device_dies(self):
|
||||
with self.assertRaises(SystemExit):
|
||||
self._run(system="Linux", exists=False, access=False)
|
||||
|
||||
def test_linux_no_access_dies(self):
|
||||
with self.assertRaises(SystemExit):
|
||||
self._run(system="Linux", exists=True, access=False)
|
||||
|
||||
def test_linux_missing_device_message(self):
|
||||
import io
|
||||
import sys
|
||||
captured = io.StringIO()
|
||||
with patch.object(sys, "stderr", captured):
|
||||
with self.assertRaises(SystemExit):
|
||||
self._run(system="Linux", exists=False, access=False)
|
||||
msg = captured.getvalue()
|
||||
self.assertIn("/dev/kvm", msg)
|
||||
self.assertIn("kvm-intel", msg)
|
||||
|
||||
def test_linux_no_access_message(self):
|
||||
import io
|
||||
import sys
|
||||
captured = io.StringIO()
|
||||
with patch.object(sys, "stderr", captured):
|
||||
with self.assertRaises(SystemExit):
|
||||
self._run(system="Linux", exists=True, access=False)
|
||||
msg = captured.getvalue()
|
||||
self.assertIn("kvm", msg)
|
||||
self.assertIn("group", msg)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -1,302 +0,0 @@
|
||||
"""Unit: stale-image check functions across backends, and the
|
||||
BottleBackend.launch template method (skip_stale flag).
|
||||
|
||||
No real images or containers are used — all Docker/container/smolmachine
|
||||
calls are mocked at the module boundary."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
from typing import Any, cast
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
|
||||
def _bottle_cm(bottle: Any) -> MagicMock:
|
||||
"""Return a mock context manager that yields `bottle`."""
|
||||
cm = MagicMock()
|
||||
cm.__enter__ = MagicMock(return_value=bottle)
|
||||
cm.__exit__ = MagicMock(return_value=False)
|
||||
return cm
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# BottleBackend.launch template — _image_stale_checks + skip_stale
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestBottleBackendLaunchTemplate(unittest.TestCase):
|
||||
"""Verify the concrete launch() method on BottleBackend calls
|
||||
_image_stale_checks unless skip_stale=True, then delegates to _launch_impl."""
|
||||
|
||||
def _make_backend(self) -> Any:
|
||||
from bot_bottle.backend.docker.backend import DockerBottleBackend
|
||||
return DockerBottleBackend()
|
||||
|
||||
def test_stale_checks_called_by_default(self) -> None:
|
||||
backend = self._make_backend()
|
||||
plan = cast(Any, SimpleNamespace())
|
||||
bottle = MagicMock()
|
||||
with patch.object(
|
||||
backend, "_image_stale_checks",
|
||||
) as stale_mock, patch.object(
|
||||
backend, "_launch_impl",
|
||||
return_value=_bottle_cm(bottle),
|
||||
):
|
||||
with backend.launch(plan):
|
||||
pass
|
||||
stale_mock.assert_called_once_with(plan)
|
||||
|
||||
def test_skip_stale_bypasses_stale_checks(self) -> None:
|
||||
backend = self._make_backend()
|
||||
plan = cast(Any, SimpleNamespace())
|
||||
bottle = MagicMock()
|
||||
with patch.object(
|
||||
backend, "_image_stale_checks",
|
||||
) as stale_mock, patch.object(
|
||||
backend, "_launch_impl",
|
||||
return_value=_bottle_cm(bottle),
|
||||
):
|
||||
with backend.launch(plan, skip_stale=True):
|
||||
pass
|
||||
stale_mock.assert_not_called()
|
||||
|
||||
def test_noop_default_image_stale_checks(self) -> None:
|
||||
from bot_bottle.backend.docker.backend import DockerBottleBackend
|
||||
from bot_bottle.backend import BottleBackend
|
||||
backend = DockerBottleBackend()
|
||||
# Call the base-class _image_stale_checks directly to verify it's a no-op.
|
||||
BottleBackend._image_stale_checks(backend, cast(Any, SimpleNamespace())) # type: ignore[arg-type]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Docker backend stale_checks
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestDockerStaleChecks(unittest.TestCase):
|
||||
def _plan(self, policy: str = "cached", slug: str = "dev-abc") -> Any:
|
||||
spec = SimpleNamespace(image_policy=policy)
|
||||
provision = SimpleNamespace(image="bot-bottle-agent:latest")
|
||||
return cast(Any, SimpleNamespace(
|
||||
spec=spec,
|
||||
slug=slug,
|
||||
image="bot-bottle-agent:latest",
|
||||
agent_provision=provision,
|
||||
))
|
||||
|
||||
def test_fresh_policy_is_noop(self) -> None:
|
||||
from bot_bottle.backend.docker import launch as mod
|
||||
with patch.object(mod, "read_committed_image") as rci, \
|
||||
patch.object(mod, "check_stale") as cs:
|
||||
mod.stale_checks(self._plan("fresh"))
|
||||
rci.assert_not_called()
|
||||
cs.assert_not_called()
|
||||
|
||||
def test_committed_image_present_checks_only_committed(self) -> None:
|
||||
from bot_bottle.backend.docker import launch as mod
|
||||
from datetime import datetime, timezone
|
||||
ts = datetime(2025, 1, 1, tzinfo=timezone.utc)
|
||||
with patch.object(mod, "read_committed_image", return_value="committed:latest"), \
|
||||
patch.object(mod.docker_mod, "image_exists", return_value=True), \
|
||||
patch.object(mod.docker_mod, "image_created_at", return_value=ts), \
|
||||
patch.object(mod, "check_stale") as cs:
|
||||
mod.stale_checks(self._plan())
|
||||
cs.assert_called_once()
|
||||
self.assertIn("committed:latest", cs.call_args.args[0])
|
||||
|
||||
def test_no_committed_image_checks_agent_and_sidecar(self) -> None:
|
||||
from bot_bottle.backend.docker import launch as mod
|
||||
from datetime import datetime, timezone
|
||||
ts = datetime(2025, 1, 1, tzinfo=timezone.utc)
|
||||
with patch.object(mod, "read_committed_image", return_value=""), \
|
||||
patch.object(mod.docker_mod, "image_exists", return_value=True), \
|
||||
patch.object(mod.docker_mod, "image_created_at", return_value=ts), \
|
||||
patch.object(mod, "check_stale") as cs:
|
||||
mod.stale_checks(self._plan())
|
||||
# Should have been called for both agent and sidecar images.
|
||||
self.assertEqual(2, cs.call_count)
|
||||
|
||||
def test_image_not_present_skips_check(self) -> None:
|
||||
from bot_bottle.backend.docker import launch as mod
|
||||
with patch.object(mod, "read_committed_image", return_value=""), \
|
||||
patch.object(mod.docker_mod, "image_exists", return_value=False), \
|
||||
patch.object(mod, "check_stale") as cs:
|
||||
mod.stale_checks(self._plan())
|
||||
cs.assert_not_called()
|
||||
|
||||
def test_only_sidecar_missing_checks_only_agent(self) -> None:
|
||||
from bot_bottle.backend.docker import launch as mod
|
||||
from datetime import datetime, timezone
|
||||
ts = datetime(2025, 1, 1, tzinfo=timezone.utc)
|
||||
plan = self._plan()
|
||||
|
||||
def image_exists(ref: str) -> bool:
|
||||
return ref == plan.image # Only agent present; sidecar missing.
|
||||
|
||||
with patch.object(mod, "read_committed_image", return_value=""), \
|
||||
patch.object(mod.docker_mod, "image_exists", side_effect=image_exists), \
|
||||
patch.object(mod.docker_mod, "image_created_at", return_value=ts), \
|
||||
patch.object(mod, "check_stale") as cs:
|
||||
mod.stale_checks(plan)
|
||||
self.assertEqual(1, cs.call_count)
|
||||
self.assertIn(plan.image, cs.call_args.args[0])
|
||||
|
||||
def test_backend_image_stale_checks_delegates(self) -> None:
|
||||
from bot_bottle.backend.docker.backend import DockerBottleBackend
|
||||
from bot_bottle.backend.docker import launch as mod
|
||||
backend = DockerBottleBackend()
|
||||
plan = self._plan()
|
||||
with patch.object(mod, "stale_checks") as sc:
|
||||
backend._image_stale_checks(plan) # type: ignore[arg-type]
|
||||
sc.assert_called_once_with(plan)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# smolmachines backend stale_checks
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestSmolmachinesStaleChecks(unittest.TestCase):
|
||||
def _plan(self, policy: str = "cached") -> Any:
|
||||
return cast(Any, SimpleNamespace(
|
||||
spec=SimpleNamespace(image_policy=policy),
|
||||
slug="dev-abc",
|
||||
agent_image="bot-bottle-claude:latest",
|
||||
))
|
||||
|
||||
def test_fresh_policy_is_noop(self) -> None:
|
||||
from bot_bottle.backend.smolmachines import launch as mod
|
||||
with patch.object(mod, "read_committed_image") as rci, \
|
||||
patch.object(mod, "check_stale_path") as csp:
|
||||
mod.stale_checks(self._plan("fresh"))
|
||||
rci.assert_not_called()
|
||||
csp.assert_not_called()
|
||||
|
||||
def test_committed_artifact_found_checks_it(self) -> None:
|
||||
from bot_bottle.backend.smolmachines import launch as mod
|
||||
with tempfile.NamedTemporaryFile(suffix=".smolmachine") as f:
|
||||
committed_path = f.name
|
||||
with patch.object(mod, "read_committed_image", return_value=committed_path), \
|
||||
patch.object(mod.docker_mod, "image_exists", return_value=False), \
|
||||
patch.object(mod, "check_stale_path") as csp:
|
||||
mod.stale_checks(self._plan())
|
||||
csp.assert_called_once()
|
||||
self.assertIn("agent smolmachine", csp.call_args.args[0])
|
||||
|
||||
def test_no_committed_agent_artifact_in_cache_is_checked(self) -> None:
|
||||
from bot_bottle.backend.smolmachines import launch as mod
|
||||
digest = "abcdef0123456789"
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
artifact = Path(tmp) / f"{digest}.smolmachine.smolmachine"
|
||||
artifact.write_text("")
|
||||
with patch.object(mod, "_SMOLMACHINE_CACHE_DIR", Path(tmp)), \
|
||||
patch.object(mod, "read_committed_image", return_value=""), \
|
||||
patch.object(mod.docker_mod, "image_exists", return_value=True), \
|
||||
patch.object(
|
||||
mod.docker_mod, "image_id",
|
||||
return_value=f"sha256:{digest}ffffffffffffffff",
|
||||
), \
|
||||
patch.object(mod, "check_stale_path") as csp:
|
||||
mod.stale_checks(self._plan())
|
||||
# At least the agent artifact check ran.
|
||||
self.assertGreaterEqual(csp.call_count, 1)
|
||||
|
||||
def test_sidecar_artifact_in_cache_is_checked(self) -> None:
|
||||
from bot_bottle.backend.smolmachines import launch as mod
|
||||
sidecar_digest = "fedcba9876543210"
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
sidecar_artifact = (
|
||||
Path(tmp) / f"{sidecar_digest}.smolmachine.smolmachine"
|
||||
)
|
||||
sidecar_artifact.write_text("")
|
||||
|
||||
def image_exists(ref: str) -> bool: # pylint: disable=unused-argument
|
||||
return True
|
||||
|
||||
def image_id(ref: str) -> str:
|
||||
if ref == mod._bundle.SIDECAR_BUNDLE_IMAGE: # type: ignore[attr-defined]
|
||||
return f"sha256:{sidecar_digest}ffffffff"
|
||||
return "sha256:0000000000000000ffffffff"
|
||||
|
||||
with patch.object(mod, "_SMOLMACHINE_CACHE_DIR", Path(tmp)), \
|
||||
patch.object(mod, "read_committed_image", return_value=""), \
|
||||
patch.object(mod.docker_mod, "image_exists", side_effect=image_exists), \
|
||||
patch.object(mod.docker_mod, "image_id", side_effect=image_id), \
|
||||
patch.object(mod, "check_stale_path") as csp:
|
||||
mod.stale_checks(self._plan())
|
||||
sidecar_calls = [c for c in csp.call_args_list if "sidecar" in c.args[0]]
|
||||
self.assertGreaterEqual(len(sidecar_calls), 1)
|
||||
|
||||
def test_backend_image_stale_checks_delegates(self) -> None:
|
||||
from bot_bottle.backend.smolmachines.backend import SmolmachinesBottleBackend
|
||||
from bot_bottle.backend.smolmachines import launch as mod
|
||||
backend = SmolmachinesBottleBackend()
|
||||
plan = self._plan()
|
||||
with patch.object(mod, "stale_checks") as sc:
|
||||
backend._image_stale_checks(plan) # type: ignore[arg-type]
|
||||
sc.assert_called_once_with(plan)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# macOS container backend stale_checks
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class TestMacosContainerStaleChecks(unittest.TestCase):
|
||||
def _plan(self, policy: str = "cached", slug: str = "dev-abc") -> Any:
|
||||
return cast(Any, SimpleNamespace(
|
||||
spec=SimpleNamespace(image_policy=policy),
|
||||
slug=slug,
|
||||
image="bot-bottle-agent:latest",
|
||||
))
|
||||
|
||||
def test_fresh_policy_is_noop(self) -> None:
|
||||
from bot_bottle.backend.macos_container import launch as mod
|
||||
with patch.object(mod, "read_committed_image") as rci, \
|
||||
patch.object(mod, "check_stale") as cs:
|
||||
mod.stale_checks(self._plan("fresh"))
|
||||
rci.assert_not_called()
|
||||
cs.assert_not_called()
|
||||
|
||||
def test_committed_image_present_checks_only_committed(self) -> None:
|
||||
from bot_bottle.backend.macos_container import launch as mod
|
||||
from datetime import datetime, timezone
|
||||
ts = datetime(2025, 1, 1, tzinfo=timezone.utc)
|
||||
with patch.object(mod, "read_committed_image", return_value="committed:latest"), \
|
||||
patch.object(mod.container_mod, "image_exists", return_value=True), \
|
||||
patch.object(mod.container_mod, "image_created_at", return_value=ts), \
|
||||
patch.object(mod, "check_stale") as cs:
|
||||
mod.stale_checks(self._plan())
|
||||
cs.assert_called_once()
|
||||
self.assertIn("committed:latest", cs.call_args.args[0])
|
||||
|
||||
def test_no_committed_image_checks_agent_and_sidecar(self) -> None:
|
||||
from bot_bottle.backend.macos_container import launch as mod
|
||||
from datetime import datetime, timezone
|
||||
ts = datetime(2025, 1, 1, tzinfo=timezone.utc)
|
||||
with patch.object(mod, "read_committed_image", return_value=""), \
|
||||
patch.object(mod.container_mod, "image_exists", return_value=True), \
|
||||
patch.object(mod.container_mod, "image_created_at", return_value=ts), \
|
||||
patch.object(mod, "check_stale") as cs:
|
||||
mod.stale_checks(self._plan())
|
||||
self.assertEqual(2, cs.call_count)
|
||||
|
||||
def test_image_not_present_skips_check(self) -> None:
|
||||
from bot_bottle.backend.macos_container import launch as mod
|
||||
with patch.object(mod, "read_committed_image", return_value=""), \
|
||||
patch.object(mod.container_mod, "image_exists", return_value=False), \
|
||||
patch.object(mod, "check_stale") as cs:
|
||||
mod.stale_checks(self._plan())
|
||||
cs.assert_not_called()
|
||||
|
||||
def test_backend_image_stale_checks_delegates(self) -> None:
|
||||
from bot_bottle.backend.macos_container.backend import MacosContainerBottleBackend
|
||||
from bot_bottle.backend.macos_container import launch as mod
|
||||
backend = MacosContainerBottleBackend()
|
||||
plan = self._plan()
|
||||
with patch.object(mod, "stale_checks") as sc:
|
||||
backend._image_stale_checks(plan) # type: ignore[arg-type]
|
||||
sc.assert_called_once_with(plan)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -1,5 +1,6 @@
|
||||
"""Unit: supervise queue + audit log + diff helpers (PRD 0013)."""
|
||||
|
||||
import json
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
@@ -8,9 +9,6 @@ from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
from bot_bottle import supervise
|
||||
from bot_bottle import supervise_types as sv_types
|
||||
from bot_bottle.audit_store import AuditStore
|
||||
from bot_bottle.queue_store import QueueStore
|
||||
from bot_bottle.supervise import (
|
||||
AuditEntry,
|
||||
Proposal,
|
||||
@@ -21,7 +19,7 @@ from bot_bottle.supervise import (
|
||||
TOOL_EGRESS_ALLOW,
|
||||
TOOL_GITLEAKS_ALLOW,
|
||||
archive_proposal,
|
||||
host_db_path,
|
||||
audit_log_path,
|
||||
list_pending_proposals,
|
||||
read_audit_entries,
|
||||
read_proposal,
|
||||
@@ -114,52 +112,32 @@ class TestResponseRoundtrip(unittest.TestCase):
|
||||
class TestQueueIO(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self._tmp = tempfile.TemporaryDirectory(prefix="bot-bottle-supervise-test.")
|
||||
self._home_patch = self._patch_home(Path(self._tmp.name))
|
||||
self.slug = "dev"
|
||||
QueueStore("").migrate()
|
||||
self.queue_dir = Path(self._tmp.name)
|
||||
|
||||
def tearDown(self):
|
||||
self._home_patch()
|
||||
self._tmp.cleanup()
|
||||
|
||||
def _patch_home(self, fake_home: Path):
|
||||
original_sv = supervise.bot_bottle_root
|
||||
original_svt = sv_types.bot_bottle_root
|
||||
|
||||
def fake_root() -> Path:
|
||||
return fake_home / ".bot-bottle"
|
||||
|
||||
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
|
||||
sv_types.bot_bottle_root = fake_root # type: ignore[assignment]
|
||||
|
||||
def restore() -> None:
|
||||
supervise.bot_bottle_root = original_sv # type: ignore[assignment]
|
||||
sv_types.bot_bottle_root = original_svt # type: ignore[assignment]
|
||||
|
||||
return restore
|
||||
|
||||
def test_write_and_read_proposal(self):
|
||||
p = _proposal()
|
||||
path = write_proposal(p)
|
||||
path = write_proposal(self.queue_dir, p)
|
||||
self.assertTrue(path.exists())
|
||||
self.assertEqual(host_db_path(), path)
|
||||
self.assertEqual(0o600, path.stat().st_mode & 0o777)
|
||||
loaded = read_proposal(self.slug, p.id)
|
||||
loaded = read_proposal(self.queue_dir, p.id)
|
||||
self.assertEqual(p, loaded)
|
||||
|
||||
def test_list_pending_excludes_responded(self):
|
||||
a = _proposal(justification="first")
|
||||
b = _proposal(justification="second")
|
||||
write_proposal(a)
|
||||
write_proposal(b)
|
||||
write_response(self.slug, Response(
|
||||
write_proposal(self.queue_dir, a)
|
||||
write_proposal(self.queue_dir, b)
|
||||
write_response(self.queue_dir, Response(
|
||||
proposal_id=a.id, status=STATUS_APPROVED, notes="",
|
||||
))
|
||||
pending = list_pending_proposals(self.slug)
|
||||
pending = list_pending_proposals(self.queue_dir)
|
||||
self.assertEqual([b.id], [p.id for p in pending])
|
||||
|
||||
def test_list_pending_returns_empty_for_missing_slug(self):
|
||||
self.assertEqual([], list_pending_proposals("nope"))
|
||||
def test_list_pending_returns_empty_for_missing_dir(self):
|
||||
self.assertEqual([], list_pending_proposals(self.queue_dir / "nope"))
|
||||
|
||||
def test_list_pending_sorted_by_arrival(self):
|
||||
# Fabricate two with explicit timestamps.
|
||||
@@ -176,30 +154,30 @@ class TestQueueIO(unittest.TestCase):
|
||||
now=datetime(2026, 5, 25, 14, 0, 0, tzinfo=timezone.utc),
|
||||
)
|
||||
# Write in reverse order.
|
||||
write_proposal(b)
|
||||
write_proposal(a)
|
||||
ordered = list_pending_proposals(self.slug)
|
||||
write_proposal(self.queue_dir, b)
|
||||
write_proposal(self.queue_dir, a)
|
||||
ordered = list_pending_proposals(self.queue_dir)
|
||||
self.assertEqual([a.id, b.id], [p.id for p in ordered])
|
||||
|
||||
def test_write_and_read_response(self):
|
||||
r = Response(proposal_id="xyz", status=STATUS_REJECTED, notes="no")
|
||||
write_response(self.slug, r)
|
||||
self.assertEqual(r, read_response(self.slug, "xyz"))
|
||||
write_response(self.queue_dir, r)
|
||||
self.assertEqual(r, read_response(self.queue_dir, "xyz"))
|
||||
|
||||
def test_wait_for_response_returns_when_file_appears(self):
|
||||
p = _proposal()
|
||||
write_proposal(p)
|
||||
write_proposal(self.queue_dir, p)
|
||||
|
||||
def write_after_delay():
|
||||
time.sleep(0.05)
|
||||
write_response(self.slug, Response(
|
||||
write_response(self.queue_dir, Response(
|
||||
proposal_id=p.id, status=STATUS_APPROVED, notes="ok",
|
||||
))
|
||||
|
||||
t = threading.Thread(target=write_after_delay)
|
||||
t.start()
|
||||
try:
|
||||
r = wait_for_response(self.slug, p.id, poll_interval=0.01)
|
||||
r = wait_for_response(self.queue_dir, p.id, poll_interval=0.01)
|
||||
finally:
|
||||
t.join()
|
||||
self.assertEqual(STATUS_APPROVED, r.status)
|
||||
@@ -209,51 +187,44 @@ class TestQueueIO(unittest.TestCase):
|
||||
deadline = time.monotonic() + 0.05
|
||||
with self.assertRaises(TimeoutError):
|
||||
wait_for_response(
|
||||
self.slug, "never",
|
||||
self.queue_dir, "never",
|
||||
poll_interval=0.01, deadline=deadline,
|
||||
)
|
||||
|
||||
def test_archive_proposal_hides_rows(self):
|
||||
def test_archive_proposal_moves_both_files(self):
|
||||
p = _proposal()
|
||||
write_proposal(p)
|
||||
write_response(self.slug, Response(
|
||||
write_proposal(self.queue_dir, p)
|
||||
write_response(self.queue_dir, Response(
|
||||
proposal_id=p.id, status=STATUS_APPROVED, notes="",
|
||||
))
|
||||
archive_proposal(self.slug, p.id)
|
||||
self.assertEqual([], list_pending_proposals(self.slug))
|
||||
with self.assertRaises(FileNotFoundError):
|
||||
read_response(self.slug, p.id)
|
||||
archive_proposal(self.queue_dir, p.id)
|
||||
self.assertFalse((self.queue_dir / f"{p.id}.proposal.json").exists())
|
||||
self.assertFalse((self.queue_dir / f"{p.id}.response.json").exists())
|
||||
self.assertTrue((self.queue_dir / "processed" / f"{p.id}.proposal.json").exists())
|
||||
self.assertTrue((self.queue_dir / "processed" / f"{p.id}.response.json").exists())
|
||||
|
||||
def test_archive_is_idempotent_on_missing_files(self):
|
||||
# Should not raise.
|
||||
archive_proposal(self.slug, "nope")
|
||||
archive_proposal(self.queue_dir, "nope")
|
||||
|
||||
|
||||
class TestAuditLog(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self._tmp = tempfile.TemporaryDirectory(prefix="bot-bottle-supervise-audit.")
|
||||
self._home_patch = self._patch_home(Path(self._tmp.name))
|
||||
AuditStore().migrate()
|
||||
|
||||
def tearDown(self):
|
||||
self._home_patch()
|
||||
self._tmp.cleanup()
|
||||
|
||||
def _patch_home(self, fake_home: Path):
|
||||
original_sv = supervise.bot_bottle_root
|
||||
original_svt = sv_types.bot_bottle_root
|
||||
original = supervise.bot_bottle_root
|
||||
|
||||
def fake_root() -> Path:
|
||||
return fake_home / ".bot-bottle"
|
||||
|
||||
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
|
||||
sv_types.bot_bottle_root = fake_root # type: ignore[assignment]
|
||||
|
||||
def restore() -> None:
|
||||
supervise.bot_bottle_root = original_sv # type: ignore[assignment]
|
||||
sv_types.bot_bottle_root = original_svt # type: ignore[assignment]
|
||||
|
||||
return restore
|
||||
return lambda: setattr(supervise, "bot_bottle_root", original)
|
||||
|
||||
def test_write_then_read_single_entry(self):
|
||||
e = AuditEntry(
|
||||
@@ -266,7 +237,6 @@ class TestAuditLog(unittest.TestCase):
|
||||
diff="--- before\n+++ after\n",
|
||||
)
|
||||
path = write_audit_entry(e)
|
||||
self.assertEqual(host_db_path(), path)
|
||||
self.assertEqual(0o600, path.stat().st_mode & 0o777)
|
||||
loaded = read_audit_entries("cred-proxy", "dev")
|
||||
self.assertEqual([e], loaded)
|
||||
@@ -282,13 +252,12 @@ class TestAuditLog(unittest.TestCase):
|
||||
justification="",
|
||||
diff="",
|
||||
))
|
||||
entries = read_audit_entries("egress", "dev")
|
||||
self.assertEqual(3, len(entries))
|
||||
self.assertEqual(
|
||||
["2026-05-25T12:00:00+00:00", "2026-05-25T12:00:01+00:00",
|
||||
"2026-05-25T12:00:02+00:00"],
|
||||
[entry.timestamp for entry in entries],
|
||||
)
|
||||
path = audit_log_path("egress", "dev")
|
||||
with path.open() as f:
|
||||
lines = [line for line in f if line.strip()]
|
||||
self.assertEqual(3, len(lines))
|
||||
for line in lines:
|
||||
self.assertTrue(json.loads(line)) # each line is valid JSON
|
||||
|
||||
def test_separate_logs_per_component_slug(self):
|
||||
write_audit_entry(AuditEntry(
|
||||
@@ -400,24 +369,17 @@ class TestSupervisePrepare(unittest.TestCase):
|
||||
self._tmp.cleanup()
|
||||
|
||||
def _patch_home(self, fake_home: Path):
|
||||
original_sv = supervise.bot_bottle_root
|
||||
original_svt = sv_types.bot_bottle_root
|
||||
original = supervise.bot_bottle_root
|
||||
|
||||
def fake_root() -> Path:
|
||||
return fake_home / ".bot-bottle"
|
||||
|
||||
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
|
||||
sv_types.bot_bottle_root = fake_root # type: ignore[assignment]
|
||||
|
||||
def restore() -> None:
|
||||
supervise.bot_bottle_root = original_sv # type: ignore[assignment]
|
||||
sv_types.bot_bottle_root = original_svt # type: ignore[assignment]
|
||||
|
||||
return restore
|
||||
return lambda: setattr(supervise, "bot_bottle_root", original)
|
||||
|
||||
def test_prepare_creates_queue(self):
|
||||
plan = _StubSupervise().prepare("dev", self.stage_dir)
|
||||
self.assertTrue(plan.db_path.is_file())
|
||||
self.assertTrue(plan.queue_dir.is_dir())
|
||||
self.assertEqual("dev", plan.slug)
|
||||
self.assertEqual("", plan.internal_network)
|
||||
|
||||
|
||||
@@ -12,10 +12,7 @@ from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
from bot_bottle import supervise
|
||||
from bot_bottle import supervise_types as sv_types
|
||||
from bot_bottle.audit_store import AuditStore
|
||||
from bot_bottle.cli import supervise as supervise_cli
|
||||
from bot_bottle.queue_store import QueueStore
|
||||
from bot_bottle.supervise import (
|
||||
Proposal,
|
||||
STATUS_APPROVED,
|
||||
@@ -55,22 +52,13 @@ class _FakeHomeMixin:
|
||||
|
||||
def _setup_fake_home(self):
|
||||
self._tmp = tempfile.TemporaryDirectory(prefix="supervise-test.")
|
||||
original_sv = supervise.bot_bottle_root
|
||||
original_svt = sv_types.bot_bottle_root
|
||||
original = supervise.bot_bottle_root
|
||||
|
||||
def fake_root() -> Path:
|
||||
return Path(self._tmp.name) / ".bot-bottle"
|
||||
|
||||
supervise.bot_bottle_root = fake_root # type: ignore[assignment]
|
||||
sv_types.bot_bottle_root = fake_root # type: ignore[assignment]
|
||||
|
||||
def restore() -> None:
|
||||
supervise.bot_bottle_root = original_sv # type: ignore[assignment]
|
||||
sv_types.bot_bottle_root = original_svt # type: ignore[assignment]
|
||||
|
||||
self._restore_home = restore
|
||||
QueueStore("").migrate()
|
||||
AuditStore().migrate()
|
||||
self._restore_home = lambda: setattr(supervise, "bot_bottle_root", original)
|
||||
|
||||
def _teardown_fake_home(self):
|
||||
self._restore_home()
|
||||
@@ -89,7 +77,9 @@ class TestDiscoverPending(_FakeHomeMixin, unittest.TestCase):
|
||||
|
||||
def test_walks_all_slug_subdirs(self):
|
||||
for slug in ("dev", "api"):
|
||||
supervise.write_proposal(_proposal(slug=slug))
|
||||
qdir = supervise.queue_dir_for_slug(slug)
|
||||
qdir.mkdir(parents=True)
|
||||
supervise.write_proposal(qdir, _proposal(slug=slug))
|
||||
pending = supervise_cli.discover_pending()
|
||||
self.assertEqual({"dev", "api"}, {qp.proposal.bottle_slug for qp in pending})
|
||||
|
||||
@@ -107,14 +97,18 @@ class TestDiscoverPending(_FakeHomeMixin, unittest.TestCase):
|
||||
now=datetime(2026, 5, 25, 14, 0, 0, tzinfo=timezone.utc),
|
||||
)
|
||||
for p in (late, early):
|
||||
supervise.write_proposal(p)
|
||||
qdir = supervise.queue_dir_for_slug(p.bottle_slug)
|
||||
qdir.mkdir(parents=True, exist_ok=True)
|
||||
supervise.write_proposal(qdir, p)
|
||||
pending = supervise_cli.discover_pending()
|
||||
self.assertEqual([early.id, late.id], [qp.proposal.id for qp in pending])
|
||||
|
||||
def test_excludes_already_responded(self):
|
||||
p = _proposal()
|
||||
supervise.write_proposal(p)
|
||||
supervise.write_response("dev", supervise.Response(
|
||||
qdir = supervise.queue_dir_for_slug("dev")
|
||||
qdir.mkdir(parents=True)
|
||||
supervise.write_proposal(qdir, p)
|
||||
supervise.write_response(qdir, supervise.Response(
|
||||
proposal_id=p.id, status=STATUS_APPROVED, notes="",
|
||||
))
|
||||
self.assertEqual([], supervise_cli.discover_pending())
|
||||
@@ -129,8 +123,10 @@ class TestApproveReject(_FakeHomeMixin, unittest.TestCase):
|
||||
|
||||
def _enqueue(self, tool: str = TOOL_EGRESS_ALLOW):
|
||||
p = _proposal(tool=tool)
|
||||
supervise.write_proposal(p)
|
||||
return supervise_cli.QueuedProposal(proposal=p)
|
||||
qdir = supervise.queue_dir_for_slug("dev")
|
||||
qdir.mkdir(parents=True, exist_ok=True)
|
||||
supervise.write_proposal(qdir, p)
|
||||
return supervise_cli.QueuedProposal(proposal=p, queue_dir=qdir)
|
||||
|
||||
def test_approve_writes_response(self):
|
||||
qp = self._enqueue()
|
||||
@@ -139,7 +135,7 @@ class TestApproveReject(_FakeHomeMixin, unittest.TestCase):
|
||||
return_value=("routes: []\n", "routes:\n - host: example.com\n"),
|
||||
):
|
||||
supervise_cli.approve(qp)
|
||||
resp = read_response(qp.proposal.bottle_slug, qp.proposal.id)
|
||||
resp = read_response(qp.queue_dir, qp.proposal.id)
|
||||
self.assertEqual(STATUS_APPROVED, resp.status)
|
||||
self.assertIsNone(resp.final_file)
|
||||
|
||||
@@ -154,7 +150,7 @@ class TestApproveReject(_FakeHomeMixin, unittest.TestCase):
|
||||
final_file="routes:\n - host: edited.example.com\n",
|
||||
notes="tweaked",
|
||||
)
|
||||
resp = read_response(qp.proposal.bottle_slug, qp.proposal.id)
|
||||
resp = read_response(qp.queue_dir, qp.proposal.id)
|
||||
self.assertEqual(STATUS_MODIFIED, resp.status)
|
||||
self.assertEqual("routes:\n - host: edited.example.com\n", resp.final_file)
|
||||
self.assertEqual("tweaked", resp.notes)
|
||||
@@ -162,7 +158,7 @@ class TestApproveReject(_FakeHomeMixin, unittest.TestCase):
|
||||
def test_reject_writes_rejection(self):
|
||||
qp = self._enqueue()
|
||||
supervise_cli.reject(qp, reason="nope")
|
||||
resp = read_response(qp.proposal.bottle_slug, qp.proposal.id)
|
||||
resp = read_response(qp.queue_dir, qp.proposal.id)
|
||||
self.assertEqual(STATUS_REJECTED, resp.status)
|
||||
self.assertEqual("nope", resp.notes)
|
||||
|
||||
@@ -185,33 +181,36 @@ class TestApproveReject(_FakeHomeMixin, unittest.TestCase):
|
||||
def test_approve_gitleaks_allow_leaves_response_for_gate(self):
|
||||
qp = self._enqueue(tool=TOOL_GITLEAKS_ALLOW)
|
||||
supervise_cli.approve(qp, notes="dummy fixture")
|
||||
# Gate polls the DB for the response; TUI must not archive it.
|
||||
resp = read_response(qp.proposal.bottle_slug, qp.proposal.id)
|
||||
# Gate polls the queue dir for the response; TUI must not archive it.
|
||||
resp = read_response(qp.queue_dir, qp.proposal.id)
|
||||
self.assertEqual(STATUS_APPROVED, resp.status)
|
||||
self.assertEqual("dummy fixture", resp.notes)
|
||||
self.assertFalse((qp.queue_dir / "processed").exists())
|
||||
|
||||
def test_tui_gitleaks_allow_requires_reason(self):
|
||||
qp = self._enqueue(tool=TOOL_GITLEAKS_ALLOW)
|
||||
with patch.object(supervise_cli, "_prompt", return_value=""):
|
||||
status = supervise_cli._approve_from_tui(None, qp) # type: ignore[arg-type]
|
||||
self.assertEqual("approve aborted (empty reason)", status)
|
||||
self.assertFalse((qp.queue_dir / "processed").exists())
|
||||
|
||||
def test_tui_gitleaks_allow_writes_reason(self):
|
||||
qp = self._enqueue(tool=TOOL_GITLEAKS_ALLOW)
|
||||
with patch.object(supervise_cli, "_prompt", return_value="test fixture"):
|
||||
status = supervise_cli._approve_from_tui(None, qp) # type: ignore[arg-type]
|
||||
self.assertIn("approved gitleaks-allow", status)
|
||||
resp = read_response(qp.proposal.bottle_slug, qp.proposal.id)
|
||||
resp = read_response(qp.queue_dir, qp.proposal.id)
|
||||
self.assertEqual("test fixture", resp.notes)
|
||||
|
||||
def test_approve_token_allow_leaves_response_for_egress(self):
|
||||
qp = self._enqueue(tool=TOOL_EGRESS_TOKEN_ALLOW)
|
||||
supervise_cli.approve(qp, notes="false positive")
|
||||
# The egress addon polls the DB for the response; the TUI must
|
||||
# The egress addon polls the queue dir for the response; the TUI must
|
||||
# not archive it (the addon archives after reading).
|
||||
resp = read_response(qp.proposal.bottle_slug, qp.proposal.id)
|
||||
resp = read_response(qp.queue_dir, qp.proposal.id)
|
||||
self.assertEqual(STATUS_APPROVED, resp.status)
|
||||
self.assertEqual("false positive", resp.notes)
|
||||
self.assertFalse((qp.queue_dir / "processed").exists())
|
||||
|
||||
def test_token_allow_writes_no_audit_log(self):
|
||||
qp = self._enqueue(tool=TOOL_EGRESS_TOKEN_ALLOW)
|
||||
@@ -223,13 +222,14 @@ class TestApproveReject(_FakeHomeMixin, unittest.TestCase):
|
||||
with patch.object(supervise_cli, "_prompt", return_value=""):
|
||||
status = supervise_cli._approve_from_tui(None, qp) # type: ignore[arg-type]
|
||||
self.assertEqual("approve aborted (empty reason)", status)
|
||||
self.assertFalse((qp.queue_dir / "processed").exists())
|
||||
|
||||
def test_tui_token_allow_writes_reason(self):
|
||||
qp = self._enqueue(tool=TOOL_EGRESS_TOKEN_ALLOW)
|
||||
with patch.object(supervise_cli, "_prompt", return_value="legit"):
|
||||
status = supervise_cli._approve_from_tui(None, qp) # type: ignore[arg-type]
|
||||
self.assertIn("approved egress-token-allow", status)
|
||||
resp = read_response(qp.proposal.bottle_slug, qp.proposal.id)
|
||||
resp = read_response(qp.queue_dir, qp.proposal.id)
|
||||
self.assertEqual("legit", resp.notes)
|
||||
|
||||
def test_suffix_for_token_allow_is_txt(self):
|
||||
|
||||
@@ -4,6 +4,7 @@ fallback paths."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import tempfile
|
||||
import time
|
||||
import unittest
|
||||
@@ -11,19 +12,14 @@ from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
from bot_bottle import supervise
|
||||
from bot_bottle.audit_store import AuditStore
|
||||
from bot_bottle.queue_store import QueueStore
|
||||
from bot_bottle.supervise import (
|
||||
AuditEntry,
|
||||
Proposal,
|
||||
STATUS_APPROVED,
|
||||
TOOL_EGRESS_ALLOW,
|
||||
list_pending_proposals,
|
||||
read_audit_entries,
|
||||
read_proposal,
|
||||
read_response,
|
||||
wait_for_response,
|
||||
write_audit_entry,
|
||||
)
|
||||
|
||||
|
||||
@@ -41,59 +37,58 @@ class TestPathHelpers(unittest.TestCase):
|
||||
def test_bot_bottle_root(self) -> None:
|
||||
self.assertTrue(str(supervise.bot_bottle_root()).endswith(".bot-bottle"))
|
||||
|
||||
def test_queue_dir_for_slug(self) -> None:
|
||||
self.assertIn("slug", str(supervise.queue_dir_for_slug("slug")))
|
||||
|
||||
def test_id_from_non_proposal_filename(self) -> None:
|
||||
self.assertIsNone(supervise._id_from_proposal_filename(Path("x.response.json")))
|
||||
|
||||
|
||||
class TestReadMalformed(unittest.TestCase):
|
||||
def test_read_proposal_missing_row(self) -> None:
|
||||
def test_read_proposal_non_dict(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
with patch.dict("os.environ", {"HOME": d}):
|
||||
QueueStore("slug").migrate()
|
||||
with self.assertRaises(FileNotFoundError):
|
||||
read_proposal("slug", "p")
|
||||
(Path(d) / "p.proposal.json").write_text("[]")
|
||||
with self.assertRaises(ValueError):
|
||||
read_proposal(Path(d), "p")
|
||||
|
||||
def test_read_response_missing_row(self) -> None:
|
||||
def test_read_response_non_dict(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
with patch.dict("os.environ", {"HOME": d}):
|
||||
QueueStore("slug").migrate()
|
||||
with self.assertRaises(FileNotFoundError):
|
||||
read_response("slug", "p")
|
||||
(Path(d) / "p.response.json").write_text("[]")
|
||||
with self.assertRaises(ValueError):
|
||||
read_response(Path(d), "p")
|
||||
|
||||
def test_list_pending_reads_db_only(self) -> None:
|
||||
def test_list_pending_skips_malformed(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
with patch.dict("os.environ", {"HOME": d}):
|
||||
QueueStore("slug").migrate()
|
||||
supervise.write_proposal(_proposal())
|
||||
pending = list_pending_proposals("slug")
|
||||
qd = Path(d)
|
||||
(qd / "bad.proposal.json").write_text("{ not json")
|
||||
(qd / "arr.proposal.json").write_text("[]")
|
||||
(qd / "incomplete.proposal.json").write_text("{}") # from_dict raises
|
||||
supervise.write_proposal(qd, _proposal()) # one valid
|
||||
pending = list_pending_proposals(qd)
|
||||
self.assertEqual(1, len(pending))
|
||||
self.assertEqual("slug", pending[0].bottle_slug)
|
||||
|
||||
def test_list_pending_skips_when_response_present(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
with patch.dict("os.environ", {"HOME": d}):
|
||||
QueueStore("slug").migrate()
|
||||
p = _proposal()
|
||||
supervise.write_proposal(p)
|
||||
supervise.write_response("slug", supervise.Response(
|
||||
proposal_id=p.id,
|
||||
status=STATUS_APPROVED,
|
||||
notes="",
|
||||
))
|
||||
self.assertEqual([], list_pending_proposals("slug"))
|
||||
qd = Path(d)
|
||||
p = _proposal()
|
||||
supervise.write_proposal(qd, p)
|
||||
(qd / f"{p.id}.response.json").write_text("{}") # response exists -> skipped
|
||||
self.assertEqual([], list_pending_proposals(qd))
|
||||
|
||||
|
||||
class TestWaitForResponse(unittest.TestCase):
|
||||
def test_missing_response_times_out(self) -> None:
|
||||
def test_malformed_response_then_timeout(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
with patch.dict("os.environ", {"HOME": d}):
|
||||
QueueStore("slug").migrate()
|
||||
with self.assertRaises(TimeoutError):
|
||||
wait_for_response("slug", "p", deadline=time.monotonic())
|
||||
(Path(d) / "p.response.json").write_text("{ not json")
|
||||
with self.assertRaises(TimeoutError):
|
||||
wait_for_response(Path(d), "p", deadline=time.monotonic())
|
||||
|
||||
def test_empty_db_response_does_not_count(self) -> None:
|
||||
def test_incomplete_response_then_timeout(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
with patch.dict("os.environ", {"HOME": d}):
|
||||
QueueStore("slug").migrate()
|
||||
with self.assertRaises(TimeoutError):
|
||||
wait_for_response("slug", "p", deadline=time.monotonic())
|
||||
(Path(d) / "p.response.json").write_text("{}") # dict but from_dict raises
|
||||
with self.assertRaises(TimeoutError):
|
||||
wait_for_response(Path(d), "p", deadline=time.monotonic())
|
||||
|
||||
|
||||
class TestReadAuditEntries(unittest.TestCase):
|
||||
@@ -102,102 +97,35 @@ class TestReadAuditEntries(unittest.TestCase):
|
||||
patch.dict("os.environ", {"HOME": home}):
|
||||
self.assertEqual([], read_audit_entries("egress", "nope"))
|
||||
|
||||
def test_reads_entries_from_db(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as home, \
|
||||
patch.dict("os.environ", {"HOME": home}):
|
||||
AuditStore().migrate()
|
||||
write_audit_entry(AuditEntry(
|
||||
timestamp="t",
|
||||
bottle_slug="slug",
|
||||
component="egress",
|
||||
operator_action="approve",
|
||||
operator_notes="",
|
||||
justification="",
|
||||
diff="",
|
||||
))
|
||||
write_audit_entry(AuditEntry(
|
||||
timestamp="t",
|
||||
bottle_slug="other",
|
||||
component="egress",
|
||||
operator_action="reject",
|
||||
operator_notes="",
|
||||
justification="",
|
||||
diff="",
|
||||
))
|
||||
entries = read_audit_entries("egress", "slug")
|
||||
self.assertEqual(1, len(entries))
|
||||
self.assertEqual("approve", entries[0].operator_action)
|
||||
|
||||
def test_legacy_audit_log_file_does_not_count(self) -> None:
|
||||
def test_skips_malformed_lines(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as home, \
|
||||
patch.dict("os.environ", {"HOME": home}):
|
||||
path = supervise.audit_log_path("egress", "slug")
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(
|
||||
valid = (
|
||||
'{"timestamp": "t", "bottle_slug": "slug", "component": "egress",'
|
||||
' "operator_action": "approve", "operator_notes": "",'
|
||||
' "justification": "", "diff": ""}\n'
|
||||
' "justification": "", "diff": ""}'
|
||||
)
|
||||
path.write_text(
|
||||
"\n" # blank line skipped
|
||||
"{ not json\n" # JSONDecodeError skipped
|
||||
"[]\n" # not a dict skipped
|
||||
"{}\n" # missing fields -> ValueError skipped
|
||||
+ valid + "\n"
|
||||
)
|
||||
entries = read_audit_entries("egress", "slug")
|
||||
self.assertEqual([], entries)
|
||||
self.assertEqual(1, len(entries))
|
||||
self.assertEqual("approve", entries[0].operator_action)
|
||||
|
||||
|
||||
class TestStoreGuardBranches(unittest.TestCase):
|
||||
"""Direct QueueStore / AuditStore construction and early-return guard branches."""
|
||||
|
||||
def test_queue_store_explicit_db_path(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
db = Path(d) / "q.db"
|
||||
store = QueueStore("key", db_path=db)
|
||||
store.migrate()
|
||||
self.assertTrue(db.is_file())
|
||||
self.assertEqual(db, store.db_path)
|
||||
|
||||
def test_queue_store_missing_db_list_pending_returns_empty(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
db = Path(d) / "q.db"
|
||||
store = QueueStore("key", db_path=db)
|
||||
store.migrate()
|
||||
db.unlink()
|
||||
self.assertEqual([], store.list_pending_proposals())
|
||||
|
||||
def test_queue_store_missing_db_list_all_returns_empty(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
db = Path(d) / "q.db"
|
||||
store = QueueStore("key", db_path=db)
|
||||
store.migrate()
|
||||
db.unlink()
|
||||
self.assertEqual([], store.list_all_pending_proposals())
|
||||
|
||||
def test_queue_store_missing_db_archive_is_noop(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
db = Path(d) / "q.db"
|
||||
store = QueueStore("key", db_path=db)
|
||||
store.migrate()
|
||||
db.unlink()
|
||||
store.archive_proposal("anything") # must not raise
|
||||
|
||||
def test_queue_store_chmod_oserror_is_swallowed(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
db = Path(d) / "q.db"
|
||||
store = QueueStore("key", db_path=db)
|
||||
with patch("pathlib.Path.chmod", side_effect=OSError("ro")):
|
||||
store.migrate() # must not raise
|
||||
|
||||
def test_audit_store_missing_db_read_returns_empty(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
db = Path(d) / "a.db"
|
||||
store = AuditStore(db_path=db)
|
||||
store.migrate()
|
||||
db.unlink()
|
||||
self.assertEqual([], store.read_audit_entries("egress", "slug"))
|
||||
|
||||
def test_audit_store_chmod_oserror_is_swallowed(self):
|
||||
with tempfile.TemporaryDirectory() as d:
|
||||
db = Path(d) / "a.db"
|
||||
store = AuditStore(db_path=db)
|
||||
with patch("pathlib.Path.chmod", side_effect=OSError("ro")):
|
||||
store.migrate() # must not raise
|
||||
class TestFlockFallback(unittest.TestCase):
|
||||
def test_flock_on_closed_fd_is_swallowed(self) -> None:
|
||||
# flock on a closed fd raises OSError(EBADF), which the helpers swallow.
|
||||
fd = os.open(os.devnull, os.O_RDONLY)
|
||||
os.close(fd)
|
||||
supervise._try_flock(fd)
|
||||
supervise._try_funlock(fd)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -17,12 +17,8 @@ from unittest.mock import patch
|
||||
# bare name `supervise`.
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent.parent.parent / "bot_bottle"))
|
||||
import supervise as _sv # noqa: E402 # type: ignore
|
||||
import queue_store as _qs # noqa: E402 # type: ignore
|
||||
import audit_store as _as # noqa: E402 # type: ignore
|
||||
import supervise_types as svt_flat # noqa: E402 # type: ignore
|
||||
|
||||
from bot_bottle import supervise_server # noqa: E402
|
||||
from bot_bottle import supervise_types as svt_pkg # noqa: E402
|
||||
from bot_bottle.supervise_server import (
|
||||
ERR_INTERNAL,
|
||||
ERR_INVALID_PARAMS,
|
||||
@@ -116,7 +112,7 @@ class TestRpcErrorTaxonomy(unittest.TestCase):
|
||||
validate_proposed_file(_sv.TOOL_EGRESS_ALLOW, "routes: nope\n")
|
||||
|
||||
def test_unknown_tool_in_tools_call_is_client_error(self):
|
||||
config = ServerConfig(bottle_slug="dev")
|
||||
config = ServerConfig(bottle_slug="dev", queue_dir=Path("/unused"))
|
||||
with self.assertRaises(_RpcClientError) as cm:
|
||||
handle_tools_call({"name": "no-such-tool", "arguments": {}}, config)
|
||||
self.assertEqual(ERR_INVALID_PARAMS, cm.exception.code)
|
||||
@@ -126,9 +122,9 @@ class TestRpcInternalErrorOnIoFailure(unittest.TestCase):
|
||||
def test_write_proposal_os_error_raises_internal(self):
|
||||
config = ServerConfig(
|
||||
bottle_slug="dev",
|
||||
queue_dir=Path("/dev/null/cannot-exist"),
|
||||
)
|
||||
with patch.object(_sv, "write_proposal", side_effect=OSError("disk full")), \
|
||||
self.assertRaises(_RpcInternalError) as cm:
|
||||
with self.assertRaises(_RpcInternalError) as cm:
|
||||
handle_tools_call(
|
||||
{
|
||||
"name": _sv.TOOL_EGRESS_ALLOW,
|
||||
@@ -269,43 +265,21 @@ class TestHandleToolsList(unittest.TestCase):
|
||||
class TestHandleToolsCall(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self._tmp = tempfile.TemporaryDirectory(prefix="supervise-server-test.")
|
||||
self._home_patch = self._patch_home(Path(self._tmp.name))
|
||||
self.config = ServerConfig(bottle_slug="dev")
|
||||
_qs.QueueStore("dev").migrate()
|
||||
_as.AuditStore().migrate()
|
||||
self.queue_dir = Path(self._tmp.name)
|
||||
self.config = ServerConfig(bottle_slug="dev", queue_dir=self.queue_dir)
|
||||
|
||||
def tearDown(self):
|
||||
self._home_patch()
|
||||
self._tmp.cleanup()
|
||||
|
||||
def _patch_home(self, fake_home: Path):
|
||||
original_sv = _sv.bot_bottle_root
|
||||
original_flat = svt_flat.bot_bottle_root
|
||||
original_pkg = svt_pkg.bot_bottle_root
|
||||
|
||||
def fake_root() -> Path:
|
||||
return fake_home / ".bot-bottle"
|
||||
|
||||
_sv.bot_bottle_root = fake_root # type: ignore[assignment]
|
||||
svt_flat.bot_bottle_root = fake_root # type: ignore[assignment]
|
||||
svt_pkg.bot_bottle_root = fake_root # type: ignore[assignment]
|
||||
|
||||
def restore() -> None:
|
||||
_sv.bot_bottle_root = original_sv # type: ignore[assignment]
|
||||
svt_flat.bot_bottle_root = original_flat # type: ignore[assignment]
|
||||
svt_pkg.bot_bottle_root = original_pkg # type: ignore[assignment]
|
||||
|
||||
return restore
|
||||
|
||||
def _respond_when_proposal_appears(self, status: str, notes: str = "") -> threading.Thread:
|
||||
"""Background thread: poll the queue for a fresh proposal, write a
|
||||
matching response. Returns the thread so the test can join it."""
|
||||
def runner():
|
||||
for _ in range(200):
|
||||
pending = _sv.list_pending_proposals("dev")
|
||||
pending = _sv.list_pending_proposals(self.queue_dir)
|
||||
if pending:
|
||||
p = pending[0]
|
||||
_sv.write_response("dev", _sv.Response(
|
||||
_sv.write_response(self.queue_dir, _sv.Response(
|
||||
proposal_id=p.id, status=status, notes=notes,
|
||||
))
|
||||
return
|
||||
@@ -438,11 +412,15 @@ class TestHandleToolsCall(unittest.TestCase):
|
||||
finally:
|
||||
responder.join()
|
||||
# No pending proposals left after archive.
|
||||
self.assertEqual([], _sv.list_pending_proposals("dev"))
|
||||
self.assertEqual([], _sv.list_pending_proposals(self.queue_dir))
|
||||
# Both files moved to processed/.
|
||||
processed = list((self.queue_dir / "processed").glob("*.json"))
|
||||
self.assertEqual(2, len(processed))
|
||||
|
||||
def test_pending_response_times_out_without_archive(self):
|
||||
config = ServerConfig(
|
||||
bottle_slug="dev",
|
||||
queue_dir=self.queue_dir,
|
||||
response_timeout_seconds=0.05,
|
||||
)
|
||||
result = handle_tools_call(
|
||||
@@ -460,7 +438,8 @@ class TestHandleToolsCall(unittest.TestCase):
|
||||
text = result["content"][0]["text"] # type: ignore[index]
|
||||
self.assertIn("status: pending", text)
|
||||
self.assertIn("proposal remains queued", text)
|
||||
self.assertEqual(1, len(_sv.list_pending_proposals("dev")))
|
||||
self.assertEqual(1, len(_sv.list_pending_proposals(self.queue_dir)))
|
||||
self.assertFalse((self.queue_dir / "processed").exists())
|
||||
|
||||
|
||||
class TestHandleListEgressRoutes(unittest.TestCase):
|
||||
@@ -482,7 +461,7 @@ class TestHandleListEgressRoutes(unittest.TestCase):
|
||||
with patch.object(supervise_server.urllib.request, "build_opener", return_value=_Opener()):
|
||||
result = handle_list_egress_routes(
|
||||
{},
|
||||
ServerConfig(bottle_slug="dev"),
|
||||
ServerConfig(bottle_slug="dev", queue_dir=Path("/unused")),
|
||||
)
|
||||
|
||||
self.assertFalse(result["isError"]) # type: ignore[index]
|
||||
@@ -497,7 +476,7 @@ class TestHandleListEgressRoutes(unittest.TestCase):
|
||||
with patch.object(supervise_server.urllib.request, "build_opener", return_value=_Opener()):
|
||||
result = handle_list_egress_routes(
|
||||
{},
|
||||
ServerConfig(bottle_slug="dev"),
|
||||
ServerConfig(bottle_slug="dev", queue_dir=Path("/unused")),
|
||||
)
|
||||
|
||||
self.assertTrue(result["isError"]) # type: ignore[index]
|
||||
@@ -565,6 +544,7 @@ class TestHttpEndToEnd(unittest.TestCase):
|
||||
|
||||
def setUp(self):
|
||||
self._tmp = tempfile.TemporaryDirectory(prefix="supervise-http-test.")
|
||||
self.queue_dir = Path(self._tmp.name)
|
||||
# Pick a random port by binding to :0 first.
|
||||
import socket
|
||||
s = socket.socket()
|
||||
@@ -572,7 +552,7 @@ class TestHttpEndToEnd(unittest.TestCase):
|
||||
self.port = s.getsockname()[1]
|
||||
s.close()
|
||||
self.server = MCPServer(("127.0.0.1", self.port), MCPHandler)
|
||||
self.server.config = ServerConfig(bottle_slug="dev")
|
||||
self.server.config = ServerConfig(bottle_slug="dev", queue_dir=self.queue_dir)
|
||||
self.thread = threading.Thread(
|
||||
target=self.server.serve_forever, daemon=True,
|
||||
)
|
||||
|
||||
@@ -6,7 +6,7 @@ import textwrap
|
||||
import unittest
|
||||
|
||||
from bot_bottle.yaml_subset import YamlSubsetError
|
||||
from bot_bottle.yaml_subset import parse_frontmatter, parse_yaml_subset, serialize_yaml_subset
|
||||
from bot_bottle.yaml_subset import parse_frontmatter, parse_yaml_subset
|
||||
|
||||
|
||||
def _y(s: str):
|
||||
@@ -457,141 +457,5 @@ class TestEdgeAndErrorBranches(unittest.TestCase):
|
||||
self.assertEqual(({}, ""), parse_frontmatter(""))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# serialize_yaml_subset
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestSerializeYamlSubset(unittest.TestCase):
|
||||
def _roundtrip(self, text: str) -> None:
|
||||
"""Parse then serialize then re-parse; result must equal original parse."""
|
||||
parsed = parse_yaml_subset(text)
|
||||
serialized = serialize_yaml_subset(parsed)
|
||||
self.assertEqual(parsed, parse_yaml_subset(serialized))
|
||||
|
||||
# --- scalar values ---------------------------------------------------------
|
||||
|
||||
def test_string_bare(self) -> None:
|
||||
self.assertEqual("key: value\n", serialize_yaml_subset({"key": "value"}))
|
||||
|
||||
def test_string_empty_is_quoted(self) -> None:
|
||||
out = serialize_yaml_subset({"key": ""})
|
||||
self.assertIn("key: ''", out)
|
||||
self._roundtrip("key: ''\n")
|
||||
|
||||
def test_string_true_like_is_quoted(self) -> None:
|
||||
out = serialize_yaml_subset({"key": "true"})
|
||||
self.assertIn("'true'", out)
|
||||
self.assertEqual({"key": "true"}, parse_yaml_subset(out))
|
||||
|
||||
def test_string_int_like_is_quoted(self) -> None:
|
||||
out = serialize_yaml_subset({"key": "42"})
|
||||
self.assertIn("'42'", out)
|
||||
self.assertEqual({"key": "42"}, parse_yaml_subset(out))
|
||||
|
||||
def test_string_special_start_char_is_quoted(self) -> None:
|
||||
for ch in ('"', "'", "[", "{", "!", "&", "*", "#", "|", ">"):
|
||||
out = serialize_yaml_subset({"key": ch + "rest"})
|
||||
self.assertIn("'", out, msg=f"expected quoting for {ch!r}")
|
||||
|
||||
def test_none_emits_null(self) -> None:
|
||||
self.assertEqual("key: null\n", serialize_yaml_subset({"key": None}))
|
||||
|
||||
def test_bool_true(self) -> None:
|
||||
self.assertEqual("key: true\n", serialize_yaml_subset({"key": True}))
|
||||
|
||||
def test_bool_false(self) -> None:
|
||||
self.assertEqual("key: false\n", serialize_yaml_subset({"key": False}))
|
||||
|
||||
def test_int(self) -> None:
|
||||
self.assertEqual("key: 42\n", serialize_yaml_subset({"key": 42}))
|
||||
|
||||
# --- nested dict -----------------------------------------------------------
|
||||
|
||||
def test_nested_dict(self) -> None:
|
||||
data: dict[str, object] = {"outer": {"inner": "val"}}
|
||||
out = serialize_yaml_subset(data)
|
||||
self.assertIn("outer:", out)
|
||||
self.assertIn(" inner: val", out)
|
||||
self.assertEqual(data, parse_yaml_subset(out))
|
||||
|
||||
def test_empty_dict_value_inline(self) -> None:
|
||||
out = serialize_yaml_subset({"key": {}})
|
||||
self.assertIn("key: {}", out)
|
||||
|
||||
# --- lists -----------------------------------------------------------------
|
||||
|
||||
def test_list_of_scalars(self) -> None:
|
||||
data: dict[str, object] = {"items": ["a", "b", "c"]}
|
||||
out = serialize_yaml_subset(data)
|
||||
self.assertEqual(data, parse_yaml_subset(out))
|
||||
|
||||
def test_empty_list_value_inline(self) -> None:
|
||||
out = serialize_yaml_subset({"key": []})
|
||||
self.assertIn("key: []", out)
|
||||
|
||||
def test_list_of_mappings(self) -> None:
|
||||
data: dict[str, object] = {"items": [{"name": "alpha", "val": 1}, {"name": "beta", "val": 2}]}
|
||||
out = serialize_yaml_subset(data)
|
||||
self.assertEqual(data, parse_yaml_subset(out))
|
||||
|
||||
def test_list_item_mapping_with_nested_dict(self) -> None:
|
||||
data: dict[str, object] = {"items": [{"name": "x", "sub": {"k": "v"}}]}
|
||||
out = serialize_yaml_subset(data)
|
||||
self.assertEqual(data, parse_yaml_subset(out))
|
||||
|
||||
def test_list_item_first_key_is_nested_dict(self) -> None:
|
||||
# Covers the branch where the FIRST key of a mapping list item
|
||||
# has a nested dict/list value (not a scalar).
|
||||
data: dict[str, object] = {"items": [{"cfg": {"a": 1, "b": 2}, "name": "x"}]}
|
||||
out = serialize_yaml_subset(data)
|
||||
self.assertEqual(data, parse_yaml_subset(out))
|
||||
|
||||
def test_list_item_mapping_continuation_keys(self) -> None:
|
||||
data: dict[str, object] = {"items": [{"a": 1, "b": 2, "c": 3}]}
|
||||
out = serialize_yaml_subset(data)
|
||||
self.assertEqual(data, parse_yaml_subset(out))
|
||||
|
||||
# --- empty / trivial -------------------------------------------------------
|
||||
|
||||
def test_empty_dict(self) -> None:
|
||||
self.assertEqual("", serialize_yaml_subset({}))
|
||||
|
||||
# --- realistic bottle frontmatter round-trip --------------------------------
|
||||
|
||||
def test_bottle_frontmatter_round_trip(self) -> None:
|
||||
text = textwrap.dedent("""\
|
||||
git-gate:
|
||||
repos:
|
||||
myrepo:
|
||||
url: ssh://git@host/org/repo.git
|
||||
key:
|
||||
provider: static
|
||||
path: /dev/null
|
||||
""")
|
||||
self._roundtrip(text)
|
||||
|
||||
def test_host_key_inserted_at_repo_level(self) -> None:
|
||||
"""The canonical use-case: host_key lands as a sibling of url/key."""
|
||||
from typing import cast
|
||||
text = textwrap.dedent("""\
|
||||
git-gate:
|
||||
repos:
|
||||
myrepo:
|
||||
url: ssh://git@host/org/repo.git
|
||||
key:
|
||||
provider: static
|
||||
path: /dev/null
|
||||
""")
|
||||
data = parse_yaml_subset(text)
|
||||
repos = cast(dict[str, object], cast(dict[str, object], data["git-gate"])["repos"])
|
||||
cast(dict[str, object], repos["myrepo"])["host_key"] = "ssh-ed25519 AAAA"
|
||||
out = serialize_yaml_subset(data)
|
||||
parsed_back = parse_yaml_subset(out)
|
||||
repo = cast(dict[str, object], cast(dict[str, object], cast(dict[str, object], parsed_back["git-gate"])["repos"])["myrepo"])
|
||||
self.assertEqual("ssh-ed25519 AAAA", repo["host_key"])
|
||||
self.assertIn("provider", cast(dict[str, object], repo["key"]))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
Reference in New Issue
Block a user