Compare commits
145 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| bd663196dc | |||
| 6b0de88be6 | |||
| 9a941e59be | |||
| d7a3539755 | |||
| cfe57a50d0 | |||
| e5d551861c | |||
| 369d332204 | |||
| 31cde11b0d | |||
| c41751f3b9 | |||
| e2422c20a0 | |||
| de71533a17 | |||
| 88c4f61901 | |||
| c666eaa63f | |||
| 83eb9e4041 | |||
| 33333ac4d9 | |||
| 4d56f515bc | |||
| c48c3688b8 | |||
| 6040b20e6e | |||
| f2775101a0 | |||
| dd99c495f4 | |||
| eb64a52ffa | |||
| d11e3940fa | |||
| a32c0c7865 | |||
| ccb2956562 | |||
| c6362fda7b | |||
| cb321f7ad4 | |||
| 311cd46185 | |||
| 28335f453f | |||
| a1aa8feb85 | |||
| cb3bb209d6 | |||
| 6e73cc4d86 | |||
| 64fac71025 | |||
| f8ac22c316 | |||
| 9465857a99 | |||
| 200306f1cf | |||
| 77bdaf0a96 | |||
| 7e344bbb53 | |||
| 5eb27cd9a8 | |||
| 5808d0b828 | |||
| 7a991e1f5e | |||
| 5606797ac2 | |||
| ebbb4053cf | |||
| eb3e64ea8f | |||
| 0ec1085238 | |||
| 4c39b45e34 | |||
| 3ea35ba5d2 | |||
| 7c6ab62e26 | |||
| da42740156 | |||
| 56ef71060a | |||
| 294a6ed023 | |||
| 468ab8c290 | |||
| 2596c18954 | |||
| 3ccd09ed0d | |||
| 996a260a98 | |||
| 3375df3f52 | |||
| c9842ce831 | |||
| d314ccf455 | |||
| 31b29631b6 | |||
| 1c11110da5 | |||
| 25ca14a8a2 | |||
| b5b7f15ef9 | |||
| 85e64b5134 | |||
| 1a5b6e25f8 | |||
| 54760964cf | |||
| e463670649 | |||
| 6e6890ebd9 | |||
| 609b3ed090 | |||
| 65faa40b9a | |||
| 9f97de115b | |||
| 8f21f4df19 | |||
| ff7a52c1d2 | |||
| 4ed6b84863 | |||
| 7a124d7d25 | |||
| f00c567469 | |||
| 6f0e5b4589 | |||
| 5da4d05bf2 | |||
| 1a8718ca9d | |||
| c1c225aa05 | |||
| dc7c10d6fe | |||
| a827b0841e | |||
| a9c93ea9df | |||
| bb69af31f8 | |||
| 7644da4280 | |||
| 13e4af421d | |||
| f2d5307573 | |||
| bc9a22b46a | |||
| 932e71c0bf | |||
| d3b0b330aa | |||
| 5e927bcd13 | |||
| 890a146413 | |||
| afdf0779a1 | |||
| eb7cae1fea | |||
| fe82dc7f2b | |||
| b00b0ba4aa | |||
| 3f04567290 | |||
| acb9cd67c6 | |||
| d90ab7e646 | |||
| 8ea90adcaf | |||
| de803e1e76 | |||
| 019efab804 | |||
| 957d37f51f | |||
| 8e084262a0 | |||
| 504144eb9c | |||
| 86374ab293 | |||
| 199edb228c | |||
| 598a20a3f0 | |||
| c8b5ba3812 | |||
| 5ea9fda69b | |||
| 4f7cfc0418 | |||
| 1f38a96561 | |||
| 660b9b3810 | |||
| 328069809b | |||
| b1551045dc | |||
| d02226aab9 | |||
| 39811c9b32 | |||
| f7f161e60f | |||
| e6040fc824 | |||
| 17fc44d0d8 | |||
| 1bebb7467f | |||
| cc1d986a74 | |||
| fabcd026af | |||
| aff042855a | |||
| 39b0c4f720 | |||
| 43a5700ae6 | |||
| 7acdabaf96 | |||
| dfd2d5f620 | |||
| f24e2857ab | |||
| d38432f640 | |||
| 4e570e3e2b | |||
| a64e3170cd | |||
| 4da4babcf4 | |||
| 384e496a1b | |||
| b38c6110f2 | |||
| 74efb1c143 | |||
| f23b2b9683 | |||
| 423003aa05 | |||
| af82f2ba20 | |||
| fe8e15d211 | |||
| b098556757 | |||
| 5c5f277d6d | |||
| 2fa5229695 | |||
| c3caa3ea94 | |||
| ee0607f022 | |||
| afe5d43a9a | |||
| dd332a5759 |
@@ -26,7 +26,7 @@ jobs:
|
||||
- name: Run pylint
|
||||
run: |
|
||||
# Run pylint on all Python files in the repo
|
||||
find . -name '*.py' -not -path './.venv/*' -not -path './.git/*' | xargs pylint --fail-under=8.0 || true
|
||||
find . -name '*.py' -not -path './.venv/*' -not -path './.git/*' | xargs pylint --fail-under=8.0
|
||||
|
||||
- name: Run pyright
|
||||
run: |
|
||||
|
||||
@@ -2,11 +2,18 @@
|
||||
|
||||
## What this is
|
||||
|
||||
bot-bottle spins up an isolated container for running AI coding agents with a
|
||||
curated set of skills and env vars. The point is to run agents with broad
|
||||
permissions inside a sandbox, so a misbehaving agent cannot reach the host.
|
||||
A Python CLI (entry point `cli.py`, package `bot_bottle/`) orchestrates
|
||||
the container lifecycle and the copying of skills and env vars into it.
|
||||
bot-bottle spins up an isolated backend runtime for running AI coding agents
|
||||
with a curated set of skills and env vars. The point is to run agents with
|
||||
broad permissions inside a sandbox, so a misbehaving agent cannot reach the
|
||||
host. A Python CLI (entry point `cli.py`, package `bot_bottle/`) orchestrates
|
||||
the runtime lifecycle and the copying of skills and env vars into it.
|
||||
The default backend on compatible macOS hosts is macos-container:
|
||||
agents and sidecar bundles run through Apple's `container` CLI without
|
||||
requiring Docker. The smolmachines backend remains available with
|
||||
`BOT_BOTTLE_BACKEND=smolmachines` or `--backend=smolmachines`; agents
|
||||
run in a libkrun micro-VM, while the sidecar bundle still uses Docker.
|
||||
The legacy Docker backend remains available with `BOT_BOTTLE_BACKEND=docker`
|
||||
or `--backend=docker`.
|
||||
|
||||
## Goals
|
||||
|
||||
@@ -17,7 +24,7 @@ the container lifecycle and the copying of skills and env vars into it.
|
||||
## Non-goals
|
||||
|
||||
- Communicating between agents directly
|
||||
- Self hosted VMs (v1 uses local Docker containers, not VMs)
|
||||
- Removing the Docker backend
|
||||
- Advanced agent auditing (lean on git history for auditing)
|
||||
|
||||
## Repository layout
|
||||
|
||||
@@ -5,29 +5,52 @@
|
||||
# bot-bottle
|
||||
|
||||
[](https://gitea.dideric.is/didericis/bot-bottle/actions?workflow=test.yml)
|
||||
[](https://github.com/PyCQA/pylint)
|
||||
[](https://github.com/PyCQA/pylint)
|
||||
[](https://github.com/microsoft/pyright)
|
||||
|
||||
**Problem:** Developer wants to run a coding agent without supervision, but they don't want a prompt injected or misbehaving agent wrecking their environment or exfiltrating sensitive data.
|
||||
**Run any coding agent like it might be compromised — and lose nothing when it is.**
|
||||
|
||||
**Solution:** Ephemeral, per agent "bottles" the agent cannot modify that scan all traffic for data exfiltration and limit capabilities and egress to only what the agent needs.
|
||||
bot-bottle is a provider-neutral, security-first substrate for autonomous agents. Bring Claude Code, Codex, or your own harness; each one runs in an ephemeral, per-agent "bottle" it cannot modify, where every byte of egress is scanned for exfiltration and capabilities are narrowed to exactly what the task declares.
|
||||
|
||||
## Features
|
||||
**Problem:** You want to let a coding agent run unsupervised, but a prompt-injected or misbehaving agent — or a poisoned repo, MCP server, or skill — can wreck your environment or exfiltrate your secrets. Locking yourself to one vendor's cloud doesn't fix that; it just moves the blast radius.
|
||||
|
||||
- **Per-bottle egress allowlist** — TLS-bumped HTTP/HTTPS chokepoint with a per-manifest host allowlist and request-body DLP scanner; DoH and arbitrary hosts blocked by default.
|
||||
**Solution:** A neutral control plane that runs *whatever agent you choose* inside an isolation boundary the agent can't touch: TLS-bumped egress allowlisting, outbound/inbound DLP, gitleaks-gated pushes, and host secrets the agent never sees. Swap the agent; keep the guarantees.
|
||||
|
||||
## Why bot-bottle
|
||||
|
||||
### A neutral substrate — bring your own agent
|
||||
|
||||
- **Provider-agnostic by design** — Claude and Codex ship built in; any other agent (Gemini, Aider, a local-model wrapper) is a drop-in plugin at `~/.bot-bottle/contrib/<name>/` — no fork, no PR against this repo. The manifest accepts any provider template, and the isolation, egress, and git guarantees are identical across all of them.
|
||||
- **One control plane, every harness** — the same bottle, egress policy, and supervise flow wrap whichever agent you run, so switching or mixing providers doesn't change your security posture.
|
||||
- **Composable bottles (`extends:`)** — keep provider/runtime policy in one base bottle (e.g. `claude.md`) and overlay task bottles on top.
|
||||
|
||||
### An isolation boundary the agent can't touch
|
||||
|
||||
- **Per-bottle egress allowlist** — TLS-bumped HTTP/HTTPS chokepoint with a per-manifest host allowlist; per-route path/method/header `matches` filtering; outbound DLP scanning for known tokens and secrets, inbound DLP scanning for prompt-injection attempts; DoH and arbitrary hosts blocked by default.
|
||||
- **Tokens the agent never sees** — host secrets live in a sidecar; the agent dials `http://sidecar:9099/<path>` and the proxy strips inbound `Authorization` and injects the real token before forwarding. `printenv` in the agent shows proxy URLs only.
|
||||
- **Gitleaks-scanned push (git-gate)** — `bottle.git` remotes route through a per-bottle `git daemon` that gitleaks-scans incoming refs pre-receive and forwards clean refs upstream over SSH. The agent never holds the upstream credential.
|
||||
- **Manifest-scoped skills + secrets** — each bottle declares its skills, env, git identity, remotes, and egress routes; unknown keys die at load.
|
||||
- **Trust boundary at `$HOME`** — bottles (credentials, egress, remotes) live only under `~/.bot-bottle/bottles/`. Repos may ship agents but not bottles, so a cloned repo can't redirect an env var to an attacker host.
|
||||
- **Composable bottles (`extends:`)** — keep provider/runtime policy in one base bottle (e.g. `claude.md`) and overlay task bottles on top.
|
||||
- **Parallel, isolated bottles** — each bottle is its own per-agent Docker `--internal` network; bottles don't share state or talk to each other.
|
||||
- **Provider templates (Claude, Codex)** — `Dockerfile.claude` / `Dockerfile.codex`, or a bottle-supplied Dockerfile. Claude auth via long-lived OAuth token; Codex via opt-in host device-auth forwarding.
|
||||
|
||||
### Isolation that matches your host
|
||||
|
||||
- **Parallel, isolated bottles** — each bottle runs in its own backend-owned isolation boundary; bottles don't share state or talk to each other.
|
||||
- **gVisor auto-detect** — on Linux hosts where `runsc` is registered with Docker, every bottle launches under it for a userspace syscall barrier; no manifest config required.
|
||||
- **Smolmachines backend (macOS)** — opt-in `BOT_BOTTLE_BACKEND=smolmachines` runs the agent in a libkrun micro-VM with the sidecar bundle still in Docker.
|
||||
- **Apple Container backend (macOS default when available)** — runs the agent and sidecar bundle with Apple's `container` CLI, using a host-only agent network plus a separate sidecar egress network.
|
||||
- **Smolmachines backend** — runs the agent in a libkrun micro-VM while the sidecar bundle stays in Docker. TSI and smolmachines DNS filtering close the raw DNS exfiltration gap that exists in the legacy Docker backend.
|
||||
- **Legacy Docker backend** — still available for examples, CI, and hosts without Apple Container via `BOT_BOTTLE_BACKEND=docker` or `--backend=docker`.
|
||||
|
||||
Per-provider auth (Claude long-lived OAuth token; Codex opt-in host device-auth forwarding) and per-provider images (`Dockerfile.claude` / `Dockerfile.codex`, or a bottle-supplied Dockerfile) are configured on the bottle — see [Manifest](#manifest).
|
||||
|
||||
## Architecture
|
||||
|
||||
A bottle is two containers per agent: an `agent` container, and a `sidecars` container that bundles egress + git-gate + supervise behind a Python init supervisor. They share a per-agent Docker `--internal` network; the agent has no default route off-box.
|
||||
On the default macOS Apple Container backend, a bottle is an agent container on a host-only internal network plus a sidecar bundle attached to both that internal network and a NAT egress network. The agent gets HTTP(S)_PROXY and CA bundle env vars pointing at the sidecar's internal-network IP, so HTTP/HTTPS traffic flows through the sidecar instead of direct egress. `bottle.git` / git-gate is intentionally deferred on this backend until a safe Apple Container key-delivery path exists.
|
||||
|
||||
On the smolmachines backend, a bottle is an agent micro-VM plus a Docker sidecar bundle for egress, git-gate, and supervise. The VM reaches the sidecars through a per-bottle loopback alias allowed by TSI; smolmachines handles DNS filtering below the guest OS.
|
||||
|
||||
On the legacy Docker backend, the same logical bottle is two containers per agent: an `agent` container and a `sidecars` container. They share a per-agent Docker `--internal` network; the agent has no default route off-box.
|
||||
|
||||
The Docker topology looks like this:
|
||||
|
||||
```
|
||||
host ( ./cli.py )
|
||||
@@ -60,9 +83,32 @@ A bottle is two containers per agent: an `agent` container, and a `sidecars` con
|
||||
|
||||
When the agent exits, `cli.py` tears down every sidecar and both networks; nothing about a bottle persists between runs.
|
||||
|
||||
## Install
|
||||
|
||||
Install the CLI with the bootstrap script:
|
||||
|
||||
```sh
|
||||
curl -fsSL https://gitea.dideric.is/didericis/bot-bottle/raw/branch/main/install.sh | sh
|
||||
```
|
||||
|
||||
The script checks Python 3.11+, checks Docker daemon reachability, creates the `~/.bot-bottle/` config directories, installs the Python package with `pipx` when available or `pip --user` otherwise, then runs:
|
||||
|
||||
```sh
|
||||
bot-bottle doctor
|
||||
```
|
||||
|
||||
Python-native installers can use the package metadata directly:
|
||||
|
||||
```sh
|
||||
pipx install git+https://gitea.dideric.is/didericis/bot-bottle.git
|
||||
uv tool install git+https://gitea.dideric.is/didericis/bot-bottle.git
|
||||
```
|
||||
|
||||
## Quickstart
|
||||
|
||||
Requires Docker on the host and a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
|
||||
On compatible macOS hosts, the default backend requires Apple's `container` CLI and does not require Docker. The smolmachines backend requires Docker on the host for the sidecar bundle plus smolvm. The legacy Docker backend requires Docker. Claude bottles also need a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
|
||||
|
||||
Use `BOT_BOTTLE_BACKEND=docker ./cli.py start <agent>` on hosts where Apple Container is not installed and Docker is the desired backend.
|
||||
|
||||
```sh
|
||||
./cli.py start <agent> # builds the image on first run, drops you into claude
|
||||
@@ -96,8 +142,15 @@ egress:
|
||||
routes:
|
||||
- host: gitea.dideric.is
|
||||
auth:
|
||||
scheme: token
|
||||
scheme: token # Bearer | token
|
||||
token_ref: BOT_BOTTLE_GITEA_TOKEN
|
||||
matches: # optional — restrict to specific paths/methods/headers
|
||||
- paths:
|
||||
- {type: prefix, value: /api/v1/}
|
||||
methods: [GET, POST, PATCH, DELETE]
|
||||
dlp: # optional — per-route detector overrides (default: all on)
|
||||
outbound_detectors: [token_patterns, known_secrets]
|
||||
inbound_detectors: false # disable response scanning for this host
|
||||
---
|
||||
|
||||
The `gitea-dev` bottle. Provider auth via the inherited Claude route;
|
||||
@@ -116,6 +169,23 @@ skills:
|
||||
You help maintain Gitea-hosted projects.
|
||||
````
|
||||
|
||||
**Egress route fields:**
|
||||
|
||||
| Field | Required | Description |
|
||||
|---|---|---|
|
||||
| `host` | yes | Hostname to allowlist. One entry per host. |
|
||||
| `role` | no | Reserved for future use. The key is recognised but any value is currently rejected at load. Provider auth routes (e.g. Claude's `api.anthropic.com`) are injected automatically from `agent_provider.auth_token`, not via `role`. |
|
||||
| `auth.scheme` | when `auth` present | `Bearer` or `token`. Injected by the proxy; the agent never sees the value. |
|
||||
| `auth.token_ref` | when `auth` present | Env-var name holding the secret on the host. |
|
||||
| `matches` | no | Array of `{paths, methods, headers}` filters. A request must match at least one entry (if any are given) to be forwarded. |
|
||||
| `matches[].paths` | no | Array of `{type, value}`. `type` is `prefix` (default), `exact`, or `regex`. |
|
||||
| `matches[].methods` | no | Array of HTTP method strings, e.g. `[GET, POST]`. |
|
||||
| `matches[].headers` | no | Array of `{name, value, type}`. `type` is `exact` (default) or `regex`. |
|
||||
| `dlp` | no | Per-route DLP overrides. Omit to use defaults (all detectors on). |
|
||||
| `dlp.outbound_detectors` | no | `false` disables outbound scanning; list restricts to named detectors (`token_patterns`, `known_secrets`). |
|
||||
| `dlp.inbound_detectors` | no | `false` disables inbound scanning; list restricts to named detectors (`naive_injection_detection`). |
|
||||
| `git.fetch` | no | `true` permits smart HTTP clone/fetch (`git-upload-pack`) for this host. Push (`git-receive-pack`) remains blocked. |
|
||||
|
||||
More examples in `examples/`. Full design lives under `docs/prds/`; the trust-boundary rationale is in `docs/prds/0011-per-file-md-manifest.md`.
|
||||
|
||||
## Trademarks
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
# Per-bottle sidecar bundle image (PRD 0024).
|
||||
#
|
||||
# Collapses the prior per-sidecar images (egress, git-gate,
|
||||
# supervise) into one. A small stdlib-Python init supervisor at
|
||||
# /app/sidecar_init.py spawns all daemons, forwards SIGTERM, and
|
||||
# propagates per-daemon stdout/stderr to the container log with a
|
||||
# `[name]` prefix. See PRD 0024 for the rationale.
|
||||
#
|
||||
# Layout:
|
||||
#
|
||||
# /usr/bin/gitleaks gitleaks binary
|
||||
# /app/egress_addon.py + siblings mitmproxy addon (egress)
|
||||
# /app/egress-entrypoint.sh mitmdump launcher
|
||||
# /app/supervise_server.py + .py supervise MCP server
|
||||
# /app/sidecar_init.py PID 1 supervisor
|
||||
# /etc/egress/routes.yaml bind-mounted at run time
|
||||
# /etc/git-gate/pre-receive docker-cp'd at start time
|
||||
# /git-gate-entrypoint.sh docker-cp'd at start time
|
||||
# /git-gate/creds/* docker-cp'd at start time
|
||||
# /git/* bare repos, populated at runtime
|
||||
# /run/supervise/queue/ bind-mounted at run time
|
||||
# /home/mitmproxy/.mitmproxy/ mitmproxy CA dir
|
||||
#
|
||||
# Exposed ports inside the container:
|
||||
# 9099 egress (mitmproxy, agent-facing HTTPS proxy)
|
||||
# 9418 git-gate (git-daemon)
|
||||
# 9420 git-gate smart HTTP (smolmachines agent-facing transport)
|
||||
# 9100 supervise (MCP HTTP)
|
||||
|
||||
# Stage 1: gitleaks binary. The upstream gitleaks image is alpine
|
||||
# with the binary at /usr/bin/gitleaks. Pinned by digest in lockstep
|
||||
# with Dockerfile.git-gate's prior base (now deleted at chunk 3).
|
||||
FROM zricethezav/gitleaks@sha256:c00b6bd0aeb3071cbcb79009cb16a60dd9e0a7c60e2be9ab65d25e6bc8abbb7f AS gitleaks-src
|
||||
|
||||
# Stage 2: assembly. mitmproxy/mitmproxy is debian-slim-based with
|
||||
# Python + mitmdump pre-installed — heavier than the others, so
|
||||
# this stage starts there and pulls the standalone binaries in.
|
||||
FROM mitmproxy/mitmproxy:11.1.3
|
||||
|
||||
# Run as root inside the bundle. The bundle is the isolation
|
||||
# boundary; per-daemon user separation inside it is not load-bearing
|
||||
# and complicates the supervisor's spawn path.
|
||||
USER root
|
||||
|
||||
# Runtime system deps:
|
||||
# git supplies the `git daemon` subcommand (no separate package)
|
||||
# plus the core `git` binary the pre-receive hook invokes.
|
||||
# openssh-client supplies the upstream SSH transport the
|
||||
# pre-receive hook uses to forward accepted refs.
|
||||
# ca-certificates is needed for mitmdump upstream TLS (the
|
||||
# base image already has it; listed for explicitness).
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends \
|
||||
git openssh-client ca-certificates \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Pull the standalone binaries into the final image.
|
||||
COPY --from=gitleaks-src /usr/bin/gitleaks /usr/bin/gitleaks
|
||||
|
||||
# Project Python: addon + server modules + the init supervisor.
|
||||
# Kept flat under /app/ so mitmdump's loader resolves them as
|
||||
# top-level siblings (absolute imports), matching the prior
|
||||
# Dockerfile.egress / Dockerfile.supervise layout.
|
||||
COPY bot_bottle/egress_addon_core.py /app/egress_addon_core.py
|
||||
COPY bot_bottle/egress_addon.py /app/egress_addon.py
|
||||
COPY bot_bottle/dlp_detectors.py /app/dlp_detectors.py
|
||||
COPY bot_bottle/yaml_subset.py /app/yaml_subset.py
|
||||
COPY bot_bottle/supervise.py /app/supervise.py
|
||||
COPY bot_bottle/supervise_server.py /app/supervise_server.py
|
||||
COPY bot_bottle/sidecar_init.py /app/sidecar_init.py
|
||||
COPY bot_bottle/git_http_backend.py /app/git_http_backend.py
|
||||
COPY bot_bottle/egress_entrypoint.sh /app/egress-entrypoint.sh
|
||||
RUN chmod +x /app/egress-entrypoint.sh
|
||||
|
||||
# Pre-create runtime directories the compose renderer + start
|
||||
# step expect to exist. `docker cp` does not create intermediate
|
||||
# dirs, and bind mounts won't either if the parent is missing.
|
||||
RUN mkdir -p \
|
||||
/etc/egress \
|
||||
/etc/git-gate \
|
||||
/git-gate/creds \
|
||||
/git \
|
||||
/run/supervise/queue \
|
||||
/home/mitmproxy/.mitmproxy
|
||||
|
||||
# Documentation only — the compose renderer publishes whichever
|
||||
# subset the bottle uses.
|
||||
EXPOSE 8888 9099 9418 9420 9100
|
||||
|
||||
# WORKDIR matches Dockerfile.supervise's prior layout so the
|
||||
# in-app same-dir import in supervise_server.py stays deterministic.
|
||||
WORKDIR /app
|
||||
|
||||
# PID 1 is the supervisor. It owns signal handling and exit-code
|
||||
# propagation; no `exec` chain in the entrypoint itself.
|
||||
ENTRYPOINT ["python3", "/app/sidecar_init.py"]
|
||||
@@ -20,6 +20,7 @@ Per PRD 0050 the per-provider implementations live under
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib.util
|
||||
import inspect
|
||||
import os
|
||||
import shlex
|
||||
import tempfile
|
||||
@@ -37,13 +38,19 @@ if TYPE_CHECKING:
|
||||
|
||||
PROVIDER_CLAUDE = "claude"
|
||||
PROVIDER_CODEX = "codex"
|
||||
PROVIDER_TEMPLATES = frozenset({PROVIDER_CLAUDE, PROVIDER_CODEX})
|
||||
PROVIDER_PI = "pi"
|
||||
PROVIDER_TEMPLATES = frozenset({PROVIDER_CLAUDE, PROVIDER_CODEX, PROVIDER_PI})
|
||||
|
||||
# Hosts that egress injects the host ChatGPT bearer on when Codex
|
||||
# forward_host_credentials is enabled. Pipelock must pass these through
|
||||
# (no TLS MITM) or its header DLP blocks the injected JWT.
|
||||
CODEX_HOST_CREDENTIAL_HOSTS = ("api.openai.com", "chatgpt.com")
|
||||
PromptMode = Literal["append_file", "read_prompt_file"]
|
||||
PromptMode = Literal[
|
||||
"append_file",
|
||||
"read_prompt_file",
|
||||
"print_read_prompt_file",
|
||||
"append_system_prompt",
|
||||
]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -51,7 +58,6 @@ class AgentProviderRuntime:
|
||||
template: str
|
||||
command: str
|
||||
image: str
|
||||
dockerfile: str
|
||||
prompt_mode: PromptMode
|
||||
bypass_args: tuple[str, ...]
|
||||
resume_args: tuple[str, ...]
|
||||
@@ -103,7 +109,12 @@ class AgentProvisionPlan:
|
||||
prompt_mode: PromptMode
|
||||
image: str
|
||||
dockerfile: str
|
||||
guest_home: str
|
||||
instance_name: str
|
||||
prompt_file: Path
|
||||
guest_env: dict[str, str]
|
||||
has_prompt: bool = False
|
||||
startup_args: tuple[str, ...] = ()
|
||||
env_vars: dict[str, str] = field(default_factory=dict)
|
||||
dirs: tuple[AgentProvisionDir, ...] = ()
|
||||
files: tuple[AgentProvisionFile, ...] = ()
|
||||
@@ -127,13 +138,31 @@ class AgentProvider(ABC):
|
||||
"""The static command / image / prompt-mode table for this
|
||||
template."""
|
||||
|
||||
@property
|
||||
def guest_home(self) -> str:
|
||||
"""In-guest home directory for the agent user. Defaults to
|
||||
`/home/node` to match the Debian-based bot-bottle-* images
|
||||
(USER node). Override for plugins whose image runs as a
|
||||
different user."""
|
||||
return "/home/node"
|
||||
|
||||
@property
|
||||
def dockerfile(self) -> Path:
|
||||
"""Path to the provider's Dockerfile.
|
||||
|
||||
Default: the `Dockerfile` file next to this provider's
|
||||
`agent_provider.py` module. Override to point at a non-standard
|
||||
path."""
|
||||
return Path(inspect.getfile(type(self))).parent / "Dockerfile"
|
||||
|
||||
@abstractmethod
|
||||
def provision_plan(
|
||||
self,
|
||||
*,
|
||||
dockerfile: str,
|
||||
state_dir: Path,
|
||||
guest_home: str,
|
||||
instance_name: str,
|
||||
prompt_file: Path,
|
||||
guest_env: dict[str, str] | None = None,
|
||||
auth_token: str = "",
|
||||
forward_host_credentials: bool = False,
|
||||
@@ -141,6 +170,7 @@ class AgentProvider(ABC):
|
||||
trusted_project_path: str = "",
|
||||
label: str = "",
|
||||
color: str = "",
|
||||
provider_settings: dict[str, object] | None = None,
|
||||
) -> AgentProvisionPlan:
|
||||
"""Build the declarative AgentProvisionPlan for one launch.
|
||||
Backends call this during `prepare` and consume the result as
|
||||
@@ -205,25 +235,12 @@ class AgentProvider(ABC):
|
||||
def provision_git(self, bottle: "Bottle", plan: "BottlePlan") -> None:
|
||||
"""Configure git inside the agent container.
|
||||
|
||||
Default: Debian/node — copies .git when --cwd is set, writes the
|
||||
git-gate insteadOf gitconfig, sets user.name/email as node.
|
||||
Override for images that run as a different user or use a
|
||||
non-standard home directory."""
|
||||
Default: Debian/node — writes the git-gate insteadOf gitconfig
|
||||
and sets user.name/email as node. Workspace copy runs through
|
||||
BottleBackend.provision_workspace against the running bottle."""
|
||||
from .log import info
|
||||
workspace = plan.workspace_plan
|
||||
if workspace.enabled and workspace.copy_git and workspace.has_host_git_dir:
|
||||
guest_workspace_git = f"{workspace.guest_path}/.git"
|
||||
host_git = str(workspace.host_path / ".git")
|
||||
info(f"copying {host_git} -> {bottle.name}:{guest_workspace_git}")
|
||||
bottle.exec(f"mkdir -p {shlex.quote(workspace.guest_path)}", user="root")
|
||||
bottle.cp_in(host_git, guest_workspace_git)
|
||||
bottle.exec(
|
||||
f"chown -R {shlex.quote(workspace.owner)} "
|
||||
f"{shlex.quote(guest_workspace_git)}",
|
||||
user="root",
|
||||
)
|
||||
|
||||
manifest_bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
||||
manifest_bottle = plan.manifest.bottle
|
||||
if manifest_bottle.git:
|
||||
from .git_gate import GIT_GATE_HOSTNAME, git_gate_render_gitconfig
|
||||
gate_host = getattr(plan, "git_gate_insteadof_host", GIT_GATE_HOSTNAME)
|
||||
@@ -310,6 +327,9 @@ def get_provider(template: str) -> AgentProvider:
|
||||
if template == PROVIDER_CODEX:
|
||||
from .contrib.codex.agent_provider import CodexAgentProvider
|
||||
return CodexAgentProvider()
|
||||
if template == PROVIDER_PI:
|
||||
from .contrib.pi.agent_provider import PiAgentProvider
|
||||
return PiAgentProvider()
|
||||
raise ValueError(f"unknown agent provider template: {template!r}")
|
||||
|
||||
|
||||
@@ -317,12 +337,13 @@ def runtime_for(template: str) -> AgentProviderRuntime:
|
||||
return get_provider(template).runtime
|
||||
|
||||
|
||||
def agent_provision_plan(
|
||||
def build_agent_provision_plan(
|
||||
*,
|
||||
template: str,
|
||||
dockerfile: str,
|
||||
state_dir: Path,
|
||||
guest_home: str,
|
||||
instance_name: str,
|
||||
prompt_file: Path,
|
||||
guest_env: dict[str, str] | None = None,
|
||||
auth_token: str = "",
|
||||
forward_host_credentials: bool = False,
|
||||
@@ -330,13 +351,15 @@ def agent_provision_plan(
|
||||
trusted_project_path: str = "",
|
||||
label: str = "",
|
||||
color: str = "",
|
||||
provider_settings: dict[str, object] | None = None,
|
||||
) -> AgentProvisionPlan:
|
||||
"""Back-compat shim — `prepare` callers stay the same; the work
|
||||
now lives on the provider plugin."""
|
||||
return get_provider(template).provision_plan(
|
||||
dockerfile=dockerfile,
|
||||
state_dir=state_dir,
|
||||
guest_home=guest_home,
|
||||
instance_name=instance_name,
|
||||
prompt_file=prompt_file,
|
||||
guest_env=guest_env,
|
||||
auth_token=auth_token,
|
||||
forward_host_credentials=forward_host_credentials,
|
||||
@@ -344,6 +367,7 @@ def agent_provision_plan(
|
||||
trusted_project_path=trusted_project_path,
|
||||
label=label,
|
||||
color=color,
|
||||
provider_settings=provider_settings,
|
||||
)
|
||||
|
||||
|
||||
@@ -361,4 +385,8 @@ def prompt_args(
|
||||
if argv and "resume" in argv:
|
||||
return []
|
||||
return [f"Read and follow the instructions in {prompt_path}."]
|
||||
if prompt_mode == "print_read_prompt_file":
|
||||
return ["-p", f"Read and follow the instructions in {prompt_path}."]
|
||||
if prompt_mode == "append_system_prompt":
|
||||
return ["--append-system-prompt", prompt_path]
|
||||
raise ValueError(f"unknown provider prompt mode: {prompt_mode}")
|
||||
|
||||
+173
-49
@@ -24,14 +24,16 @@ backend exposes five methods:
|
||||
enough metadata for callers (CLI `list active`, dashboard
|
||||
agents pane) to render a row.
|
||||
|
||||
Selection is driven by `--backend` on `start` or
|
||||
BOT_BOTTLE_BACKEND (env var; default "docker"). Per PRD 0003 the
|
||||
manifest does not carry a backend field; the host picks.
|
||||
Selection is driven by `--backend` on `start` or BOT_BOTTLE_BACKEND
|
||||
(env var). When neither is set, compatible macOS hosts default to
|
||||
`macos-container`; other hosts default to `smolmachines`. Per PRD 0003
|
||||
the manifest does not carry a backend field; the host picks.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shlex
|
||||
import sys
|
||||
from abc import ABC, abstractmethod
|
||||
from contextlib import AbstractContextManager
|
||||
@@ -39,14 +41,15 @@ from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any, Generic, Sequence, TypeVar
|
||||
|
||||
from ..agent_provider import AgentProvisionPlan, get_provider
|
||||
from ..agent_provider import AgentProvisionPlan, get_provider, build_agent_provision_plan
|
||||
from ..egress import EgressPlan
|
||||
from ..git_gate import GitGatePlan
|
||||
from ..log import die, info
|
||||
from ..manifest import GitEntry, Manifest
|
||||
from ..manifest import Manifest, ManifestIndex
|
||||
from ..supervise import SupervisePlan
|
||||
from ..util import expand_tilde
|
||||
from ..workspace import WorkspacePlan
|
||||
from ..env import resolve_env, ResolvedEnv
|
||||
from ..workspace import WorkspacePlan, workspace_plan
|
||||
from .print_util import print_multi, visible_agent_env_names
|
||||
from .util import host_skill_dir
|
||||
|
||||
@@ -58,7 +61,7 @@ class BottleSpec:
|
||||
Resolved values (image names, container name, scratch paths, runsc
|
||||
availability) live on the plan, not the spec."""
|
||||
|
||||
manifest: Manifest
|
||||
manifest: ManifestIndex
|
||||
agent_name: str
|
||||
copy_cwd: bool
|
||||
user_cwd: str
|
||||
@@ -77,10 +80,14 @@ class BottlePlan(ABC):
|
||||
(e.g. DockerBottlePlan) add backend-specific resolved fields."""
|
||||
|
||||
spec: BottleSpec
|
||||
manifest: Manifest
|
||||
stage_dir: Path
|
||||
guest_home: str
|
||||
git_gate_plan: GitGatePlan
|
||||
|
||||
@property
|
||||
def guest_home(self) -> str:
|
||||
return self.agent_provision.guest_home
|
||||
|
||||
@property
|
||||
def git_gate_insteadof_host(self) -> str:
|
||||
"""Host (and optional port) used in git-gate insteadOf URLs.
|
||||
@@ -97,15 +104,18 @@ class BottlePlan(ABC):
|
||||
egress_plan: EgressPlan
|
||||
supervise_plan: SupervisePlan | None
|
||||
agent_provision: AgentProvisionPlan
|
||||
workspace_plan: WorkspacePlan
|
||||
|
||||
@property
|
||||
def workspace_plan(self) -> WorkspacePlan:
|
||||
return workspace_plan(self.spec, guest_home=self.guest_home)
|
||||
|
||||
def print(self, *, remote_control: bool) -> None:
|
||||
"""Render the y/N preflight summary to stderr."""
|
||||
del remote_control
|
||||
spec = self.spec
|
||||
manifest = spec.manifest
|
||||
agent = manifest.agents[spec.agent_name]
|
||||
bottle = manifest.bottle_for(spec.agent_name)
|
||||
manifest = self.manifest
|
||||
agent = manifest.agent
|
||||
bottle = manifest.bottle
|
||||
|
||||
env_names = visible_agent_env_names(
|
||||
sorted(
|
||||
@@ -122,7 +132,7 @@ class BottlePlan(ABC):
|
||||
print_multi("skills ", list(agent.skills))
|
||||
info(f"bottle : {agent.bottle}")
|
||||
|
||||
identity = manifest.git_identity_summary(spec.agent_name)
|
||||
identity = manifest.git_identity_summary()
|
||||
if identity:
|
||||
info(f" git identity : {identity}")
|
||||
|
||||
@@ -182,7 +192,7 @@ class ActiveAgent:
|
||||
of sidecar daemons currently up for this bottle (`egress`,
|
||||
`git-gate`, `supervise`); the dashboard uses it to
|
||||
gate edit verbs. `backend_name` is the matching key in
|
||||
`_BACKENDS` (`docker` / `smolmachines`) — used by the active-
|
||||
`_BACKENDS` (`docker` / `smolmachines` / `macos-container`) — used by the active-
|
||||
list rendering to disambiguate and by the dashboard's
|
||||
re-attach path."""
|
||||
|
||||
@@ -263,27 +273,101 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
|
||||
|
||||
name: str
|
||||
|
||||
def prepare(self, spec: BottleSpec, *, stage_dir: Path) -> PlanT:
|
||||
def prepare(self, spec: BottleSpec, stage_dir: Path) -> PlanT:
|
||||
"""Template method: run cross-backend host-side validation, then
|
||||
delegate to the subclass's `_resolve_plan` for the
|
||||
backend-specific resolution (names, scratch files, etc.). The
|
||||
validation step is enforced here so a future backend cannot
|
||||
accidentally skip it. No remote/runtime resources are created."""
|
||||
self._validate(spec)
|
||||
return self._resolve_plan(spec, stage_dir=stage_dir)
|
||||
from .resolve_common import (
|
||||
merge_provision_env_vars,
|
||||
mint_slug,
|
||||
prepare_agent_state_dir,
|
||||
prepare_egress,
|
||||
prepare_git_gate,
|
||||
prepare_supervise,
|
||||
resolve_manifest_dockerfile,
|
||||
write_launch_metadata,
|
||||
)
|
||||
|
||||
def _validate(self, spec: BottleSpec) -> None:
|
||||
"""Cross-backend pre-launch checks. Confirms the agent exists,
|
||||
the named skills are present on the host, and every git
|
||||
IdentityFile resolves. Subclasses with additional preconditions
|
||||
should override and call `super()._validate(spec)` first."""
|
||||
manifest = spec.manifest
|
||||
manifest.require_agent(spec.agent_name)
|
||||
agent = manifest.agents[spec.agent_name]
|
||||
bottle = manifest.bottle_for(spec.agent_name)
|
||||
self._validate_skills(agent.skills)
|
||||
self._validate_git_entries(bottle.git)
|
||||
self._validate_agent_provider_dockerfile(spec)
|
||||
manifest = self._validate(spec)
|
||||
|
||||
self._preflight()
|
||||
|
||||
manifest_bottle = manifest.bottle
|
||||
manifest_agent_provider = manifest_bottle.agent_provider
|
||||
agent_provider = get_provider(manifest_agent_provider.template)
|
||||
resolved_env = resolve_env(manifest)
|
||||
workspace = workspace_plan(spec, guest_home=agent_provider.guest_home)
|
||||
|
||||
slug = mint_slug(spec)
|
||||
write_launch_metadata(slug, spec, compose_project="", backend=self.name)
|
||||
|
||||
# Manifest may override the Dockerfile per-bottle; otherwise fall
|
||||
# back to the provider plugin's bundled Dockerfile (next to its
|
||||
# agent_provider.py module).
|
||||
if manifest_agent_provider.dockerfile:
|
||||
agent_dockerfile_path = resolve_manifest_dockerfile(
|
||||
manifest_agent_provider.dockerfile, spec,
|
||||
)
|
||||
else:
|
||||
agent_dockerfile_path = str(agent_provider.dockerfile)
|
||||
|
||||
agent_dir, prompt_file = prepare_agent_state_dir(slug, manifest)
|
||||
|
||||
agent_provision_plan = build_agent_provision_plan(
|
||||
template=manifest_agent_provider.template,
|
||||
dockerfile=agent_dockerfile_path,
|
||||
state_dir=agent_dir,
|
||||
instance_name=f"bot-bottle-{slug}",
|
||||
prompt_file=prompt_file,
|
||||
guest_env=self._build_guest_env(resolved_env),
|
||||
forward_host_credentials=manifest_agent_provider.forward_host_credentials,
|
||||
auth_token=manifest_agent_provider.auth_token,
|
||||
host_env=dict(os.environ),
|
||||
trusted_project_path=workspace.workdir,
|
||||
label=spec.label,
|
||||
color=spec.color,
|
||||
provider_settings=manifest_agent_provider.settings,
|
||||
)
|
||||
agent_provision_plan = merge_provision_env_vars(agent_provision_plan)
|
||||
egress_plan = prepare_egress(manifest_bottle, slug, agent_provision_plan)
|
||||
supervise_plan = prepare_supervise(manifest_bottle, slug)
|
||||
git_gate_plan = prepare_git_gate(manifest_bottle, slug)
|
||||
|
||||
return self._resolve_plan(
|
||||
spec,
|
||||
manifest=manifest,
|
||||
slug=slug,
|
||||
resolved_env=resolved_env,
|
||||
agent_provision_plan=agent_provision_plan,
|
||||
egress_plan=egress_plan,
|
||||
supervise_plan=supervise_plan,
|
||||
git_gate_plan=git_gate_plan,
|
||||
stage_dir=stage_dir,
|
||||
)
|
||||
|
||||
def _build_guest_env(self, resolved_env: ResolvedEnv) -> dict[str, str]:
|
||||
return {}
|
||||
|
||||
def _preflight(self) -> None:
|
||||
"""
|
||||
tasks to do before resolving a plan
|
||||
"""
|
||||
pass
|
||||
|
||||
def _validate(self, spec: BottleSpec) -> Manifest:
|
||||
"""Cross-backend pre-launch checks. Parses the selected agent and
|
||||
its bottle (raising ManifestError on invalid content), confirms
|
||||
skills are present on the host, and every git IdentityFile resolves.
|
||||
|
||||
Returns the loaded Manifest for the selected agent. Subclasses with
|
||||
additional preconditions should override and call
|
||||
`super()._validate(spec)` first."""
|
||||
manifest = spec.manifest.load_for_agent(spec.agent_name)
|
||||
self._validate_skills(manifest.agent.skills)
|
||||
self._validate_agent_provider_dockerfile(spec, manifest)
|
||||
return manifest
|
||||
|
||||
def _validate_skills(self, skills: Sequence[str]) -> None:
|
||||
"""Each named skill must be a directory under the host's
|
||||
@@ -297,18 +381,8 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
|
||||
f"Create it under ~/.claude/skills/, then re-run."
|
||||
)
|
||||
|
||||
def _validate_git_entries(self, entries: Sequence[GitEntry]) -> None:
|
||||
"""Each entry's IdentityFile must exist on the host (after
|
||||
expanding leading ~) — the git-gate copies it in at start time
|
||||
to authenticate the upstream push (PRD 0008). Shape is already
|
||||
enforced by Manifest validation; this only checks presence."""
|
||||
for entry in entries:
|
||||
key = expand_tilde(entry.IdentityFile)
|
||||
if not os.path.isfile(key):
|
||||
die(f"git upstream key file not found for '{entry.Name}': {key}")
|
||||
|
||||
def _validate_agent_provider_dockerfile(self, spec: BottleSpec) -> None:
|
||||
bottle = spec.manifest.bottle_for(spec.agent_name)
|
||||
def _validate_agent_provider_dockerfile(self, spec: BottleSpec, manifest: Manifest) -> None:
|
||||
bottle = manifest.bottle
|
||||
dockerfile = bottle.agent_provider.dockerfile
|
||||
if not dockerfile:
|
||||
return
|
||||
@@ -318,14 +392,26 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
|
||||
if not path.is_file():
|
||||
die(
|
||||
f"agent_provider.dockerfile for bottle "
|
||||
f"'{spec.manifest.agents[spec.agent_name].bottle}' not found: {path}"
|
||||
f"'{manifest.agent.bottle}' not found: {path}"
|
||||
)
|
||||
|
||||
@abstractmethod
|
||||
def _resolve_plan(self, spec: BottleSpec, *, stage_dir: Path) -> PlanT:
|
||||
def _resolve_plan(self,
|
||||
spec: BottleSpec,
|
||||
*,
|
||||
manifest: Manifest,
|
||||
slug: str,
|
||||
resolved_env: ResolvedEnv,
|
||||
agent_provision_plan: AgentProvisionPlan,
|
||||
egress_plan: EgressPlan,
|
||||
git_gate_plan: GitGatePlan,
|
||||
supervise_plan: SupervisePlan | None,
|
||||
stage_dir: Path) -> PlanT:
|
||||
"""Backend-specific plan resolution: image/container names,
|
||||
env-file, prompt-file, proxy plan, runtime detection. Called by
|
||||
`prepare` after `_validate` succeeds."""
|
||||
`prepare` after `_validate` succeeds. Instance name, image,
|
||||
prompt file, Dockerfile path, and guest home all live on
|
||||
`agent_provision_plan` — the source of truth."""
|
||||
|
||||
@abstractmethod
|
||||
def launch(self, plan: PlanT) -> AbstractContextManager[Bottle]:
|
||||
@@ -369,9 +455,30 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
|
||||
return prompt_path
|
||||
|
||||
def provision_workspace(self, plan: PlanT, bottle: "Bottle") -> None:
|
||||
"""Copy the operator workspace into the running bottle when
|
||||
the backend cannot bake it into the agent image. Default is
|
||||
no-op for backends like Docker that handle this before launch."""
|
||||
"""Copy the operator workspace into the running bottle.
|
||||
|
||||
This is the only supported workspace-provisioning path: Docker
|
||||
does not build a derived image containing the current
|
||||
workspace."""
|
||||
workspace = plan.workspace_plan
|
||||
if not (workspace.enabled and workspace.copy_contents):
|
||||
return
|
||||
|
||||
guest_parent = workspace.guest_path.rsplit("/", 1)[0] or "/"
|
||||
guest_path = shlex.quote(workspace.guest_path)
|
||||
guest_parent = shlex.quote(guest_parent)
|
||||
owner = shlex.quote(workspace.owner)
|
||||
mode = shlex.quote(workspace.mode)
|
||||
info(f"copying {workspace.host_path} -> {bottle.name}:{workspace.guest_path}")
|
||||
bottle.exec(
|
||||
f"rm -rf {guest_path} && mkdir -p {guest_parent}",
|
||||
user="root",
|
||||
)
|
||||
bottle.cp_in(str(workspace.host_path), workspace.guest_path)
|
||||
bottle.exec(
|
||||
f"chown -R {owner} {guest_path} && chmod {mode} {guest_path}",
|
||||
user="root",
|
||||
)
|
||||
|
||||
def supervise_mcp_url(self, plan: PlanT) -> str:
|
||||
"""Return the agent-side URL of the per-bottle supervise
|
||||
@@ -416,8 +523,14 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
|
||||
# each backend module can pull BottleSpec / BottlePlan / BottleBackend
|
||||
# via `from . import ...` without hitting a partially-initialized module.
|
||||
from .docker import DockerBottleBackend # noqa: E402 # pylint: disable=wrong-import-position
|
||||
from .macos_container import MacosContainerBottleBackend # noqa: E402 # pylint: disable=wrong-import-position
|
||||
from .smolmachines import SmolmachinesBottleBackend # noqa: E402 # pylint: disable=wrong-import-position
|
||||
|
||||
# Freezer is imported after the backend classes for the same reason:
|
||||
# Freezer.commit_slug constructs ActiveAgent, which must be fully
|
||||
# defined first.
|
||||
from .freeze import CommitCancelled, Freezer, get_freezer # noqa: E402 # pylint: disable=wrong-import-position
|
||||
|
||||
|
||||
# The dict is heterogeneous: each value is a BottleBackend specialized
|
||||
# over its own plan type. Concrete plan types are erased here because
|
||||
@@ -425,6 +538,7 @@ from .smolmachines import SmolmachinesBottleBackend # noqa: E402 # pylint: dis
|
||||
# unparameterized methods (prepare → plan → launch(plan), cleanup, etc.).
|
||||
_BACKENDS: dict[str, BottleBackend[Any, Any]] = {
|
||||
"docker": DockerBottleBackend(),
|
||||
"macos-container": MacosContainerBottleBackend(),
|
||||
"smolmachines": SmolmachinesBottleBackend(),
|
||||
}
|
||||
|
||||
@@ -437,17 +551,24 @@ def get_bottle_backend(
|
||||
`name` precedence:
|
||||
1. explicit arg (CLI `--backend=<name>` passes through here)
|
||||
2. BOT_BOTTLE_BACKEND env var
|
||||
3. default `docker`
|
||||
3. `macos-container` on compatible macOS hosts
|
||||
4. default `smolmachines`
|
||||
|
||||
Dies with a pointer at the known backends if the chosen name
|
||||
isn't implemented."""
|
||||
resolved = name or os.environ.get("BOT_BOTTLE_BACKEND") or "docker"
|
||||
resolved = name or os.environ.get("BOT_BOTTLE_BACKEND") or _default_backend_name()
|
||||
if resolved not in _BACKENDS:
|
||||
known = ", ".join(sorted(_BACKENDS))
|
||||
die(f"unknown backend {resolved!r}; known backends: {known}")
|
||||
return _BACKENDS[resolved]
|
||||
|
||||
|
||||
def _default_backend_name() -> str:
|
||||
if has_backend("macos-container"):
|
||||
return "macos-container"
|
||||
return "smolmachines"
|
||||
|
||||
|
||||
def known_backend_names() -> tuple[str, ...]:
|
||||
"""Sorted tuple of all backend keys in `_BACKENDS`. Used by
|
||||
argparse (`--backend` choices) and the dashboard's backend
|
||||
@@ -497,9 +618,12 @@ __all__ = [
|
||||
"BottleCleanupPlan",
|
||||
"BottlePlan",
|
||||
"BottleSpec",
|
||||
"CommitCancelled",
|
||||
"ExecResult",
|
||||
"Freezer",
|
||||
"enumerate_active_agents",
|
||||
"get_bottle_backend",
|
||||
"get_freezer",
|
||||
"has_backend",
|
||||
"known_backend_names",
|
||||
]
|
||||
|
||||
@@ -2,10 +2,10 @@
|
||||
|
||||
This module is a thin façade. The real work lives in four siblings:
|
||||
|
||||
- prepare.py — host-side resolution into a DockerBottlePlan
|
||||
- launch.py — bring-up + teardown context manager
|
||||
- cleanup.py — orphan enumeration + removal
|
||||
- enumerate.py — active-agent listing
|
||||
- resolve_plan.py — Docker-specific resolution into a DockerBottlePlan
|
||||
- launch.py — bring-up + teardown context manager
|
||||
- cleanup.py — orphan enumeration + removal
|
||||
- enumerate.py — active-agent listing
|
||||
|
||||
The base class's `prepare` template runs cross-backend host-side
|
||||
validation before calling `_resolve_plan` here.
|
||||
@@ -25,17 +25,23 @@ from pathlib import Path
|
||||
from typing import Generator, Sequence
|
||||
|
||||
from ...supervise import SUPERVISE_HOSTNAME, SUPERVISE_PORT
|
||||
from ...agent_provider import AgentProvisionPlan
|
||||
from ...egress import EgressPlan
|
||||
from ...env import ResolvedEnv
|
||||
from ...git_gate import GitGatePlan
|
||||
from ...supervise import SupervisePlan
|
||||
from ...manifest import Manifest
|
||||
from .. import ActiveAgent, BottleBackend, BottleSpec
|
||||
from . import cleanup as _cleanup
|
||||
from . import enumerate as _enumerate
|
||||
from . import launch as _launch
|
||||
from . import prepare as _prepare
|
||||
from . import resolve_plan as _resolve_plan
|
||||
from .bottle import DockerBottle
|
||||
from .bottle_cleanup_plan import DockerBottleCleanupPlan
|
||||
from .bottle_plan import DockerBottlePlan
|
||||
class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanupPlan"]):
|
||||
"""Docker backend implementation. Selected by BOT_BOTTLE_BACKEND
|
||||
(default)."""
|
||||
when set to `docker`; retained as a legacy/example backend."""
|
||||
|
||||
name = "docker"
|
||||
|
||||
@@ -48,8 +54,36 @@ class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanup
|
||||
launch."""
|
||||
return shutil.which("docker") is not None
|
||||
|
||||
def _resolve_plan(self, spec: BottleSpec, *, stage_dir: Path) -> DockerBottlePlan:
|
||||
return _prepare.resolve_plan(spec, stage_dir=stage_dir)
|
||||
def _preflight(self) -> None:
|
||||
_resolve_plan.preflight()
|
||||
|
||||
def _build_guest_env(self, resolved_env: ResolvedEnv) -> dict[str, str]:
|
||||
return _resolve_plan.build_guest_env(resolved_env)
|
||||
|
||||
def _resolve_plan(
|
||||
self,
|
||||
spec: BottleSpec,
|
||||
*,
|
||||
manifest: Manifest,
|
||||
slug: str,
|
||||
resolved_env: ResolvedEnv,
|
||||
agent_provision_plan: AgentProvisionPlan,
|
||||
egress_plan: EgressPlan,
|
||||
git_gate_plan: GitGatePlan,
|
||||
supervise_plan: SupervisePlan | None,
|
||||
stage_dir: Path,
|
||||
) -> DockerBottlePlan:
|
||||
return _resolve_plan.resolve_plan(
|
||||
spec,
|
||||
manifest=manifest,
|
||||
slug=slug,
|
||||
resolved_env=resolved_env,
|
||||
agent_provision_plan=agent_provision_plan,
|
||||
egress_plan=egress_plan,
|
||||
supervise_plan=supervise_plan,
|
||||
git_gate_plan=git_gate_plan,
|
||||
stage_dir=stage_dir,
|
||||
)
|
||||
|
||||
@contextmanager
|
||||
def launch(self, plan: DockerBottlePlan) -> Generator[DockerBottle, None, None]:
|
||||
|
||||
@@ -9,6 +9,7 @@ from typing import cast
|
||||
|
||||
from ...agent_provider import PromptMode, prompt_args
|
||||
from .. import Bottle, ExecResult
|
||||
from ..terminal import exec_shell_script
|
||||
|
||||
|
||||
class DockerBottle(Bottle):
|
||||
@@ -22,15 +23,20 @@ class DockerBottle(Bottle):
|
||||
*,
|
||||
agent_command: str = "claude",
|
||||
agent_prompt_mode: PromptMode = "append_file",
|
||||
agent_provider_template: str = "claude",
|
||||
terminal_title: str = "",
|
||||
terminal_color: str = "",
|
||||
agent_workdir: str = "/home/node",
|
||||
):
|
||||
self.name = container
|
||||
self._teardown = teardown
|
||||
self.prompt_path = prompt_path_in_container
|
||||
self._agent_prompt_mode = agent_prompt_mode
|
||||
self.agent_command = agent_command
|
||||
self.agent_provider_template = (
|
||||
"codex" if agent_command == "codex" else "claude"
|
||||
)
|
||||
self.terminal_title = terminal_title
|
||||
self.terminal_color = terminal_color
|
||||
self.agent_provider_template = agent_provider_template
|
||||
self.agent_workdir = agent_workdir
|
||||
self._closed = False
|
||||
|
||||
def agent_argv(
|
||||
@@ -43,13 +49,17 @@ class DockerBottle(Bottle):
|
||||
cmd = ["docker", "exec"]
|
||||
if tty:
|
||||
cmd.append("-it")
|
||||
if self.agent_workdir and self.agent_workdir != "/home/node":
|
||||
cmd.extend(["-w", self.agent_workdir])
|
||||
cmd.extend([self.name, self.agent_command, *full_argv])
|
||||
return cmd
|
||||
|
||||
def exec_agent(self, argv: list[str], *, tty: bool = True) -> int:
|
||||
return subprocess.run(
|
||||
self.agent_argv(argv, tty=tty), check=False,
|
||||
).returncode
|
||||
agent_argv = self.agent_argv(argv, tty=tty)
|
||||
script = exec_shell_script(agent_argv, self.terminal_title, self.terminal_color) if tty else None
|
||||
if script is None:
|
||||
return subprocess.run(agent_argv, check=False).returncode
|
||||
return subprocess.run(["sh", "-lc", script], check=False).returncode
|
||||
|
||||
def exec(self, script: str, *, user: str = "node") -> ExecResult:
|
||||
# Pipe via stdin to `sh -s` so the caller never has to worry
|
||||
|
||||
@@ -22,25 +22,32 @@ class DockerBottlePlan(BottlePlan):
|
||||
`agent_provision` from BottlePlan."""
|
||||
|
||||
slug: str
|
||||
container_name: str
|
||||
container_name_pinned: bool
|
||||
image: str
|
||||
derived_image: str # "" -> no derived image
|
||||
runtime_image: str # image actually launched (derived or base)
|
||||
# Absolute path to the Dockerfile that builds `image`. Empty means
|
||||
# use the repo's default Dockerfile. Populated to a per-bottle
|
||||
# state file (~/.bot-bottle/state/<slug>/Dockerfile) after a
|
||||
# capability-block remediation (PRD 0016).
|
||||
dockerfile_path: str
|
||||
env_file: Path # docker --env-file: NAME=VALUE literals
|
||||
# name -> value for vars forwarded into the docker-run child process
|
||||
# via subprocess env (so values never land on argv or in a file).
|
||||
# repr=False keeps secret/interpolated/OAuth values out of any
|
||||
# accidental log of the plan dataclass.
|
||||
forwarded_env: dict[str, str] = field(repr=False)
|
||||
prompt_file: Path
|
||||
use_runsc: bool
|
||||
|
||||
@property
|
||||
def container_name(self) -> str:
|
||||
return self.agent_provision.instance_name
|
||||
|
||||
@property
|
||||
def image(self) -> str:
|
||||
return self.agent_provision.image
|
||||
|
||||
@property
|
||||
def dockerfile_path(self) -> str:
|
||||
"""Absolute path to the Dockerfile that builds `image`. Sourced
|
||||
from the agent provision plan — the manifest may override per
|
||||
bottle; otherwise the provider plugin's bundled Dockerfile."""
|
||||
return self.agent_provision.dockerfile
|
||||
|
||||
@property
|
||||
def prompt_file(self) -> Path:
|
||||
return self.agent_provision.prompt_file
|
||||
|
||||
@property
|
||||
def agent_command(self) -> str:
|
||||
return self.agent_provision.command
|
||||
|
||||
@@ -32,10 +32,10 @@ from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
from ...agent_provider import get_provider
|
||||
from ...log import info, warn
|
||||
from .bottle_state import (
|
||||
from ...bottle_state import (
|
||||
mark_preserved,
|
||||
per_bottle_dockerfile,
|
||||
transcript_snapshot_dir,
|
||||
@@ -93,11 +93,11 @@ def fetch_current_dockerfile(slug: str) -> str:
|
||||
override = per_bottle_dockerfile(slug)
|
||||
if override is not None:
|
||||
return override
|
||||
repo_dockerfile = _repo_dockerfile_path()
|
||||
repo_dockerfile = get_provider("claude").dockerfile
|
||||
if repo_dockerfile.is_file():
|
||||
return repo_dockerfile.read_text()
|
||||
raise CapabilityApplyError(
|
||||
f"no per-bottle Dockerfile for {slug} and no repo Dockerfile at "
|
||||
f"no per-bottle Dockerfile for {slug} and no provider Dockerfile at "
|
||||
f"{repo_dockerfile}"
|
||||
)
|
||||
|
||||
@@ -125,13 +125,6 @@ def apply_capability_change(slug: str, new_dockerfile: str) -> tuple[str, str]:
|
||||
# --- Internals -------------------------------------------------------------
|
||||
|
||||
|
||||
def _repo_dockerfile_path() -> Path:
|
||||
"""Path to the repo's Claude Dockerfile (one dir above this module's
|
||||
package root). Resolved at call time so the path is correct
|
||||
regardless of where this module is imported from."""
|
||||
# bot_bottle/backend/docker/capability_apply.py -> repo root
|
||||
return Path(__file__).resolve().parent.parent.parent.parent / "Dockerfile.claude"
|
||||
|
||||
|
||||
def snapshot_transcript(slug: str) -> None:
|
||||
"""`docker cp` /home/node/.claude out of the agent container into
|
||||
|
||||
@@ -31,7 +31,7 @@ from ... import supervise as _supervise
|
||||
from ...log import info, warn
|
||||
from . import util as docker_mod
|
||||
from .bottle_cleanup_plan import DockerBottleCleanupPlan
|
||||
from .bottle_state import bottle_state_dir, is_preserved
|
||||
from ...bottle_state import bottle_state_dir, is_preserved
|
||||
from .compose import COMPOSE_PROJECT_PREFIX, list_compose_projects
|
||||
|
||||
|
||||
|
||||
@@ -58,10 +58,17 @@ from .sidecar_bundle import (
|
||||
)
|
||||
|
||||
|
||||
# Repo root, used as the build context for the bundle Dockerfile.
|
||||
# Repo root or installed site-packages root, used as the build context for
|
||||
# Dockerfiles that COPY bot_bottle source files.
|
||||
_REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent)
|
||||
|
||||
|
||||
def _sidecar_bundle_dockerfile() -> str:
|
||||
if (Path(_REPO_DIR) / SIDECAR_BUNDLE_DOCKERFILE).is_file():
|
||||
return SIDECAR_BUNDLE_DOCKERFILE
|
||||
return f"bot_bottle/{SIDECAR_BUNDLE_DOCKERFILE}"
|
||||
|
||||
|
||||
def bottle_plan_to_compose(plan: DockerBottlePlan) -> dict[str, Any]:
|
||||
"""Render a Compose v2 spec dict from a fully-resolved
|
||||
DockerBottlePlan.
|
||||
@@ -134,7 +141,7 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
||||
ep = plan.egress_plan
|
||||
volumes.append(_bind(ep.mitmproxy_ca_host_path, EGRESS_CA_IN_CONTAINER))
|
||||
if ep.routes:
|
||||
volumes.append(_bind(ep.routes_path, EGRESS_ROUTES_IN_CONTAINER))
|
||||
volumes.append(_bind(ep.routes_path.parent, str(Path(EGRESS_ROUTES_IN_CONTAINER).parent)))
|
||||
for token_env in sorted(ep.token_env_map.keys()):
|
||||
env.append(token_env)
|
||||
|
||||
@@ -183,7 +190,7 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
||||
"image": SIDECAR_BUNDLE_IMAGE,
|
||||
"build": {
|
||||
"context": _REPO_DIR,
|
||||
"dockerfile": SIDECAR_BUNDLE_DOCKERFILE,
|
||||
"dockerfile": _sidecar_bundle_dockerfile(),
|
||||
},
|
||||
"container_name": sidecar_bundle_container_name(plan.slug),
|
||||
"networks": {
|
||||
@@ -222,7 +229,7 @@ def _agent_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
||||
env.append(name)
|
||||
|
||||
service: dict[str, Any] = {
|
||||
"image": plan.runtime_image,
|
||||
"image": plan.image,
|
||||
"container_name": plan.container_name,
|
||||
"command": ["sleep", "infinity"],
|
||||
"networks": {"internal": None},
|
||||
@@ -230,8 +237,6 @@ def _agent_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
||||
}
|
||||
if plan.use_runsc:
|
||||
service["runtime"] = "runsc"
|
||||
if plan.env_file and plan.env_file.exists() and plan.env_file.stat().st_size > 0:
|
||||
service["env_file"] = [str(plan.env_file)]
|
||||
|
||||
volumes: list[dict[str, Any]] = []
|
||||
if plan.supervise_plan is not None:
|
||||
|
||||
@@ -1,24 +1,21 @@
|
||||
"""Host-side helper for egress sidecar inspection (issue #198).
|
||||
"""Host-side helper for egress sidecar inspection and live updates.
|
||||
|
||||
`_merge_single_route`, `add_route`, and `apply_routes_change` were
|
||||
removed when the egress-block MCP tool was dropped. The remaining
|
||||
helpers support runtime inspection and validation of the routes file
|
||||
without modifying it at runtime.
|
||||
The approve path uses this module to validate a proposed routes file,
|
||||
write it to the bottle's live egress state dir, and signal the sidecar
|
||||
bundle so the mitmproxy addon reloads it.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
|
||||
from ...egress import EGRESS_ROUTES_IN_CONTAINER
|
||||
from ...egress_addon_core import load_routes
|
||||
from ...log import warn
|
||||
from ..egress_apply import EgressApplicator, EgressApplyError
|
||||
from .sidecar_bundle import sidecar_bundle_container_name
|
||||
|
||||
|
||||
class EgressApplyError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
def fetch_current_routes(slug: str) -> str:
|
||||
container = sidecar_bundle_container_name(slug)
|
||||
r = subprocess.run(
|
||||
@@ -33,17 +30,31 @@ def fetch_current_routes(slug: str) -> str:
|
||||
return r.stdout
|
||||
|
||||
|
||||
def validate_routes_content(content: str) -> None:
|
||||
try:
|
||||
load_routes(content)
|
||||
except ValueError as e:
|
||||
raise EgressApplyError(
|
||||
f"proposed routes.yaml is not valid: {e}"
|
||||
) from e
|
||||
class DockerEgressApplicator(EgressApplicator):
|
||||
def _signal_bundle_reload(self, slug: str) -> None:
|
||||
container = sidecar_bundle_container_name(slug)
|
||||
result = subprocess.run(
|
||||
["docker", "kill", "--signal", "HUP", container],
|
||||
capture_output=True, text=True, check=False, env=os.environ,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
last_error = (result.stderr or "").strip() or (result.stdout or "").strip()
|
||||
warn(
|
||||
f"egress: routes updated on disk for {slug}, but bundle reload failed: "
|
||||
f"{last_error or 'docker kill failed'}"
|
||||
)
|
||||
raise EgressApplyError(
|
||||
f"could not reload egress bundle {container}: "
|
||||
f"{last_error or 'docker kill failed'}"
|
||||
)
|
||||
|
||||
|
||||
applicator = DockerEgressApplicator()
|
||||
|
||||
|
||||
__all__ = [
|
||||
"DockerEgressApplicator",
|
||||
"EgressApplyError",
|
||||
"applicator",
|
||||
"fetch_current_routes",
|
||||
"validate_routes_content",
|
||||
]
|
||||
|
||||
@@ -15,7 +15,7 @@ from __future__ import annotations
|
||||
import subprocess
|
||||
|
||||
from .. import ActiveAgent
|
||||
from .bottle_state import read_metadata
|
||||
from ...bottle_state import read_metadata
|
||||
from .compose import compose_project_name, list_active_slugs
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
"""DockerFreezer — snapshot a Docker bottle via `docker commit`."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from .. import ActiveAgent
|
||||
from ..freeze import Freezer
|
||||
from .util import commit_container
|
||||
from ...log import info
|
||||
|
||||
|
||||
class DockerFreezer(Freezer):
|
||||
"""Freezes a Docker bottle by running `docker commit`."""
|
||||
|
||||
backend_name = "docker"
|
||||
|
||||
def _freeze(self, agent: ActiveAgent) -> str:
|
||||
container = f"bot-bottle-{agent.slug}"
|
||||
image_tag = f"bot-bottle-committed-{agent.slug}:latest"
|
||||
commit_container(container, image_tag)
|
||||
return image_tag
|
||||
|
||||
def _export_hint(self, slug: str, image_ref: str) -> None:
|
||||
info(f"to export for migration: docker save {image_ref} -o {slug}.tar")
|
||||
@@ -4,8 +4,8 @@ PRD 0018 chunk 3: each instance is one `docker compose` project.
|
||||
|
||||
The flow is:
|
||||
|
||||
1. Build the agent's base + derived image (compose builds the
|
||||
sidecar images via the `build:` directive on first up).
|
||||
1. Build the agent image from the provider Dockerfile (compose
|
||||
builds the sidecar images via the `build:` directive on first up).
|
||||
2. Mint the per-bottle egress CA (chunk 2 writes it under
|
||||
state/<slug>/egress/).
|
||||
3. Populate the inner plans with launch-time fields so the
|
||||
@@ -15,8 +15,8 @@ The flow is:
|
||||
7. `docker compose up -d` (token + OAuth values flow into the
|
||||
compose subprocess env so `environment: [NAME]` bare-name
|
||||
entries inherit without rendering values into the file).
|
||||
8. Provision (CA install, prompt copy, skills, git, supervise
|
||||
config) — unchanged, uses `docker exec`.
|
||||
8. Provision (CA install, prompt copy, skills, workspace, git,
|
||||
supervise config) — unchanged, uses `docker exec` / `docker cp`.
|
||||
9. Yield a DockerBottle handle. `exec_agent` runs claude via
|
||||
`docker exec -it` exactly like the pre-compose world.
|
||||
|
||||
@@ -43,10 +43,11 @@ from . import network as network_mod
|
||||
from . import util as docker_mod
|
||||
from .bottle import DockerBottle
|
||||
from .bottle_plan import DockerBottlePlan
|
||||
from .bottle_state import (
|
||||
from ...bottle_state import (
|
||||
bottle_state_dir,
|
||||
egress_state_dir,
|
||||
git_gate_state_dir,
|
||||
read_committed_image,
|
||||
)
|
||||
from .compose import (
|
||||
bottle_plan_to_compose,
|
||||
@@ -75,7 +76,7 @@ def launch(
|
||||
Teardown on exit."""
|
||||
stack = ExitStack()
|
||||
|
||||
_bottle_for_revoke = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
||||
_bottle_for_revoke = plan.manifest.bottle
|
||||
_git_gate_dir_for_revoke = git_gate_state_dir(plan.slug)
|
||||
|
||||
def teardown() -> None:
|
||||
@@ -91,15 +92,21 @@ def launch(
|
||||
)
|
||||
|
||||
try:
|
||||
# Step 1: agent image build. Sidecar images get built lazily by
|
||||
# `docker compose up` via the renderer's `build:` directives.
|
||||
docker_mod.build_image(
|
||||
plan.image, _REPO_DIR,
|
||||
dockerfile=plan.dockerfile_path,
|
||||
)
|
||||
if plan.derived_image:
|
||||
docker_mod.build_image_with_cwd(
|
||||
plan.derived_image, plan.image, plan.workspace_plan
|
||||
# Step 1: agent image. Use a committed snapshot when one exists
|
||||
# and is present in the local daemon; otherwise build from the
|
||||
# Dockerfile. Sidecar images get built lazily by `docker compose
|
||||
# up` via the renderer's `build:` directives.
|
||||
committed = read_committed_image(plan.slug)
|
||||
if committed and docker_mod.image_exists(committed):
|
||||
info(f"using committed image {committed!r}")
|
||||
plan = dataclasses.replace(
|
||||
plan,
|
||||
agent_provision=dataclasses.replace(plan.agent_provision, image=committed),
|
||||
)
|
||||
else:
|
||||
docker_mod.build_image(
|
||||
plan.image, _REPO_DIR,
|
||||
dockerfile=plan.dockerfile_path,
|
||||
)
|
||||
|
||||
internal_network = network_mod.network_name_for_slug(plan.slug)
|
||||
@@ -179,6 +186,10 @@ def launch(
|
||||
None,
|
||||
agent_command=plan.agent_command,
|
||||
agent_prompt_mode=plan.agent_prompt_mode,
|
||||
agent_provider_template=plan.agent_provider_template,
|
||||
terminal_title=f"{plan.spec.label} ({plan.spec.agent_name})" if plan.spec.label else plan.spec.agent_name,
|
||||
terminal_color=plan.spec.color,
|
||||
agent_workdir=plan.workspace_plan.workdir,
|
||||
)
|
||||
bottle.prompt_path = provision(plan, bottle)
|
||||
|
||||
|
||||
@@ -1,280 +0,0 @@
|
||||
"""Prepare step for the Docker bottle backend.
|
||||
|
||||
`resolve_plan` does all host-side resolution (image and container
|
||||
names, env-file, prompt-file, proxy plan, runtime detection) and
|
||||
returns a frozen DockerBottlePlan. No Docker resources are created;
|
||||
the only side effects are scratch files under `stage_dir` and a probe
|
||||
of `docker info`. Cross-backend host-side validation has already run
|
||||
via the base class's `prepare` template before this is called.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from datetime import datetime, timezone
|
||||
from dataclasses import replace
|
||||
from pathlib import Path
|
||||
|
||||
from ...agent_provider import PROVIDER_TEMPLATES, agent_provision_plan, runtime_for
|
||||
from ...egress import Egress
|
||||
from ...env import ResolvedEnv, resolve_env
|
||||
from ...git_gate import GitGate
|
||||
from ...log import die
|
||||
from ...supervise import Supervise
|
||||
from ...workspace import workspace_plan as resolve_workspace_plan
|
||||
from .. import BottleSpec
|
||||
from . import util as docker_mod
|
||||
from .bottle_plan import DockerBottlePlan
|
||||
from .bottle_state import (
|
||||
BottleMetadata,
|
||||
agent_state_dir,
|
||||
bottle_identity,
|
||||
clear_preserve_marker,
|
||||
egress_state_dir,
|
||||
git_gate_state_dir,
|
||||
per_bottle_dockerfile,
|
||||
per_bottle_dockerfile_path,
|
||||
per_bottle_image_tag,
|
||||
supervise_state_dir,
|
||||
write_metadata,
|
||||
)
|
||||
from .sidecar_bundle import sidecar_bundle_container_name
|
||||
|
||||
|
||||
def resolve_plan(
|
||||
spec: BottleSpec,
|
||||
*,
|
||||
stage_dir: Path,
|
||||
) -> DockerBottlePlan:
|
||||
"""Resolve Docker-specific names and write scratch files. Trusts
|
||||
that the agent and its skills/git-gate keys are present —
|
||||
validation already ran in the base class."""
|
||||
docker_mod.require_docker()
|
||||
|
||||
git_gate = GitGate()
|
||||
egress = Egress()
|
||||
supervise = Supervise()
|
||||
|
||||
manifest = spec.manifest
|
||||
agent = manifest.agents[spec.agent_name]
|
||||
bottle = manifest.bottle_for(spec.agent_name)
|
||||
provider = bottle.agent_provider
|
||||
provider_runtime = runtime_for(provider.template)
|
||||
guest_home = "/home/node"
|
||||
workspace_plan = resolve_workspace_plan(spec, guest_home=guest_home)
|
||||
|
||||
# PRD 0016 follow-up: identity, not bare slug. A fresh `start`
|
||||
# mints a random-suffixed identity (so parallel runs of the same
|
||||
# agent in the same cwd don't collide on container/network
|
||||
# names); a `resume` passes the recorded identity in via
|
||||
# spec.identity to continue an existing bottle's state.
|
||||
slug = spec.identity or bottle_identity(spec.agent_name)
|
||||
# Record the launch metadata so `cli.py resume <identity>` can
|
||||
# reconstruct the spec. Idempotent — re-writes on resume with a
|
||||
# refreshed started_at.
|
||||
write_metadata(BottleMetadata(
|
||||
identity=slug,
|
||||
agent_name=spec.agent_name,
|
||||
cwd=spec.user_cwd if spec.copy_cwd else "",
|
||||
copy_cwd=spec.copy_cwd,
|
||||
started_at=datetime.now(timezone.utc).isoformat(),
|
||||
compose_project=f"bot-bottle-{slug}",
|
||||
backend="docker",
|
||||
label=spec.label,
|
||||
color=spec.color,
|
||||
))
|
||||
# Clear any leftover preserve marker from a prior capability-block
|
||||
# so this fresh launch can be cleaned up at session-end unless
|
||||
# the agent triggers another capability-block.
|
||||
clear_preserve_marker(slug)
|
||||
|
||||
# PRD 0016 capability-block: if a per-bottle Dockerfile has been
|
||||
# written (via apply_capability_change), the base image becomes
|
||||
# per_bottle_image_tag(slug) built from that file. --cwd still
|
||||
# layers a derived image on top.
|
||||
dockerfile_path = ""
|
||||
if per_bottle_dockerfile(slug) is not None:
|
||||
image_default = per_bottle_image_tag(slug)
|
||||
dockerfile_path = str(per_bottle_dockerfile_path(slug))
|
||||
elif provider.dockerfile:
|
||||
image_default = f"bot-bottle-{provider.template}:{slug}"
|
||||
dockerfile_path = _resolve_manifest_dockerfile(provider.dockerfile, spec)
|
||||
elif provider_runtime.dockerfile:
|
||||
image_default = provider_runtime.image
|
||||
dockerfile_path = provider_runtime.dockerfile
|
||||
elif provider.template not in PROVIDER_TEMPLATES:
|
||||
user_dockerfile = (
|
||||
Path.home() / ".bot-bottle" / "contrib" / provider.template / "Dockerfile"
|
||||
)
|
||||
if user_dockerfile.is_file():
|
||||
image_default = f"bot-bottle-{provider.template}:{slug}"
|
||||
dockerfile_path = str(user_dockerfile)
|
||||
else:
|
||||
image_default = provider_runtime.image
|
||||
else:
|
||||
image_default = provider_runtime.image
|
||||
image = os.environ.get("BOT_BOTTLE_IMAGE", image_default)
|
||||
derived_image = ""
|
||||
runtime_image = image
|
||||
if spec.copy_cwd:
|
||||
derived_image = os.environ.get(
|
||||
"BOT_BOTTLE_DERIVED_IMAGE", f"bot-bottle-cwd:{slug}"
|
||||
)
|
||||
runtime_image = derived_image
|
||||
|
||||
default_container = f"bot-bottle-{slug}"
|
||||
pinned_container = os.environ.get("BOT_BOTTLE_CONTAINER", "")
|
||||
container_name_pinned = bool(pinned_container)
|
||||
if container_name_pinned:
|
||||
container_name = pinned_container
|
||||
if docker_mod.container_exists(container_name):
|
||||
die(
|
||||
f"container '{container_name}' already exists "
|
||||
f"(pinned via BOT_BOTTLE_CONTAINER). "
|
||||
f"Remove it with 'docker rm -f {container_name}' or unset the override."
|
||||
)
|
||||
else:
|
||||
container_name = ""
|
||||
for candidate in docker_mod.container_name_candidates(default_container):
|
||||
if not docker_mod.container_exists(candidate):
|
||||
container_name = candidate
|
||||
break
|
||||
if not container_name:
|
||||
die(
|
||||
f"could not find a free container name after "
|
||||
f"{default_container}-{docker_mod.MAX_CONTAINER_SUFFIX}; "
|
||||
f"clean up old containers with 'docker rm -f <name>'"
|
||||
)
|
||||
|
||||
# Probe the sidecar-bundle container name for an orphan from a
|
||||
# previous run. Otherwise a stale bundle surfaces as a
|
||||
# docker-create conflict deep inside launch() with no actionable
|
||||
# hint; failing fast here points at the cleanup command.
|
||||
bundle_name = sidecar_bundle_container_name(slug)
|
||||
if docker_mod.container_exists(bundle_name):
|
||||
die(
|
||||
f"sidecar bundle container '{bundle_name}' already exists. "
|
||||
f"This is an orphan from a previous run; clean it up with "
|
||||
f"'./cli.py cleanup' (or 'docker rm -f {bundle_name}') and "
|
||||
f"retry."
|
||||
)
|
||||
|
||||
# PRD 0018 chunk 2: prepare-time scratch files live under
|
||||
# ~/.bot-bottle/state/<slug>/<service>/ so chunk 3's compose
|
||||
# bind-mounts can point at stable paths. The state subdirs are
|
||||
# cleaned up by start.py's session-end teardown unless something
|
||||
# explicitly preserves the state dir (capability-block, crash).
|
||||
agent_dir = agent_state_dir(slug)
|
||||
agent_dir.mkdir(parents=True, exist_ok=True)
|
||||
env_file = agent_dir / "agent.env"
|
||||
prompt_file = agent_dir / "prompt.txt"
|
||||
prompt_file.write_text("")
|
||||
prompt_file.chmod(0o600)
|
||||
|
||||
git_gate_dir = git_gate_state_dir(slug)
|
||||
git_gate_dir.mkdir(parents=True, exist_ok=True)
|
||||
git_gate_plan = git_gate.prepare(bottle, slug, git_gate_dir)
|
||||
|
||||
resolved = resolve_env(manifest, spec.agent_name)
|
||||
# Everything that should reach the bottle by-name (so its value
|
||||
# never lands on argv or in env_file) goes into one dict. Nothing
|
||||
# mutates the host os.environ.
|
||||
forwarded_env: dict[str, str] = dict(resolved.forwarded)
|
||||
_write_env_file(resolved, env_file)
|
||||
prompt_file.write_text(agent.prompt)
|
||||
|
||||
use_runsc = docker_mod.runsc_available()
|
||||
agent_provision = agent_provision_plan(
|
||||
template=provider.template,
|
||||
dockerfile=dockerfile_path,
|
||||
state_dir=agent_dir,
|
||||
guest_home=guest_home,
|
||||
forward_host_credentials=provider.forward_host_credentials,
|
||||
auth_token=provider.auth_token,
|
||||
host_env=dict(os.environ),
|
||||
trusted_project_path=workspace_plan.workdir,
|
||||
label=spec.label,
|
||||
color=spec.color,
|
||||
)
|
||||
guest_env = dict(agent_provision.guest_env)
|
||||
for key, val in agent_provision.env_vars.items():
|
||||
guest_env.setdefault(key, val)
|
||||
agent_provision = replace(agent_provision, guest_env=guest_env)
|
||||
|
||||
egress_dir = egress_state_dir(slug)
|
||||
egress_dir.mkdir(parents=True, exist_ok=True)
|
||||
egress_plan = egress.prepare(
|
||||
bottle, slug, egress_dir, agent_provision.egress_routes,
|
||||
)
|
||||
|
||||
supervise_plan = None
|
||||
if bottle.supervise:
|
||||
# Current Dockerfile for the agent image. Read from the repo
|
||||
# root; for `--cwd` derived images the base Dockerfile is what
|
||||
# the agent should propose changes against (the derived layer
|
||||
# is just a workspace copy).
|
||||
# (routes.yaml used to land here too but PRD 0017 chunk 3
|
||||
# moved it behind the `list-egress-routes` MCP tool so the
|
||||
# agent gets live state rather than a launch-time snapshot.)
|
||||
supervise_dockerfile_path = (
|
||||
Path(dockerfile_path)
|
||||
if dockerfile_path
|
||||
else Path(__file__).resolve().parent.parent.parent.parent / "Dockerfile.claude"
|
||||
)
|
||||
dockerfile_content = (
|
||||
supervise_dockerfile_path.read_text(encoding="utf-8")
|
||||
if supervise_dockerfile_path.is_file()
|
||||
else ""
|
||||
)
|
||||
supervise_dir = supervise_state_dir(slug)
|
||||
supervise_dir.mkdir(parents=True, exist_ok=True)
|
||||
supervise_plan = supervise.prepare(
|
||||
slug, supervise_dir,
|
||||
dockerfile_content=dockerfile_content,
|
||||
)
|
||||
|
||||
return DockerBottlePlan(
|
||||
spec=spec,
|
||||
stage_dir=stage_dir,
|
||||
guest_home=guest_home,
|
||||
slug=slug,
|
||||
container_name=container_name,
|
||||
container_name_pinned=container_name_pinned,
|
||||
image=image,
|
||||
derived_image=derived_image,
|
||||
runtime_image=runtime_image,
|
||||
dockerfile_path=dockerfile_path,
|
||||
env_file=env_file,
|
||||
forwarded_env=forwarded_env,
|
||||
prompt_file=prompt_file,
|
||||
git_gate_plan=git_gate_plan,
|
||||
egress_plan=egress_plan,
|
||||
supervise_plan=supervise_plan,
|
||||
use_runsc=use_runsc,
|
||||
agent_provision=agent_provision,
|
||||
workspace_plan=workspace_plan,
|
||||
)
|
||||
|
||||
|
||||
def _write_env_file(resolved: ResolvedEnv, env_file: Path) -> None:
|
||||
"""Serialize the literal portion of a ResolvedEnv into docker's
|
||||
`--env-file` syntax (NAME=VALUE per line, mode 600 since the file
|
||||
may carry verbatim values from the manifest). Forwarded names ride
|
||||
on the plan as a structured tuple instead."""
|
||||
env_lines: list[str] = []
|
||||
for name, value in resolved.literals.items():
|
||||
if "\n" in value:
|
||||
die(
|
||||
f"env entry {name} (literal) contains a newline; "
|
||||
f"docker --env-file cannot represent multi-line values."
|
||||
)
|
||||
env_lines.append(f"{name}={value}")
|
||||
env_file.write_text("\n".join(env_lines) + ("\n" if env_lines else ""))
|
||||
env_file.chmod(0o600)
|
||||
|
||||
|
||||
def _resolve_manifest_dockerfile(path_value: str, spec: BottleSpec) -> str:
|
||||
path = Path(os.path.expanduser(path_value))
|
||||
if not path.is_absolute():
|
||||
path = Path(spec.user_cwd) / path
|
||||
return str(path)
|
||||
@@ -0,0 +1,62 @@
|
||||
"""Prepare step for the Docker bottle backend.
|
||||
|
||||
`resolve_plan` does all host-side resolution (image and container
|
||||
names, prompt-file, proxy plan, runtime detection) and returns a
|
||||
frozen DockerBottlePlan. No Docker resources are created; the only
|
||||
side effects are scratch files under `stage_dir` and a probe of
|
||||
`docker info`. Cross-backend host-side validation has already run
|
||||
via the base class's `prepare` template before this is called.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from . import util as docker_mod
|
||||
from .bottle_plan import DockerBottlePlan
|
||||
from .. import BottleSpec
|
||||
from ...env import ResolvedEnv
|
||||
from ...agent_provider import AgentProvisionPlan
|
||||
from ...egress import EgressPlan
|
||||
from ...manifest import Manifest
|
||||
from ...supervise import SupervisePlan
|
||||
from ...git_gate import GitGatePlan
|
||||
|
||||
def preflight() -> None:
|
||||
docker_mod.require_docker()
|
||||
|
||||
|
||||
def build_guest_env(resolved_env: ResolvedEnv) -> dict[str, str]:
|
||||
return dict(resolved_env.literals)
|
||||
|
||||
|
||||
def resolve_plan(
|
||||
spec: BottleSpec,
|
||||
manifest: Manifest,
|
||||
slug: str,
|
||||
resolved_env: ResolvedEnv,
|
||||
agent_provision_plan: AgentProvisionPlan,
|
||||
egress_plan: EgressPlan,
|
||||
supervise_plan: SupervisePlan | None,
|
||||
git_gate_plan: GitGatePlan,
|
||||
stage_dir: Path,
|
||||
) -> DockerBottlePlan:
|
||||
"""Resolve Docker-specific names and write scratch files. Trusts
|
||||
that the agent and its skills/git-gate keys are present —
|
||||
validation already ran in the base class."""
|
||||
|
||||
# ==== docker specific setup ====
|
||||
use_runsc = docker_mod.runsc_available()
|
||||
|
||||
return DockerBottlePlan(
|
||||
spec=spec,
|
||||
manifest=manifest,
|
||||
stage_dir=stage_dir,
|
||||
slug=slug,
|
||||
forwarded_env=dict(resolved_env.forwarded),
|
||||
git_gate_plan=git_gate_plan,
|
||||
egress_plan=egress_plan,
|
||||
supervise_plan=supervise_plan,
|
||||
use_runsc=use_runsc,
|
||||
agent_provision=agent_provision_plan,
|
||||
)
|
||||
@@ -12,9 +12,10 @@ from __future__ import annotations
|
||||
import os
|
||||
|
||||
|
||||
# Bundle image. Defaults to a built-locally tag (built from the
|
||||
# repo's Dockerfile.sidecars via compose `build:`). Operators
|
||||
# pinning to a published digest can override via env.
|
||||
# Bundle image. Defaults to a built-locally tag. Source checkouts
|
||||
# build from the repo-root Dockerfile.sidecars; installed packages
|
||||
# build from the packaged copy under bot_bottle/.
|
||||
# Operators pinning to a published digest can override via env.
|
||||
SIDECAR_BUNDLE_IMAGE = os.environ.get(
|
||||
"BOT_BOTTLE_SIDECAR_IMAGE",
|
||||
"bot-bottle-sidecars:latest",
|
||||
|
||||
@@ -7,11 +7,10 @@ from __future__ import annotations
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
from typing import Iterable, Iterator
|
||||
|
||||
from ...log import die, info
|
||||
from ...workspace import WorkspacePlan
|
||||
# from ...workspace import WorkspacePlan
|
||||
|
||||
|
||||
# Cap on the suffix the container-name conflict logic will try before
|
||||
@@ -118,39 +117,54 @@ def build_image(ref: str, context: str, *, dockerfile: str = "") -> None:
|
||||
subprocess.run(args, check=True)
|
||||
|
||||
|
||||
def build_image_with_cwd(
|
||||
derived: str,
|
||||
base: str,
|
||||
workspace: WorkspacePlan,
|
||||
) -> None:
|
||||
"""Build a thin derived image that copies the workspace into
|
||||
the plan's guest path and sets the plan's workdir."""
|
||||
import os
|
||||
# def build_image_with_cwd(
|
||||
# derived: str,
|
||||
# base: str,
|
||||
# workspace: "WorkspacePlan",
|
||||
# ) -> None:
|
||||
# """Build a thin derived image that copies the workspace into
|
||||
# the plan's guest path and sets the plan's workdir."""
|
||||
# import os
|
||||
#
|
||||
# cwd = str(workspace.host_path)
|
||||
# if not os.path.isdir(cwd):
|
||||
# die(f"cwd not found at {cwd}")
|
||||
# info(f"building image {derived} from {base} with {cwd} -> {workspace.guest_path}")
|
||||
# with tempfile.TemporaryDirectory(prefix="bot-bottle-cwd.") as tmp:
|
||||
# context_dir = os.path.join(tmp, "context")
|
||||
# staged_workspace = os.path.join(context_dir, "workspace")
|
||||
# shutil.copytree(
|
||||
# cwd,
|
||||
# staged_workspace,
|
||||
# symlinks=True,
|
||||
# ignore=shutil.ignore_patterns(".git"),
|
||||
# )
|
||||
# dockerfile = (
|
||||
# f"FROM {base}\n"
|
||||
# f"COPY --chown=node:node workspace/. {workspace.guest_path}\n"
|
||||
# f"WORKDIR {workspace.workdir}\n"
|
||||
# )
|
||||
# subprocess.run(
|
||||
# ["docker", "build", "-t", derived, "-f", "-", context_dir],
|
||||
# input=dockerfile,
|
||||
# text=True,
|
||||
# check=True,
|
||||
# )
|
||||
|
||||
cwd = str(workspace.host_path)
|
||||
if not os.path.isdir(cwd):
|
||||
die(f"cwd not found at {cwd}")
|
||||
info(f"building image {derived} from {base} with {cwd} -> {workspace.guest_path}")
|
||||
with tempfile.TemporaryDirectory(prefix="bot-bottle-cwd.") as tmp:
|
||||
context_dir = os.path.join(tmp, "context")
|
||||
staged_workspace = os.path.join(context_dir, "workspace")
|
||||
shutil.copytree(
|
||||
cwd,
|
||||
staged_workspace,
|
||||
symlinks=True,
|
||||
ignore=shutil.ignore_patterns(".git"),
|
||||
)
|
||||
dockerfile = (
|
||||
f"FROM {base}\n"
|
||||
f"COPY --chown=node:node workspace/. {workspace.guest_path}\n"
|
||||
f"WORKDIR {workspace.workdir}\n"
|
||||
)
|
||||
subprocess.run(
|
||||
["docker", "build", "-t", derived, "-f", "-", context_dir],
|
||||
input=dockerfile,
|
||||
text=True,
|
||||
check=True,
|
||||
|
||||
def commit_container(container_name: str, image_tag: str) -> None:
|
||||
"""Run `docker commit <container_name> <image_tag>` to snapshot the
|
||||
running container's filesystem state as a local Docker image."""
|
||||
result = subprocess.run(
|
||||
["docker", "commit", container_name, image_tag],
|
||||
capture_output=True, text=True, check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
die(
|
||||
f"docker commit {container_name!r} → {image_tag!r} failed: "
|
||||
f"{(result.stderr or '').strip() or '<no stderr>'}"
|
||||
)
|
||||
info(f"committed {container_name!r} → {image_tag!r}")
|
||||
|
||||
|
||||
def image_id(ref: str) -> str:
|
||||
|
||||
@@ -0,0 +1,50 @@
|
||||
"""Shared base class for host-side egress apply across backends.
|
||||
|
||||
Each backend subclasses EgressApplicator and overrides _signal_bundle_reload
|
||||
with the backend-specific kill command.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from abc import ABC, abstractmethod
|
||||
from pathlib import Path
|
||||
|
||||
from ..bottle_state import egress_state_dir
|
||||
from ..egress import EGRESS_ROUTES_FILENAME
|
||||
from ..egress_addon_core import load_routes
|
||||
|
||||
|
||||
class EgressApplyError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
class EgressApplicator(ABC):
|
||||
def apply_routes_change(self, slug: str, content: str) -> tuple[str, str]:
|
||||
"""Persist `content` to the live routes file and reload egress."""
|
||||
self.validate_routes_content(content)
|
||||
routes_path = self._routes_path(slug)
|
||||
routes_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
before = routes_path.read_text(encoding="utf-8") if routes_path.exists() else ""
|
||||
routes_path.write_text(content, encoding="utf-8")
|
||||
routes_path.chmod(0o600)
|
||||
self._signal_bundle_reload(slug)
|
||||
return before, content
|
||||
|
||||
@staticmethod
|
||||
def validate_routes_content(content: str) -> None:
|
||||
try:
|
||||
load_routes(content)
|
||||
except ValueError as e:
|
||||
raise EgressApplyError(
|
||||
f"proposed routes.yaml is not valid: {e}"
|
||||
) from e
|
||||
|
||||
@staticmethod
|
||||
def _routes_path(slug: str) -> Path:
|
||||
return egress_state_dir(slug) / EGRESS_ROUTES_FILENAME
|
||||
|
||||
@abstractmethod
|
||||
def _signal_bundle_reload(self, slug: str) -> None: ...
|
||||
|
||||
|
||||
__all__ = ["EgressApplicator", "EgressApplyError"]
|
||||
@@ -0,0 +1,100 @@
|
||||
"""Freezer — snapshot a running bottle to a resumable artifact.
|
||||
|
||||
Follows the same pattern as BottleBackend: a shared base class with
|
||||
common post-freeze steps (write committed-image path, mark preserved,
|
||||
print resume hint) and backend-specific subclasses in their respective
|
||||
backend directories.
|
||||
|
||||
Entry points:
|
||||
Freezer.commit(agent) — freeze by ActiveAgent
|
||||
Freezer.commit_slug(slug) — convenience wrapper for cmd_commit
|
||||
get_freezer(backend_name) — factory
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from abc import ABC, abstractmethod
|
||||
|
||||
from . import ActiveAgent
|
||||
from ..bottle_state import mark_preserved, write_committed_image
|
||||
from ..log import die, info
|
||||
|
||||
|
||||
class CommitCancelled(Exception):
|
||||
"""Raised by Freezer._freeze when the user declines a confirmation prompt."""
|
||||
|
||||
|
||||
class Freezer(ABC):
|
||||
"""Freezes a running bottle to a resumable artifact.
|
||||
|
||||
The base class owns the shared post-commit steps:
|
||||
- write_committed_image — records the artifact path in per-bottle state
|
||||
- mark_preserved — prevents teardown from removing the state dir
|
||||
- resume hint — printed to stderr after the snapshot
|
||||
|
||||
Subclasses implement _freeze with the backend-specific snapshot
|
||||
operation and optionally override _export_hint for migration hints.
|
||||
"""
|
||||
|
||||
backend_name: str
|
||||
|
||||
def commit(self, agent: ActiveAgent) -> None:
|
||||
"""Freeze the bottle for `agent` to a resumable artifact.
|
||||
|
||||
Calls _freeze for the backend-specific snapshot, then writes the
|
||||
committed image reference to per-bottle state and marks the bottle
|
||||
preserved so the next `./cli.py resume` boots from the snapshot.
|
||||
|
||||
Raises CommitCancelled if the user declines an interactive
|
||||
confirmation prompt (e.g. the macos-container stop prompt).
|
||||
"""
|
||||
image_ref = self._freeze(agent)
|
||||
write_committed_image(agent.slug, image_ref)
|
||||
mark_preserved(agent.slug)
|
||||
info(f"to resume from this snapshot: ./cli.py resume {agent.slug}")
|
||||
self._export_hint(agent.slug, image_ref)
|
||||
|
||||
@abstractmethod
|
||||
def _freeze(self, agent: ActiveAgent) -> str:
|
||||
"""Backend-specific snapshot. Returns the image tag or artifact path
|
||||
stored by write_committed_image. Raises CommitCancelled if the user
|
||||
declines a stop-confirmation prompt."""
|
||||
|
||||
def _export_hint(self, slug: str, image_ref: str) -> None:
|
||||
"""Optionally print an export-for-migration hint after committing.
|
||||
Overridden by backends that provide a meaningful export command."""
|
||||
|
||||
def commit_slug(self, slug: str) -> None:
|
||||
"""Convenience entry for cmd_commit when only a slug is available."""
|
||||
from ..bottle_state import read_metadata
|
||||
metadata = read_metadata(slug)
|
||||
agent = ActiveAgent(
|
||||
backend_name=self.backend_name,
|
||||
slug=slug,
|
||||
agent_name=metadata.agent_name if metadata else "",
|
||||
started_at=metadata.started_at if metadata else "",
|
||||
services=(),
|
||||
)
|
||||
self.commit(agent)
|
||||
|
||||
|
||||
def get_freezer(backend_name: str) -> Freezer:
|
||||
"""Return the Freezer for the named backend.
|
||||
|
||||
backend_name "" is treated as "docker" for backward compatibility
|
||||
with state dirs written before the backend field was added."""
|
||||
resolved = backend_name or "docker"
|
||||
if resolved == "docker":
|
||||
from .docker.freezer import DockerFreezer
|
||||
return DockerFreezer()
|
||||
if resolved == "macos-container":
|
||||
from .macos_container.freezer import MacosContainerFreezer
|
||||
return MacosContainerFreezer()
|
||||
if resolved == "smolmachines":
|
||||
from .smolmachines.freezer import SmolmachinesFreezer
|
||||
return SmolmachinesFreezer()
|
||||
die(
|
||||
f"commit is only supported for docker, macos-container, and "
|
||||
f"smolmachines; backend {backend_name!r} has no freezer"
|
||||
)
|
||||
raise AssertionError("unreachable")
|
||||
@@ -0,0 +1,10 @@
|
||||
"""macOS Apple Container backend.
|
||||
|
||||
Selectable via `BOT_BOTTLE_BACKEND=macos-container`. This package owns
|
||||
the Apple `container` CLI integration; launch remains gated until the
|
||||
sidecar network enforcement shape is implemented.
|
||||
"""
|
||||
|
||||
from .backend import MacosContainerBottleBackend
|
||||
|
||||
__all__ = ["MacosContainerBottleBackend"]
|
||||
@@ -0,0 +1,87 @@
|
||||
"""MacosContainerBottleBackend — Apple Container implementation."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from contextlib import contextmanager
|
||||
from pathlib import Path
|
||||
from typing import Generator, Sequence
|
||||
|
||||
from ...agent_provider import AgentProvisionPlan
|
||||
from ...egress import EgressPlan
|
||||
from ...env import ResolvedEnv
|
||||
from ...git_gate import GitGatePlan
|
||||
from ...supervise import SupervisePlan
|
||||
from ...manifest import Manifest
|
||||
from .. import ActiveAgent, BottleBackend, BottleSpec
|
||||
from . import cleanup as _cleanup
|
||||
from . import enumerate as _enumerate
|
||||
from . import launch as _launch
|
||||
from . import resolve_plan as _resolve_plan
|
||||
from . import util as _container
|
||||
from .bottle import MacosContainerBottle
|
||||
from .bottle_cleanup_plan import MacosContainerBottleCleanupPlan
|
||||
from .bottle_plan import MacosContainerBottlePlan
|
||||
|
||||
|
||||
class MacosContainerBottleBackend(
|
||||
BottleBackend["MacosContainerBottlePlan", "MacosContainerBottleCleanupPlan"]
|
||||
):
|
||||
"""Apple Container backend. Selected by
|
||||
`BOT_BOTTLE_BACKEND=macos-container` or
|
||||
`--backend=macos-container`."""
|
||||
|
||||
name = "macos-container"
|
||||
|
||||
@classmethod
|
||||
def is_available(cls) -> bool:
|
||||
return _container.is_available()
|
||||
|
||||
def _preflight(self) -> None:
|
||||
_resolve_plan.preflight()
|
||||
|
||||
def _build_guest_env(self, resolved_env: ResolvedEnv) -> dict[str, str]:
|
||||
return _resolve_plan.build_guest_env(resolved_env)
|
||||
|
||||
def _resolve_plan(
|
||||
self,
|
||||
spec: BottleSpec,
|
||||
*,
|
||||
manifest: Manifest,
|
||||
slug: str,
|
||||
resolved_env: ResolvedEnv,
|
||||
agent_provision_plan: AgentProvisionPlan,
|
||||
egress_plan: EgressPlan,
|
||||
git_gate_plan: GitGatePlan,
|
||||
supervise_plan: SupervisePlan | None,
|
||||
stage_dir: Path,
|
||||
) -> MacosContainerBottlePlan:
|
||||
return _resolve_plan.resolve_plan(
|
||||
spec,
|
||||
manifest=manifest,
|
||||
slug=slug,
|
||||
resolved_env=resolved_env,
|
||||
agent_provision_plan=agent_provision_plan,
|
||||
egress_plan=egress_plan,
|
||||
supervise_plan=supervise_plan,
|
||||
git_gate_plan=git_gate_plan,
|
||||
stage_dir=stage_dir,
|
||||
)
|
||||
|
||||
@contextmanager
|
||||
def launch(
|
||||
self, plan: MacosContainerBottlePlan
|
||||
) -> Generator[MacosContainerBottle, None, None]:
|
||||
with _launch.launch(plan, provision=self.provision) as bottle:
|
||||
yield bottle
|
||||
|
||||
def prepare_cleanup(self) -> MacosContainerBottleCleanupPlan:
|
||||
return _cleanup.prepare_cleanup()
|
||||
|
||||
def cleanup(self, plan: MacosContainerBottleCleanupPlan) -> None:
|
||||
_cleanup.cleanup(plan)
|
||||
|
||||
def enumerate_active(self) -> Sequence[ActiveAgent]:
|
||||
return _enumerate.enumerate_active()
|
||||
|
||||
def supervise_mcp_url(self, plan: MacosContainerBottlePlan) -> str:
|
||||
return plan.agent_supervise_url
|
||||
@@ -0,0 +1,131 @@
|
||||
"""Bottle handle for Apple's `container` CLI."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
from typing import Callable, cast
|
||||
|
||||
from ...agent_provider import PromptMode, prompt_args
|
||||
from .. import Bottle, ExecResult
|
||||
from ..terminal import exec_shell_script
|
||||
from . import pty_forward as _pty_forward
|
||||
|
||||
|
||||
_PTY_FORWARD_SCRIPT = _pty_forward.__file__
|
||||
_TERMINAL_ENV_NAMES = (
|
||||
"TERM",
|
||||
"COLORTERM",
|
||||
"TERM_PROGRAM",
|
||||
"TERM_PROGRAM_VERSION",
|
||||
"KITTY_WINDOW_ID",
|
||||
"KITTY_PID",
|
||||
"WEZTERM_PANE",
|
||||
"WEZTERM_UNIX_SOCKET",
|
||||
"GHOSTTY_BIN_DIR",
|
||||
"GHOSTTY_RESOURCES_DIR",
|
||||
"ITERM_SESSION_ID",
|
||||
"VTE_VERSION",
|
||||
"KONSOLE_VERSION",
|
||||
"ALACRITTY_WINDOW_ID",
|
||||
)
|
||||
|
||||
|
||||
def _terminal_env_names() -> tuple[str, ...]:
|
||||
return tuple(
|
||||
name for name in _TERMINAL_ENV_NAMES
|
||||
if name == "TERM" or os.environ.get(name)
|
||||
)
|
||||
|
||||
|
||||
class MacosContainerBottle(Bottle):
|
||||
def __init__(
|
||||
self,
|
||||
container: str,
|
||||
teardown: Callable[[], None],
|
||||
prompt_path_in_container: str | None,
|
||||
*,
|
||||
agent_command: str = "claude",
|
||||
agent_prompt_mode: PromptMode = "append_file",
|
||||
agent_provider_template: str = "claude",
|
||||
terminal_title: str = "",
|
||||
terminal_color: str = "",
|
||||
agent_workdir: str = "/home/node",
|
||||
):
|
||||
self.name = container
|
||||
self._teardown = teardown
|
||||
self.prompt_path = prompt_path_in_container
|
||||
self._agent_prompt_mode = agent_prompt_mode
|
||||
self.agent_command = agent_command
|
||||
self.terminal_title = terminal_title
|
||||
self.terminal_color = terminal_color
|
||||
self.agent_provider_template = agent_provider_template
|
||||
self.agent_workdir = agent_workdir
|
||||
self._closed = False
|
||||
|
||||
def agent_argv(self, argv: list[str], *, tty: bool = True) -> list[str]:
|
||||
full_argv = list(argv)
|
||||
full_argv.extend(
|
||||
prompt_args(
|
||||
cast(PromptMode, self._agent_prompt_mode),
|
||||
self.prompt_path,
|
||||
argv=full_argv,
|
||||
)
|
||||
)
|
||||
container_exec = ["container", "exec"]
|
||||
if tty:
|
||||
container_exec.extend(["--interactive", "--tty"])
|
||||
# Forward terminal capability hints so TUIs can enable modified-key
|
||||
# protocols. Use bare env names: values stay in the child env, not
|
||||
# on argv, and pty_forward supplies a TERM fallback when needed.
|
||||
for name in _terminal_env_names():
|
||||
container_exec.extend(["--env", name])
|
||||
if self.agent_workdir and self.agent_workdir != "/home/node":
|
||||
container_exec.extend(["--workdir", self.agent_workdir])
|
||||
container_exec.extend([self.name, self.agent_command, *full_argv])
|
||||
if tty:
|
||||
# Wrap with the raw-mode forwarder: container exec does not put
|
||||
# the host terminal into raw mode itself, so the line discipline
|
||||
# buffers modifier-key sequences until CR. The wrapper sets raw
|
||||
# mode before exec and restores it on exit.
|
||||
return [sys.executable, _PTY_FORWARD_SCRIPT, "--", *container_exec]
|
||||
return container_exec
|
||||
|
||||
def exec_agent(self, argv: list[str], *, tty: bool = True) -> int:
|
||||
agent_argv = self.agent_argv(argv, tty=tty)
|
||||
script = (
|
||||
exec_shell_script(agent_argv, self.terminal_title, self.terminal_color)
|
||||
if tty else None
|
||||
)
|
||||
if script is None:
|
||||
return subprocess.run(agent_argv, check=False).returncode
|
||||
return subprocess.run(["sh", "-lc", script], check=False).returncode
|
||||
|
||||
def exec(self, script: str, *, user: str = "node") -> ExecResult:
|
||||
result = subprocess.run(
|
||||
["container", "exec", "--user", user, "--interactive",
|
||||
self.name, "sh", "-s"],
|
||||
input=script,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
return ExecResult(
|
||||
returncode=result.returncode,
|
||||
stdout=result.stdout,
|
||||
stderr=result.stderr,
|
||||
)
|
||||
|
||||
def cp_in(self, host_path: str, container_path: str) -> None:
|
||||
subprocess.run(
|
||||
["container", "cp", host_path, f"{self.name}:{container_path}"],
|
||||
stdout=subprocess.DEVNULL,
|
||||
check=True,
|
||||
)
|
||||
|
||||
def close(self) -> None:
|
||||
if self._closed:
|
||||
return
|
||||
self._closed = True
|
||||
self._teardown()
|
||||
@@ -0,0 +1,27 @@
|
||||
"""Cleanup plan for the macOS Apple Container backend."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
from ...log import info
|
||||
from .. import BottleCleanupPlan
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class MacosContainerBottleCleanupPlan(BottleCleanupPlan):
|
||||
containers: tuple[str, ...] = ()
|
||||
networks: tuple[str, ...] = ()
|
||||
|
||||
def print(self) -> None:
|
||||
if not self.containers and not self.networks:
|
||||
info("macos-container cleanup: nothing to remove")
|
||||
return
|
||||
for name in self.containers:
|
||||
info(f"macos-container container: {name}")
|
||||
for name in self.networks:
|
||||
info(f"macos-container network: {name}")
|
||||
|
||||
@property
|
||||
def empty(self) -> bool:
|
||||
return not self.containers and not self.networks
|
||||
@@ -0,0 +1,58 @@
|
||||
"""Plan type for the macOS Apple Container backend."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
from ...agent_provider import PromptMode
|
||||
from .. import BottlePlan
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class MacosContainerBottlePlan(BottlePlan):
|
||||
slug: str
|
||||
forwarded_env: dict[str, str] = field(repr=False)
|
||||
agent_proxy_url: str = ""
|
||||
agent_git_gate_url: str = ""
|
||||
agent_supervise_url: str = ""
|
||||
|
||||
@property
|
||||
def container_name(self) -> str:
|
||||
return self.agent_provision.instance_name
|
||||
|
||||
@property
|
||||
def image(self) -> str:
|
||||
return self.agent_provision.image
|
||||
|
||||
@property
|
||||
def dockerfile_path(self) -> str:
|
||||
return self.agent_provision.dockerfile
|
||||
|
||||
@property
|
||||
def prompt_file(self) -> Path:
|
||||
return self.agent_provision.prompt_file
|
||||
|
||||
@property
|
||||
def agent_command(self) -> str:
|
||||
return self.agent_provision.command
|
||||
|
||||
@property
|
||||
def agent_prompt_mode(self) -> PromptMode:
|
||||
return self.agent_provision.prompt_mode
|
||||
|
||||
@property
|
||||
def agent_provider_template(self) -> str:
|
||||
return self.agent_provision.template
|
||||
|
||||
@property
|
||||
def git_gate_insteadof_host(self) -> str:
|
||||
if self.agent_git_gate_url.startswith("http://"):
|
||||
return self.agent_git_gate_url.removeprefix("http://").rstrip("/")
|
||||
return super().git_gate_insteadof_host
|
||||
|
||||
@property
|
||||
def git_gate_insteadof_scheme(self) -> str:
|
||||
if self.agent_git_gate_url.startswith("http://"):
|
||||
return "http"
|
||||
return super().git_gate_insteadof_scheme
|
||||
@@ -0,0 +1,70 @@
|
||||
"""Cleanup for the macOS Apple Container backend."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
|
||||
from ...log import info, warn
|
||||
from . import util as container_mod
|
||||
from .bottle_cleanup_plan import MacosContainerBottleCleanupPlan
|
||||
|
||||
_PREFIX = "bot-bottle-"
|
||||
_BUNDLE_PREFIX = "bot-bottle-sidecars-"
|
||||
|
||||
|
||||
def _list_prefixed_containers() -> list[str]:
|
||||
result = subprocess.run(
|
||||
["container", "list", "--all", "--quiet"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
warn(f"container list failed: {result.stderr.strip()}")
|
||||
return []
|
||||
return sorted(
|
||||
name for name in (line.strip() for line in result.stdout.splitlines())
|
||||
if name.startswith(_PREFIX) or name.startswith(_BUNDLE_PREFIX)
|
||||
)
|
||||
|
||||
|
||||
def _list_prefixed_networks() -> list[str]:
|
||||
result = subprocess.run(
|
||||
["container", "network", "list", "--quiet"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return []
|
||||
return sorted(
|
||||
name for name in (line.strip() for line in result.stdout.splitlines())
|
||||
if name.startswith(_PREFIX)
|
||||
)
|
||||
|
||||
|
||||
def prepare_cleanup() -> MacosContainerBottleCleanupPlan:
|
||||
container_mod.require_container()
|
||||
return MacosContainerBottleCleanupPlan(
|
||||
containers=tuple(_list_prefixed_containers()),
|
||||
networks=tuple(_list_prefixed_networks()),
|
||||
)
|
||||
|
||||
|
||||
def cleanup(plan: MacosContainerBottleCleanupPlan) -> None:
|
||||
for name in plan.containers:
|
||||
info(f"container delete --force {name}")
|
||||
subprocess.run(
|
||||
["container", "delete", "--force", name],
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
)
|
||||
for name in plan.networks:
|
||||
info(f"container network delete {name}")
|
||||
subprocess.run(
|
||||
["container", "network", "delete", name],
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
)
|
||||
@@ -0,0 +1,39 @@
|
||||
"""Host-side egress apply for the macos-container backend.
|
||||
|
||||
Uses `container kill --signal HUP` (Apple Container framework) instead
|
||||
of `docker kill` to signal the sidecar bundle.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
|
||||
from ...log import warn
|
||||
from ..egress_apply import EgressApplicator, EgressApplyError
|
||||
from .launch import sidecar_container_name
|
||||
|
||||
|
||||
class MacOSContainerEgressApplicator(EgressApplicator):
|
||||
def _signal_bundle_reload(self, slug: str) -> None:
|
||||
container = sidecar_container_name(slug)
|
||||
result = subprocess.run(
|
||||
["container", "kill", "--signal", "HUP", container],
|
||||
capture_output=True, text=True, check=False, env=os.environ,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
last_error = (result.stderr or "").strip() or (result.stdout or "").strip()
|
||||
warn(
|
||||
f"egress: routes updated on disk for {slug}, but bundle reload failed: "
|
||||
f"{last_error or 'container kill failed'}"
|
||||
)
|
||||
raise EgressApplyError(
|
||||
f"could not reload egress bundle {container}: "
|
||||
f"{last_error or 'container kill failed'}"
|
||||
)
|
||||
|
||||
|
||||
applicator = MacOSContainerEgressApplicator()
|
||||
|
||||
|
||||
__all__ = ["MacOSContainerEgressApplicator", "EgressApplyError", "applicator"]
|
||||
@@ -0,0 +1,40 @@
|
||||
"""Active-agent enumeration for the macOS Apple Container backend."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
|
||||
from ...bottle_state import read_metadata
|
||||
from .. import ActiveAgent
|
||||
|
||||
_PREFIX = "bot-bottle-"
|
||||
_SIDECAR_PREFIX = "bot-bottle-sidecars-"
|
||||
|
||||
|
||||
def enumerate_active() -> list[ActiveAgent]:
|
||||
result = subprocess.run(
|
||||
["container", "list", "--quiet"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return []
|
||||
out: list[ActiveAgent] = []
|
||||
for name in sorted(line.strip() for line in result.stdout.splitlines()):
|
||||
if not name.startswith(_PREFIX):
|
||||
continue
|
||||
if name.startswith(_SIDECAR_PREFIX):
|
||||
continue
|
||||
slug = name[len(_PREFIX):]
|
||||
metadata = read_metadata(slug)
|
||||
out.append(ActiveAgent(
|
||||
backend_name="macos-container",
|
||||
slug=slug,
|
||||
agent_name=metadata.agent_name if metadata else "?",
|
||||
started_at=metadata.started_at if metadata else "",
|
||||
services=(),
|
||||
label=metadata.label if metadata else "",
|
||||
color=metadata.color if metadata else "",
|
||||
))
|
||||
return out
|
||||
@@ -0,0 +1,31 @@
|
||||
"""MacosContainerFreezer — snapshot a macOS container bottle.
|
||||
|
||||
Apple Container removes containers when they stop, making stop-then-export
|
||||
impossible. Instead, commit_container execs into the running container and
|
||||
streams the root filesystem via tar. The bottle continues running after commit.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from .. import ActiveAgent
|
||||
from ..freeze import Freezer
|
||||
from .util import commit_container
|
||||
from ...log import info
|
||||
|
||||
|
||||
class MacosContainerFreezer(Freezer):
|
||||
"""Freezes a macOS-container bottle via exec-tar + image rebuild."""
|
||||
|
||||
backend_name = "macos-container"
|
||||
|
||||
def _freeze(self, agent: ActiveAgent) -> str:
|
||||
container = f"bot-bottle-{agent.slug}"
|
||||
image_tag = f"bot-bottle-committed-{agent.slug}:latest"
|
||||
commit_container(container, image_tag)
|
||||
return image_tag
|
||||
|
||||
def _export_hint(self, slug: str, image_ref: str) -> None:
|
||||
info(
|
||||
f"to export for migration: "
|
||||
f"container image save {image_ref} -o {slug}.tar"
|
||||
)
|
||||
@@ -0,0 +1,428 @@
|
||||
"""Launch flow for the macOS Apple Container backend.
|
||||
|
||||
This backend keeps the explicit proxy-env enforcement model for v1:
|
||||
the agent container is attached only to a host-only Apple Container
|
||||
network, while the sidecar bundle is attached to a NAT network first
|
||||
and the host-only network second. The sidecar's host-only IP is
|
||||
discovered from `container inspect` and stamped into the agent's
|
||||
HTTP_PROXY / HTTPS_PROXY env vars.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import dataclasses
|
||||
import os
|
||||
import subprocess
|
||||
from contextlib import ExitStack, contextmanager
|
||||
from pathlib import Path
|
||||
from typing import Callable, Generator
|
||||
|
||||
from ...bottle_state import (
|
||||
egress_state_dir,
|
||||
git_gate_state_dir,
|
||||
read_committed_image,
|
||||
)
|
||||
from ...egress import EGRESS_ROUTES_IN_CONTAINER, egress_resolve_token_values
|
||||
from ...git_gate import revoke_git_gate_provisioned_keys
|
||||
from ...log import die, info, warn
|
||||
from ...supervise import QUEUE_DIR_IN_CONTAINER, SUPERVISE_PORT
|
||||
from ...util import expand_tilde
|
||||
from ..docker.egress import EGRESS_CA_IN_CONTAINER, EGRESS_PORT
|
||||
from ..docker.git_gate import (
|
||||
GIT_GATE_ACCESS_HOOK_IN_CONTAINER,
|
||||
GIT_GATE_CREDS_DIR_IN_CONTAINER,
|
||||
GIT_GATE_ENTRYPOINT_IN_CONTAINER,
|
||||
GIT_GATE_HOOK_IN_CONTAINER,
|
||||
)
|
||||
from ..docker.sidecar_bundle import (
|
||||
SIDECAR_BUNDLE_DOCKERFILE,
|
||||
SIDECAR_BUNDLE_IMAGE,
|
||||
)
|
||||
from ..docker.egress import egress_tls_init
|
||||
from ..util import AGENT_CA_BUNDLE, AGENT_CA_PATH
|
||||
from . import util as container_mod
|
||||
from .bottle import MacosContainerBottle
|
||||
from .bottle_plan import MacosContainerBottlePlan
|
||||
|
||||
|
||||
_REPO_DIR = str(Path(__file__).resolve().parent.parent.parent.parent)
|
||||
_AGENT_SLEEP_SECONDS = "2147483647"
|
||||
_GIT_HTTP_PORT = 9420
|
||||
_GIT_GATE_READY_FILE = "/run/git-gate/ready"
|
||||
|
||||
|
||||
def internal_network_name(slug: str) -> str:
|
||||
return f"bot-bottle-net-{slug}"
|
||||
|
||||
|
||||
def egress_network_name(slug: str) -> str:
|
||||
return f"bot-bottle-egress-{slug}"
|
||||
|
||||
|
||||
def sidecar_container_name(slug: str) -> str:
|
||||
return f"bot-bottle-sidecars-{slug}"
|
||||
|
||||
|
||||
@contextmanager
|
||||
def launch(
|
||||
plan: MacosContainerBottlePlan,
|
||||
*,
|
||||
provision: Callable[[MacosContainerBottlePlan, "MacosContainerBottle"], str | None],
|
||||
) -> Generator[MacosContainerBottle, None, None]:
|
||||
"""Build, run, provision, and yield an Apple Container bottle."""
|
||||
stack = ExitStack()
|
||||
bottle_for_revoke = plan.manifest.bottle
|
||||
git_gate_dir_for_revoke = git_gate_state_dir(plan.slug)
|
||||
|
||||
def teardown() -> None:
|
||||
teardown_exc: BaseException | None = None
|
||||
try:
|
||||
stack.close()
|
||||
except BaseException as exc: # noqa: W0718 - teardown must continue
|
||||
teardown_exc = exc
|
||||
warn(f"macos-container teardown failed: {exc!r}")
|
||||
revoke_git_gate_provisioned_keys(bottle_for_revoke, git_gate_dir_for_revoke)
|
||||
if teardown_exc is not None:
|
||||
raise teardown_exc
|
||||
|
||||
try:
|
||||
plan = _mint_certs(plan)
|
||||
plan = _build_images(plan)
|
||||
|
||||
internal_network = internal_network_name(plan.slug)
|
||||
egress_network = egress_network_name(plan.slug)
|
||||
_create_networks(internal_network, egress_network, stack)
|
||||
|
||||
sidecar_name = sidecar_container_name(plan.slug)
|
||||
container_mod.force_remove_container(sidecar_name)
|
||||
_start_sidecar_bundle(plan, sidecar_name, internal_network, egress_network)
|
||||
stack.callback(container_mod.force_remove_container, sidecar_name)
|
||||
_stage_git_gate(plan, sidecar_name)
|
||||
|
||||
sidecar_ip = container_mod.container_ipv4_on_network(
|
||||
sidecar_name, internal_network,
|
||||
)
|
||||
plan = _stamp_agent_urls(plan, sidecar_ip)
|
||||
|
||||
container_mod.force_remove_container(plan.container_name)
|
||||
_start_agent(plan, internal_network, sidecar_ip)
|
||||
stack.callback(container_mod.force_remove_container, plan.container_name)
|
||||
|
||||
bottle = MacosContainerBottle(
|
||||
plan.container_name,
|
||||
teardown,
|
||||
None,
|
||||
agent_command=plan.agent_command,
|
||||
agent_prompt_mode=plan.agent_prompt_mode,
|
||||
agent_provider_template=plan.agent_provider_template,
|
||||
terminal_title=f"{plan.spec.label} ({plan.spec.agent_name})" if plan.spec.label else plan.spec.agent_name,
|
||||
terminal_color=plan.spec.color,
|
||||
agent_workdir=plan.workspace_plan.workdir,
|
||||
)
|
||||
bottle.prompt_path = provision(plan, bottle)
|
||||
|
||||
yield bottle
|
||||
finally:
|
||||
teardown()
|
||||
|
||||
|
||||
def _mint_certs(plan: MacosContainerBottlePlan) -> MacosContainerBottlePlan:
|
||||
egress_ca_host, egress_ca_cert_only = egress_tls_init(
|
||||
egress_state_dir(plan.slug),
|
||||
)
|
||||
egress_plan = dataclasses.replace(
|
||||
plan.egress_plan,
|
||||
mitmproxy_ca_host_path=egress_ca_host,
|
||||
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
|
||||
)
|
||||
return dataclasses.replace(plan, egress_plan=egress_plan)
|
||||
|
||||
|
||||
def _build_images(plan: MacosContainerBottlePlan) -> MacosContainerBottlePlan:
|
||||
container_mod.build_image(
|
||||
SIDECAR_BUNDLE_IMAGE,
|
||||
_REPO_DIR,
|
||||
dockerfile=SIDECAR_BUNDLE_DOCKERFILE,
|
||||
)
|
||||
committed = read_committed_image(plan.slug)
|
||||
if committed and container_mod.image_exists(committed):
|
||||
info(f"using committed image {committed!r}")
|
||||
return dataclasses.replace(
|
||||
plan,
|
||||
agent_provision=dataclasses.replace(
|
||||
plan.agent_provision,
|
||||
image=committed,
|
||||
),
|
||||
)
|
||||
container_mod.build_image(
|
||||
plan.image,
|
||||
_REPO_DIR,
|
||||
dockerfile=plan.dockerfile_path,
|
||||
)
|
||||
return plan
|
||||
|
||||
|
||||
def _create_networks(
|
||||
internal_network: str,
|
||||
egress_network: str,
|
||||
stack: ExitStack,
|
||||
) -> None:
|
||||
container_mod.create_network(internal_network, internal=True)
|
||||
stack.callback(container_mod.remove_network, internal_network)
|
||||
container_mod.create_network(egress_network)
|
||||
stack.callback(container_mod.remove_network, egress_network)
|
||||
|
||||
|
||||
def _start_sidecar_bundle(
|
||||
plan: MacosContainerBottlePlan,
|
||||
sidecar_name: str,
|
||||
internal_network: str,
|
||||
egress_network: str,
|
||||
) -> None:
|
||||
argv = _sidecar_run_argv(plan, sidecar_name, internal_network, egress_network)
|
||||
effective_env = {**dict(os.environ), **plan.agent_provision.provisioned_env}
|
||||
token_values = egress_resolve_token_values(
|
||||
plan.egress_plan.token_env_map, effective_env,
|
||||
)
|
||||
env = {**os.environ, **token_values}
|
||||
info(f"container run sidecar bundle {sidecar_name}")
|
||||
result = subprocess.run(
|
||||
argv, capture_output=True, text=True, env=env, check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
die(
|
||||
f"container run for sidecar bundle {sidecar_name} failed: "
|
||||
f"{(result.stderr or '').strip() or '<no stderr>'}"
|
||||
)
|
||||
|
||||
|
||||
def _start_agent(
|
||||
plan: MacosContainerBottlePlan,
|
||||
internal_network: str,
|
||||
sidecar_ip: str,
|
||||
) -> None:
|
||||
argv = _agent_run_argv(plan, internal_network, sidecar_ip)
|
||||
env = {
|
||||
**os.environ,
|
||||
**plan.forwarded_env,
|
||||
}
|
||||
info(f"container run agent {plan.container_name}")
|
||||
result = subprocess.run(
|
||||
argv, capture_output=True, text=True, env=env, check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
die(
|
||||
f"container run for agent {plan.container_name} failed: "
|
||||
f"{(result.stderr or '').strip() or '<no stderr>'}"
|
||||
)
|
||||
|
||||
|
||||
def _stamp_agent_urls(
|
||||
plan: MacosContainerBottlePlan,
|
||||
sidecar_ip: str,
|
||||
) -> MacosContainerBottlePlan:
|
||||
proxy_url = f"http://{sidecar_ip}:{EGRESS_PORT}"
|
||||
supervise_url = ""
|
||||
if plan.supervise_plan is not None:
|
||||
supervise_url = f"http://{sidecar_ip}:{SUPERVISE_PORT}/"
|
||||
git_gate_url = ""
|
||||
if plan.git_gate_plan.upstreams:
|
||||
git_gate_url = f"http://{sidecar_ip}:{_GIT_HTTP_PORT}"
|
||||
return dataclasses.replace(
|
||||
plan,
|
||||
agent_proxy_url=proxy_url,
|
||||
agent_git_gate_url=git_gate_url,
|
||||
agent_supervise_url=supervise_url,
|
||||
)
|
||||
|
||||
|
||||
def _stage_git_gate(plan: MacosContainerBottlePlan, sidecar_name: str) -> None:
|
||||
gp = plan.git_gate_plan
|
||||
if not gp.upstreams:
|
||||
return
|
||||
|
||||
container_mod.exec_container(
|
||||
sidecar_name,
|
||||
[
|
||||
"mkdir",
|
||||
"-p",
|
||||
str(Path(GIT_GATE_HOOK_IN_CONTAINER).parent),
|
||||
GIT_GATE_CREDS_DIR_IN_CONTAINER,
|
||||
"/git",
|
||||
str(Path(_GIT_GATE_READY_FILE).parent),
|
||||
],
|
||||
)
|
||||
|
||||
for host_path, container_path in _git_gate_files(plan):
|
||||
container_mod.copy_into_container(
|
||||
sidecar_name, host_path, container_path,
|
||||
)
|
||||
|
||||
container_mod.exec_container(
|
||||
sidecar_name,
|
||||
[
|
||||
"sh",
|
||||
"-c",
|
||||
"chmod 755 "
|
||||
f"{GIT_GATE_ENTRYPOINT_IN_CONTAINER} "
|
||||
f"{GIT_GATE_HOOK_IN_CONTAINER} "
|
||||
f"{GIT_GATE_ACCESS_HOOK_IN_CONTAINER} && "
|
||||
f"chmod 600 {GIT_GATE_CREDS_DIR_IN_CONTAINER}/* && "
|
||||
f"touch {_GIT_GATE_READY_FILE}",
|
||||
],
|
||||
)
|
||||
|
||||
|
||||
def _git_gate_files(
|
||||
plan: MacosContainerBottlePlan,
|
||||
) -> tuple[tuple[str, str], ...]:
|
||||
gp = plan.git_gate_plan
|
||||
files: list[tuple[str, str]] = [
|
||||
(str(gp.entrypoint_script), GIT_GATE_ENTRYPOINT_IN_CONTAINER),
|
||||
(str(gp.hook_script), GIT_GATE_HOOK_IN_CONTAINER),
|
||||
(str(gp.access_hook_script), GIT_GATE_ACCESS_HOOK_IN_CONTAINER),
|
||||
]
|
||||
for upstream in gp.upstreams:
|
||||
files.append((
|
||||
expand_tilde(upstream.identity_file),
|
||||
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{upstream.name}-key",
|
||||
))
|
||||
if upstream.known_hosts_file:
|
||||
files.append((
|
||||
str(upstream.known_hosts_file),
|
||||
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{upstream.name}-known_hosts",
|
||||
))
|
||||
return tuple(files)
|
||||
|
||||
|
||||
def _sidecar_run_argv(
|
||||
plan: MacosContainerBottlePlan,
|
||||
sidecar_name: str,
|
||||
internal_network: str,
|
||||
egress_network: str,
|
||||
) -> list[str]:
|
||||
argv = [
|
||||
"container", "run",
|
||||
"--name", sidecar_name,
|
||||
"--detach",
|
||||
"--rm",
|
||||
"--network", egress_network,
|
||||
"--network", internal_network,
|
||||
"--dns", _sidecar_dns(),
|
||||
"--env", f"BOT_BOTTLE_SIDECAR_DAEMONS={','.join(_sidecar_daemons(plan))}",
|
||||
]
|
||||
for entry in _sidecar_env_entries(plan):
|
||||
argv += ["--env", entry]
|
||||
for host_path, container_path, read_only in _sidecar_mounts(plan):
|
||||
argv += ["--mount", _mount_spec(host_path, container_path, read_only)]
|
||||
argv.append(SIDECAR_BUNDLE_IMAGE)
|
||||
return argv
|
||||
|
||||
|
||||
def _agent_run_argv(
|
||||
plan: MacosContainerBottlePlan,
|
||||
internal_network: str,
|
||||
sidecar_ip: str,
|
||||
) -> list[str]:
|
||||
argv = [
|
||||
"container", "run",
|
||||
"--name", plan.container_name,
|
||||
"--detach",
|
||||
"--network", internal_network,
|
||||
]
|
||||
for entry in _agent_env_entries(plan, sidecar_ip):
|
||||
argv += ["--env", entry]
|
||||
argv += [plan.image, "sleep", _AGENT_SLEEP_SECONDS]
|
||||
return argv
|
||||
|
||||
|
||||
def _sidecar_dns() -> str:
|
||||
return container_mod.dns_server()
|
||||
|
||||
|
||||
def _sidecar_daemons(plan: MacosContainerBottlePlan) -> tuple[str, ...]:
|
||||
daemons = ["egress"]
|
||||
if plan.git_gate_plan.upstreams:
|
||||
daemons += ["git-gate", "git-http"]
|
||||
if plan.supervise_plan is not None:
|
||||
daemons.append("supervise")
|
||||
return tuple(daemons)
|
||||
|
||||
|
||||
def _sidecar_env_entries(plan: MacosContainerBottlePlan) -> tuple[str, ...]:
|
||||
env: list[str] = []
|
||||
if plan.egress_plan.routes:
|
||||
env.extend(sorted(plan.egress_plan.token_env_map.keys()))
|
||||
if plan.git_gate_plan.upstreams:
|
||||
env.append(f"BOT_BOTTLE_GIT_GATE_READY_FILE={_GIT_GATE_READY_FILE}")
|
||||
if plan.supervise_plan is not None:
|
||||
env += [
|
||||
f"SUPERVISE_BOTTLE_SLUG={plan.slug}",
|
||||
f"SUPERVISE_QUEUE_DIR={QUEUE_DIR_IN_CONTAINER}",
|
||||
f"SUPERVISE_PORT={SUPERVISE_PORT}",
|
||||
]
|
||||
return tuple(env)
|
||||
|
||||
|
||||
def _sidecar_mounts(
|
||||
plan: MacosContainerBottlePlan,
|
||||
) -> tuple[tuple[str, str, bool], ...]:
|
||||
mounts: list[tuple[str, str, bool]] = []
|
||||
|
||||
ep = plan.egress_plan
|
||||
mounts.append((
|
||||
str(ep.mitmproxy_ca_host_path.parent),
|
||||
str(Path(EGRESS_CA_IN_CONTAINER).parent),
|
||||
False,
|
||||
))
|
||||
if ep.routes:
|
||||
mounts.append((
|
||||
str(ep.routes_path.parent),
|
||||
str(Path(EGRESS_ROUTES_IN_CONTAINER).parent),
|
||||
True,
|
||||
))
|
||||
|
||||
sp = plan.supervise_plan
|
||||
if sp is not None:
|
||||
mounts.append((str(sp.queue_dir), QUEUE_DIR_IN_CONTAINER, False))
|
||||
|
||||
return tuple(mounts)
|
||||
|
||||
def _mount_spec(host_path: str, container_path: str, read_only: bool) -> str:
|
||||
spec = f"type=bind,source={host_path},target={container_path}"
|
||||
if read_only:
|
||||
spec += ",readonly"
|
||||
return spec
|
||||
|
||||
|
||||
def _agent_env_entries(
|
||||
plan: MacosContainerBottlePlan,
|
||||
sidecar_ip: str,
|
||||
) -> tuple[str, ...]:
|
||||
proxy_url = f"http://{sidecar_ip}:{EGRESS_PORT}"
|
||||
no_proxy = _agent_no_proxy(plan, sidecar_ip)
|
||||
env = [
|
||||
f"HTTPS_PROXY={proxy_url}",
|
||||
f"HTTP_PROXY={proxy_url}",
|
||||
f"https_proxy={proxy_url}",
|
||||
f"http_proxy={proxy_url}",
|
||||
f"NO_PROXY={no_proxy}",
|
||||
f"no_proxy={no_proxy}",
|
||||
f"NODE_EXTRA_CA_CERTS={AGENT_CA_PATH}",
|
||||
f"SSL_CERT_FILE={AGENT_CA_BUNDLE}",
|
||||
f"REQUESTS_CA_BUNDLE={AGENT_CA_BUNDLE}",
|
||||
]
|
||||
if plan.agent_git_gate_url:
|
||||
env.append(f"GIT_GATE_URL={plan.agent_git_gate_url}")
|
||||
if plan.agent_supervise_url:
|
||||
env.append(f"MCP_SUPERVISE_URL={plan.agent_supervise_url}")
|
||||
for name, value in sorted(plan.agent_provision.guest_env.items()):
|
||||
env.append(f"{name}={value}")
|
||||
for name in sorted(plan.forwarded_env.keys()):
|
||||
env.append(name)
|
||||
return tuple(env)
|
||||
|
||||
|
||||
def _agent_no_proxy(plan: MacosContainerBottlePlan, sidecar_ip: str) -> str:
|
||||
hosts = ["localhost", "127.0.0.1", sidecar_ip]
|
||||
return ",".join(hosts)
|
||||
@@ -0,0 +1,70 @@
|
||||
"""Host-side raw-mode wrapper for `container exec --interactive --tty`.
|
||||
|
||||
Apple's `container exec --interactive --tty` does not set the host terminal to
|
||||
raw mode before starting its I/O relay. Without raw mode the kernel line
|
||||
discipline buffers modifier-key escape sequences (e.g. Shift+Enter in
|
||||
modifyOtherKeys mode produces \\x1b[13;2~) until a carriage-return arrives, so
|
||||
they never reach Claude Code inside the container.
|
||||
|
||||
This module sets the host terminal to raw mode, spawns the inner argv (the
|
||||
container exec command), and restores the original terminal attributes on
|
||||
exit. When stdin is not a TTY (piped invocations, CI) it falls through to a
|
||||
bare subprocess.run so callers do not need to special-case non-interactive
|
||||
contexts.
|
||||
|
||||
Usage (the `--` separator is the API contract — everything after it is the
|
||||
inner command):
|
||||
|
||||
python pty_forward.py -- container exec --interactive --tty <name> <cmd>
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import termios
|
||||
import tty
|
||||
|
||||
|
||||
def _inner_env() -> dict[str, str]:
|
||||
env = dict(os.environ)
|
||||
env.setdefault("TERM", "xterm-256color")
|
||||
return env
|
||||
|
||||
|
||||
def _run_inner(inner: list[str]) -> int:
|
||||
return subprocess.run(inner, check=False, env=_inner_env()).returncode
|
||||
|
||||
|
||||
def main(argv: list[str]) -> int:
|
||||
"""Entry point. ``argv`` shape: ``-- <inner-argv...>``."""
|
||||
if len(argv) < 2 or argv[0] != "--":
|
||||
sys.stderr.write(
|
||||
"usage: python pty_forward.py -- <container-exec-argv...>\n"
|
||||
)
|
||||
return 2
|
||||
inner = argv[1:]
|
||||
|
||||
try:
|
||||
fd = sys.stdin.fileno()
|
||||
except OSError:
|
||||
return _run_inner(inner)
|
||||
|
||||
if not os.isatty(fd):
|
||||
return _run_inner(inner)
|
||||
|
||||
try:
|
||||
old = termios.tcgetattr(fd)
|
||||
except termios.error:
|
||||
return _run_inner(inner)
|
||||
|
||||
try:
|
||||
tty.setraw(fd)
|
||||
return _run_inner(inner)
|
||||
finally:
|
||||
termios.tcsetattr(fd, termios.TCSADRAIN, old)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv[1:]))
|
||||
@@ -0,0 +1,47 @@
|
||||
"""Prepare step for the macOS Apple Container backend."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from ...agent_provider import AgentProvisionPlan
|
||||
from ...egress import EgressPlan
|
||||
from ...env import ResolvedEnv
|
||||
from ...git_gate import GitGatePlan
|
||||
from ...supervise import SupervisePlan
|
||||
from ...manifest import Manifest
|
||||
from .. import BottleSpec
|
||||
from . import util as container_mod
|
||||
from .bottle_plan import MacosContainerBottlePlan
|
||||
|
||||
|
||||
def preflight() -> None:
|
||||
container_mod.require_container()
|
||||
|
||||
|
||||
def build_guest_env(resolved_env: ResolvedEnv) -> dict[str, str]:
|
||||
return dict(resolved_env.literals)
|
||||
|
||||
|
||||
def resolve_plan(
|
||||
spec: BottleSpec,
|
||||
manifest: Manifest,
|
||||
slug: str,
|
||||
resolved_env: ResolvedEnv,
|
||||
agent_provision_plan: AgentProvisionPlan,
|
||||
egress_plan: EgressPlan,
|
||||
supervise_plan: SupervisePlan | None,
|
||||
git_gate_plan: GitGatePlan,
|
||||
stage_dir: Path,
|
||||
) -> MacosContainerBottlePlan:
|
||||
return MacosContainerBottlePlan(
|
||||
spec=spec,
|
||||
manifest=manifest,
|
||||
stage_dir=stage_dir,
|
||||
slug=slug,
|
||||
forwarded_env=dict(resolved_env.forwarded),
|
||||
git_gate_plan=git_gate_plan,
|
||||
egress_plan=egress_plan,
|
||||
supervise_plan=supervise_plan,
|
||||
agent_provision=agent_provision_plan,
|
||||
)
|
||||
@@ -0,0 +1,466 @@
|
||||
"""Host-side primitives for Apple's `container` CLI."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import ipaddress
|
||||
import platform
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
import time
|
||||
from typing import Iterable
|
||||
|
||||
from ...log import die, info
|
||||
|
||||
|
||||
_CONTAINER = "container"
|
||||
_DEFAULT_DNS = "1.1.1.1"
|
||||
|
||||
|
||||
def is_macos() -> bool:
|
||||
return platform.system() == "Darwin"
|
||||
|
||||
|
||||
def is_available() -> bool:
|
||||
return is_macos() and shutil.which(_CONTAINER) is not None
|
||||
|
||||
|
||||
def require_container() -> None:
|
||||
"""Fail with an install pointer if Apple Container is unavailable."""
|
||||
if not is_macos():
|
||||
info("BOT_BOTTLE_BACKEND=macos-container requires macOS.")
|
||||
die("macos-container backend is only supported on macOS")
|
||||
if shutil.which(_CONTAINER) is None:
|
||||
info("Apple Container is required but was not found on PATH.")
|
||||
info("Install: https://github.com/apple/container/releases")
|
||||
die("container not found")
|
||||
_require_container_service()
|
||||
|
||||
|
||||
def _require_container_service() -> None:
|
||||
result = subprocess.run(
|
||||
[_CONTAINER, "system", "status"],
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
info("Apple Container system service is not running.")
|
||||
info("Start it with: container system start")
|
||||
die("container system service not running")
|
||||
|
||||
|
||||
def dns_server() -> str:
|
||||
override = os.environ.get("BOT_BOTTLE_MACOS_CONTAINER_DNS", "").strip()
|
||||
if override:
|
||||
return override
|
||||
return _host_ipv4_dns() or _DEFAULT_DNS
|
||||
|
||||
|
||||
def build_image(ref: str, context: str, *, dockerfile: str = "") -> None:
|
||||
"""Build an OCI image with Apple's BuildKit-backed `container build`."""
|
||||
info(
|
||||
f"building image {ref} from {context} with Apple Container "
|
||||
"(layer cache keeps repeat builds fast)"
|
||||
)
|
||||
_ensure_builder_dns()
|
||||
args = [_CONTAINER, "build", "-t", ref, "--dns", dns_server()]
|
||||
if dockerfile:
|
||||
args.extend(["-f", dockerfile])
|
||||
args.append(context)
|
||||
subprocess.run(args, check=True)
|
||||
|
||||
|
||||
def commit_container(container_name: str, image_tag: str) -> None:
|
||||
"""Snapshot a running Apple Container as a local image.
|
||||
|
||||
`container export` requires a stopped container, but Apple Container
|
||||
removes containers when they stop, making stop-then-export impossible.
|
||||
Instead, exec into the running container as root and stream the root
|
||||
filesystem out via tar, then build a new image from that archive.
|
||||
The bottle continues running after commit.
|
||||
"""
|
||||
with tempfile.TemporaryDirectory(prefix="bot-bottle-container-commit.") as tmp:
|
||||
rootfs_tar = os.path.join(tmp, "rootfs.tar")
|
||||
dockerfile = os.path.join(tmp, "Dockerfile")
|
||||
with open(rootfs_tar, "wb") as tar_out:
|
||||
result = subprocess.run(
|
||||
[
|
||||
_CONTAINER, "exec",
|
||||
"--user", "root",
|
||||
container_name,
|
||||
"tar", "--create",
|
||||
"--exclude=./proc",
|
||||
"--exclude=./sys",
|
||||
"--exclude=./dev",
|
||||
"--exclude=./run",
|
||||
"--file=-",
|
||||
"--directory=/",
|
||||
".",
|
||||
],
|
||||
stdout=tar_out,
|
||||
stderr=subprocess.PIPE,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
die(
|
||||
f"container exec tar {container_name!r} failed: "
|
||||
f"{(result.stderr or b'').decode().strip() or '<no stderr>'}"
|
||||
)
|
||||
with open(dockerfile, "w", encoding="utf-8") as f:
|
||||
f.write(
|
||||
"FROM scratch\n"
|
||||
"ADD rootfs.tar /\n"
|
||||
"USER node\n"
|
||||
"WORKDIR /home/node\n"
|
||||
)
|
||||
build_image(image_tag, tmp, dockerfile=dockerfile)
|
||||
info(f"committed {container_name!r} → {image_tag!r}")
|
||||
|
||||
|
||||
def _ensure_builder_dns() -> None:
|
||||
dns = dns_server()
|
||||
status = _builder_status()
|
||||
override = os.environ.get("BOT_BOTTLE_MACOS_CONTAINER_DNS", "").strip()
|
||||
if _builder_running(status) and _builder_resolves_build_hosts():
|
||||
if override and not _builder_has_dns(status, dns):
|
||||
_restart_builder_with_dns(dns)
|
||||
return
|
||||
_restart_builder_with_dns(dns)
|
||||
|
||||
|
||||
def _restart_builder_with_dns(dns: str) -> None:
|
||||
subprocess.run(
|
||||
[_CONTAINER, "builder", "stop"],
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
)
|
||||
subprocess.run(
|
||||
[_CONTAINER, "builder", "start", "--dns", dns],
|
||||
check=True,
|
||||
)
|
||||
|
||||
|
||||
def _host_ipv4_dns() -> str:
|
||||
if not is_macos():
|
||||
return ""
|
||||
result = subprocess.run(
|
||||
["scutil", "--dns"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return ""
|
||||
blocks: list[list[str]] = []
|
||||
current: list[str] = []
|
||||
for line in result.stdout.splitlines():
|
||||
if line.startswith("resolver #") and current:
|
||||
blocks.append(current)
|
||||
current = []
|
||||
current.append(line)
|
||||
if current:
|
||||
blocks.append(current)
|
||||
for direct_only in (True, False):
|
||||
for block in blocks:
|
||||
text = "\n".join(block)
|
||||
if direct_only and "Directly Reachable Address" not in text:
|
||||
continue
|
||||
for line in block:
|
||||
if "nameserver[" not in line or ":" not in line:
|
||||
continue
|
||||
candidate = line.split(":", 1)[1].strip()
|
||||
if _usable_ipv4(candidate):
|
||||
return candidate
|
||||
return ""
|
||||
|
||||
|
||||
def _usable_ipv4(value: str) -> bool:
|
||||
try:
|
||||
address = ipaddress.ip_address(value)
|
||||
except ValueError:
|
||||
return False
|
||||
return (
|
||||
address.version == 4
|
||||
and not address.is_loopback
|
||||
and not address.is_link_local
|
||||
and not address.is_multicast
|
||||
and not address.is_unspecified
|
||||
)
|
||||
|
||||
|
||||
def _builder_status() -> list[dict[str, object]]:
|
||||
result = subprocess.run(
|
||||
[_CONTAINER, "builder", "status", "--format", "json"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return []
|
||||
try:
|
||||
data = json.loads(result.stdout or "[]")
|
||||
except json.JSONDecodeError:
|
||||
return []
|
||||
if isinstance(data, list):
|
||||
return [entry for entry in data if isinstance(entry, dict)]
|
||||
if isinstance(data, dict):
|
||||
return [data]
|
||||
return []
|
||||
|
||||
|
||||
def _builder_running(status: list[dict[str, object]]) -> bool:
|
||||
for entry in status:
|
||||
entry_status = entry.get("status")
|
||||
if isinstance(entry_status, dict) and entry_status.get("state") == "running":
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _builder_dns_nameservers(status: list[dict[str, object]]) -> list[str]:
|
||||
out: list[str] = []
|
||||
for entry in status:
|
||||
config = entry.get("configuration")
|
||||
config_dns = config.get("dns") if isinstance(config, dict) else None
|
||||
nameservers = (
|
||||
config_dns.get("nameservers")
|
||||
if isinstance(config_dns, dict)
|
||||
else None
|
||||
)
|
||||
if not isinstance(nameservers, list):
|
||||
continue
|
||||
out.extend(name for name in nameservers if isinstance(name, str))
|
||||
return out
|
||||
|
||||
|
||||
def _builder_has_dns(status: list[dict[str, object]], dns: str) -> bool:
|
||||
return dns in _builder_dns_nameservers(status)
|
||||
|
||||
|
||||
def _builder_resolves_build_hosts() -> bool:
|
||||
result = subprocess.run(
|
||||
[_CONTAINER, "exec", "buildkit", "getent", "hosts", "deb.debian.org"],
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
)
|
||||
return result.returncode == 0
|
||||
|
||||
|
||||
def image_exists(ref: str) -> bool:
|
||||
return _silent_run([_CONTAINER, "image", "inspect", ref]) == 0
|
||||
|
||||
|
||||
def container_exists(name: str) -> bool:
|
||||
result = subprocess.run(
|
||||
[_CONTAINER, "list", "--all", "--quiet"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return False
|
||||
return name in {line.strip() for line in result.stdout.splitlines()}
|
||||
|
||||
|
||||
def container_is_running(name: str) -> bool:
|
||||
"""Return True if the named container is currently running.
|
||||
|
||||
`container list` without `--all` lists only running containers."""
|
||||
result = subprocess.run(
|
||||
[_CONTAINER, "list", "--quiet"],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return False
|
||||
return name in {line.strip() for line in result.stdout.splitlines()}
|
||||
|
||||
|
||||
def stop_container(name: str) -> None:
|
||||
"""Stop the named container without deleting it."""
|
||||
result = subprocess.run(
|
||||
[_CONTAINER, "stop", name],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
die(
|
||||
f"container stop {name!r} failed: "
|
||||
f"{(result.stderr or '').strip() or '<no stderr>'}"
|
||||
)
|
||||
|
||||
|
||||
def force_remove_container(name: str) -> None:
|
||||
if container_exists(name):
|
||||
subprocess.run(
|
||||
[_CONTAINER, "delete", "--force", name],
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
)
|
||||
|
||||
|
||||
def copy_into_container(name: str, host_path: str, container_path: str) -> None:
|
||||
cmd = [_CONTAINER, "cp", host_path, f"{name}:{container_path}"]
|
||||
result = _run_container_op(cmd)
|
||||
if result.returncode != 0:
|
||||
die(
|
||||
f"container cp into {name}:{container_path} failed: "
|
||||
f"{(result.stderr or '').strip() or '<no stderr>'}"
|
||||
)
|
||||
|
||||
|
||||
def exec_container(name: str, argv: list[str]) -> None:
|
||||
result = _run_container_op([_CONTAINER, "exec", name, *argv])
|
||||
if result.returncode != 0:
|
||||
die(
|
||||
f"container exec in {name} failed: "
|
||||
f"{(result.stderr or '').strip() or '<no stderr>'}"
|
||||
)
|
||||
|
||||
|
||||
def _run_container_op(cmd: list[str]) -> subprocess.CompletedProcess[str]:
|
||||
result = subprocess.run(
|
||||
cmd,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
for _ in range(19):
|
||||
if result.returncode == 0:
|
||||
return result
|
||||
time.sleep(0.1)
|
||||
result = subprocess.run(
|
||||
cmd,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
def create_network(name: str, *, internal: bool = False) -> None:
|
||||
args = [
|
||||
_CONTAINER, "network", "create",
|
||||
"--label", "bot-bottle.backend=macos-container",
|
||||
]
|
||||
if internal:
|
||||
args.append("--internal")
|
||||
args.append(name)
|
||||
result = subprocess.run(
|
||||
args, capture_output=True, text=True, check=False,
|
||||
)
|
||||
if result.returncode == 0:
|
||||
return
|
||||
if "already exists" in (result.stderr or "").lower():
|
||||
return
|
||||
die(
|
||||
f"container network create {name} failed: "
|
||||
f"{(result.stderr or '').strip() or '<no stderr>'}"
|
||||
)
|
||||
|
||||
|
||||
def remove_network(name: str) -> None:
|
||||
result = subprocess.run(
|
||||
[_CONTAINER, "network", "delete", name],
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return
|
||||
|
||||
|
||||
def inspect_container(name: str) -> dict[str, object]:
|
||||
result = subprocess.run(
|
||||
[_CONTAINER, "inspect", name],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
die(
|
||||
f"container inspect {name} failed: "
|
||||
f"{(result.stderr or '').strip() or '<no stderr>'}"
|
||||
)
|
||||
try:
|
||||
data = json.loads(result.stdout or "[]")
|
||||
except json.JSONDecodeError as exc:
|
||||
die(f"container inspect {name} returned malformed JSON: {exc}")
|
||||
if isinstance(data, list) and data and isinstance(data[0], dict):
|
||||
return data[0]
|
||||
if isinstance(data, dict):
|
||||
return data
|
||||
die(f"container inspect {name} returned an unexpected shape")
|
||||
raise AssertionError("unreachable")
|
||||
|
||||
|
||||
def container_ipv4_on_network(name: str, network: str) -> str:
|
||||
data = inspect_container(name)
|
||||
status = data.get("status")
|
||||
networks = status.get("networks") if isinstance(status, dict) else None
|
||||
if not isinstance(networks, list):
|
||||
die(f"container inspect {name} did not include status.networks")
|
||||
for entry in networks:
|
||||
if not isinstance(entry, dict):
|
||||
continue
|
||||
if entry.get("network") != network:
|
||||
continue
|
||||
raw = entry.get("ipv4Address")
|
||||
if not isinstance(raw, str) or not raw:
|
||||
die(f"container {name} has no IPv4 address on {network}")
|
||||
return raw.split("/", 1)[0]
|
||||
die(f"container {name} is not attached to network {network}")
|
||||
raise AssertionError("unreachable")
|
||||
|
||||
|
||||
def image_id(ref: str) -> str:
|
||||
"""Return the image digest/ID from `container image inspect`.
|
||||
|
||||
The command returns JSON on current Apple Container releases. Keep
|
||||
parsing narrow and fatal so callers do not cache on an empty key.
|
||||
"""
|
||||
import json
|
||||
|
||||
result = subprocess.run(
|
||||
[_CONTAINER, "image", "inspect", ref],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
die(
|
||||
f"container image inspect for {ref!r} failed: "
|
||||
f"{(result.stderr or '').strip() or '<no stderr>'}"
|
||||
)
|
||||
try:
|
||||
data = json.loads(result.stdout or "{}")
|
||||
except json.JSONDecodeError as exc:
|
||||
die(f"container image inspect for {ref!r} returned malformed JSON: {exc}")
|
||||
if isinstance(data, list) and data:
|
||||
data = data[0]
|
||||
if isinstance(data, dict):
|
||||
value = data.get("id") or data.get("digest") or data.get("ID")
|
||||
if value:
|
||||
return str(value)
|
||||
die(f"container image inspect for {ref!r} did not include an image id")
|
||||
raise AssertionError("unreachable")
|
||||
|
||||
|
||||
def save(ref: str, output: str) -> None:
|
||||
subprocess.run([_CONTAINER, "image", "save", ref, "-o", output], check=True)
|
||||
|
||||
|
||||
def _silent_run(cmd: Iterable[str]) -> int:
|
||||
return subprocess.run(
|
||||
list(cmd),
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
).returncode
|
||||
@@ -0,0 +1,131 @@
|
||||
"""Shared helpers used by both backends' resolve_plan steps.
|
||||
|
||||
Each helper owns one well-defined step of the per-bottle plan
|
||||
resolution so docker and smolmachines don't repeat the same logic.
|
||||
Backend-specific steps (container names, env-file, per-bottle
|
||||
Dockerfile overrides, subnet allocation) stay in the backend's own
|
||||
resolve_plan.py.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from dataclasses import replace
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
from ..agent_provider import AgentProvisionPlan
|
||||
from ..bottle_state import (
|
||||
BottleMetadata,
|
||||
agent_state_dir,
|
||||
bottle_identity,
|
||||
egress_state_dir,
|
||||
git_gate_state_dir,
|
||||
supervise_state_dir,
|
||||
write_metadata,
|
||||
)
|
||||
from ..egress import Egress, EgressPlan
|
||||
from ..git_gate import GitGate, GitGatePlan
|
||||
from ..manifest import Manifest, ManifestBottle
|
||||
from ..supervise import Supervise, SupervisePlan
|
||||
from . import BottleSpec
|
||||
|
||||
|
||||
def mint_slug(spec: BottleSpec) -> str:
|
||||
"""Return the bottle identity: the recorded identity for a resume,
|
||||
or a freshly minted one for a new start.
|
||||
|
||||
When a label is provided it becomes the full slug (no random suffix),
|
||||
so two launches with the same label collide by design. When no label
|
||||
is given the identity is minted with a random suffix to avoid
|
||||
collisions between anonymous launches of the same agent."""
|
||||
if spec.identity:
|
||||
return spec.identity
|
||||
if spec.label:
|
||||
from .docker import util as docker_mod
|
||||
return docker_mod.slugify(spec.label)
|
||||
return bottle_identity(spec.agent_name)
|
||||
|
||||
|
||||
def write_launch_metadata(
|
||||
slug: str, spec: BottleSpec, *, compose_project: str, backend: str,
|
||||
) -> None:
|
||||
"""Persist launch metadata so `cli.py resume <identity>` can
|
||||
reconstruct the spec. Idempotent — re-writes on resume with a
|
||||
refreshed started_at."""
|
||||
write_metadata(BottleMetadata(
|
||||
identity=slug,
|
||||
agent_name=spec.agent_name,
|
||||
cwd=spec.user_cwd if spec.copy_cwd else "",
|
||||
copy_cwd=spec.copy_cwd,
|
||||
started_at=datetime.now(timezone.utc).isoformat(),
|
||||
compose_project=compose_project,
|
||||
backend=backend,
|
||||
label=spec.label,
|
||||
color=spec.color,
|
||||
))
|
||||
|
||||
|
||||
def prepare_agent_state_dir(slug: str, manifest: Manifest) -> tuple[Path, Path]:
|
||||
"""Create the agent state subdir, write the prompt file.
|
||||
Returns (agent_dir, prompt_file)."""
|
||||
agent = manifest.agent
|
||||
agent_dir = agent_state_dir(slug)
|
||||
agent_dir.mkdir(parents=True, exist_ok=True)
|
||||
prompt_file = agent_dir / "prompt.txt"
|
||||
prompt_file.write_text(agent.prompt or "")
|
||||
prompt_file.chmod(0o600)
|
||||
return agent_dir, prompt_file
|
||||
|
||||
|
||||
def prepare_git_gate(bottle: ManifestBottle, slug: str) -> GitGatePlan:
|
||||
git_gate_dir = git_gate_state_dir(slug)
|
||||
git_gate_dir.mkdir(parents=True, exist_ok=True)
|
||||
return GitGate().prepare(bottle, slug, git_gate_dir)
|
||||
|
||||
|
||||
def prepare_egress(
|
||||
bottle: ManifestBottle, slug: str, provision: AgentProvisionPlan,
|
||||
) -> EgressPlan:
|
||||
egress_dir = egress_state_dir(slug)
|
||||
egress_dir.mkdir(parents=True, exist_ok=True)
|
||||
return Egress().prepare(bottle, slug, egress_dir, provision.egress_routes)
|
||||
|
||||
|
||||
def prepare_supervise(bottle: ManifestBottle, slug: str) -> SupervisePlan | None:
|
||||
"""Prepare the supervise sidecar state dir. Returns None when
|
||||
bottle.supervise is falsy."""
|
||||
if not bottle.supervise:
|
||||
return None
|
||||
supervise_dir = supervise_state_dir(slug)
|
||||
supervise_dir.mkdir(parents=True, exist_ok=True)
|
||||
return Supervise().prepare(slug, supervise_dir)
|
||||
|
||||
|
||||
def merge_provision_env_vars(provision: AgentProvisionPlan) -> AgentProvisionPlan:
|
||||
"""Fold provision.env_vars into guest_env (setdefault semantics)
|
||||
and return a new plan with the merged guest_env."""
|
||||
merged = dict(provision.guest_env)
|
||||
for key, val in provision.env_vars.items():
|
||||
merged.setdefault(key, val)
|
||||
return replace(provision, guest_env=merged)
|
||||
|
||||
|
||||
def resolve_manifest_dockerfile(path_value: str, spec: BottleSpec) -> str:
|
||||
"""Resolve a manifest-supplied dockerfile path relative to user_cwd."""
|
||||
path = Path(os.path.expanduser(path_value))
|
||||
if not path.is_absolute():
|
||||
path = Path(spec.user_cwd) / path
|
||||
return str(path)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"merge_provision_env_vars",
|
||||
"mint_slug",
|
||||
"prepare_agent_state_dir",
|
||||
"prepare_egress",
|
||||
"prepare_git_gate",
|
||||
"prepare_supervise",
|
||||
"resolve_manifest_dockerfile",
|
||||
"write_launch_metadata",
|
||||
]
|
||||
@@ -13,16 +13,21 @@ from contextlib import contextmanager
|
||||
from pathlib import Path
|
||||
from typing import Generator, Sequence
|
||||
|
||||
from .. import ActiveAgent, Bottle, BottleBackend, BottleSpec
|
||||
from ...agent_provider import AgentProvisionPlan
|
||||
from ...egress import EgressPlan
|
||||
from ...env import ResolvedEnv
|
||||
from ...git_gate import GitGatePlan
|
||||
from ...supervise import SupervisePlan
|
||||
from ...manifest import Manifest
|
||||
from .. import ActiveAgent, BottleBackend, BottleSpec
|
||||
from . import cleanup as _cleanup
|
||||
from . import enumerate as _enumerate
|
||||
from . import launch as _launch
|
||||
from . import prepare as _prepare
|
||||
from . import resolve_plan as _resolve_plan
|
||||
from . import smolvm as _smolvm
|
||||
from .bottle import SmolmachinesBottle
|
||||
from .bottle_cleanup_plan import SmolmachinesBottleCleanupPlan
|
||||
from .bottle_plan import SmolmachinesBottlePlan
|
||||
from .provision import workspace as _workspace
|
||||
|
||||
|
||||
class SmolmachinesBottleBackend(
|
||||
@@ -41,10 +46,36 @@ class SmolmachinesBottleBackend(
|
||||
runtime check happens at `prepare`."""
|
||||
return _smolvm.is_available()
|
||||
|
||||
def _preflight(self) -> None:
|
||||
_resolve_plan.preflight()
|
||||
|
||||
def _build_guest_env(self, resolved_env: ResolvedEnv) -> dict[str, str]:
|
||||
return _resolve_plan.build_guest_env(resolved_env)
|
||||
|
||||
def _resolve_plan(
|
||||
self, spec: BottleSpec, *, stage_dir: Path
|
||||
self,
|
||||
spec: BottleSpec,
|
||||
*,
|
||||
manifest: Manifest,
|
||||
slug: str,
|
||||
resolved_env: ResolvedEnv,
|
||||
agent_provision_plan: AgentProvisionPlan,
|
||||
egress_plan: EgressPlan,
|
||||
git_gate_plan: GitGatePlan,
|
||||
supervise_plan: SupervisePlan | None,
|
||||
stage_dir: Path,
|
||||
) -> SmolmachinesBottlePlan:
|
||||
return _prepare.resolve_plan(spec, stage_dir=stage_dir)
|
||||
return _resolve_plan.resolve_plan(
|
||||
spec,
|
||||
manifest=manifest,
|
||||
slug=slug,
|
||||
resolved_env=resolved_env,
|
||||
agent_provision_plan=agent_provision_plan,
|
||||
egress_plan=egress_plan,
|
||||
supervise_plan=supervise_plan,
|
||||
git_gate_plan=git_gate_plan,
|
||||
stage_dir=stage_dir,
|
||||
)
|
||||
|
||||
@contextmanager
|
||||
def launch(
|
||||
@@ -53,11 +84,6 @@ class SmolmachinesBottleBackend(
|
||||
with _launch.launch(plan, provision=self.provision) as bottle:
|
||||
yield bottle
|
||||
|
||||
def provision_workspace(
|
||||
self, plan: SmolmachinesBottlePlan, bottle: Bottle
|
||||
) -> None:
|
||||
_workspace.provision_workspace(plan, bottle)
|
||||
|
||||
def supervise_mcp_url(self, plan: SmolmachinesBottlePlan) -> str:
|
||||
"""The smolmachines guest reaches the supervise sidecar via a
|
||||
host-published random port the launch step pinned earlier
|
||||
|
||||
@@ -20,10 +20,12 @@ from __future__ import annotations
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
import shlex
|
||||
from typing import Mapping, cast
|
||||
|
||||
from ...agent_provider import PromptMode, prompt_args
|
||||
from .. import Bottle, ExecResult
|
||||
from ..terminal import exec_shell_script
|
||||
from . import pty_resize as _pty_resize
|
||||
from . import smolvm as _smolvm
|
||||
|
||||
@@ -68,6 +70,10 @@ class SmolmachinesBottle(Bottle):
|
||||
guest_env: Mapping[str, str] | None = None,
|
||||
agent_command: str = "claude",
|
||||
agent_prompt_mode: PromptMode = "append_file",
|
||||
agent_provider_template: str = "claude",
|
||||
terminal_title: str = "",
|
||||
terminal_color: str = "",
|
||||
agent_workdir: str = "/home/node",
|
||||
) -> None:
|
||||
self.name = machine_name
|
||||
# In-VM path to the agent's prompt file. None when the
|
||||
@@ -81,9 +87,10 @@ class SmolmachinesBottle(Bottle):
|
||||
self._guest_env = dict(guest_env or {})
|
||||
self._agent_prompt_mode = agent_prompt_mode
|
||||
self.agent_command = agent_command
|
||||
self.agent_provider_template = (
|
||||
"codex" if agent_command == "codex" else "claude"
|
||||
)
|
||||
self.terminal_title = terminal_title
|
||||
self.terminal_color = terminal_color
|
||||
self.agent_provider_template = agent_provider_template
|
||||
self.agent_workdir = agent_workdir
|
||||
|
||||
def agent_argv(
|
||||
self, argv: list[str], *, tty: bool = True,
|
||||
@@ -91,8 +98,14 @@ class SmolmachinesBottle(Bottle):
|
||||
flags = ["smolvm", "machine", "exec", "--name", self.name]
|
||||
if tty:
|
||||
flags += ["-i", "-t"]
|
||||
agent_tail = ["env", *_env_assignments_for("node", self._guest_env),
|
||||
self.agent_command]
|
||||
agent_tail = ["env", *_env_assignments_for("node", self._guest_env)]
|
||||
if self.agent_workdir and self.agent_workdir != _HOME_FOR["node"]:
|
||||
agent_tail += [
|
||||
"sh", "-lc",
|
||||
f"cd {shlex.quote(self.agent_workdir)} && exec \"$@\"",
|
||||
"bot-bottle-agent",
|
||||
]
|
||||
agent_tail.append(self.agent_command)
|
||||
provider_prompt_args = prompt_args(
|
||||
cast(PromptMode, self._agent_prompt_mode), self.prompt_path, argv=argv,
|
||||
)
|
||||
@@ -128,9 +141,16 @@ class SmolmachinesBottle(Bottle):
|
||||
UID switches via `runuser -u node --` (not `-l`) so we
|
||||
avoid login-shell wiring. HOME / USER come from `smolvm
|
||||
-e` instead, which sets them on the process env."""
|
||||
return subprocess.run(
|
||||
self.agent_argv(argv, tty=tty), check=False,
|
||||
).returncode
|
||||
agent_argv = self.agent_argv(argv, tty=tty)
|
||||
script = exec_shell_script(agent_argv, self.terminal_title, self.terminal_color) if tty else None
|
||||
if script is None:
|
||||
return subprocess.run(agent_argv, check=False).returncode
|
||||
# Use sh -c (not -lc) so the script inherits PATH from the calling
|
||||
# process. sh -l sources login-shell init files (e.g. /etc/profile)
|
||||
# which may NOT include smolvm's location when it was installed via
|
||||
# homebrew. The calling process (./cli.py) already has smolvm on PATH
|
||||
# (provision steps succeed), so -c is sufficient.
|
||||
return subprocess.run(["sh", "-c", script], check=False).returncode
|
||||
|
||||
# smolvm/libkrun can SIGKILL an otherwise-normal exec during
|
||||
# early-VM provisioning. Retry once after a short settle so
|
||||
|
||||
@@ -29,27 +29,6 @@ class SmolmachinesBottlePlan(BottlePlan):
|
||||
bundle_subnet: str
|
||||
bundle_gateway: str
|
||||
bundle_ip: str
|
||||
# smolvm machine name + agent image source. machine_create
|
||||
# boots from a packed `.smolmachine` artifact (pre-baked at
|
||||
# prepare time via `smolvm pack create`); using `--from`
|
||||
# instead of `--image` avoids the registry-pull race we hit
|
||||
# when machine_start tried to fetch on-demand and the libkrun
|
||||
# agent's network attempt got refused by macOS.
|
||||
#
|
||||
# Chunk 2d ships with a public placeholder image (alpine)
|
||||
# since bot-bottle-claude:latest lives in the operator's local
|
||||
# docker daemon and smolvm's crane backend can't read from
|
||||
# there; chunk 4 resolves the agent-image-conversion gap
|
||||
# (push to a registry first, or smolvm grows a docker-daemon
|
||||
# transport).
|
||||
machine_name: str
|
||||
# Agent image ref (docker tag). `launch` runs the
|
||||
# build → save → registry push → smolvm pack pipeline against
|
||||
# this and feeds the resulting `.smolmachine` artifact to
|
||||
# `machine_create --from`. The pipeline runs at launch time
|
||||
# (not prepare time) so the docker build output doesn't garble
|
||||
# the dashboard's preflight modal.
|
||||
agent_image_ref: str
|
||||
# In-guest env vars (HTTPS_PROXY etc) — IP-literal URLs since
|
||||
# the guest has no DNS resolver inside the TSI allowlist.
|
||||
# Passed to `smolvm machine create` as `-e K=V` flags.
|
||||
@@ -57,11 +36,6 @@ class SmolmachinesBottlePlan(BottlePlan):
|
||||
# `--smolfile` is mutually exclusive with `--from`, and
|
||||
# `--from` is the path that avoids the registry-pull race).
|
||||
guest_env: dict[str, str]
|
||||
# Path to the agent's prompt file on the host. Always written
|
||||
# (mode 0o600) so the in-VM path always exists; the file is
|
||||
# empty when the agent has no prompt — claude-code reads it
|
||||
# via --append-system-prompt-file only when non-empty.
|
||||
prompt_file: Path
|
||||
# Inner Plans for the sidecar bundle daemons. The same shape the
|
||||
# docker backend uses — same `.prepare()` calls produced
|
||||
# them — but our launch step doesn't populate the
|
||||
@@ -82,6 +56,34 @@ class SmolmachinesBottlePlan(BottlePlan):
|
||||
agent_git_gate_host: str = ""
|
||||
agent_supervise_url: str = ""
|
||||
|
||||
@property
|
||||
def machine_name(self) -> str:
|
||||
"""smolvm machine name. `machine_create` boots from a packed
|
||||
`.smolmachine` artifact (pre-baked at prepare time via
|
||||
`smolvm pack create`); using `--from` instead of `--image`
|
||||
avoids the registry-pull race we hit when machine_start tried
|
||||
to fetch on-demand and the libkrun agent's network attempt
|
||||
got refused by macOS."""
|
||||
return self.agent_provision.instance_name
|
||||
|
||||
@property
|
||||
def agent_image(self) -> str:
|
||||
"""Agent image ref (docker tag). `launch` runs the
|
||||
build → save → registry push → smolvm pack pipeline against
|
||||
this and feeds the resulting `.smolmachine` artifact to
|
||||
`machine_create --from`. The pipeline runs at launch time
|
||||
(not prepare time) so the docker build output doesn't garble
|
||||
the dashboard's preflight modal."""
|
||||
return self.agent_provision.image
|
||||
|
||||
@property
|
||||
def prompt_file(self) -> Path:
|
||||
"""Path to the agent's prompt file on the host. Always written
|
||||
(mode 0o600) so the in-VM path always exists; the file is
|
||||
empty when the agent has no prompt — claude-code reads it
|
||||
via --append-system-prompt-file only when non-empty."""
|
||||
return self.agent_provision.prompt_file
|
||||
|
||||
@property
|
||||
def git_gate_insteadof_host(self) -> str:
|
||||
return self.agent_git_gate_host
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
"""Egress apply for the smolmachines backend.
|
||||
|
||||
The smolmachines sidecar bundle runs as a host-side Docker container,
|
||||
so egress signalling is identical to the docker backend.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from ..docker.egress_apply import ( # noqa: F401
|
||||
DockerEgressApplicator,
|
||||
EgressApplyError,
|
||||
applicator,
|
||||
fetch_current_routes,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"DockerEgressApplicator",
|
||||
"EgressApplyError",
|
||||
"applicator",
|
||||
"fetch_current_routes",
|
||||
]
|
||||
@@ -23,7 +23,7 @@ import json
|
||||
import subprocess
|
||||
|
||||
from .. import ActiveAgent
|
||||
from ..docker.bottle_state import read_metadata
|
||||
from ...bottle_state import read_metadata
|
||||
from . import sidecar_bundle as _bundle
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,145 @@
|
||||
"""SmolmachinesFreezer — snapshot a smolmachines bottle.
|
||||
|
||||
`smolvm pack create --from-vm` requires the VM to be stopped, and smolvm
|
||||
removes VMs when stopped (same issue as Apple Container). Instead, exec
|
||||
into the running VM as root to write a gzip-compressed tar of the root
|
||||
filesystem to /var/tmp, then copy it to the host with `smolvm machine cp`,
|
||||
build a Docker image from the archive, convert it to a smolmachine artifact
|
||||
via the existing registry pipeline, and record the sidecar path. The VM
|
||||
stays running throughout."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
from .. import ActiveAgent
|
||||
from ..freeze import Freezer
|
||||
from ..docker import util as docker_mod
|
||||
from .local_registry import crane_push_tarball, ephemeral_registry
|
||||
from .smolvm import machine_cp, machine_exec, pack_create
|
||||
from ...bottle_state import bottle_state_dir
|
||||
from ...log import die, info
|
||||
|
||||
|
||||
# Temp file written inside the VM during commit. Lives in /var/tmp
|
||||
# (on-disk, unlike tmpfs /tmp) to survive for machine_cp.
|
||||
_VM_COMMIT_TAR = "/var/tmp/.bot-bottle-commit.tar.gz"
|
||||
|
||||
|
||||
class SmolmachinesFreezer(Freezer):
|
||||
"""Freezes a smolmachines bottle via exec-tar + Docker image + smolmachine pack.
|
||||
|
||||
The VM is NOT stopped. We exec into the running VM to write a compressed
|
||||
tar of the root filesystem to /var/tmp, copy it to the host with
|
||||
machine_cp, build a Docker image (Docker's ADD decompresses .tar.gz
|
||||
automatically), then run the same image→registry→pack_create pipeline
|
||||
that _ensure_smolmachine uses for fresh builds."""
|
||||
|
||||
backend_name = "smolmachines"
|
||||
|
||||
def _freeze(self, agent: ActiveAgent) -> str:
|
||||
machine = f"bot-bottle-{agent.slug}"
|
||||
image_ref = f"bot-bottle-committed-{agent.slug}:latest"
|
||||
output_dir = bottle_state_dir(agent.slug)
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
binary = output_dir / "committed-smolmachine"
|
||||
sidecar = output_dir / "committed-smolmachine.smolmachine"
|
||||
_snapshot_running_vm(machine, image_ref, binary)
|
||||
return str(sidecar)
|
||||
|
||||
def _export_hint(self, slug: str, image_ref: str) -> None:
|
||||
info(f"to export for migration: cp {image_ref} {slug}.smolmachine")
|
||||
|
||||
|
||||
def _snapshot_running_vm(machine: str, image_ref: str, binary: Path) -> None:
|
||||
"""Exec-tar the running VM, build a Docker image, and pack to a smolmachine.
|
||||
|
||||
binary: destination for the launcher (sibling .smolmachine is the artifact
|
||||
that machine_create --from consumes, same convention as pack_create).
|
||||
"""
|
||||
with tempfile.TemporaryDirectory(prefix="bot-bottle-vm-commit.") as tmp:
|
||||
tmp_path = Path(tmp)
|
||||
# Use .tar.gz — Docker ADD decompresses automatically and the
|
||||
# compressed archive fits in the VM's /var/tmp more easily.
|
||||
rootfs_tar_gz = tmp_path / "rootfs.tar.gz"
|
||||
dockerfile = tmp_path / "Dockerfile"
|
||||
|
||||
_exec_tar_to_file(machine, rootfs_tar_gz)
|
||||
|
||||
dockerfile.write_text(
|
||||
"FROM scratch\n"
|
||||
"ADD rootfs.tar.gz /\n"
|
||||
"USER node\n"
|
||||
"WORKDIR /home/node\n"
|
||||
)
|
||||
docker_mod.build_image(image_ref, str(tmp_path), dockerfile=str(dockerfile))
|
||||
|
||||
image_tarball = binary.parent / "committed.image.tar"
|
||||
docker_mod.save(image_ref, str(image_tarball))
|
||||
try:
|
||||
with ephemeral_registry() as handle:
|
||||
digest = docker_mod.image_id(image_ref).split(":", 1)[-1][:16]
|
||||
push_ref = f"{handle.push_endpoint}/bot-bottle-committed:{digest}"
|
||||
pack_ref = f"{handle.pull_endpoint}/bot-bottle-committed:{digest}"
|
||||
crane_push_tarball(handle, str(image_tarball), push_ref)
|
||||
pack_create(pack_ref, binary)
|
||||
finally:
|
||||
image_tarball.unlink(missing_ok=True)
|
||||
|
||||
|
||||
def _exec_tar_to_file(machine: str, dest: Path) -> None:
|
||||
"""Snapshot the running VM's root filesystem to dest (.tar.gz).
|
||||
|
||||
Writes a gzip-compressed tar to _VM_COMMIT_TAR inside the VM via
|
||||
machine_exec (same mechanism as provisioning), then copies it to the
|
||||
host with machine_cp. This avoids binary-stdout piping through the
|
||||
smolvm exec channel, which does not reliably handle large binary output.
|
||||
|
||||
A connectivity probe (machine_exec true) runs first so a concurrent-exec
|
||||
limitation (smolvm may reject a second exec while -i -t is active) is
|
||||
reported clearly rather than as a silent failure."""
|
||||
# Connectivity probe — if smolvm rejects concurrent exec while an
|
||||
# interactive session is running, fail clearly here.
|
||||
probe = machine_exec(machine, ["true"])
|
||||
if probe.returncode != 0:
|
||||
die(
|
||||
f"smolvm exec is not available for {machine!r} "
|
||||
f"(exit {probe.returncode}: {probe.stderr.strip() or probe.stdout.strip() or '<no output>'}). "
|
||||
f"If an interactive session is active, smolvm may not support concurrent exec."
|
||||
)
|
||||
|
||||
# Create the compressed tar inside the VM.
|
||||
# tar exits 1 when files change during archiving (normal for a live
|
||||
# filesystem); only treat exit > 1 as fatal.
|
||||
tar_result = machine_exec(
|
||||
machine,
|
||||
[
|
||||
"tar", "--create", "--gzip",
|
||||
"--exclude=./proc",
|
||||
"--exclude=./sys",
|
||||
"--exclude=./dev",
|
||||
"--exclude=./run",
|
||||
# /tmp and /var/tmp are ephemeral. Their stale contents
|
||||
# (e.g. /tmp/claude-<uid>) have uid remapped by smolvm's
|
||||
# pack process, causing Claude Code to refuse to use them
|
||||
# on resume. Exclude both; _init_vm recreates them with
|
||||
# mkdir -p + correct ownership on every boot.
|
||||
"--exclude=./tmp",
|
||||
"--exclude=./var/tmp",
|
||||
f"--file={_VM_COMMIT_TAR}",
|
||||
"--directory=/",
|
||||
".",
|
||||
],
|
||||
)
|
||||
if tar_result.returncode > 1:
|
||||
die(
|
||||
f"smolvm exec tar {machine!r} failed (exit {tar_result.returncode}): "
|
||||
f"{tar_result.stderr.strip() or tar_result.stdout.strip() or '<no output>'}"
|
||||
)
|
||||
|
||||
# Copy from VM to host, then clean up.
|
||||
try:
|
||||
machine_cp(f"{machine}:{_VM_COMMIT_TAR}", str(dest))
|
||||
finally:
|
||||
machine_exec(machine, ["rm", "-f", _VM_COMMIT_TAR])
|
||||
@@ -40,8 +40,12 @@ from ..docker.git_gate import (
|
||||
GIT_GATE_HOOK_IN_CONTAINER,
|
||||
)
|
||||
from ...git_gate import revoke_git_gate_provisioned_keys
|
||||
from ...log import warn
|
||||
from ..docker.bottle_state import egress_state_dir, git_gate_state_dir
|
||||
from ...log import info, warn
|
||||
from ...bottle_state import (
|
||||
egress_state_dir,
|
||||
git_gate_state_dir,
|
||||
read_committed_image,
|
||||
)
|
||||
from . import loopback_alias as _loopback
|
||||
from . import sidecar_bundle as _bundle
|
||||
from . import smolvm as _smolvm
|
||||
@@ -85,14 +89,7 @@ def launch(
|
||||
plan = _start_bundle(plan, network, loopback_ip, stack)
|
||||
plan = _discover_urls(plan, loopback_ip)
|
||||
|
||||
# Build the agent image and pack it into a `.smolmachine`
|
||||
# artifact (or hit the per-Dockerfile-digest cache). Runs
|
||||
# here, not in prepare, so the docker-build output doesn't
|
||||
# garble the dashboard's preflight modal.
|
||||
agent_from_path = _ensure_smolmachine(
|
||||
plan.agent_image_ref,
|
||||
dockerfile=plan.agent_dockerfile_path,
|
||||
)
|
||||
agent_from_path = _agent_from_path(plan)
|
||||
|
||||
_launch_vm(plan, agent_from_path, loopback_ip, stack)
|
||||
_init_vm(plan)
|
||||
@@ -103,6 +100,10 @@ def launch(
|
||||
guest_env=plan.guest_env,
|
||||
agent_command=plan.agent_command,
|
||||
agent_prompt_mode=plan.agent_prompt_mode,
|
||||
agent_provider_template=plan.agent_provider_template,
|
||||
terminal_title=f"{plan.spec.label} ({plan.spec.agent_name})" if plan.spec.label else plan.spec.agent_name,
|
||||
terminal_color=plan.spec.color,
|
||||
agent_workdir=plan.workspace_plan.workdir,
|
||||
)
|
||||
bottle.prompt_path = provision(plan, bottle)
|
||||
|
||||
@@ -126,7 +127,7 @@ def _teardown_smolmachines(
|
||||
except BaseException as exc: # noqa: W0718 — teardown must not fail
|
||||
teardown_exc = exc
|
||||
warn(f"smolmachines teardown failed: {exc!r}")
|
||||
bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
||||
bottle = plan.manifest.bottle
|
||||
revoke_git_gate_provisioned_keys(bottle, git_gate_state_dir(plan.slug))
|
||||
if teardown_exc is not None:
|
||||
raise teardown_exc
|
||||
@@ -213,11 +214,15 @@ def _discover_urls(
|
||||
agent_supervise_url = f"http://{loopback_ip}:{supervise_host_port}/"
|
||||
|
||||
existing_no_proxy = plan.guest_env.get("NO_PROXY", "localhost,127.0.0.1")
|
||||
no_proxy = f"{existing_no_proxy},{loopback_ip}"
|
||||
guest_env = {
|
||||
**plan.guest_env,
|
||||
"HTTPS_PROXY": agent_proxy_url,
|
||||
"HTTP_PROXY": agent_proxy_url,
|
||||
"NO_PROXY": f"{existing_no_proxy},{loopback_ip}",
|
||||
"https_proxy": agent_proxy_url,
|
||||
"http_proxy": agent_proxy_url,
|
||||
"NO_PROXY": no_proxy,
|
||||
"no_proxy": no_proxy,
|
||||
}
|
||||
if agent_git_gate_host:
|
||||
guest_env["GIT_GATE_URL"] = f"http://{agent_git_gate_host}"
|
||||
@@ -271,10 +276,16 @@ def _init_vm(plan: SmolmachinesBottlePlan) -> None:
|
||||
All folded into one sh -c to avoid back-to-back exec calls
|
||||
immediately after machine_start (libkrun exec-channel race).
|
||||
|
||||
mkdir -p guards: when booting from a committed snapshot, /tmp and
|
||||
/var/tmp are excluded from the archive (they're ephemeral and their
|
||||
stale contents would have wrong uid after smolvm's uid remap). The
|
||||
directories must be created before chown/chmod can set permissions.
|
||||
|
||||
wait_exec_ready polls until the exec channel is ready for the
|
||||
subsequent provision calls, replacing the empirical sleep."""
|
||||
_smolvm.machine_exec(plan.machine_name, [
|
||||
"sh", "-c",
|
||||
"mkdir -p /tmp /var/tmp && "
|
||||
"chown -R node:node /home/node && "
|
||||
"chown root:root /tmp /var/tmp && "
|
||||
"chmod 1777 /tmp /var/tmp",
|
||||
@@ -304,7 +315,7 @@ def _bundle_launch_spec(
|
||||
ep = plan.egress_plan
|
||||
volumes.append((str(ep.mitmproxy_ca_host_path), EGRESS_CA_IN_CONTAINER, True))
|
||||
if ep.routes:
|
||||
volumes.append((str(ep.routes_path), EGRESS_ROUTES_IN_CONTAINER, True))
|
||||
volumes.append((str(ep.routes_path.parent), str(Path(EGRESS_ROUTES_IN_CONTAINER).parent), True))
|
||||
# Bare-name entries for upstream-token slots. Their values
|
||||
# come from the docker-run subprocess env (inherited from
|
||||
# the operator's shell), never landing on argv.
|
||||
@@ -378,6 +389,30 @@ def _resolve_token_env(
|
||||
return egress_resolve_token_values(plan.egress_plan.token_env_map, effective_env)
|
||||
|
||||
|
||||
def _agent_from_path(plan: SmolmachinesBottlePlan) -> Path:
|
||||
"""Return the `.smolmachine` artifact used for `machine create --from`.
|
||||
|
||||
Prefer a committed VM artifact when one is recorded and still
|
||||
present. If the file was removed, fall back to the normal image
|
||||
build + pack cache path.
|
||||
"""
|
||||
committed = read_committed_image(plan.slug)
|
||||
if committed:
|
||||
committed_path = Path(committed)
|
||||
if committed_path.is_file():
|
||||
info(f"using committed smolmachine {str(committed_path)!r}")
|
||||
return committed_path
|
||||
|
||||
# Build the agent image and pack it into a `.smolmachine`
|
||||
# artifact (or hit the per-Dockerfile-digest cache). Runs here,
|
||||
# not in prepare, so the docker-build output doesn't garble the
|
||||
# dashboard's preflight modal.
|
||||
return _ensure_smolmachine(
|
||||
plan.agent_image,
|
||||
dockerfile=plan.agent_dockerfile_path,
|
||||
)
|
||||
|
||||
|
||||
def _ensure_smolmachine(image_ref: str, *, dockerfile: str = "") -> Path:
|
||||
"""Build the agent docker image and convert it into a
|
||||
`.smolmachine` artifact, caching the result under
|
||||
|
||||
@@ -1,185 +0,0 @@
|
||||
"""smolmachines `_resolve_plan` (PRD 0023 chunks 2d + 4c).
|
||||
|
||||
Resolves the per-bottle docker subnet + bundle IP and assembles
|
||||
the guest env. The agent's docker image build → smolmachine
|
||||
pack pipeline runs in `launch.launch`, not here, so the
|
||||
dashboard's preflight modal isn't garbled by docker-build output
|
||||
before the operator has confirmed.
|
||||
|
||||
No VM bringup — that's `launch.launch`'s job."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from datetime import datetime, timezone
|
||||
from dataclasses import replace
|
||||
from pathlib import Path
|
||||
|
||||
from ...agent_provider import agent_provision_plan, runtime_for
|
||||
from ...backend import BottleSpec
|
||||
from ...backend.docker.bottle_state import (
|
||||
BottleMetadata,
|
||||
agent_state_dir,
|
||||
bottle_identity,
|
||||
egress_state_dir,
|
||||
git_gate_state_dir,
|
||||
supervise_state_dir,
|
||||
write_metadata,
|
||||
)
|
||||
from ...egress import Egress
|
||||
from ...env import resolve_env
|
||||
from ...git_gate import GitGate
|
||||
from ...supervise import Supervise
|
||||
from ...workspace import workspace_plan as resolve_workspace_plan
|
||||
from .bottle_plan import SmolmachinesBottlePlan
|
||||
from .util import smolmachines_bundle_subnet, smolmachines_preflight
|
||||
|
||||
|
||||
# Gateway ports the bundle exposes inside its container — git-gate's
|
||||
# git-daemon, supervise's MCP. The agent inside the smolvm guest
|
||||
# dials these on the bundle's pinned IP.
|
||||
_BUNDLE_GIT_GATE_PORT = 9418
|
||||
_BUNDLE_SUPERVISE_PORT = 9100
|
||||
|
||||
|
||||
def resolve_plan(
|
||||
spec: BottleSpec, *, stage_dir: Path
|
||||
) -> SmolmachinesBottlePlan:
|
||||
"""Materialize the smolmachines plan. The bundle's docker
|
||||
subnet + pinned IP are derived from the slug; the agent's
|
||||
`.smolmachine` artifact is built (or cache-hit) here so
|
||||
launch's `machine create --from` boots without a registry
|
||||
pull. Per-bottle guest env + the TSI allow_cidrs land on the
|
||||
plan for launch to pass straight through to
|
||||
`machine create` flags."""
|
||||
smolmachines_preflight()
|
||||
|
||||
manifest = spec.manifest
|
||||
bottle = manifest.bottle_for(spec.agent_name)
|
||||
provider = bottle.agent_provider
|
||||
provider_runtime = runtime_for(provider.template)
|
||||
guest_home = "/home/node"
|
||||
workspace_plan = resolve_workspace_plan(spec, guest_home=guest_home)
|
||||
|
||||
slug = spec.identity or bottle_identity(spec.agent_name)
|
||||
|
||||
# Record minimal metadata so `cli.py resume` can recover the
|
||||
# slug. Same schema as the docker backend.
|
||||
write_metadata(BottleMetadata(
|
||||
identity=slug,
|
||||
agent_name=spec.agent_name,
|
||||
cwd=spec.user_cwd if spec.copy_cwd else "",
|
||||
copy_cwd=spec.copy_cwd,
|
||||
started_at=datetime.now(timezone.utc).isoformat(),
|
||||
compose_project="",
|
||||
backend="smolmachines",
|
||||
label=spec.label,
|
||||
color=spec.color,
|
||||
))
|
||||
|
||||
subnet, gateway, bundle_ip = smolmachines_bundle_subnet(slug)
|
||||
|
||||
# Agent's env: resolve through resolve_env() so ?prompt entries
|
||||
# are prompted and ${HOST_VAR} entries are interpolated — matching
|
||||
# the Docker backend's contract. Forwarded (secret/interpolated)
|
||||
# values still reach the guest as -e K=V smolvm flags because
|
||||
# smolvm 0.8.0 has no env-file or stdin injection path; this is
|
||||
# the known argv-exposure gap documented in PRD 0038.
|
||||
# HTTPS_PROXY / GIT_GATE_URL / MCP_SUPERVISE_URL are populated
|
||||
# in launch.py after bundle bringup.
|
||||
resolved = resolve_env(manifest, spec.agent_name)
|
||||
guest_env: dict[str, str] = {
|
||||
**resolved.literals,
|
||||
**resolved.forwarded,
|
||||
"NO_PROXY": "localhost,127.0.0.1",
|
||||
"NODE_EXTRA_CA_CERTS": "/etc/ssl/certs/ca-certificates.crt",
|
||||
"SSL_CERT_FILE": "/etc/ssl/certs/ca-certificates.crt",
|
||||
"REQUESTS_CA_BUNDLE": "/etc/ssl/certs/ca-certificates.crt",
|
||||
}
|
||||
|
||||
git_gate_dir = git_gate_state_dir(slug)
|
||||
git_gate_dir.mkdir(parents=True, exist_ok=True)
|
||||
git_gate_plan = GitGate().prepare(bottle, slug, git_gate_dir)
|
||||
|
||||
# Prompt file is always written (mode 0o600) so the in-VM
|
||||
# path always exists. Content is the agent's `prompt`
|
||||
# field (markdown body) — empty for agents with no prompt.
|
||||
# claude-code reads it via --append-system-prompt-file only
|
||||
# when non-empty, but the file must exist either way to
|
||||
# match the docker backend's contract.
|
||||
agent_dir = agent_state_dir(slug)
|
||||
agent_dir.mkdir(parents=True, exist_ok=True)
|
||||
prompt_file = agent_dir / "prompt.txt"
|
||||
agent = manifest.agents[spec.agent_name]
|
||||
prompt_file.write_text(agent.prompt or "")
|
||||
prompt_file.chmod(0o600)
|
||||
|
||||
machine_name = f"bot-bottle-{slug}"
|
||||
# Stash the agent image ref — `launch.launch` runs the
|
||||
# build → pack pipeline at bringup. Honors BOT_BOTTLE_IMAGE
|
||||
# to match the docker backend's `resolve_plan` default.
|
||||
agent_dockerfile_path = ""
|
||||
if provider.dockerfile:
|
||||
agent_dockerfile_path = _resolve_manifest_dockerfile(provider.dockerfile, spec)
|
||||
image_default = f"bot-bottle-{provider.template}:{slug}"
|
||||
elif provider_runtime.dockerfile:
|
||||
agent_dockerfile_path = provider_runtime.dockerfile
|
||||
image_default = provider_runtime.image
|
||||
else:
|
||||
image_default = provider_runtime.image
|
||||
agent_image_ref = os.environ.get("BOT_BOTTLE_IMAGE", image_default)
|
||||
agent_provision = agent_provision_plan(
|
||||
template=provider.template,
|
||||
dockerfile=agent_dockerfile_path,
|
||||
state_dir=agent_dir,
|
||||
guest_home=guest_home,
|
||||
guest_env=guest_env,
|
||||
forward_host_credentials=provider.forward_host_credentials,
|
||||
auth_token=provider.auth_token,
|
||||
host_env=dict(os.environ),
|
||||
trusted_project_path=workspace_plan.workdir,
|
||||
label=spec.label,
|
||||
color=spec.color,
|
||||
)
|
||||
merged_guest_env = dict(agent_provision.guest_env)
|
||||
for key, val in agent_provision.env_vars.items():
|
||||
merged_guest_env.setdefault(key, val)
|
||||
agent_provision = replace(agent_provision, guest_env=merged_guest_env)
|
||||
|
||||
egress_dir = egress_state_dir(slug)
|
||||
egress_dir.mkdir(parents=True, exist_ok=True)
|
||||
egress_plan = Egress().prepare(
|
||||
bottle, slug, egress_dir, agent_provision.egress_routes,
|
||||
)
|
||||
|
||||
supervise_plan = None
|
||||
if bottle.supervise:
|
||||
supervise_dir = supervise_state_dir(slug)
|
||||
supervise_dir.mkdir(parents=True, exist_ok=True)
|
||||
supervise_plan = Supervise().prepare(slug, supervise_dir)
|
||||
|
||||
return SmolmachinesBottlePlan(
|
||||
spec=spec,
|
||||
stage_dir=stage_dir,
|
||||
guest_home=guest_home,
|
||||
slug=slug,
|
||||
bundle_subnet=subnet,
|
||||
bundle_gateway=gateway,
|
||||
bundle_ip=bundle_ip,
|
||||
machine_name=machine_name,
|
||||
agent_image_ref=agent_image_ref,
|
||||
guest_env=agent_provision.guest_env,
|
||||
prompt_file=prompt_file,
|
||||
git_gate_plan=git_gate_plan,
|
||||
egress_plan=egress_plan,
|
||||
supervise_plan=supervise_plan,
|
||||
agent_provision=agent_provision,
|
||||
workspace_plan=workspace_plan,
|
||||
)
|
||||
|
||||
|
||||
def _resolve_manifest_dockerfile(path_value: str, spec: BottleSpec) -> str:
|
||||
path = Path(os.path.expanduser(path_value))
|
||||
if not path.is_absolute():
|
||||
path = Path(spec.user_cwd) / path
|
||||
return str(path)
|
||||
@@ -6,8 +6,7 @@ the `AgentProvider` plugin under `bot_bottle/contrib/`. CA and git
|
||||
provisioning also moved to the AgentProvider ABC (with Debian/node
|
||||
defaults); user plugins override them for non-standard images.
|
||||
|
||||
The module left in this subpackage handles the remaining backend-
|
||||
specific step:
|
||||
|
||||
- workspace.py — copy the operator workspace into the guest
|
||||
No modules remain in this subpackage. Workspace copying now runs
|
||||
through `BottleBackend.provision_workspace` against the running
|
||||
bottle for every backend.
|
||||
"""
|
||||
|
||||
@@ -1,32 +0,0 @@
|
||||
"""Copy the operator workspace into a smolmachines guest."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shlex
|
||||
|
||||
from ....log import info
|
||||
from ... import Bottle
|
||||
from ..bottle_plan import SmolmachinesBottlePlan
|
||||
|
||||
|
||||
def provision_workspace(plan: SmolmachinesBottlePlan, bottle: Bottle) -> None:
|
||||
"""Copy host cwd contents to the planned guest workspace."""
|
||||
workspace = plan.workspace_plan
|
||||
if not (workspace.enabled and workspace.copy_contents):
|
||||
return
|
||||
|
||||
guest_parent = workspace.guest_path.rsplit("/", 1)[0] or "/"
|
||||
guest_path_q = shlex.quote(workspace.guest_path)
|
||||
guest_parent_q = shlex.quote(guest_parent)
|
||||
owner_q = shlex.quote(workspace.owner)
|
||||
mode_q = shlex.quote(workspace.mode)
|
||||
info(f"copying {workspace.host_path} -> {bottle.name}:{workspace.guest_path}")
|
||||
bottle.exec(
|
||||
f"rm -rf {guest_path_q} && mkdir -p {guest_parent_q}",
|
||||
user="root",
|
||||
)
|
||||
bottle.cp_in(str(workspace.host_path), workspace.guest_path)
|
||||
bottle.exec(
|
||||
f"chown -R {owner_q} {guest_path_q} && chmod {mode_q} {guest_path_q}",
|
||||
user="root",
|
||||
)
|
||||
@@ -0,0 +1,83 @@
|
||||
"""smolmachines `_resolve_plan` (PRD 0023 chunks 2d + 4c).
|
||||
|
||||
Resolves the per-bottle docker subnet + bundle IP and assembles
|
||||
the guest env. The agent's docker image build → smolmachine
|
||||
pack pipeline runs in `launch.launch`, not here, so the
|
||||
dashboard's preflight modal isn't garbled by docker-build output
|
||||
before the operator has confirmed.
|
||||
|
||||
No VM bringup — that's `launch.launch`'s job."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from .. import BottleSpec
|
||||
from ...manifest import Manifest
|
||||
from ...env import ResolvedEnv
|
||||
from ...agent_provider import AgentProvisionPlan
|
||||
from ...egress import EgressPlan
|
||||
from ...supervise import SupervisePlan
|
||||
from ...git_gate import GitGatePlan
|
||||
from .bottle_plan import SmolmachinesBottlePlan
|
||||
from .util import smolmachines_bundle_subnet, smolmachines_preflight
|
||||
|
||||
def preflight() -> None:
|
||||
smolmachines_preflight()
|
||||
|
||||
|
||||
def build_guest_env(resolved_env: ResolvedEnv) -> dict[str, str]:
|
||||
# Agent's env: resolve through resolve_env() so ?prompt entries
|
||||
# are prompted and ${HOST_VAR} entries are interpolated — matching
|
||||
# the Docker backend's contract. Forwarded (secret/interpolated)
|
||||
# values still reach the guest as -e K=V smolvm flags because
|
||||
# smolvm 0.8.0 has no env-file or stdin injection path; this is
|
||||
# the known argv-exposure gap documented in PRD 0038.
|
||||
# HTTPS_PROXY / GIT_GATE_URL / MCP_SUPERVISE_URL are populated
|
||||
# in launch.py after bundle bringup.
|
||||
return {
|
||||
**resolved_env.literals,
|
||||
**resolved_env.forwarded,
|
||||
"NO_PROXY": "localhost,127.0.0.1",
|
||||
"NODE_EXTRA_CA_CERTS": "/etc/ssl/certs/ca-certificates.crt",
|
||||
"SSL_CERT_FILE": "/etc/ssl/certs/ca-certificates.crt",
|
||||
"REQUESTS_CA_BUNDLE": "/etc/ssl/certs/ca-certificates.crt",
|
||||
}
|
||||
|
||||
|
||||
def resolve_plan(
|
||||
spec: BottleSpec,
|
||||
manifest: Manifest,
|
||||
slug: str,
|
||||
resolved_env: ResolvedEnv,
|
||||
agent_provision_plan: AgentProvisionPlan,
|
||||
egress_plan: EgressPlan,
|
||||
supervise_plan: SupervisePlan | None,
|
||||
git_gate_plan: GitGatePlan,
|
||||
stage_dir: Path,
|
||||
) -> SmolmachinesBottlePlan:
|
||||
"""Materialize the smolmachines plan. The bundle's docker
|
||||
subnet + pinned IP are derived from the slug; the agent's
|
||||
`.smolmachine` artifact is built (or cache-hit) here so
|
||||
launch's `machine create --from` boots without a registry
|
||||
pull. Per-bottle guest env + the TSI allow_cidrs land on the
|
||||
plan for launch to pass straight through to
|
||||
`machine create` flags."""
|
||||
|
||||
# ==== smolmachines specific setup ====
|
||||
subnet, gateway, bundle_ip = smolmachines_bundle_subnet(slug)
|
||||
|
||||
return SmolmachinesBottlePlan(
|
||||
spec=spec,
|
||||
manifest=manifest,
|
||||
stage_dir=stage_dir,
|
||||
slug=slug,
|
||||
bundle_subnet=subnet,
|
||||
bundle_gateway=gateway,
|
||||
bundle_ip=bundle_ip,
|
||||
guest_env=agent_provision_plan.guest_env,
|
||||
git_gate_plan=git_gate_plan,
|
||||
egress_plan=egress_plan,
|
||||
supervise_plan=supervise_plan,
|
||||
agent_provision=agent_provision_plan,
|
||||
)
|
||||
@@ -25,6 +25,7 @@ smolvm binary."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import shutil
|
||||
import subprocess
|
||||
import time
|
||||
@@ -94,6 +95,16 @@ def pack_create(image: str, output: Path) -> None:
|
||||
_smolvm("pack", "create", "--image", image, "-o", str(output))
|
||||
|
||||
|
||||
def pack_create_from_vm(name: str, output: Path) -> None:
|
||||
"""`smolvm pack create --from-vm <name> -o <output>`.
|
||||
|
||||
Snapshots an existing persistent VM into a pack artifact. As
|
||||
with `pack_create`, smolvm writes a launcher at `output` and the
|
||||
bootable sidecar at `output.smolmachine`.
|
||||
"""
|
||||
_smolvm("pack", "create", "--from-vm", name, "-o", str(output))
|
||||
|
||||
|
||||
# --- Machine lifecycle ---------------------------------------------------
|
||||
|
||||
|
||||
@@ -143,6 +154,21 @@ def machine_create(
|
||||
_smolvm(*args)
|
||||
|
||||
|
||||
def machine_is_running(name: str) -> bool:
|
||||
"""Return True if the named VM is in the 'running' state."""
|
||||
result = _smolvm("machine", "ls", "--json", check=False)
|
||||
if result.returncode != 0:
|
||||
return False
|
||||
try:
|
||||
machines = json.loads(result.stdout or "[]")
|
||||
except ValueError:
|
||||
return False
|
||||
return any(
|
||||
isinstance(m, dict) and m.get("name") == name and m.get("state") == "running"
|
||||
for m in machines
|
||||
)
|
||||
|
||||
|
||||
def machine_start(name: str) -> None:
|
||||
"""`smolvm machine start --name NAME`."""
|
||||
_smolvm("machine", "start", "--name", name)
|
||||
|
||||
@@ -21,7 +21,9 @@ def smolmachines_preflight() -> None:
|
||||
die(
|
||||
"BOT_BOTTLE_BACKEND=smolmachines requires `smolvm` on "
|
||||
"PATH. Install with: "
|
||||
"curl -sSL https://smolmachines.com/install.sh | sh"
|
||||
"curl -sSL https://smolmachines.com/install.sh | sh. "
|
||||
"To use the legacy Docker backend instead, set "
|
||||
"BOT_BOTTLE_BACKEND=docker or pass --backend=docker."
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,71 @@
|
||||
"""Terminal escape-sequence helpers shared across all bottle backends."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shlex
|
||||
|
||||
|
||||
# color name → (normal_idx, normal_hex, bright_idx, bright_hex, dark_bg_hex)
|
||||
# OSC 4 sets indexed palette entries (affects syntax-highlighted code and any
|
||||
# TUI content that uses indexed colors). dark_bg_hex is used for OSC 11
|
||||
# (default background) — a very dark tint that's visible even when the TUI
|
||||
# uses true/24-bit colors for its own chrome, which would otherwise bypass
|
||||
# the palette entirely.
|
||||
_COLORS: dict[str, tuple[int, str, int, str, str]] = {
|
||||
"red": (9, "#e74c3c", 1, "#c0392b", "#200808"),
|
||||
"green": (10, "#2ecc71", 2, "#27ae60", "#082008"),
|
||||
"yellow": (11, "#f1c40f", 3, "#d4ac0d", "#201808"),
|
||||
"blue": (12, "#3498db", 4, "#2471a3", "#080820"),
|
||||
"magenta": (13, "#9b59b6", 5, "#7d3c98", "#160820"),
|
||||
}
|
||||
|
||||
# OSC 104 resets all indexed palette entries; OSC 111 resets default background.
|
||||
_RESET_PRINTF = "printf '\\033]104\\007\\033]111\\007'"
|
||||
|
||||
|
||||
def palette_printf(color: str) -> str:
|
||||
"""Shell `printf` command that emits OSC 4 + OSC 11 to tint the terminal
|
||||
for *color*: sets the normal/bright palette entries AND the default
|
||||
background to a dark shade of that color. Returns '' if unknown."""
|
||||
entry = _COLORS.get(color)
|
||||
if not entry:
|
||||
return ""
|
||||
n_idx, n_hex, b_idx, b_hex, bg_hex = entry
|
||||
seq = (
|
||||
f"\\033]4;{n_idx};{n_hex}\\007"
|
||||
f"\\033]4;{b_idx};{b_hex}\\007"
|
||||
f"\\033]11;{bg_hex}\\007"
|
||||
)
|
||||
return f"printf '{seq}'"
|
||||
|
||||
|
||||
def exec_shell_script(
|
||||
agent_argv: list[str],
|
||||
terminal_title: str = "",
|
||||
terminal_color: str = "",
|
||||
) -> str | None:
|
||||
"""Build a shell script string that optionally sets the terminal
|
||||
title and/or palette before running *agent_argv*, and resets the
|
||||
palette + background on exit. Returns None when no decoration is
|
||||
needed — callers should run *agent_argv* directly in that case."""
|
||||
title_cmd = (
|
||||
f"printf '\\033]0;%s\\007' {shlex.quote(terminal_title)}"
|
||||
if terminal_title else ""
|
||||
)
|
||||
pal_cmd = palette_printf(terminal_color)
|
||||
|
||||
if not title_cmd and not pal_cmd:
|
||||
return None
|
||||
|
||||
parts: list[str] = []
|
||||
if title_cmd:
|
||||
parts.append(title_cmd)
|
||||
if pal_cmd:
|
||||
parts.append(pal_cmd)
|
||||
parts.append(shlex.join(agent_argv))
|
||||
parts.append(_RESET_PRINTF)
|
||||
else:
|
||||
# No palette change — exec so the agent replaces the shell.
|
||||
parts.append(f"exec {shlex.join(agent_argv)}")
|
||||
|
||||
return "; ".join(parts)
|
||||
@@ -37,13 +37,13 @@ from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import cast
|
||||
|
||||
from ... import supervise as _supervise
|
||||
from . import util as docker_mod
|
||||
from . import supervise as _supervise
|
||||
|
||||
|
||||
# Directory layout: ~/.bot-bottle/state/<identity>/...
|
||||
_STATE_SUBDIR = "state"
|
||||
_PER_BOTTLE_DOCKERFILE_NAME = "Dockerfile"
|
||||
_COMMITTED_IMAGE_NAME = "committed-image"
|
||||
_TRANSCRIPT_SUBDIR = "transcript"
|
||||
# Per-sidecar scratch subdirs. PRD 0018 chunk 2: bind-mount sources
|
||||
# live here so chunk 3's `docker compose up` can find them at stable
|
||||
@@ -82,6 +82,7 @@ def bottle_identity(agent_name: str) -> str:
|
||||
To continue an existing bottle's state, use the recorded
|
||||
identity from BottleMetadata via `cli.py resume <identity>`,
|
||||
not this function."""
|
||||
from .backend.docker import util as docker_mod
|
||||
slug = docker_mod.slugify(agent_name)
|
||||
suffix = "".join(secrets.choice(_SUFFIX_ALPHABET) for _ in range(_RANDOM_SUFFIX_LEN))
|
||||
return f"{slug}-{suffix}"
|
||||
@@ -179,6 +180,32 @@ def write_per_bottle_dockerfile(identity: str, content: str) -> Path:
|
||||
return p
|
||||
|
||||
|
||||
def committed_image_path(identity: str) -> Path:
|
||||
return bottle_state_dir(identity) / _COMMITTED_IMAGE_NAME
|
||||
|
||||
|
||||
def write_committed_image(identity: str, image_tag: str) -> Path:
|
||||
"""Persist the committed image tag for `identity`. The next
|
||||
`cli.py resume <identity>` will boot from this image instead of
|
||||
rebuilding from the Dockerfile."""
|
||||
path = committed_image_path(identity)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(image_tag.strip() + "\n")
|
||||
path.chmod(0o644)
|
||||
return path
|
||||
|
||||
|
||||
def read_committed_image(identity: str) -> str | None:
|
||||
"""Return the committed image tag for `identity`, or None if no
|
||||
commit has been recorded. Used by the Docker launch step to skip
|
||||
the Dockerfile build when a committed snapshot exists."""
|
||||
path = committed_image_path(identity)
|
||||
if not path.is_file():
|
||||
return None
|
||||
tag = path.read_text().strip()
|
||||
return tag or None
|
||||
|
||||
|
||||
def per_bottle_image_tag(identity: str) -> str:
|
||||
"""Image tag for a rebuilt bottle. Distinct from the base
|
||||
bot-bottle-claude:latest so per-bottle rebuilds don't collide in
|
||||
@@ -314,6 +341,7 @@ __all__ = [
|
||||
"bottle_state_dir",
|
||||
"cleanup_state",
|
||||
"clear_preserve_marker",
|
||||
"committed_image_path",
|
||||
"egress_state_dir",
|
||||
"git_gate_state_dir",
|
||||
"is_preserved",
|
||||
@@ -323,9 +351,11 @@ __all__ = [
|
||||
"per_bottle_dockerfile_path",
|
||||
"per_bottle_image_tag",
|
||||
"preserve_marker_path",
|
||||
"read_committed_image",
|
||||
"read_metadata",
|
||||
"supervise_state_dir",
|
||||
"transcript_snapshot_dir",
|
||||
"write_committed_image",
|
||||
"write_metadata",
|
||||
"write_per_bottle_dockerfile",
|
||||
]
|
||||
@@ -1,6 +1,6 @@
|
||||
"""Main CLI dispatcher.
|
||||
|
||||
Commands: cleanup, edit, info, init, list, resume, start, supervise
|
||||
Commands: cleanup, commit, doctor, edit, info, init, list, resume, start, supervise
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -12,6 +12,8 @@ from ..manifest import ManifestError
|
||||
from ._common import PROG
|
||||
from . import list as _list_mod
|
||||
from .cleanup import cmd_cleanup
|
||||
from .commit import cmd_commit
|
||||
from .doctor import cmd_doctor
|
||||
from .edit import cmd_edit
|
||||
from .info import cmd_info
|
||||
from .init import cmd_init
|
||||
@@ -23,6 +25,8 @@ cmd_list = _list_mod.cmd_list
|
||||
|
||||
COMMANDS = {
|
||||
"cleanup": cmd_cleanup,
|
||||
"commit": cmd_commit,
|
||||
"doctor": cmd_doctor,
|
||||
"edit": cmd_edit,
|
||||
"info": cmd_info,
|
||||
"init": cmd_init,
|
||||
@@ -37,6 +41,8 @@ def usage() -> None:
|
||||
sys.stderr.write(f"usage: {PROG} <command> [args...]\n\n")
|
||||
sys.stderr.write("Commands:\n")
|
||||
sys.stderr.write(" cleanup stop and remove all active bot-bottle containers\n")
|
||||
sys.stderr.write(" commit snapshot a running bottle's container state to a Docker image\n")
|
||||
sys.stderr.write(" doctor check Python, Docker, and bot-bottle config prerequisites\n")
|
||||
sys.stderr.write(" edit open an agent in vim for editing\n")
|
||||
sys.stderr.write(" info print env, skills, and prompt details for a named agent\n")
|
||||
sys.stderr.write(" init interactively create a new agent and add it to bot-bottle.json\n")
|
||||
|
||||
@@ -6,7 +6,7 @@ import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
PROG = "cli.py"
|
||||
PROG = Path(sys.argv[0]).name or "bot-bottle"
|
||||
USER_CWD = os.getcwd()
|
||||
REPO_DIR = str(Path(__file__).resolve().parent.parent.parent)
|
||||
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
"""commit: freeze a running bottle's state to a resumable artifact.
|
||||
|
||||
Docker bottles are committed to a local Docker image. Macos-container
|
||||
bottles are exported and rebuilt as a local Apple Container image.
|
||||
Smolmachines bottles are packed from the running VM into a
|
||||
`.smolmachine` artifact. The resulting reference is stored in
|
||||
per-bottle state so the next `./cli.py resume <slug>` boots from the
|
||||
snapshot instead of rebuilding from the Dockerfile.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
|
||||
from ..backend import enumerate_active_agents
|
||||
from ..backend.freeze import CommitCancelled, get_freezer
|
||||
from ..bottle_state import read_metadata
|
||||
from ..log import die
|
||||
from ._common import PROG
|
||||
from . import tui
|
||||
|
||||
|
||||
def cmd_commit(argv: list[str]) -> int:
|
||||
parser = argparse.ArgumentParser(prog=f"{PROG} commit", add_help=True)
|
||||
parser.add_argument(
|
||||
"slug",
|
||||
nargs="?",
|
||||
default=None,
|
||||
help=(
|
||||
"bottle slug from `cli.py list active` "
|
||||
"(omit to pick interactively)"
|
||||
),
|
||||
)
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
slug = args.slug
|
||||
if slug is None:
|
||||
active = enumerate_active_agents()
|
||||
if not active:
|
||||
die("no active bottles; start one with `./cli.py start`")
|
||||
choices = [a.slug for a in active]
|
||||
slug = tui.filter_select(choices, title="Select bottle to commit")
|
||||
if slug is None:
|
||||
return 0
|
||||
|
||||
metadata = read_metadata(slug)
|
||||
backend = metadata.backend if metadata else ""
|
||||
|
||||
try:
|
||||
get_freezer(backend).commit_slug(slug)
|
||||
except CommitCancelled:
|
||||
return 0
|
||||
return 0
|
||||
@@ -0,0 +1,73 @@
|
||||
"""doctor: validate host prerequisites for running bot-bottle."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from ._common import PROG
|
||||
|
||||
|
||||
def _ok(label: str, detail: str) -> None:
|
||||
print(f"ok: {label}: {detail}")
|
||||
|
||||
|
||||
def _fail(label: str, detail: str) -> None:
|
||||
print(f"fail: {label}: {detail}")
|
||||
|
||||
|
||||
def _check_python() -> bool:
|
||||
version = sys.version_info
|
||||
detail = f"{version.major}.{version.minor}.{version.micro}"
|
||||
if version >= (3, 11):
|
||||
_ok("python", detail)
|
||||
return True
|
||||
_fail("python", f"{detail}; need 3.11 or newer")
|
||||
return False
|
||||
|
||||
|
||||
def _check_docker() -> bool:
|
||||
docker = shutil.which("docker")
|
||||
if not docker:
|
||||
_fail("docker", "docker command not found")
|
||||
return False
|
||||
try:
|
||||
result = subprocess.run(
|
||||
[docker, "info"],
|
||||
stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.DEVNULL,
|
||||
check=False,
|
||||
timeout=10,
|
||||
)
|
||||
except (OSError, subprocess.TimeoutExpired) as exc:
|
||||
_fail("docker", f"daemon check failed: {exc}")
|
||||
return False
|
||||
if result.returncode == 0:
|
||||
_ok("docker", "daemon reachable")
|
||||
return True
|
||||
_fail("docker", "daemon not reachable")
|
||||
return False
|
||||
|
||||
|
||||
def _check_config_dir() -> bool:
|
||||
config = Path.home() / ".bot-bottle"
|
||||
if config.is_dir():
|
||||
_ok("config", str(config))
|
||||
return True
|
||||
_fail("config", f"{config} does not exist")
|
||||
return False
|
||||
|
||||
|
||||
def cmd_doctor(argv: list[str]) -> int:
|
||||
parser = argparse.ArgumentParser(prog=f"{PROG} doctor", add_help=True)
|
||||
parser.parse_args(argv)
|
||||
|
||||
checks = (
|
||||
_check_python(),
|
||||
_check_docker(),
|
||||
_check_config_dir(),
|
||||
)
|
||||
return 0 if all(checks) else 1
|
||||
@@ -5,7 +5,7 @@ from __future__ import annotations
|
||||
import argparse
|
||||
|
||||
from ..log import info
|
||||
from ..manifest import Manifest
|
||||
from ..manifest import ManifestIndex
|
||||
from ._common import PROG, USER_CWD
|
||||
|
||||
|
||||
@@ -14,11 +14,12 @@ def cmd_info(argv: list[str]) -> int:
|
||||
parser.add_argument("name", help="agent name defined in bot-bottle.json")
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
manifest = Manifest.resolve(USER_CWD)
|
||||
manifest.require_agent(args.name)
|
||||
names = ManifestIndex.resolve(USER_CWD)
|
||||
names.require_agent(args.name)
|
||||
manifest = names.load_for_agent(args.name)
|
||||
|
||||
agent = manifest.agents[args.name]
|
||||
bottle = manifest.bottle_for(args.name)
|
||||
agent = manifest.agent
|
||||
bottle = manifest.bottle
|
||||
env_names = list(bottle.env.keys())
|
||||
prompt_first_line = agent.prompt.splitlines()[0] if agent.prompt else ""
|
||||
|
||||
@@ -31,7 +32,7 @@ def cmd_info(argv: list[str]) -> int:
|
||||
f"first line: {prompt_first_line or '(empty)'}"
|
||||
)
|
||||
info(f"bottle : {agent.bottle}")
|
||||
identity = manifest.git_identity_summary(args.name)
|
||||
identity = manifest.git_identity_summary()
|
||||
if identity:
|
||||
info(f" git identity : {identity}")
|
||||
if bottle.git:
|
||||
|
||||
+9
-20
@@ -7,26 +7,15 @@ import os
|
||||
import sys
|
||||
|
||||
from ..backend import enumerate_active_agents
|
||||
from ..manifest import Manifest
|
||||
from ..manifest import ManifestIndex
|
||||
from ._common import PROG, USER_CWD
|
||||
|
||||
_ANSI_COLOR_CODES: dict[str, str] = {
|
||||
"black": "\033[30m",
|
||||
"red": "\033[31m",
|
||||
"green": "\033[32m",
|
||||
"yellow": "\033[33m",
|
||||
"blue": "\033[34m",
|
||||
"magenta": "\033[35m",
|
||||
"cyan": "\033[36m",
|
||||
"white": "\033[37m",
|
||||
"bright-black": "\033[90m",
|
||||
"bright-red": "\033[91m",
|
||||
"bright-green": "\033[92m",
|
||||
"bright-yellow": "\033[93m",
|
||||
"bright-blue": "\033[94m",
|
||||
"bright-magenta": "\033[95m",
|
||||
"bright-cyan": "\033[96m",
|
||||
"bright-white": "\033[97m",
|
||||
"red": "\033[91m",
|
||||
"green": "\033[92m",
|
||||
"yellow": "\033[93m",
|
||||
"blue": "\033[94m",
|
||||
"magenta": "\033[95m",
|
||||
}
|
||||
_ANSI_RESET = "\033[0m"
|
||||
|
||||
@@ -51,8 +40,8 @@ def cmd_list(argv: list[str]) -> int:
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
if args.scope == "available":
|
||||
manifest = Manifest.resolve(USER_CWD)
|
||||
for name in manifest.agents.keys():
|
||||
manifest = ManifestIndex.resolve(USER_CWD)
|
||||
for name in manifest.all_agent_names:
|
||||
print(name)
|
||||
return 0
|
||||
|
||||
@@ -66,7 +55,7 @@ def cmd_list(argv: list[str]) -> int:
|
||||
# Tab-separated keeps the format stable for shell pipelines.
|
||||
for b in active:
|
||||
services = ",".join(b.services) if b.services else "-"
|
||||
display_name = b.label if b.label else b.agent_name
|
||||
display_name = f"{b.label} ({b.agent_name})" if b.label else b.agent_name
|
||||
colored_name = _ansi_label(display_name, b.color)
|
||||
print(f"{b.backend_name}\t{b.slug}\t{colored_name}\t{services}")
|
||||
return 0
|
||||
|
||||
@@ -18,9 +18,9 @@ from __future__ import annotations
|
||||
import argparse
|
||||
|
||||
from ..backend import BottleSpec
|
||||
from ..backend.docker.bottle_state import read_metadata
|
||||
from ..bottle_state import read_metadata
|
||||
from ..log import die
|
||||
from ..manifest import Manifest
|
||||
from ..manifest import ManifestIndex
|
||||
from ._common import PROG, USER_CWD
|
||||
from .start import _launch_bottle
|
||||
|
||||
@@ -42,7 +42,7 @@ def cmd_resume(argv: list[str]) -> int:
|
||||
f"check ~/.bot-bottle/state/ or run `cli.py start` to create a new bottle"
|
||||
)
|
||||
|
||||
manifest = Manifest.resolve(USER_CWD)
|
||||
manifest = ManifestIndex.resolve(USER_CWD)
|
||||
manifest.require_agent(metadata.agent_name)
|
||||
|
||||
spec = BottleSpec(
|
||||
|
||||
+31
-17
@@ -20,18 +20,20 @@ from ..agent_provider import runtime_for
|
||||
from ..backend import (
|
||||
Bottle,
|
||||
BottleSpec,
|
||||
enumerate_active_agents,
|
||||
get_bottle_backend,
|
||||
known_backend_names,
|
||||
)
|
||||
from ..backend.docker import util as docker_mod
|
||||
from ..backend.docker.bottle_plan import DockerBottlePlan
|
||||
from ..backend.docker.bottle_state import (
|
||||
from ..bottle_state import (
|
||||
cleanup_state,
|
||||
is_preserved,
|
||||
mark_preserved,
|
||||
)
|
||||
from ..backend.docker.capability_apply import snapshot_transcript
|
||||
# from ..backend.docker.capability_apply import snapshot_transcript
|
||||
from ..log import info
|
||||
from ..manifest import Manifest
|
||||
from ..manifest import ManifestIndex
|
||||
from ._common import PROG, USER_CWD, read_tty_line
|
||||
from . import tui
|
||||
|
||||
@@ -39,7 +41,7 @@ from . import tui
|
||||
def cmd_start(argv: list[str]) -> int:
|
||||
parser = argparse.ArgumentParser(prog=f"{PROG} start", add_help=True)
|
||||
parser.add_argument("--dry-run", action="store_true")
|
||||
parser.add_argument("--cwd", action="store_true", help="copy host cwd into a derived image")
|
||||
parser.add_argument("--cwd", action="store_true", help="copy host cwd into the running bottle")
|
||||
parser.add_argument("--remote-control", action="store_true")
|
||||
parser.add_argument(
|
||||
"--backend",
|
||||
@@ -47,7 +49,7 @@ def cmd_start(argv: list[str]) -> int:
|
||||
default=None,
|
||||
help=(
|
||||
"backend to launch the bottle on (default: $BOT_BOTTLE_BACKEND "
|
||||
"or 'docker'). Overrides the env var when set."
|
||||
"or host auto-selection). Overrides the env var when set."
|
||||
),
|
||||
)
|
||||
parser.add_argument(
|
||||
@@ -60,27 +62,21 @@ def cmd_start(argv: list[str]) -> int:
|
||||
|
||||
dry_run = args.dry_run or os.environ.get("BOT_BOTTLE_DRY_RUN") == "1"
|
||||
|
||||
manifest = Manifest.resolve(USER_CWD)
|
||||
manifest = ManifestIndex.resolve(USER_CWD)
|
||||
|
||||
agent_name: str | None = args.name
|
||||
if agent_name is None:
|
||||
agent_name = tui.filter_select(
|
||||
sorted(manifest.agents.keys()),
|
||||
manifest.all_agent_names,
|
||||
title="Select agent",
|
||||
)
|
||||
if agent_name is None:
|
||||
return 0
|
||||
|
||||
backend_name: str | None = args.backend
|
||||
if backend_name is None and "BOT_BOTTLE_BACKEND" not in os.environ:
|
||||
backend_name = tui.filter_select(
|
||||
list(known_backend_names()),
|
||||
title="Select backend",
|
||||
)
|
||||
if backend_name is None:
|
||||
return 0
|
||||
|
||||
label, color = tui.name_color_modal(default_label=agent_name)
|
||||
label, color = _resolve_unique_label(label, color)
|
||||
|
||||
spec = BottleSpec(
|
||||
manifest=manifest,
|
||||
@@ -114,8 +110,8 @@ def prepare_with_preflight(
|
||||
injected callable, prompt y/N via the injected callable.
|
||||
|
||||
`backend_name` selects which backend prepares the plan
|
||||
(`None` → `$BOT_BOTTLE_BACKEND` → `docker`). The CLI passes
|
||||
whatever `--backend` resolved to.
|
||||
(`None` → `$BOT_BOTTLE_BACKEND` → host auto-selection). The CLI
|
||||
passes whatever `--backend` resolved to.
|
||||
|
||||
Returns `(plan, identity)`. `plan` is None on dry-run or
|
||||
operator-N, but `identity` is set as soon as `backend.prepare`
|
||||
@@ -140,6 +136,7 @@ def prepare_with_preflight(
|
||||
def attach_agent(
|
||||
bottle: Bottle, *, remote_control: bool = False, resume: bool = False,
|
||||
agent_provider_template: str = "claude",
|
||||
startup_args: tuple[str, ...] = (),
|
||||
) -> int:
|
||||
"""Run the selected provider CLI inside `bottle` as an
|
||||
interactive session. Blocks until the session ends; returns the
|
||||
@@ -158,6 +155,7 @@ def attach_agent(
|
||||
agent_args = list(runtime.bypass_args)
|
||||
if remote_control:
|
||||
agent_args.extend(runtime.remote_control_args)
|
||||
agent_args.extend(startup_args)
|
||||
if resume:
|
||||
agent_args.extend(runtime.resume_args)
|
||||
return bottle.exec_agent(agent_args, tty=True)
|
||||
@@ -172,7 +170,7 @@ def capture_claude_session_state(identity: str, exit_code: int) -> None:
|
||||
# instead of relying on each agent's transcript layout.
|
||||
if not identity:
|
||||
return
|
||||
snapshot_transcript(identity)
|
||||
# snapshot_transcript(identity)
|
||||
if exit_code != 0:
|
||||
mark_preserved(identity)
|
||||
|
||||
@@ -196,6 +194,21 @@ def _identity_from_plan(plan: object) -> str:
|
||||
return getattr(plan, "slug", "")
|
||||
|
||||
|
||||
def _resolve_unique_label(label: str, color: str) -> tuple[str, str]:
|
||||
"""Re-prompt with a disclaimer until the label's slug is not already
|
||||
in use among running bottles. Passes through unchanged when no
|
||||
collision is found on the first check."""
|
||||
while True:
|
||||
slug_candidate = docker_mod.slugify(label)
|
||||
active_slugs = {a.slug for a in enumerate_active_agents()}
|
||||
if slug_candidate not in active_slugs:
|
||||
return label, color
|
||||
label, color = tui.name_color_modal(
|
||||
default_label=label,
|
||||
disclaimer=f'"{label}" is already in use',
|
||||
)
|
||||
|
||||
|
||||
def _text_prompt_yes() -> bool:
|
||||
"""Default `prompt_yes` for CLI use: reads y/N from the
|
||||
controlling tty via stderr prompt + tty-line read."""
|
||||
@@ -242,6 +255,7 @@ def _launch_bottle(
|
||||
bottle,
|
||||
remote_control=remote_control,
|
||||
agent_provider_template=agent_provider_template,
|
||||
startup_args=plan.agent_provision.startup_args,
|
||||
)
|
||||
info(
|
||||
f"session ended (exit {exit_code}); "
|
||||
|
||||
+86
-22
@@ -3,7 +3,8 @@ act on them (approve / modify / reject).
|
||||
|
||||
Curses-based TUI; modify-then-approve shells out to $EDITOR. The
|
||||
approval handler wires to PRD 0016 (capability-block), which rebuilds
|
||||
the bottle Dockerfile. The egress-block tool was removed in issue #198.
|
||||
the bottle Dockerfile. Egress proposals are queued for operator review
|
||||
as full routes.yaml updates.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -20,12 +21,27 @@ from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
from .. import supervise as _supervise
|
||||
from ..backend.docker.bottle_state import read_metadata
|
||||
from ..backend.docker.capability_apply import (
|
||||
CapabilityApplyError,
|
||||
apply_capability_change,
|
||||
from ..bottle_state import read_metadata
|
||||
# from ..backend.docker.capability_apply import (
|
||||
# CapabilityApplyError,
|
||||
# apply_capability_change,
|
||||
# )
|
||||
from ..backend.docker.egress_apply import (
|
||||
EgressApplyError,
|
||||
applicator as _docker_applicator,
|
||||
)
|
||||
from ..backend.macos_container.egress_apply import (
|
||||
applicator as _macos_applicator,
|
||||
)
|
||||
from ..backend.smolmachines.egress_apply import (
|
||||
applicator as _smolmachines_applicator,
|
||||
)
|
||||
from ..log import Die, error, info
|
||||
|
||||
|
||||
class CapabilityApplyError(RuntimeError):
|
||||
"""Placeholder while capability_apply is disabled."""
|
||||
|
||||
from ..supervise import (
|
||||
COMPONENT_FOR_TOOL,
|
||||
AuditEntry,
|
||||
@@ -35,6 +51,9 @@ from ..supervise import (
|
||||
STATUS_MODIFIED,
|
||||
STATUS_REJECTED,
|
||||
TOOL_CAPABILITY_BLOCK,
|
||||
TOOL_ALLOW,
|
||||
TOOL_EGRESS_BLOCK,
|
||||
TOOL_GITLEAKS_ALLOW,
|
||||
archive_proposal,
|
||||
list_pending_proposals,
|
||||
render_diff,
|
||||
@@ -58,7 +77,17 @@ class QueuedProposal:
|
||||
# Errors any remediation engine may raise. Caught by the TUI key
|
||||
# handlers and surfaced in the status line so a failed apply keeps
|
||||
# the proposal pending rather than crashing curses.
|
||||
ApplyError = (CapabilityApplyError,)
|
||||
ApplyError = (CapabilityApplyError, EgressApplyError)
|
||||
|
||||
|
||||
def apply_routes_change(slug: str, content: str) -> tuple[str, str]:
|
||||
meta = read_metadata(slug)
|
||||
backend = meta.backend if meta is not None else ""
|
||||
if backend == "macos-container":
|
||||
return _macos_applicator.apply_routes_change(slug, content)
|
||||
if backend == "smolmachines":
|
||||
return _smolmachines_applicator.apply_routes_change(slug, content)
|
||||
return _docker_applicator.apply_routes_change(slug, content)
|
||||
|
||||
|
||||
def discover_pending() -> list[QueuedProposal]:
|
||||
@@ -110,6 +139,10 @@ def _detail_lines(
|
||||
def _suffix_for_tool(tool: str) -> str:
|
||||
if tool == TOOL_CAPABILITY_BLOCK:
|
||||
return ".dockerfile"
|
||||
if tool in (TOOL_ALLOW, TOOL_EGRESS_BLOCK):
|
||||
return ".yaml"
|
||||
if tool == TOOL_GITLEAKS_ALLOW:
|
||||
return ".txt"
|
||||
return ".txt"
|
||||
|
||||
|
||||
@@ -127,16 +160,21 @@ def approve(
|
||||
file_to_apply = final_file if final_file is not None else qp.proposal.proposed_file
|
||||
|
||||
diff_before, diff_after = "", ""
|
||||
if qp.proposal.tool == TOOL_CAPABILITY_BLOCK:
|
||||
_meta = read_metadata(qp.proposal.bottle_slug)
|
||||
if _meta is not None and not _meta.compose_project:
|
||||
raise CapabilityApplyError(
|
||||
"capability-block remediation is not supported for smolmachines "
|
||||
"bottles. Reject this proposal or handle the capability change "
|
||||
"manually, then restart the bottle."
|
||||
)
|
||||
diff_before, diff_after = apply_capability_change(
|
||||
qp.proposal.bottle_slug, file_to_apply,
|
||||
# if qp.proposal.tool == TOOL_CAPABILITY_BLOCK:
|
||||
# _meta = read_metadata(qp.proposal.bottle_slug)
|
||||
# if _meta is not None and not _meta.compose_project:
|
||||
# raise CapabilityApplyError(
|
||||
# "capability-block remediation is not supported for smolmachines "
|
||||
# "bottles. Reject this proposal or handle the capability change "
|
||||
# "manually, then restart the bottle."
|
||||
# )
|
||||
# diff_before, diff_after = apply_capability_change(
|
||||
# qp.proposal.bottle_slug, file_to_apply,
|
||||
# )
|
||||
if qp.proposal.tool in (TOOL_ALLOW, TOOL_EGRESS_BLOCK):
|
||||
diff_before, diff_after = apply_routes_change(
|
||||
qp.proposal.bottle_slug,
|
||||
file_to_apply,
|
||||
)
|
||||
|
||||
response = Response(
|
||||
@@ -166,6 +204,23 @@ def reject(qp: QueuedProposal, *, reason: str) -> None:
|
||||
_write_audit(qp, action=STATUS_REJECTED, notes=reason, diff_before="", diff_after="")
|
||||
|
||||
|
||||
def _approve_from_tui(
|
||||
stdscr: "curses._CursesWindow", # type: ignore
|
||||
qp: QueuedProposal,
|
||||
*,
|
||||
final_file: str | None = None,
|
||||
notes: str = "",
|
||||
) -> str:
|
||||
"""Approve from curses, prompting for any tool-specific audit note."""
|
||||
if qp.proposal.tool == TOOL_GITLEAKS_ALLOW and final_file is None:
|
||||
notes = _prompt(stdscr, "allow reason (test fixture/false positive): ")
|
||||
if not notes:
|
||||
return "approve aborted (empty reason)"
|
||||
approve(qp, final_file=final_file, notes=notes)
|
||||
verb = "modified+approved" if final_file is not None else "approved"
|
||||
return _approval_status(qp, verb)
|
||||
|
||||
|
||||
def _write_audit(
|
||||
qp: QueuedProposal,
|
||||
*,
|
||||
@@ -349,18 +404,22 @@ def _main_loop(stdscr: "curses._CursesWindow") -> None: # type: ignore
|
||||
_detail_view(stdscr, qp, green_attr=green_attr)
|
||||
elif key == ord("a"):
|
||||
try:
|
||||
approve(qp)
|
||||
status_line = _approval_status(qp, "approved")
|
||||
status_line = _approve_from_tui(stdscr, qp)
|
||||
except ApplyError as e:
|
||||
status_line = f"apply failed: {e}"
|
||||
elif key == ord("m"):
|
||||
if qp.proposal.tool == TOOL_GITLEAKS_ALLOW:
|
||||
status_line = "modify unavailable for gitleaks-allow"
|
||||
continue
|
||||
edited = _modify(stdscr, qp)
|
||||
if edited is None:
|
||||
status_line = "modify aborted (no change)"
|
||||
else:
|
||||
try:
|
||||
approve(qp, final_file=edited, notes="operator modified before approving")
|
||||
status_line = _approval_status(qp, "modified+approved")
|
||||
status_line = _approve_from_tui(
|
||||
stdscr, qp, final_file=edited,
|
||||
notes="operator modified before approving",
|
||||
)
|
||||
except ApplyError as e:
|
||||
status_line = f"apply failed: {e}"
|
||||
elif key == ord("r"):
|
||||
@@ -458,15 +517,20 @@ def _detail_view(
|
||||
offset = max(0, len(lines) - 1)
|
||||
elif key == ord("a"):
|
||||
try:
|
||||
approve(qp)
|
||||
_approve_from_tui(stdscr, qp)
|
||||
except ApplyError:
|
||||
pass
|
||||
return
|
||||
elif key == ord("m"):
|
||||
if qp.proposal.tool == TOOL_GITLEAKS_ALLOW:
|
||||
return
|
||||
edited = _modify(stdscr, qp)
|
||||
if edited is not None:
|
||||
try:
|
||||
approve(qp, final_file=edited, notes="operator modified before approving")
|
||||
_approve_from_tui(
|
||||
stdscr, qp, final_file=edited,
|
||||
notes="operator modified before approving",
|
||||
)
|
||||
except ApplyError:
|
||||
pass
|
||||
return
|
||||
|
||||
+19
-19
@@ -226,20 +226,15 @@ def _addstr_safe(screen: Any, row: int, col: int, text: str, attr: int = curses.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_ANSI_COLORS = [
|
||||
"red", "green", "blue", "yellow", "magenta", "cyan", "white", "black",
|
||||
"bright-red", "bright-green", "bright-blue", "bright-yellow",
|
||||
"bright-magenta", "bright-cyan", "bright-white", "bright-black",
|
||||
"red", "green", "yellow", "blue", "magenta",
|
||||
]
|
||||
|
||||
_CURSES_COLOR_MAP: dict[str, int] = {
|
||||
"black": curses.COLOR_BLACK,
|
||||
"red": curses.COLOR_RED,
|
||||
"green": curses.COLOR_GREEN,
|
||||
"yellow": curses.COLOR_YELLOW,
|
||||
"blue": curses.COLOR_BLUE,
|
||||
"magenta": curses.COLOR_MAGENTA,
|
||||
"cyan": curses.COLOR_CYAN,
|
||||
"white": curses.COLOR_WHITE,
|
||||
}
|
||||
|
||||
_COLOR_NONE = "(none)"
|
||||
@@ -248,11 +243,15 @@ _COLOR_NONE = "(none)"
|
||||
def name_color_modal(
|
||||
default_label: str,
|
||||
*,
|
||||
disclaimer: str = "",
|
||||
tty_path: str = "/dev/tty",
|
||||
) -> tuple[str, str]:
|
||||
"""Present a two-step curses modal: first edit the agent label,
|
||||
then optionally pick a color.
|
||||
|
||||
``disclaimer`` is shown below the input field — use it to surface
|
||||
an error from a previous attempt (e.g. name already in use).
|
||||
|
||||
Returns ``(label, color)`` where ``color`` is one of the 16 ANSI
|
||||
color name strings or ``""`` for no color. Falls back to
|
||||
``(default_label, "")`` on any error (terminal too small, not a tty).
|
||||
@@ -264,14 +263,14 @@ def name_color_modal(
|
||||
|
||||
try:
|
||||
fd_dup = os.dup(tty_fd.fileno())
|
||||
return _run_name_color(default_label, tty_fd=fd_dup)
|
||||
return _run_name_color(default_label, tty_fd=fd_dup, disclaimer=disclaimer)
|
||||
except Exception: # noqa: BLE001 # pylint: disable=broad-exception-caught
|
||||
return default_label, ""
|
||||
finally:
|
||||
tty_fd.close()
|
||||
|
||||
|
||||
def _run_name_color(default_label: str, *, tty_fd: int) -> tuple[str, str]:
|
||||
def _run_name_color(default_label: str, *, tty_fd: int, disclaimer: str = "") -> tuple[str, str]:
|
||||
import io
|
||||
orig_stdin = sys.__stdin__
|
||||
orig_stdout = sys.__stdout__
|
||||
@@ -286,7 +285,7 @@ def _run_name_color(default_label: str, *, tty_fd: int) -> tuple[str, str]:
|
||||
curses.cbreak()
|
||||
screen.keypad(True)
|
||||
try:
|
||||
label = _label_step(screen, default_label)
|
||||
label = _label_step(screen, default_label, disclaimer=disclaimer)
|
||||
color = _color_step(screen, label)
|
||||
finally:
|
||||
screen.keypad(False)
|
||||
@@ -299,14 +298,14 @@ def _run_name_color(default_label: str, *, tty_fd: int) -> tuple[str, str]:
|
||||
return label, color
|
||||
|
||||
|
||||
def _label_step(screen: Any, default_label: str) -> str:
|
||||
def _label_step(screen: Any, default_label: str, *, disclaimer: str = "") -> str:
|
||||
"""Step 1: edit the label. First printable key replaces the
|
||||
pre-fill; subsequent keys append. Enter confirms."""
|
||||
text = default_label
|
||||
replaced = False # True once the user has typed their first char
|
||||
|
||||
while True:
|
||||
_render_label(screen, text)
|
||||
_render_label(screen, text, disclaimer=disclaimer)
|
||||
try:
|
||||
key = screen.getch()
|
||||
except KeyboardInterrupt:
|
||||
@@ -330,7 +329,7 @@ def _label_step(screen: Any, default_label: str) -> str:
|
||||
text += chr(key)
|
||||
|
||||
|
||||
def _render_label(screen: Any, text: str) -> None:
|
||||
def _render_label(screen: Any, text: str, *, disclaimer: str = "") -> None:
|
||||
screen.erase()
|
||||
rows, cols = screen.getmaxyx()
|
||||
sep = "─" * min(cols - 1, 40)
|
||||
@@ -338,8 +337,12 @@ def _render_label(screen: Any, text: str) -> None:
|
||||
_addstr_safe(screen, 1, 0, sep)
|
||||
_addstr_safe(screen, 2, 0, text[:cols - 1], curses.A_REVERSE)
|
||||
_addstr_safe(screen, 3, 0, sep)
|
||||
if rows > 5:
|
||||
_addstr_safe(screen, 5, 0, "[any key] edit [Enter] confirm", curses.A_DIM)
|
||||
row = 4
|
||||
if disclaimer and rows > row + 1:
|
||||
_addstr_safe(screen, row, 0, disclaimer[:cols - 1], curses.A_BOLD)
|
||||
row += 1
|
||||
if rows > row + 1:
|
||||
_addstr_safe(screen, row, 0, "[any key] edit [Enter] confirm", curses.A_DIM)
|
||||
screen.refresh()
|
||||
|
||||
|
||||
@@ -379,13 +382,10 @@ def _init_color_pairs() -> dict[str, int]:
|
||||
curses.use_default_colors()
|
||||
pair_idx = 2 # pair 1 reserved for other uses
|
||||
for name in _ANSI_COLORS:
|
||||
base = name.replace("bright-", "")
|
||||
fg = _CURSES_COLOR_MAP.get(base, curses.COLOR_WHITE)
|
||||
fg = _CURSES_COLOR_MAP.get(name, curses.COLOR_WHITE)
|
||||
try:
|
||||
curses.init_pair(pair_idx, fg, -1)
|
||||
attr = curses.color_pair(pair_idx)
|
||||
if name.startswith("bright-"):
|
||||
attr |= curses.A_BOLD
|
||||
attr = curses.color_pair(pair_idx) | curses.A_BOLD
|
||||
attrs[name] = attr
|
||||
pair_idx += 1
|
||||
except curses.error:
|
||||
|
||||
@@ -36,7 +36,7 @@ RUN apt-get update \
|
||||
# build (`claude --version` returns 2.1.126). Bump deliberately when
|
||||
# rolling forward; an unpinned install would mean rebuilds silently pick
|
||||
# up new behavior.
|
||||
RUN npm install -g --no-fund --no-audit @anthropic-ai/claude-code@2.1.126 \
|
||||
RUN npm install -g --no-fund --no-audit @anthropic-ai/claude-code@2.1.172 \
|
||||
&& npm cache clean --force
|
||||
|
||||
# Run as a non-root user. The node image already provides a `node` user
|
||||
@@ -17,9 +17,11 @@ from typing import TYPE_CHECKING
|
||||
from ...agent_provider import (
|
||||
AgentProvider,
|
||||
AgentProviderRuntime,
|
||||
AgentProvisionDir,
|
||||
AgentProvisionFile,
|
||||
AgentProvisionPlan,
|
||||
)
|
||||
from ...backend.docker import util as docker_mod
|
||||
from ...egress import EgressRoute
|
||||
from ...log import die, info, warn
|
||||
|
||||
@@ -28,8 +30,6 @@ if TYPE_CHECKING:
|
||||
from ...backend import Bottle, BottlePlan
|
||||
|
||||
|
||||
_REPO_ROOT = Path(__file__).resolve().parents[3]
|
||||
|
||||
_SUPERVISE_MCP_NAME = "supervise"
|
||||
|
||||
|
||||
@@ -40,11 +40,53 @@ def _skills_dir(guest_home: str) -> str:
|
||||
def _prompt_path(guest_home: str) -> str:
|
||||
return f"{guest_home}/.bot-bottle-prompt.txt"
|
||||
|
||||
|
||||
_STATUS_LINE_COLORS = {
|
||||
"red": "\033[91m",
|
||||
"green": "\033[92m",
|
||||
"yellow": "\033[93m",
|
||||
"blue": "\033[94m",
|
||||
"magenta": "\033[95m",
|
||||
}
|
||||
|
||||
_CLAUDE_THEME_COLORS = {
|
||||
"red": "redBright",
|
||||
"green": "greenBright",
|
||||
"yellow": "yellowBright",
|
||||
"blue": "blueBright",
|
||||
"magenta": "magentaBright",
|
||||
}
|
||||
|
||||
|
||||
def _status_line_script(label: str, color: str) -> str:
|
||||
if not label:
|
||||
return "#!/bin/sh\nprintf '\\n'\n"
|
||||
label_q = shlex.quote(label)
|
||||
if color and color in _STATUS_LINE_COLORS:
|
||||
return (
|
||||
"#!/bin/sh\n"
|
||||
f"printf '%b%s%b\\n' '{_STATUS_LINE_COLORS[color]}' {label_q} '\\033[0m'\n"
|
||||
)
|
||||
return f"#!/bin/sh\nprintf '%s\\n' {label_q}\n"
|
||||
|
||||
|
||||
def _custom_theme_payload(color: str) -> dict[str, object] | None:
|
||||
theme_color = _CLAUDE_THEME_COLORS.get(color)
|
||||
if not theme_color:
|
||||
return None
|
||||
return {
|
||||
"name": f"Bot-bottle {color}",
|
||||
"base": "dark",
|
||||
"overrides": {
|
||||
"claude": f"ansi:{theme_color}",
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
_RUNTIME = AgentProviderRuntime(
|
||||
template="claude",
|
||||
command="claude",
|
||||
image="bot-bottle-claude:latest",
|
||||
dockerfile=str(_REPO_ROOT / "Dockerfile.claude"),
|
||||
prompt_mode="append_file",
|
||||
bypass_args=("--dangerously-skip-permissions",),
|
||||
resume_args=("--continue",),
|
||||
@@ -62,7 +104,8 @@ class ClaudeAgentProvider(AgentProvider):
|
||||
*,
|
||||
dockerfile: str,
|
||||
state_dir: Path,
|
||||
guest_home: str,
|
||||
instance_name: str,
|
||||
prompt_file: Path,
|
||||
guest_env: dict[str, str] | None = None,
|
||||
auth_token: str = "",
|
||||
forward_host_credentials: bool = False,
|
||||
@@ -70,15 +113,21 @@ class ClaudeAgentProvider(AgentProvider):
|
||||
trusted_project_path: str = "",
|
||||
label: str = "",
|
||||
color: str = "",
|
||||
provider_settings: dict[str, object] | None = None,
|
||||
) -> AgentProvisionPlan:
|
||||
del forward_host_credentials, host_env # Codex-only knobs
|
||||
del forward_host_credentials, host_env, provider_settings
|
||||
resolved_guest_env = dict(guest_env or {})
|
||||
guest_home = self.guest_home
|
||||
trusted_path = trusted_project_path or guest_home
|
||||
|
||||
env_vars: dict[str, str] = {
|
||||
"CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC": "1",
|
||||
"DISABLE_ERROR_REPORTING": "1",
|
||||
}
|
||||
dirs = (
|
||||
AgentProvisionDir(f"{guest_home}/.claude"),
|
||||
AgentProvisionDir(f"{guest_home}/.claude/themes"),
|
||||
)
|
||||
claude_config = state_dir / "claude.json"
|
||||
claude_projects = {guest_home: {"hasTrustDialogAccepted": True}}
|
||||
claude_projects[trusted_path] = {"hasTrustDialogAccepted": True}
|
||||
@@ -88,15 +137,45 @@ class ClaudeAgentProvider(AgentProvider):
|
||||
"bypassPermissionsModeAccepted": True,
|
||||
"projects": claude_projects,
|
||||
}
|
||||
if label:
|
||||
payload["name"] = label
|
||||
if color:
|
||||
payload["color"] = color
|
||||
claude_config.write_text(json.dumps(payload, indent=2) + "\n")
|
||||
claude_config.chmod(0o600)
|
||||
files = (
|
||||
files = [
|
||||
AgentProvisionFile(claude_config, f"{guest_home}/.claude.json"),
|
||||
)
|
||||
]
|
||||
|
||||
claude_settings = state_dir / "claude-settings.json"
|
||||
claude_settings_payload: dict[str, object] = {}
|
||||
if label or color:
|
||||
statusline_script = state_dir / "claude-statusline.sh"
|
||||
statusline_script.write_text(_status_line_script(label, color))
|
||||
statusline_script.chmod(0o755)
|
||||
files.append(AgentProvisionFile(
|
||||
statusline_script,
|
||||
f"{guest_home}/.claude/statusline.sh",
|
||||
mode="755",
|
||||
))
|
||||
claude_settings_payload["statusLine"] = {
|
||||
"type": "command",
|
||||
"command": "~/.claude/statusline.sh",
|
||||
}
|
||||
theme_payload = _custom_theme_payload(color)
|
||||
if theme_payload is not None:
|
||||
theme_name = f"bot-bottle-{docker_mod.slugify(label or color)}"
|
||||
theme_file = state_dir / f"{theme_name}.json"
|
||||
theme_file.write_text(json.dumps(theme_payload, indent=2) + "\n")
|
||||
theme_file.chmod(0o644)
|
||||
files.append(AgentProvisionFile(
|
||||
theme_file,
|
||||
f"{guest_home}/.claude/themes/{theme_name}.json",
|
||||
))
|
||||
claude_settings_payload["theme"] = f"custom:{theme_name}"
|
||||
if claude_settings_payload:
|
||||
claude_settings.write_text(json.dumps(claude_settings_payload, indent=2) + "\n")
|
||||
claude_settings.chmod(0o600)
|
||||
files.append(AgentProvisionFile(
|
||||
claude_settings,
|
||||
f"{guest_home}/.claude/settings.json",
|
||||
))
|
||||
egress_routes = (EgressRoute(
|
||||
host="api.anthropic.com",
|
||||
auth_scheme="Bearer" if auth_token else "",
|
||||
@@ -107,15 +186,21 @@ class ClaudeAgentProvider(AgentProvider):
|
||||
env_vars["CLAUDE_CODE_OAUTH_TOKEN"] = "egress-placeholder"
|
||||
hidden_env_names = frozenset({"CLAUDE_CODE_OAUTH_TOKEN"})
|
||||
|
||||
has_prompt = prompt_file.exists() and bool(prompt_file.read_text())
|
||||
return AgentProvisionPlan(
|
||||
template=_RUNTIME.template,
|
||||
command=_RUNTIME.command,
|
||||
prompt_mode=_RUNTIME.prompt_mode,
|
||||
image=_RUNTIME.image,
|
||||
dockerfile=dockerfile,
|
||||
guest_home=guest_home,
|
||||
instance_name=instance_name,
|
||||
prompt_file=prompt_file,
|
||||
env_vars=env_vars,
|
||||
guest_env=resolved_guest_env,
|
||||
files=files,
|
||||
has_prompt=has_prompt,
|
||||
dirs=dirs,
|
||||
files=tuple(files),
|
||||
egress_routes=egress_routes,
|
||||
hidden_env_names=hidden_env_names,
|
||||
)
|
||||
@@ -126,7 +211,7 @@ class ClaudeAgentProvider(AgentProvider):
|
||||
when the agent has no skills."""
|
||||
from ...backend.util import host_skill_dir
|
||||
|
||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||
agent = plan.manifest.agent
|
||||
if not agent.skills:
|
||||
return
|
||||
skills_dir = _skills_dir(plan.guest_home)
|
||||
@@ -155,8 +240,8 @@ class ClaudeAgentProvider(AgentProvider):
|
||||
f"chown node:node {prompt_path} && chmod 600 {prompt_path}",
|
||||
user="root",
|
||||
)
|
||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||
return prompt_path if agent.prompt else None
|
||||
agent = plan.manifest.agent
|
||||
return prompt_path if plan.agent_provision.has_prompt or agent.prompt else None
|
||||
|
||||
def provision(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
||||
"""Apply the claude-side declarative provision steps from
|
||||
|
||||
@@ -18,8 +18,8 @@ from ...agent_provider import (
|
||||
CODEX_HOST_CREDENTIAL_HOSTS,
|
||||
AgentProvider,
|
||||
AgentProviderRuntime,
|
||||
AgentProvisionCommand,
|
||||
AgentProvisionDir,
|
||||
AgentProvisionCommand,
|
||||
AgentProvisionFile,
|
||||
AgentProvisionPlan,
|
||||
)
|
||||
@@ -32,8 +32,6 @@ if TYPE_CHECKING:
|
||||
from ...backend import Bottle, BottlePlan
|
||||
|
||||
|
||||
_REPO_ROOT = Path(__file__).resolve().parents[3]
|
||||
|
||||
_SUPERVISE_MCP_NAME = "supervise"
|
||||
|
||||
|
||||
@@ -48,11 +46,11 @@ def _skills_dir(guest_home: str) -> str:
|
||||
def _prompt_path(guest_home: str) -> str:
|
||||
return f"{guest_home}/.bot-bottle-prompt.txt"
|
||||
|
||||
|
||||
_RUNTIME = AgentProviderRuntime(
|
||||
template="codex",
|
||||
command="codex",
|
||||
image="bot-bottle-codex:latest",
|
||||
dockerfile=str(_REPO_ROOT / "Dockerfile.codex"),
|
||||
prompt_mode="read_prompt_file",
|
||||
bypass_args=("--dangerously-bypass-approvals-and-sandbox",),
|
||||
resume_args=("resume", "--last"),
|
||||
@@ -70,7 +68,8 @@ class CodexAgentProvider(AgentProvider):
|
||||
*,
|
||||
dockerfile: str,
|
||||
state_dir: Path,
|
||||
guest_home: str,
|
||||
instance_name: str,
|
||||
prompt_file: Path,
|
||||
guest_env: dict[str, str] | None = None,
|
||||
auth_token: str = "",
|
||||
forward_host_credentials: bool = False,
|
||||
@@ -78,9 +77,11 @@ class CodexAgentProvider(AgentProvider):
|
||||
trusted_project_path: str = "",
|
||||
label: str = "",
|
||||
color: str = "",
|
||||
provider_settings: dict[str, object] | None = None,
|
||||
) -> AgentProvisionPlan:
|
||||
del auth_token, label, color # Claude-only knobs
|
||||
del auth_token, label, color, provider_settings
|
||||
resolved_guest_env = dict(guest_env or {})
|
||||
guest_home = self.guest_home
|
||||
trusted_path = trusted_project_path or guest_home
|
||||
|
||||
env_vars: dict[str, str] = {
|
||||
@@ -102,6 +103,11 @@ class CodexAgentProvider(AgentProvider):
|
||||
config_file.write_text(
|
||||
f'[projects."{toml_path}"]\n'
|
||||
'trust_level = "trusted"\n'
|
||||
"\n"
|
||||
"[tui]\n"
|
||||
'status_line = ["model-with-reasoning"]\n'
|
||||
'terminal_title = ["spinner", "project"]\n'
|
||||
'theme = "ansi"\n'
|
||||
)
|
||||
config_file.chmod(0o600)
|
||||
files.append(AgentProvisionFile(config_file, config_path))
|
||||
@@ -144,14 +150,19 @@ class CodexAgentProvider(AgentProvider):
|
||||
"guest, but Codex did not accept it"
|
||||
)))
|
||||
|
||||
has_prompt = prompt_file.exists() and bool(prompt_file.read_text())
|
||||
return AgentProvisionPlan(
|
||||
template=_RUNTIME.template,
|
||||
command=_RUNTIME.command,
|
||||
prompt_mode=_RUNTIME.prompt_mode,
|
||||
image=_RUNTIME.image,
|
||||
dockerfile=dockerfile,
|
||||
guest_home=guest_home,
|
||||
instance_name=instance_name,
|
||||
prompt_file=prompt_file,
|
||||
env_vars=env_vars,
|
||||
guest_env=resolved_guest_env,
|
||||
has_prompt=has_prompt,
|
||||
dirs=tuple(dirs),
|
||||
files=tuple(files),
|
||||
pre_copy=tuple(pre_copy),
|
||||
@@ -166,7 +177,7 @@ class CodexAgentProvider(AgentProvider):
|
||||
skills."""
|
||||
from ...backend.util import host_skill_dir
|
||||
|
||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||
agent = plan.manifest.agent
|
||||
if not agent.skills:
|
||||
return
|
||||
skills_dir = _skills_dir(plan.guest_home)
|
||||
@@ -195,8 +206,8 @@ class CodexAgentProvider(AgentProvider):
|
||||
f"chown node:node {prompt_path} && chmod 600 {prompt_path}",
|
||||
user="root",
|
||||
)
|
||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||
return prompt_path if agent.prompt else None
|
||||
agent = plan.manifest.agent
|
||||
return prompt_path if plan.agent_provision.has_prompt or agent.prompt else None
|
||||
|
||||
def provision(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
||||
"""Apply the codex-side declarative provision steps from
|
||||
@@ -250,8 +261,8 @@ class CodexAgentProvider(AgentProvider):
|
||||
return
|
||||
info(f"registering supervise MCP server in agent codex config → {supervise_url}")
|
||||
r = bottle.exec(
|
||||
f"codex mcp add --transport http "
|
||||
f"{_SUPERVISE_MCP_NAME} {supervise_url}",
|
||||
f"codex mcp add {_SUPERVISE_MCP_NAME} --url "
|
||||
f"{shlex.quote(supervise_url)}",
|
||||
user="node",
|
||||
)
|
||||
if r.returncode != 0:
|
||||
@@ -259,7 +270,7 @@ class CodexAgentProvider(AgentProvider):
|
||||
f"`codex mcp add supervise` failed (exit {r.returncode}): "
|
||||
f"{(r.stderr or r.stdout or '').strip()}. Inside the bottle, "
|
||||
f"register manually with: "
|
||||
f"codex mcp add --transport http supervise {supervise_url}"
|
||||
f"codex mcp add supervise --url {shlex.quote(supervise_url)}"
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -2,7 +2,13 @@
|
||||
|
||||
Generates ed25519 keypairs via `ssh-keygen` and registers / deletes
|
||||
them using the Gitea deploy-key HTTP API. No new Python dependencies —
|
||||
only stdlib `urllib.request` and `subprocess`."""
|
||||
only stdlib `urllib.request` and `subprocess`.
|
||||
|
||||
Required token permissions (Gitea "Applications" → "Generate Token"):
|
||||
- Repository: Read & Write
|
||||
Grants POST /api/v1/repos/{owner}/{repo}/keys (create deploy key)
|
||||
and DELETE /api/v1/repos/{owner}/{repo}/keys/{id} (revoke deploy key).
|
||||
No other scopes are needed."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
|
||||
@@ -0,0 +1,41 @@
|
||||
# bot-bottle Pi provider image.
|
||||
#
|
||||
# Node LTS, git/network tooling, and the Pi coding-agent CLI installed globally.
|
||||
|
||||
FROM node:22-slim
|
||||
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends \
|
||||
git \
|
||||
ca-certificates \
|
||||
curl \
|
||||
fd-find \
|
||||
ripgrep \
|
||||
&& ln -s /usr/bin/fdfind /usr/local/bin/fd \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends python3 python3-pip python3-venv \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
RUN npm install -g --ignore-scripts --no-fund --no-audit @earendil-works/pi-coding-agent \
|
||||
&& npm cache clean --force
|
||||
|
||||
RUN mkdir -p /home/node/.pi/agent \
|
||||
/home/node/.pi/context-mode/sessions \
|
||||
/tmp/pi-subagents-uid-1000 \
|
||||
&& chown -R node:node /home/node/.pi /tmp \
|
||||
&& chmod -R u+rwX /tmp \
|
||||
&& chown root:root /tmp /var/tmp \
|
||||
&& chmod 1777 /tmp /var/tmp
|
||||
|
||||
USER node
|
||||
WORKDIR /home/node
|
||||
|
||||
RUN pi install npm:@harms-haus/pi-cwd \
|
||||
&& pi install npm:pi-web-access \
|
||||
&& pi install npm:context-mode \
|
||||
&& pi install npm:pi-subagents \
|
||||
&& pi install npm:pi-mcp-adapter
|
||||
|
||||
CMD ["pi"]
|
||||
@@ -0,0 +1 @@
|
||||
"""Pi agent provider package."""
|
||||
@@ -0,0 +1,319 @@
|
||||
"""Pi agent provider plugin (PRD 0058, contrib).
|
||||
|
||||
Pi uses ~/.pi/agent/models.json for custom provider/model settings.
|
||||
This provider writes an Ollama-compatible default configuration and
|
||||
lets bottles override the model endpoint and model ids via
|
||||
agent_provider.settings.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import shlex
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING
|
||||
from urllib.parse import urlparse
|
||||
|
||||
from ...agent_provider import (
|
||||
AgentProvider,
|
||||
AgentProviderRuntime,
|
||||
AgentProvisionDir,
|
||||
AgentProvisionFile,
|
||||
AgentProvisionPlan,
|
||||
)
|
||||
from ...egress import EgressRoute
|
||||
from ...log import die, info
|
||||
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from ...backend import Bottle, BottlePlan
|
||||
|
||||
|
||||
_DEFAULT_BASE_URL = "http://ollama:11434/v1"
|
||||
_DEFAULT_MODEL = "qwen2.5-coder:7b"
|
||||
_DEFAULT_PROVIDER_NAME = "ollama"
|
||||
_DEFAULT_CONTEXT_WINDOW = 4096
|
||||
_DEFAULT_MAX_TOKENS = 1024
|
||||
|
||||
|
||||
def _skills_dir(guest_home: str) -> str:
|
||||
return f"{guest_home}/.pi/agent/skills"
|
||||
|
||||
|
||||
def _prompt_path(guest_home: str) -> str:
|
||||
return f"{guest_home}/.bot-bottle-prompt.txt"
|
||||
|
||||
|
||||
def _append_system_path(guest_home: str) -> str:
|
||||
return f"{guest_home}/.pi/agent/APPEND_SYSTEM.md"
|
||||
|
||||
|
||||
def _models_path(guest_home: str) -> str:
|
||||
return f"{guest_home}/.pi/agent/models.json"
|
||||
|
||||
|
||||
def _runtime_state_repair_script(guest_home: str) -> str:
|
||||
home = shlex.quote(guest_home)
|
||||
pi_home = shlex.quote(f"{guest_home}/.pi")
|
||||
context_sessions = shlex.quote(f"{guest_home}/.pi/context-mode/sessions")
|
||||
return (
|
||||
f"mkdir -p {context_sessions} /tmp/pi-subagents-uid-1000 && "
|
||||
f"chown node:node {home} && "
|
||||
f"chown -R node:node {pi_home} /tmp && "
|
||||
"chmod -R u+rwX /tmp && "
|
||||
f"chmod 755 {home} && "
|
||||
"chown root:root /tmp /var/tmp && "
|
||||
"chmod 1777 /tmp /var/tmp"
|
||||
)
|
||||
|
||||
|
||||
def _settings_value(
|
||||
settings: dict[str, object],
|
||||
key: str,
|
||||
default: object,
|
||||
) -> object:
|
||||
value = settings.get(key)
|
||||
return default if value is None else value
|
||||
|
||||
|
||||
def _settings_int(
|
||||
settings: dict[str, object],
|
||||
key: str,
|
||||
default: int,
|
||||
) -> int:
|
||||
value = _settings_value(settings, key, default)
|
||||
if isinstance(value, bool):
|
||||
return default
|
||||
if isinstance(value, (int, str)):
|
||||
return int(value)
|
||||
return default
|
||||
|
||||
|
||||
def _pi_models_json(
|
||||
settings: dict[str, object],
|
||||
) -> tuple[dict[str, object], str, str, list[str], str]:
|
||||
provider_name = str(
|
||||
_settings_value(settings, "provider", _DEFAULT_PROVIDER_NAME)
|
||||
)
|
||||
base_url = str(_settings_value(settings, "base_url", _DEFAULT_BASE_URL))
|
||||
api = str(_settings_value(settings, "api", "openai-completions"))
|
||||
api_key = settings.get("api_key")
|
||||
api_key_env = str(settings.get("api_key_env", ""))
|
||||
models_raw = _settings_value(settings, "models", [_DEFAULT_MODEL])
|
||||
models = [str(model) for model in models_raw] # type: ignore[union-attr]
|
||||
supports_developer_role = bool(
|
||||
_settings_value(settings, "supports_developer_role", False)
|
||||
)
|
||||
supports_reasoning_effort = bool(
|
||||
_settings_value(settings, "supports_reasoning_effort", False)
|
||||
)
|
||||
max_tokens_field = str(
|
||||
_settings_value(settings, "max_tokens_field", "max_tokens")
|
||||
)
|
||||
context_window = _settings_int(
|
||||
settings, "context_window", _DEFAULT_CONTEXT_WINDOW,
|
||||
)
|
||||
max_tokens = _settings_int(settings, "max_tokens", _DEFAULT_MAX_TOKENS)
|
||||
input_context_window = max(1, context_window - max_tokens)
|
||||
provider: dict[str, object] = {
|
||||
"baseUrl": base_url,
|
||||
"api": api,
|
||||
"compat": {
|
||||
"supportsDeveloperRole": supports_developer_role,
|
||||
"supportsReasoningEffort": supports_reasoning_effort,
|
||||
"maxTokensField": max_tokens_field,
|
||||
},
|
||||
"models": [
|
||||
{
|
||||
"id": model,
|
||||
"name": model,
|
||||
"contextWindow": input_context_window,
|
||||
"maxTokens": max_tokens,
|
||||
}
|
||||
for model in models
|
||||
],
|
||||
}
|
||||
if api_key is not None:
|
||||
provider["apiKey"] = str(api_key)
|
||||
elif api_key_env:
|
||||
provider["apiKey"] = "egress-placeholder"
|
||||
elif provider_name == _DEFAULT_PROVIDER_NAME:
|
||||
provider["apiKey"] = "ollama"
|
||||
payload: dict[str, object] = {
|
||||
"providers": {
|
||||
provider_name: provider,
|
||||
}
|
||||
}
|
||||
return payload, base_url, api_key_env, models, provider_name
|
||||
|
||||
|
||||
def _route_host(base_url: str) -> str:
|
||||
parsed = urlparse(base_url)
|
||||
if not parsed.scheme or not parsed.hostname:
|
||||
die(
|
||||
"agent provider provisioning: pi settings base_url must be an "
|
||||
f"absolute URL (was {base_url!r})"
|
||||
)
|
||||
return parsed.hostname
|
||||
|
||||
|
||||
_RUNTIME = AgentProviderRuntime(
|
||||
template="pi",
|
||||
command="pi",
|
||||
image="bot-bottle-pi:latest",
|
||||
prompt_mode="append_system_prompt",
|
||||
bypass_args=(),
|
||||
resume_args=(),
|
||||
remote_control_args=(),
|
||||
)
|
||||
|
||||
|
||||
class PiAgentProvider(AgentProvider):
|
||||
@property
|
||||
def runtime(self) -> AgentProviderRuntime:
|
||||
return _RUNTIME
|
||||
|
||||
def provision_plan(
|
||||
self,
|
||||
*,
|
||||
dockerfile: str,
|
||||
state_dir: Path,
|
||||
instance_name: str,
|
||||
prompt_file: Path,
|
||||
guest_env: dict[str, str] | None = None,
|
||||
auth_token: str = "",
|
||||
forward_host_credentials: bool = False,
|
||||
host_env: dict[str, str] | None = None,
|
||||
trusted_project_path: str = "",
|
||||
label: str = "",
|
||||
color: str = "",
|
||||
provider_settings: dict[str, object] | None = None,
|
||||
) -> AgentProvisionPlan:
|
||||
del auth_token, forward_host_credentials, host_env, trusted_project_path
|
||||
del label, color
|
||||
resolved_guest_env = dict(guest_env or {})
|
||||
guest_home = self.guest_home
|
||||
settings = dict(provider_settings or {})
|
||||
|
||||
models_payload, base_url, api_key_env, models, provider_name = (
|
||||
_pi_models_json(settings)
|
||||
)
|
||||
models_file = state_dir / "pi-models.json"
|
||||
models_file.write_text(json.dumps(models_payload, indent=2) + "\n")
|
||||
models_file.chmod(0o600)
|
||||
|
||||
has_prompt = prompt_file.exists() and bool(prompt_file.read_text())
|
||||
auth_scheme = "Bearer" if api_key_env else ""
|
||||
return AgentProvisionPlan(
|
||||
template=_RUNTIME.template,
|
||||
command=_RUNTIME.command,
|
||||
prompt_mode=_RUNTIME.prompt_mode,
|
||||
image=_RUNTIME.image,
|
||||
dockerfile=dockerfile,
|
||||
guest_home=guest_home,
|
||||
instance_name=instance_name,
|
||||
prompt_file=prompt_file,
|
||||
guest_env=resolved_guest_env,
|
||||
has_prompt=has_prompt,
|
||||
startup_args=(
|
||||
"--models",
|
||||
",".join(f"{provider_name}/{model}" for model in models),
|
||||
),
|
||||
dirs=(AgentProvisionDir(f"{guest_home}/.pi/agent"),),
|
||||
files=(AgentProvisionFile(models_file, _models_path(guest_home)),),
|
||||
egress_routes=(EgressRoute(
|
||||
host=_route_host(base_url),
|
||||
auth_scheme=auth_scheme,
|
||||
token_ref=api_key_env,
|
||||
),),
|
||||
)
|
||||
|
||||
def provision_skills(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
||||
from ...backend.util import host_skill_dir
|
||||
|
||||
agent = plan.manifest.agent
|
||||
if not agent.skills:
|
||||
return
|
||||
skills_dir = _skills_dir(plan.guest_home)
|
||||
bottle.exec(f"mkdir -p {skills_dir}", user="root")
|
||||
for name in agent.skills:
|
||||
src = host_skill_dir(name)
|
||||
if not os.path.isdir(src):
|
||||
die(
|
||||
f"skill {name!r} disappeared from host between "
|
||||
f"validation and copy at {src}."
|
||||
)
|
||||
dst = f"{skills_dir}/{name}"
|
||||
info(f"copying skill {name} into {bottle.name}:{dst}")
|
||||
bottle.exec(f"rm -rf {dst} && mkdir -p {dst}", user="root")
|
||||
bottle.cp_in(f"{src}/.", f"{dst}/")
|
||||
bottle.exec(f"chown -R node:node {dst}", user="root")
|
||||
|
||||
def provision_prompt(self, plan: "BottlePlan", bottle: "Bottle") -> str | None:
|
||||
prompt_path = _prompt_path(plan.guest_home)
|
||||
append_system_path = _append_system_path(plan.guest_home)
|
||||
bottle.cp_in(str(plan.prompt_file), prompt_path) # type: ignore
|
||||
bottle.exec(
|
||||
f"mkdir -p {shlex.quote(plan.guest_home)}/.pi/agent && "
|
||||
f"cp {shlex.quote(prompt_path)} {shlex.quote(append_system_path)} && "
|
||||
f"chown node:node {shlex.quote(prompt_path)} "
|
||||
f"{shlex.quote(append_system_path)} && "
|
||||
f"chmod 600 {shlex.quote(prompt_path)} "
|
||||
f"{shlex.quote(append_system_path)}",
|
||||
user="root",
|
||||
)
|
||||
# Pi's `--append-system-prompt` takes literal text, not a file path.
|
||||
# Use its documented APPEND_SYSTEM.md discovery path instead.
|
||||
return None
|
||||
|
||||
def provision(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
||||
provision = plan.agent_provision
|
||||
_exec(
|
||||
bottle,
|
||||
_runtime_state_repair_script(plan.guest_home),
|
||||
"could not prepare pi runtime state",
|
||||
)
|
||||
for d in provision.dirs:
|
||||
path = shlex.quote(d.guest_path)
|
||||
_exec(bottle, f"mkdir -p {path}", f"could not create {d.guest_path}")
|
||||
_exec(
|
||||
bottle,
|
||||
f"chown {shlex.quote(d.owner)} {path}",
|
||||
f"could not chown {d.guest_path}",
|
||||
)
|
||||
_exec(
|
||||
bottle,
|
||||
f"chmod {shlex.quote(d.mode)} {path}",
|
||||
f"could not chmod {d.guest_path}",
|
||||
)
|
||||
for f in provision.files:
|
||||
bottle.cp_in(str(f.host_path), f.guest_path)
|
||||
path = shlex.quote(f.guest_path)
|
||||
_exec(
|
||||
bottle,
|
||||
f"chown {shlex.quote(f.owner)} {path}",
|
||||
f"could not chown {f.guest_path}",
|
||||
)
|
||||
_exec(
|
||||
bottle,
|
||||
f"chmod {shlex.quote(f.mode)} {path}",
|
||||
f"could not chmod {f.guest_path}",
|
||||
)
|
||||
|
||||
def provision_supervise_mcp(
|
||||
self,
|
||||
plan: "BottlePlan",
|
||||
bottle: "Bottle",
|
||||
supervise_url: str,
|
||||
) -> None:
|
||||
del plan, bottle, supervise_url
|
||||
|
||||
|
||||
def _exec(bottle: "Bottle", script: str, error: str) -> None:
|
||||
result = bottle.exec(script, user="root")
|
||||
if result.returncode != 0:
|
||||
detail = (result.stderr or result.stdout).strip()
|
||||
if detail:
|
||||
detail = f": {detail}"
|
||||
die(f"agent provider provisioning: {error}{detail}")
|
||||
+15
-5
@@ -24,13 +24,14 @@ from .egress_addon_core import (
|
||||
from .log import die
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from .manifest import Bottle
|
||||
from .manifest import ManifestBottle
|
||||
|
||||
CODEX_HOST_CREDENTIAL_TOKEN_REF = "BOT_BOTTLE_CODEX_HOST_ACCESS_TOKEN"
|
||||
|
||||
EGRESS_HOSTNAME = "egress"
|
||||
|
||||
EGRESS_ROUTES_IN_CONTAINER = "/etc/egress/routes.yaml"
|
||||
EGRESS_ROUTES_FILENAME = Path(EGRESS_ROUTES_IN_CONTAINER).name
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -66,7 +67,7 @@ class EgressPlan:
|
||||
|
||||
|
||||
def egress_manifest_routes(
|
||||
bottle: Bottle,
|
||||
bottle: ManifestBottle,
|
||||
) -> tuple[EgressRoute, ...]:
|
||||
out: list[EgressRoute] = []
|
||||
for r in bottle.egress.routes:
|
||||
@@ -91,6 +92,7 @@ def egress_manifest_routes(
|
||||
auth_scheme=r.AuthScheme,
|
||||
token_ref=r.TokenRef,
|
||||
roles=r.Role,
|
||||
git_fetch=r.GitFetch,
|
||||
outbound_detectors=r.OutboundDetectors,
|
||||
inbound_detectors=r.InboundDetectors,
|
||||
))
|
||||
@@ -98,7 +100,7 @@ def egress_manifest_routes(
|
||||
|
||||
|
||||
def egress_routes_for_bottle(
|
||||
bottle: Bottle,
|
||||
bottle: ManifestBottle,
|
||||
provider_routes: tuple[EgressRoute, ...] = (),
|
||||
) -> tuple[EgressRoute, ...]:
|
||||
manifest = egress_manifest_routes(bottle)
|
||||
@@ -173,6 +175,8 @@ def _route_to_yaml_fields(r: Route) -> dict[str, object]:
|
||||
entry_data["headers"] = headers_data
|
||||
matches_data.append(entry_data)
|
||||
fields["matches"] = matches_data
|
||||
if r.git_fetch:
|
||||
fields["git"] = {"fetch": True}
|
||||
if r.outbound_detectors is not None or r.inbound_detectors is not None:
|
||||
dlp: dict[str, object] = {}
|
||||
if r.outbound_detectors is not None:
|
||||
@@ -242,6 +246,11 @@ def egress_render_routes(
|
||||
lines.append(" matches:")
|
||||
for entry in f["matches"]: # type: ignore[union-attr]
|
||||
lines.extend(_render_match_entry(entry)) # type: ignore[arg-type]
|
||||
if "git" in f:
|
||||
git_dict: dict[str, object] = f["git"] # type: ignore
|
||||
lines.append(" git:")
|
||||
if git_dict.get("fetch") is True:
|
||||
lines.append(" fetch: true")
|
||||
if "dlp" in f:
|
||||
dlp_dict: dict[str, object] = f["dlp"] # type: ignore
|
||||
lines.append(" dlp:")
|
||||
@@ -280,14 +289,14 @@ def egress_resolve_token_values(
|
||||
class Egress(ABC):
|
||||
def prepare(
|
||||
self,
|
||||
bottle: Bottle,
|
||||
bottle: ManifestBottle,
|
||||
slug: str,
|
||||
stage_dir: Path,
|
||||
provider_routes: tuple[EgressRoute, ...] = (),
|
||||
) -> EgressPlan:
|
||||
routes = egress_routes_for_bottle(bottle, provider_routes)
|
||||
log = bottle.egress.Log
|
||||
routes_path = stage_dir / "egress_routes.yaml"
|
||||
routes_path = stage_dir / EGRESS_ROUTES_FILENAME
|
||||
routes_path.write_text(egress_render_routes(routes, log=log))
|
||||
routes_path.chmod(0o600)
|
||||
return EgressPlan(
|
||||
@@ -301,6 +310,7 @@ class Egress(ABC):
|
||||
__all__ = [
|
||||
"CODEX_HOST_CREDENTIAL_TOKEN_REF",
|
||||
"EGRESS_HOSTNAME",
|
||||
"EGRESS_ROUTES_FILENAME",
|
||||
"EGRESS_ROUTES_IN_CONTAINER",
|
||||
"Egress",
|
||||
"EgressPlan",
|
||||
|
||||
@@ -5,7 +5,6 @@ egress container."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import dataclasses
|
||||
import json
|
||||
import os
|
||||
import signal
|
||||
@@ -21,9 +20,13 @@ from egress_addon_core import ( # type: ignore[import-not-found] # pylint: dis
|
||||
build_inbound_scan_text,
|
||||
build_outbound_scan_text,
|
||||
decide,
|
||||
decide_git_fetch,
|
||||
is_git_fetch_request,
|
||||
is_git_push_request,
|
||||
load_config,
|
||||
match_route,
|
||||
outbound_scan_headers,
|
||||
route_to_yaml_dict,
|
||||
scan_inbound,
|
||||
scan_outbound,
|
||||
)
|
||||
@@ -79,7 +82,7 @@ class EgressAddon:
|
||||
def _serve_introspection(self, flow: http.HTTPFlow, path: str) -> None:
|
||||
if path == "/allowlist":
|
||||
payload = json.dumps(
|
||||
{"routes": [dataclasses.asdict(r) for r in self.config.routes]},
|
||||
{"routes": [route_to_yaml_dict(r) for r in self.config.routes]},
|
||||
indent=2,
|
||||
).encode("utf-8")
|
||||
flow.response = http.Response.make(
|
||||
@@ -159,7 +162,7 @@ class EgressAddon:
|
||||
flow.request.pretty_host,
|
||||
request_path,
|
||||
query,
|
||||
dict(flow.request.headers),
|
||||
outbound_scan_headers(route, dict(flow.request.headers)),
|
||||
body,
|
||||
)
|
||||
dlp_result = scan_outbound(route, scan_text, os.environ)
|
||||
@@ -180,6 +183,18 @@ class EgressAddon:
|
||||
)
|
||||
return
|
||||
|
||||
if is_git_fetch_request(request_path, query):
|
||||
git_decision = decide_git_fetch(
|
||||
self.config.routes, flow.request.pretty_host,
|
||||
)
|
||||
if git_decision.action == "block":
|
||||
self._block(
|
||||
flow,
|
||||
git_decision.reason,
|
||||
ctx=self._req_ctx(flow),
|
||||
)
|
||||
return
|
||||
|
||||
# Strip agent-set Authorization after DLP scan so smuggled tokens
|
||||
# are caught above; the route may inject sidecar-owned auth below.
|
||||
flow.request.headers.pop("authorization", None)
|
||||
|
||||
@@ -66,6 +66,7 @@ class Route:
|
||||
matches: tuple[MatchEntry, ...] = ()
|
||||
auth_scheme: str = ""
|
||||
token_env: str = ""
|
||||
git_fetch: bool = False
|
||||
outbound_detectors: tuple[str, ...] | None = None
|
||||
inbound_detectors: tuple[str, ...] | None = None
|
||||
|
||||
@@ -316,16 +317,35 @@ def _parse_one(idx: int, raw: object) -> Route:
|
||||
f"token_env={token_env!r})"
|
||||
)
|
||||
|
||||
# git-over-HTTPS policy
|
||||
git_fetch = False
|
||||
git_raw = raw_dict.get("git")
|
||||
if git_raw is not None:
|
||||
if not isinstance(git_raw, dict):
|
||||
raise ValueError(f"{label} ({host}): 'git' must be an object")
|
||||
git_dict: dict[str, object] = typing.cast(dict[str, object], git_raw)
|
||||
fetch_raw = git_dict.get("fetch", False)
|
||||
if fetch_raw is True or fetch_raw is False:
|
||||
git_fetch = fetch_raw
|
||||
else:
|
||||
raise ValueError(f"{label} ({host}): 'git.fetch' must be a boolean")
|
||||
for k in git_dict:
|
||||
if k != "fetch":
|
||||
raise ValueError(
|
||||
f"{label} ({host}): git has unknown key {k!r}; "
|
||||
"accepted key is 'fetch'"
|
||||
)
|
||||
|
||||
# dlp detectors
|
||||
outbound_detectors, inbound_detectors = _parse_detectors(
|
||||
idx, host, raw_dict,
|
||||
)
|
||||
|
||||
for k in raw_dict:
|
||||
if k not in ("host", "matches", "auth_scheme", "token_env", "dlp"):
|
||||
if k not in ("host", "matches", "auth_scheme", "token_env", "dlp", "git"):
|
||||
raise ValueError(
|
||||
f"{label} ({host}): unknown key {k!r}; accepted keys "
|
||||
f"are 'host', 'matches', 'auth_scheme', 'token_env', 'dlp'"
|
||||
f"are 'host', 'matches', 'auth_scheme', 'token_env', 'dlp', 'git'"
|
||||
)
|
||||
|
||||
return Route(
|
||||
@@ -333,11 +353,62 @@ def _parse_one(idx: int, raw: object) -> Route:
|
||||
matches=matches,
|
||||
auth_scheme=auth_scheme,
|
||||
token_env=token_env,
|
||||
git_fetch=git_fetch,
|
||||
outbound_detectors=outbound_detectors,
|
||||
inbound_detectors=inbound_detectors,
|
||||
)
|
||||
|
||||
|
||||
def _path_match_to_dict(pm: PathMatch) -> dict[str, object]:
|
||||
d: dict[str, object] = {"value": pm.value}
|
||||
if pm.type != "prefix":
|
||||
d["type"] = pm.type
|
||||
return d
|
||||
|
||||
|
||||
def _header_match_to_dict(hm: HeaderMatch) -> dict[str, object]:
|
||||
d: dict[str, object] = {"name": hm.name, "value": hm.value}
|
||||
if hm.type != "exact":
|
||||
d["type"] = hm.type
|
||||
return d
|
||||
|
||||
|
||||
def _match_entry_to_dict(me: MatchEntry) -> dict[str, object]:
|
||||
d: dict[str, object] = {}
|
||||
if me.paths:
|
||||
d["paths"] = [_path_match_to_dict(p) for p in me.paths]
|
||||
if me.methods:
|
||||
d["methods"] = list(me.methods)
|
||||
if me.headers:
|
||||
d["headers"] = [_header_match_to_dict(h) for h in me.headers]
|
||||
return d
|
||||
|
||||
|
||||
def route_to_yaml_dict(r: Route) -> dict[str, object]:
|
||||
"""Serialize a Route to YAML-schema-compatible dict.
|
||||
|
||||
Uses the same field names the YAML parser accepts, so the output
|
||||
can be round-tripped directly into an `allow` or `egress-block`
|
||||
proposal without translation. Fields that are empty/default are
|
||||
omitted so the agent doesn't copy irrelevant keys."""
|
||||
d: dict[str, object] = {"host": r.host}
|
||||
if r.auth_scheme:
|
||||
d["auth_scheme"] = r.auth_scheme
|
||||
d["token_env"] = r.token_env
|
||||
if r.matches:
|
||||
d["matches"] = [_match_entry_to_dict(m) for m in r.matches]
|
||||
if r.git_fetch:
|
||||
d["git"] = {"fetch": True}
|
||||
dlp: dict[str, object] = {}
|
||||
if r.outbound_detectors is not None:
|
||||
dlp["outbound_detectors"] = list(r.outbound_detectors)
|
||||
if r.inbound_detectors is not None:
|
||||
dlp["inbound_detectors"] = list(r.inbound_detectors)
|
||||
if dlp:
|
||||
d["dlp"] = dlp
|
||||
return d
|
||||
|
||||
|
||||
def load_routes(text: str) -> tuple[Route, ...]:
|
||||
"""Parse YAML text → routes."""
|
||||
try:
|
||||
@@ -450,6 +521,17 @@ def is_git_push_request(path: str, query: str) -> bool:
|
||||
return False
|
||||
|
||||
|
||||
def is_git_fetch_request(path: str, query: str) -> bool:
|
||||
if path.endswith("/git-upload-pack"):
|
||||
return True
|
||||
if path.endswith("/info/refs"):
|
||||
for pair in query.split("&"):
|
||||
k, _, v = pair.partition("=")
|
||||
if k == "service" and v == "git-upload-pack":
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Route lookup + decision
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -513,6 +595,24 @@ def decide(
|
||||
return Decision(action="forward")
|
||||
|
||||
|
||||
def decide_git_fetch(
|
||||
routes: typing.Sequence[Route],
|
||||
request_host: str,
|
||||
) -> Decision:
|
||||
route = match_route(routes, request_host)
|
||||
if route is not None and route.git_fetch:
|
||||
return Decision(action="forward")
|
||||
return Decision(
|
||||
action="block",
|
||||
reason=(
|
||||
"egress: git fetch/clone over HTTPS is not allowed by default; "
|
||||
"use git-gate for declared repos or set "
|
||||
"egress.routes[].git.fetch=true for explicit read-only "
|
||||
"HTTPS Git access."
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# DLP scan dispatch (PRD 0053)
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -538,6 +638,27 @@ def build_outbound_scan_text(
|
||||
return "\n".join(parts)
|
||||
|
||||
|
||||
def outbound_scan_headers(
|
||||
route: Route,
|
||||
headers: typing.Mapping[str, str],
|
||||
) -> dict[str, str]:
|
||||
"""Return request headers that should be included in outbound DLP.
|
||||
|
||||
Routes that inject sidecar-owned auth always strip the agent's
|
||||
Authorization header before forwarding. Scanning that header first
|
||||
creates false positives for provider clients that insist on sending
|
||||
their own bearer-shaped placeholder, while still not changing what
|
||||
reaches the upstream.
|
||||
"""
|
||||
out: dict[str, str] = {}
|
||||
skip_auth = bool(route.auth_scheme and route.token_env)
|
||||
for name, value in headers.items():
|
||||
if skip_auth and name.lower() == "authorization":
|
||||
continue
|
||||
out[name] = value
|
||||
return out
|
||||
|
||||
|
||||
def build_inbound_scan_text(
|
||||
headers: typing.Mapping[str, str],
|
||||
body: str,
|
||||
@@ -627,6 +748,7 @@ def scan_inbound(
|
||||
|
||||
__all__ = [
|
||||
"LOG_BLOCKS",
|
||||
"route_to_yaml_dict",
|
||||
"LOG_FULL",
|
||||
"LOG_OFF",
|
||||
"Config",
|
||||
@@ -639,11 +761,14 @@ __all__ = [
|
||||
"build_inbound_scan_text",
|
||||
"build_outbound_scan_text",
|
||||
"decide",
|
||||
"decide_git_fetch",
|
||||
"evaluate_matches",
|
||||
"is_git_push_request",
|
||||
"is_git_fetch_request",
|
||||
"load_config",
|
||||
"load_routes",
|
||||
"match_route",
|
||||
"outbound_scan_headers",
|
||||
"parse_config",
|
||||
"parse_routes",
|
||||
"scan_inbound",
|
||||
|
||||
+2
-2
@@ -114,7 +114,7 @@ def _read_secret_silent(name: str, prompt_body: str) -> str:
|
||||
return value
|
||||
|
||||
|
||||
def resolve_env(manifest: Manifest, agent: str) -> ResolvedEnv:
|
||||
def resolve_env(manifest: Manifest) -> ResolvedEnv:
|
||||
"""Iterate the agent's env entries:
|
||||
- secret: prompt at runtime; carry value in forwarded
|
||||
- interpolated: read $HOST_VAR from os.environ; carry value in forwarded
|
||||
@@ -124,7 +124,7 @@ def resolve_env(manifest: Manifest, agent: str) -> ResolvedEnv:
|
||||
backend injects forwarded values via its launcher's env parameter."""
|
||||
forwarded: dict[str, str] = {}
|
||||
literals: dict[str, str] = {}
|
||||
bottle = manifest.bottle_for(agent)
|
||||
bottle = manifest.bottle
|
||||
for name, raw in bottle.env.items():
|
||||
if not name:
|
||||
continue
|
||||
|
||||
+209
-24
@@ -37,7 +37,7 @@ from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
from .log import info
|
||||
from .manifest import Bottle, GitEntry
|
||||
from .manifest import ManifestBottle, ManifestGitEntry
|
||||
|
||||
|
||||
# Short network alias for git-gate inside the sidecar bundle. The
|
||||
@@ -96,9 +96,9 @@ class GitGatePlan:
|
||||
egress_network: str = ""
|
||||
|
||||
|
||||
def git_gate_upstreams_for_bottle(bottle: Bottle) -> tuple[GitGateUpstream, ...]:
|
||||
def git_gate_upstreams_for_bottle(bottle: ManifestBottle) -> tuple[GitGateUpstream, ...]:
|
||||
"""Lift each `bottle.git` entry into a GitGateUpstream. Unique-Name
|
||||
validation already ran in `manifest.Bottle.from_dict`."""
|
||||
validation already ran in `manifest.ManifestBottle.from_dict`."""
|
||||
return tuple(
|
||||
GitGateUpstream(
|
||||
name=e.Name,
|
||||
@@ -113,7 +113,7 @@ def git_gate_upstreams_for_bottle(bottle: Bottle) -> tuple[GitGateUpstream, ...]
|
||||
|
||||
|
||||
def git_gate_render_gitconfig(
|
||||
entries: tuple[GitEntry, ...], gate_host: str, *, scheme: str = "git",
|
||||
entries: tuple[ManifestGitEntry, ...], gate_host: str, *, scheme: str = "git",
|
||||
) -> str:
|
||||
"""Render the agent's ~/.gitconfig content for git-gate
|
||||
`insteadOf` rewrites. Pure host-side, no docker / smolvm;
|
||||
@@ -204,6 +204,7 @@ def git_gate_render_entrypoint(upstreams: tuple[GitGateUpstream, ...]) -> str:
|
||||
" git -C \"$repo\" config git-gate.identityFile \"$keyfile\"",
|
||||
" git -C \"$repo\" config git-gate.knownHosts \"$hostsfile\"",
|
||||
" git -C \"$repo\" config receive.denyCurrentBranch ignore",
|
||||
" git -C \"$repo\" config receive.advertisePushOptions true",
|
||||
" git -C \"$repo\" config http.receivepack true",
|
||||
" install -m 755 /etc/git-gate/pre-receive \"$repo/hooks/pre-receive\"",
|
||||
"}",
|
||||
@@ -246,6 +247,164 @@ cat > "$refs_file"
|
||||
|
||||
zero=0000000000000000000000000000000000000000
|
||||
|
||||
supervise_gitleaks_allow() {
|
||||
log_opts=$1
|
||||
ref=$2
|
||||
report_file=$(mktemp)
|
||||
if ! gitleaks git \
|
||||
--log-opts="$log_opts" \
|
||||
--no-banner \
|
||||
--redact \
|
||||
--ignore-gitleaks-allow \
|
||||
--report-format=json \
|
||||
--report-path="$report_file" \
|
||||
--exit-code 0 \
|
||||
1>&2; then
|
||||
rm -f "$report_file"
|
||||
echo "git-gate: gitleaks inline-suppression scan failed for $ref" >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
proposal_id=$(
|
||||
GITLEAKS_ALLOW_REF="$ref" python3 - "$report_file" <<'PY'
|
||||
import datetime
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
|
||||
report_path = Path(sys.argv[1])
|
||||
queue_dir = os.environ.get("SUPERVISE_QUEUE_DIR", "")
|
||||
slug = os.environ.get("SUPERVISE_BOTTLE_SLUG", "")
|
||||
if not queue_dir or not slug:
|
||||
sys.exit(2)
|
||||
|
||||
try:
|
||||
raw = json.loads(report_path.read_text() or "[]")
|
||||
except json.JSONDecodeError:
|
||||
sys.exit(3)
|
||||
if not isinstance(raw, list):
|
||||
sys.exit(3)
|
||||
if not raw:
|
||||
sys.exit(0)
|
||||
|
||||
ref = os.environ.get("GITLEAKS_ALLOW_REF", "")
|
||||
lines = [
|
||||
"gitleaks inline suppression requires supervisor approval",
|
||||
f"ref: {ref}",
|
||||
"",
|
||||
]
|
||||
for i, finding in enumerate(raw, 1):
|
||||
if not isinstance(finding, dict):
|
||||
continue
|
||||
file_path = finding.get("File", "")
|
||||
line_no = finding.get("StartLine", finding.get("Line", ""))
|
||||
rule_id = finding.get("RuleID", "")
|
||||
commit = finding.get("Commit", "")
|
||||
line = finding.get("Line", "")
|
||||
lines.extend([
|
||||
f"finding {i}:",
|
||||
f" file: {file_path}",
|
||||
f" line: {line_no}",
|
||||
f" rule: {rule_id}",
|
||||
f" commit: {commit}",
|
||||
f" code: {line}",
|
||||
"",
|
||||
])
|
||||
|
||||
payload = "\n".join(lines).rstrip() + "\n"
|
||||
proposal_id = str(uuid.uuid4())
|
||||
proposal = {
|
||||
"id": proposal_id,
|
||||
"bottle_slug": slug,
|
||||
"tool": "gitleaks-allow",
|
||||
"proposed_file": payload,
|
||||
"justification": (
|
||||
"git-gate found gitleaks findings hidden by # gitleaks:allow; "
|
||||
"approve only for dummy test fixtures or confirmed false positives"
|
||||
),
|
||||
"arrival_timestamp": datetime.datetime.now(
|
||||
datetime.timezone.utc
|
||||
).isoformat(),
|
||||
"current_file_hash": hashlib.sha256(payload.encode("utf-8")).hexdigest(),
|
||||
}
|
||||
queue = Path(queue_dir)
|
||||
queue.mkdir(parents=True, exist_ok=True)
|
||||
path = queue / f"{proposal_id}.proposal.json"
|
||||
tmp = path.with_suffix(path.suffix + ".tmp")
|
||||
with tmp.open("w", encoding="utf-8") as f:
|
||||
json.dump(proposal, f, indent=2)
|
||||
f.write("\n")
|
||||
os.chmod(tmp, 0o600)
|
||||
os.replace(tmp, path)
|
||||
print(proposal_id)
|
||||
PY
|
||||
)
|
||||
rc=$?
|
||||
rm -f "$report_file"
|
||||
if [ "$rc" -eq 0 ] && [ -z "$proposal_id" ]; then
|
||||
return 0
|
||||
fi
|
||||
if [ "$rc" -ne 0 ]; then
|
||||
echo "git-gate: cannot route # gitleaks:allow finding to supervisor; refusing push" >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
queue_dir=${SUPERVISE_QUEUE_DIR:-}
|
||||
response_file="$queue_dir/${proposal_id}.response.json"
|
||||
timeout=${SUPERVISE_GITLEAKS_ALLOW_TIMEOUT_SECONDS:-300}
|
||||
case "$timeout" in
|
||||
''|*[!0-9]*)
|
||||
echo "git-gate: invalid SUPERVISE_GITLEAKS_ALLOW_TIMEOUT_SECONDS=$timeout" >&2
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
echo "git-gate: queued # gitleaks:allow supervisor approval $proposal_id" >&2
|
||||
echo "git-gate: approve with './cli.py supervise' to continue this push" >&2
|
||||
waited=0
|
||||
while [ "$waited" -lt "$timeout" ]; do
|
||||
if [ -f "$response_file" ]; then
|
||||
status=$(python3 - "$response_file" <<'PY'
|
||||
import json
|
||||
import sys
|
||||
try:
|
||||
with open(sys.argv[1], encoding="utf-8") as f:
|
||||
raw = json.load(f)
|
||||
except (OSError, json.JSONDecodeError):
|
||||
sys.exit(1)
|
||||
status = raw.get("status")
|
||||
if not isinstance(status, str):
|
||||
sys.exit(1)
|
||||
print(status)
|
||||
PY
|
||||
) || status=""
|
||||
case "$status" in
|
||||
approved|modified)
|
||||
mkdir -p "$queue_dir/processed"
|
||||
mv -f "$queue_dir/${proposal_id}.proposal.json" "$queue_dir/processed/" 2>/dev/null || true
|
||||
mv -f "$queue_dir/${proposal_id}.response.json" "$queue_dir/processed/" 2>/dev/null || true
|
||||
echo "git-gate: supervisor approved # gitleaks:allow for $ref" >&2
|
||||
return 0
|
||||
;;
|
||||
rejected)
|
||||
echo "git-gate: supervisor rejected # gitleaks:allow for $ref" >&2
|
||||
return 1
|
||||
;;
|
||||
*)
|
||||
echo "git-gate: invalid supervisor response for # gitleaks:allow" >&2
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
fi
|
||||
sleep 1
|
||||
waited=$((waited + 1))
|
||||
done
|
||||
echo "git-gate: supervisor approval timed out for # gitleaks:allow; refusing push" >&2
|
||||
return 1
|
||||
}
|
||||
|
||||
# Phase 1: gitleaks scan each ref's incoming commits.
|
||||
while IFS=' ' read -r old new ref; do
|
||||
[ -z "$ref" ] && continue
|
||||
@@ -267,6 +426,9 @@ while IFS=' ' read -r old new ref; do
|
||||
echo "git-gate: gitleaks rejected push to $ref" >&2
|
||||
exit 1
|
||||
fi
|
||||
if ! supervise_gitleaks_allow "$log_opts" "$ref"; then
|
||||
exit 1
|
||||
fi
|
||||
done < "$refs_file"
|
||||
|
||||
# Phase 2: forward each ref to the upstream (`origin`, configured
|
||||
@@ -280,15 +442,32 @@ if [ ! -f "$hostsfile" ]; then
|
||||
fi
|
||||
ssh_cmd="ssh -i $keyfile -o UserKnownHostsFile=$hostsfile -o StrictHostKeyChecking=yes -o IdentitiesOnly=yes -o BatchMode=yes -o ConnectTimeout=10"
|
||||
|
||||
push_option_count=${GIT_PUSH_OPTION_COUNT:-0}
|
||||
case "$push_option_count" in
|
||||
''|*[!0-9]*)
|
||||
echo "git-gate: invalid GIT_PUSH_OPTION_COUNT=$push_option_count" >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
set --
|
||||
i=0
|
||||
while [ "$i" -lt "$push_option_count" ]; do
|
||||
opt=$(printenv "GIT_PUSH_OPTION_$i" || :)
|
||||
set -- "$@" --push-option="$opt"
|
||||
i=$((i + 1))
|
||||
done
|
||||
|
||||
while IFS=' ' read -r old new ref; do
|
||||
[ -z "$ref" ] && continue
|
||||
if [ "$new" = "$zero" ]; then
|
||||
refspec=":$ref"
|
||||
elif [ "$old" != "$zero" ] && ! git merge-base --is-ancestor "$old" "$new" 2>/dev/null; then
|
||||
refspec="+$new:$ref"
|
||||
else
|
||||
refspec="$new:$ref"
|
||||
fi
|
||||
echo "git-gate: forwarding $ref to origin" >&2
|
||||
if ! GIT_SSH_COMMAND="$ssh_cmd" git push origin "$refspec" 1>&2; then
|
||||
if ! GIT_SSH_COMMAND="$ssh_cmd" git push "$@" origin "$refspec" 1>&2; then
|
||||
echo "git-gate: upstream push failed for $ref" >&2
|
||||
exit 1
|
||||
fi
|
||||
@@ -361,7 +540,7 @@ exit 0
|
||||
|
||||
|
||||
def _provision_dynamic_key(
|
||||
entry: GitEntry,
|
||||
entry: ManifestGitEntry,
|
||||
slug: str,
|
||||
stage_dir: Path,
|
||||
) -> str:
|
||||
@@ -371,13 +550,12 @@ def _provision_dynamic_key(
|
||||
Returns the host-side path to the private key file so the caller
|
||||
can inject it into the GitGateUpstream as `identity_file`."""
|
||||
from .deploy_key_provisioner import get_provisioner
|
||||
pk = entry.ProvisionedKey
|
||||
assert pk is not None
|
||||
token = os.environ.get(pk.token_env)
|
||||
pk = entry.Key
|
||||
token = os.environ.get(pk.forge_token_env)
|
||||
if token is None:
|
||||
raise RuntimeError(
|
||||
f"git-gate.repos[{entry.Name!r}] provisioned_key.token_env"
|
||||
f" = {pk.token_env!r}: env var is not set"
|
||||
f"git-gate.repos[{entry.Name!r}] key.forge_token_env"
|
||||
f" = {pk.forge_token_env!r}: env var is not set"
|
||||
)
|
||||
api_url = pk.api_url or f"https://{entry.UpstreamHost}"
|
||||
provisioner = get_provisioner(pk.provider, token, api_url)
|
||||
@@ -402,7 +580,7 @@ def _provision_dynamic_key(
|
||||
return str(key_file)
|
||||
|
||||
|
||||
def revoke_git_gate_provisioned_keys(bottle: Bottle, stage_dir: Path) -> None:
|
||||
def revoke_git_gate_provisioned_keys(bottle: ManifestBottle, stage_dir: Path) -> None:
|
||||
"""Revoke all deploy keys provisioned for `bottle` during prepare.
|
||||
|
||||
Called at teardown after containers stop. Raises if any revocation
|
||||
@@ -410,18 +588,18 @@ def revoke_git_gate_provisioned_keys(bottle: Bottle, stage_dir: Path) -> None:
|
||||
address manually."""
|
||||
from .deploy_key_provisioner import get_provisioner
|
||||
for entry in bottle.git:
|
||||
if entry.ProvisionedKey is None:
|
||||
if entry.Key.provider != "gitea":
|
||||
continue
|
||||
pk = entry.ProvisionedKey
|
||||
pk = entry.Key
|
||||
id_file = stage_dir / f"{entry.Name}-deploy-key-id"
|
||||
if not id_file.exists():
|
||||
continue
|
||||
key_id = id_file.read_text().strip()
|
||||
token = os.environ.get(pk.token_env)
|
||||
token = os.environ.get(pk.forge_token_env)
|
||||
if token is None:
|
||||
raise RuntimeError(
|
||||
f"git-gate.repos[{entry.Name!r}] provisioned_key.token_env"
|
||||
f" = {pk.token_env!r}: env var is not set;"
|
||||
f"git-gate.repos[{entry.Name!r}] key.forge_token_env"
|
||||
f" = {pk.forge_token_env!r}: env var is not set;"
|
||||
f" cannot revoke deploy key {key_id}"
|
||||
)
|
||||
api_url = pk.api_url or f"https://{entry.UpstreamHost}"
|
||||
@@ -434,18 +612,26 @@ def revoke_git_gate_provisioned_keys(bottle: Bottle, stage_dir: Path) -> None:
|
||||
info(f"revoked deploy key {key_id} for git-gate.repos[{entry.Name!r}]")
|
||||
|
||||
|
||||
def _resolve_identity_file(entry: ManifestGitEntry, slug: str, stage_dir: Path) -> str:
|
||||
"""Return the host-side SSH identity file path for this entry.
|
||||
For gitea entries, provisions a fresh deploy key first."""
|
||||
if entry.Key.provider == "gitea":
|
||||
return _provision_dynamic_key(entry, slug, stage_dir)
|
||||
return entry.IdentityFile
|
||||
|
||||
|
||||
class GitGate(ABC):
|
||||
"""The per-agent git-gate. Encapsulates the host-side prepare
|
||||
(upstream lift + entrypoint/hook render); the sidecar's
|
||||
start/stop lifecycle is backend-specific and lives on concrete
|
||||
subclasses."""
|
||||
|
||||
def prepare(self, bottle: Bottle, slug: str, stage_dir: Path) -> GitGatePlan:
|
||||
def prepare(self, bottle: ManifestBottle, slug: str, stage_dir: Path) -> GitGatePlan:
|
||||
"""Compute the upstream table from `bottle.git` and write the
|
||||
entrypoint, pre-receive hook, and access-hook scripts (mode
|
||||
600) under `stage_dir`. Pure host-side, no docker subprocess.
|
||||
|
||||
For `provisioned_key` entries, also generates and registers
|
||||
For `gitea` key entries, also generates and registers
|
||||
a fresh deploy key via the forge API and writes the private key
|
||||
+ key ID to `stage_dir`.
|
||||
|
||||
@@ -454,11 +640,10 @@ class GitGate(ABC):
|
||||
before passing the plan to `.start`."""
|
||||
upstreams_list = list(git_gate_upstreams_for_bottle(bottle))
|
||||
for i, entry in enumerate(bottle.git):
|
||||
if entry.ProvisionedKey is not None:
|
||||
key_file = _provision_dynamic_key(entry, slug, stage_dir)
|
||||
upstreams_list[i] = dataclasses.replace(
|
||||
upstreams_list[i], identity_file=key_file
|
||||
)
|
||||
upstreams_list[i] = dataclasses.replace(
|
||||
upstreams_list[i],
|
||||
identity_file=_resolve_identity_file(entry, slug, stage_dir),
|
||||
)
|
||||
upstreams = tuple(upstreams_list)
|
||||
entrypoint = stage_dir / "git_gate_entrypoint.sh"
|
||||
entrypoint.write_text(git_gate_render_entrypoint(upstreams))
|
||||
|
||||
@@ -19,8 +19,8 @@ from urllib.parse import urlsplit
|
||||
|
||||
DEFAULT_PORT = 9420
|
||||
|
||||
# Body-size cap matching supervise_server.py's 1 MiB limit.
|
||||
MAX_BODY_BYTES = 1 * 1024 * 1024
|
||||
# Bound memory use while still allowing ordinary git push packfiles.
|
||||
MAX_BODY_BYTES = 100 * 1024 * 1024
|
||||
|
||||
|
||||
class GitHttpHandler(BaseHTTPRequestHandler):
|
||||
|
||||
+229
-133
@@ -19,7 +19,7 @@ Bottle schema (frontmatter):
|
||||
repos: { <name>: <git-gate-entry>, ... } # optional
|
||||
egress: { routes: [ <egress-route>, ... ] }
|
||||
# route keys: host, matches, auth, role, dlp
|
||||
supervise: <bool> # optional
|
||||
supervise: <bool> # optional (default true)
|
||||
|
||||
Agent schema (frontmatter):
|
||||
bottle: <bottle-name> # required
|
||||
@@ -36,10 +36,23 @@ Bottles can ONLY live under $HOME. A bottles/ dir under $CWD is a
|
||||
warn at load time and contributes nothing. The trust boundary is
|
||||
expressed as filesystem layout rather than resolver logic.
|
||||
|
||||
Validation runs once at load. Manifest.from_json_obj is preserved
|
||||
as a programmatic entry point (used by tests) that takes a dict
|
||||
with the same field names — useful for building manifests without
|
||||
on-disk files.
|
||||
Two types are exported:
|
||||
|
||||
ManifestIndex — the multi-agent/bottle collection returned by
|
||||
resolve() and from_json_obj(). Used for agent
|
||||
selection (all_agent_names), validation
|
||||
(require_agent), and lazy loading (load_for_agent).
|
||||
This is the pre-preflight form.
|
||||
|
||||
Manifest — a single-agent/bottle value type holding exactly
|
||||
one agent: ManifestAgent and one bottle:
|
||||
ManifestBottle (with the agent's git-gate.user
|
||||
already overlaid). Returned by load_for_agent().
|
||||
This is the post-preflight form passed to backends.
|
||||
|
||||
ManifestIndex.from_json_obj is preserved as a programmatic entry
|
||||
point (used by tests) that takes a dict with the same field names —
|
||||
useful for building manifests without on-disk files.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -50,26 +63,28 @@ from pathlib import Path
|
||||
from typing import Mapping
|
||||
|
||||
from .manifest_util import ManifestError, as_json_object
|
||||
from .manifest_agent import Agent, AgentProvider
|
||||
from .manifest_agent import ManifestAgent, ManifestAgentProvider
|
||||
from .manifest_egress import (
|
||||
EGRESS_AUTH_SCHEMES,
|
||||
EgressConfig,
|
||||
EgressRoute,
|
||||
ManifestEgressConfig,
|
||||
ManifestEgressRoute,
|
||||
)
|
||||
from .manifest_git import GitEntry, GitUser, parse_git_gate_config
|
||||
from .manifest_git import ManifestGitEntry, ManifestGitUser, ManifestKeyConfig, parse_git_gate_config
|
||||
from .manifest_schema import BOTTLE_KEYS
|
||||
|
||||
# Re-export everything that callers currently import from this module.
|
||||
__all__ = [
|
||||
"ManifestError",
|
||||
"GitEntry",
|
||||
"GitUser",
|
||||
"AgentProvider",
|
||||
"ManifestGitEntry",
|
||||
"ManifestGitUser",
|
||||
"ManifestKeyConfig",
|
||||
"ManifestAgentProvider",
|
||||
"EGRESS_AUTH_SCHEMES",
|
||||
"EgressRoute",
|
||||
"EgressConfig",
|
||||
"Agent",
|
||||
"Bottle",
|
||||
"ManifestEgressRoute",
|
||||
"ManifestEgressConfig",
|
||||
"ManifestAgent",
|
||||
"ManifestBottle",
|
||||
"ManifestIndex",
|
||||
"Manifest",
|
||||
]
|
||||
|
||||
@@ -86,26 +101,26 @@ def _section_dict(value: object, label: str) -> dict[str, object]:
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Bottle:
|
||||
class ManifestBottle:
|
||||
env: Mapping[str, str] = field(default_factory=_empty_str_dict)
|
||||
agent_provider: AgentProvider = field(default_factory=AgentProvider)
|
||||
git: tuple[GitEntry, ...] = ()
|
||||
agent_provider: ManifestAgentProvider = field(default_factory=ManifestAgentProvider)
|
||||
git: tuple[ManifestGitEntry, ...] = ()
|
||||
# Per-bottle git identity (issue #86). Empty default — bottles
|
||||
# that don't set `git-gate.user:` in the manifest skip the
|
||||
# `git config --global` step entirely. A bottle can declare a user
|
||||
# identity without any git-gate.repos upstreams, and vice versa.
|
||||
git_user: GitUser = field(default_factory=GitUser)
|
||||
egress: EgressConfig = field(default_factory=EgressConfig)
|
||||
# Opt-in per-bottle stuck-recovery sidecar (PRD 0013). When true,
|
||||
# the launch step brings up a supervise sidecar that exposes MCP
|
||||
# tools to the agent (egress-block, capability-block) plus mounts
|
||||
# the current-config dir read-only into the agent at
|
||||
# /etc/bot-bottle/current-config. False (the default) skips the
|
||||
# sidecar and mount.
|
||||
supervise: bool = False
|
||||
git_user: ManifestGitUser = field(default_factory=ManifestGitUser)
|
||||
egress: ManifestEgressConfig = field(default_factory=ManifestEgressConfig)
|
||||
# Per-bottle stuck-recovery sidecar (PRD 0013). When true (the
|
||||
# default, issue #249), the launch step brings up a supervise
|
||||
# sidecar that exposes MCP tools to the agent (egress-block,
|
||||
# capability-block) plus mounts the current-config dir read-only
|
||||
# into the agent at /etc/bot-bottle/current-config. Set
|
||||
# `supervise: false` to skip the sidecar and mount.
|
||||
supervise: bool = True
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, name: str, raw: object) -> "Bottle":
|
||||
def from_dict(cls, name: str, raw: object) -> "ManifestBottle":
|
||||
d = as_json_object(raw, f"bottle '{name}'")
|
||||
|
||||
if "runtime" in d:
|
||||
@@ -157,25 +172,25 @@ class Bottle:
|
||||
)
|
||||
env[var] = value
|
||||
|
||||
git: tuple[GitEntry, ...] = ()
|
||||
git_user = GitUser()
|
||||
git: tuple[ManifestGitEntry, ...] = ()
|
||||
git_user = ManifestGitUser()
|
||||
git_raw = d.get("git-gate")
|
||||
if git_raw is not None:
|
||||
git, git_user = parse_git_gate_config(name, git_raw)
|
||||
|
||||
agent_provider = (
|
||||
AgentProvider.from_dict(name, d["agent_provider"])
|
||||
ManifestAgentProvider.from_dict(name, d["agent_provider"])
|
||||
if "agent_provider" in d
|
||||
else AgentProvider()
|
||||
else ManifestAgentProvider()
|
||||
)
|
||||
|
||||
egress = (
|
||||
EgressConfig.from_dict(name, d["egress"])
|
||||
ManifestEgressConfig.from_dict(name, d["egress"])
|
||||
if "egress" in d
|
||||
else EgressConfig()
|
||||
else ManifestEgressConfig()
|
||||
)
|
||||
|
||||
supervise_raw = d.get("supervise", False)
|
||||
supervise_raw = d.get("supervise", True)
|
||||
if not isinstance(supervise_raw, bool):
|
||||
raise ManifestError(
|
||||
f"bottle '{name}' supervise must be a boolean "
|
||||
@@ -188,14 +203,64 @@ class Bottle:
|
||||
)
|
||||
|
||||
|
||||
def _merge_git_user(
|
||||
agent_user: ManifestGitUser, base_user: ManifestGitUser
|
||||
) -> ManifestGitUser:
|
||||
"""Merge the agent's git.user over the bottle's, agent-wins-on-non-empty."""
|
||||
if agent_user.is_empty():
|
||||
return base_user
|
||||
return ManifestGitUser(
|
||||
name=agent_user.name or base_user.name,
|
||||
email=agent_user.email or base_user.email,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Manifest:
|
||||
bottles: Mapping[str, Bottle]
|
||||
agents: Mapping[str, Agent]
|
||||
"""Single-agent/bottle value type. Returned by ManifestIndex.load_for_agent().
|
||||
|
||||
`bottle` is the effective bottle with the agent's git-gate.user already
|
||||
overlaid per-field (agent wins on non-empty). Backends and provisioners
|
||||
use this directly — no agent_name lookup needed."""
|
||||
|
||||
agent: ManifestAgent
|
||||
bottle: ManifestBottle
|
||||
|
||||
def git_identity_summary(self) -> str | None:
|
||||
"""One-line effective git identity with per-field provenance, e.g.
|
||||
`name=claude (agent), email=eric@dideric.is (bottle)`.
|
||||
Returns None when neither agent nor bottle sets an identity."""
|
||||
over = self.agent.git_user # agent's declared git_user (pre-merge)
|
||||
merged = self.bottle.git_user # effective git_user (post-merge)
|
||||
if merged.is_empty():
|
||||
return None
|
||||
parts: list[str] = []
|
||||
if merged.name:
|
||||
parts.append(f"name={merged.name} ({'agent' if over.name else 'bottle'})")
|
||||
if merged.email:
|
||||
parts.append(f"email={merged.email} ({'agent' if over.email else 'bottle'})")
|
||||
return ", ".join(parts)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ManifestIndex:
|
||||
"""Multi-agent/bottle collection. The pre-preflight form.
|
||||
|
||||
In lazy mode (from resolve()/from_md_dirs()) only filenames are scanned;
|
||||
no file content is read. In eager mode (from from_json_obj()) all agents
|
||||
and bottles are pre-parsed. Call load_for_agent() to get a single-value
|
||||
Manifest ready for backend use."""
|
||||
|
||||
bottles: Mapping[str, ManifestBottle]
|
||||
agents: Mapping[str, ManifestAgent]
|
||||
# Set by from_md_dirs; None in from_json_obj (test/programmatic) mode.
|
||||
# Stores the manifest root dirs so load_for_agent can locate files later.
|
||||
home_md: Path | None = field(default=None)
|
||||
cwd_md: Path | None = field(default=None)
|
||||
|
||||
@classmethod
|
||||
def resolve(cls, cwd: str, *, missing_ok: bool = False) -> "Manifest":
|
||||
"""Walk the per-file manifest tree and build a Manifest.
|
||||
def resolve(cls, cwd: str, *, missing_ok: bool = False) -> "ManifestIndex":
|
||||
"""Walk the per-file manifest tree and build a ManifestIndex.
|
||||
|
||||
Layout (PRD 0011):
|
||||
$HOME/.bot-bottle/bottles/<name>.md — bottles (home-only)
|
||||
@@ -208,7 +273,7 @@ class Manifest:
|
||||
boundary.
|
||||
|
||||
If `missing_ok` is true, a missing `$HOME/.bot-bottle/`
|
||||
returns an empty manifest instead of dying. This is for
|
||||
returns an empty index instead of dying. This is for
|
||||
passive UI surfaces like the dashboard, which can still
|
||||
monitor already-running agents without launch config.
|
||||
|
||||
@@ -247,25 +312,16 @@ class Manifest:
|
||||
cls,
|
||||
home_dir: Path,
|
||||
cwd_dir: Path | None,
|
||||
) -> "Manifest":
|
||||
"""Programmatic entry point. Loads bottles from
|
||||
`<home_dir>/bottles/`, home agents from `<home_dir>/agents/`,
|
||||
and (if `cwd_dir` is passed) cwd agents from
|
||||
`<cwd_dir>/agents/`. Cwd agents override home agents on
|
||||
name collision. A `bottles/` subdir under `cwd_dir` is
|
||||
logged as a warning and ignored.
|
||||
) -> "ManifestIndex":
|
||||
"""Return a names-only ManifestIndex. No file content is read; only
|
||||
filenames are scanned for the agent selector. Full parsing happens
|
||||
later, per-agent, via `load_for_agent`.
|
||||
|
||||
Used by tests to build a Manifest from fixture directories
|
||||
A `bottles/` subdir under `cwd_dir` is logged as a warning and
|
||||
ignored — the filesystem layout IS the trust boundary.
|
||||
|
||||
Used by tests to build a ManifestIndex from fixture directories
|
||||
without touching `os.environ`."""
|
||||
bottles_dir = home_dir / "bottles"
|
||||
from .manifest_loader import load_agents_from_dir, load_bottles_from_dir
|
||||
|
||||
bottles = load_bottles_from_dir(bottles_dir)
|
||||
|
||||
bottle_names = set(bottles.keys())
|
||||
agents_dir = home_dir / "agents"
|
||||
agents = load_agents_from_dir(agents_dir, bottle_names, source="$HOME")
|
||||
|
||||
if cwd_dir is not None:
|
||||
stale_bottles = cwd_dir / "bottles"
|
||||
if stale_bottles.is_dir():
|
||||
@@ -279,17 +335,11 @@ class Manifest:
|
||||
f"live under $HOME/.bot-bottle/bottles/ "
|
||||
f"(PRD 0011). Move them or delete."
|
||||
)
|
||||
cwd_agents_dir = cwd_dir / "agents"
|
||||
cwd_agents = load_agents_from_dir(
|
||||
cwd_agents_dir, bottle_names, source="$CWD"
|
||||
)
|
||||
agents = {**agents, **cwd_agents}
|
||||
|
||||
return cls(bottles=bottles, agents=agents)
|
||||
return cls(bottles={}, agents={}, home_md=home_dir, cwd_md=cwd_dir)
|
||||
|
||||
@classmethod
|
||||
def from_json_obj(cls, obj: object) -> "Manifest":
|
||||
"""Validate and build a Manifest from a raw JSON-like dict."""
|
||||
def from_json_obj(cls, obj: object) -> "ManifestIndex":
|
||||
"""Validate and build a ManifestIndex from a raw JSON-like dict."""
|
||||
d = as_json_object(obj, "manifest")
|
||||
raw_bottles_obj = _section_dict(d.get("bottles"), "manifest 'bottles'")
|
||||
raw_agents = _section_dict(d.get("agents"), "manifest 'agents'")
|
||||
@@ -305,80 +355,126 @@ class Manifest:
|
||||
bottles = resolve_bottles(raw_bottles)
|
||||
|
||||
bottle_names = set(bottles.keys())
|
||||
agents: dict[str, Agent] = {
|
||||
n: Agent.from_dict(n, a, bottle_names) for n, a in raw_agents.items()
|
||||
agents: dict[str, ManifestAgent] = {
|
||||
n: ManifestAgent.from_dict(n, a, bottle_names) for n, a in raw_agents.items()
|
||||
}
|
||||
return cls(bottles=bottles, agents=agents)
|
||||
|
||||
@property
|
||||
def all_agent_names(self) -> list[str]:
|
||||
"""Sorted list of all discoverable agent names.
|
||||
|
||||
In names-only mode (from resolve/from_md_dirs) this scans agent
|
||||
filenames without reading their content. In eager mode (from
|
||||
from_json_obj) it returns the pre-parsed agents' names."""
|
||||
if self.home_md is not None:
|
||||
from .manifest_loader import scan_agent_names
|
||||
home_names = set(scan_agent_names(self.home_md / "agents").keys())
|
||||
cwd_names: set[str] = set()
|
||||
if self.cwd_md is not None:
|
||||
cwd_names = set(scan_agent_names(self.cwd_md / "agents").keys())
|
||||
return sorted(home_names | cwd_names)
|
||||
return sorted(self.agents.keys())
|
||||
|
||||
def load_for_agent(self, agent_name: str) -> "Manifest":
|
||||
"""Parse the named agent and its bottle; return a single-value Manifest.
|
||||
|
||||
In lazy mode (from resolve/from_md_dirs) the agent file and its
|
||||
bottle chain are read from disk for the first time here. In eager
|
||||
mode (from_json_obj) the data is already parsed; this just filters
|
||||
down to the requested agent and its bottle.
|
||||
|
||||
The returned Manifest.bottle has the agent's git-gate.user already
|
||||
overlaid (agent wins on non-empty, per-field).
|
||||
|
||||
Always raises ManifestError if the agent is unknown or invalid.
|
||||
Backends call this at preflight inside _validate."""
|
||||
if self.home_md is None:
|
||||
# Eager manifest (from_json_obj): data already parsed; filter to
|
||||
# the one requested agent and its bottle so the returned Manifest
|
||||
# always holds exactly one agent and one bottle regardless of path.
|
||||
if agent_name not in self.agents:
|
||||
available = ", ".join(sorted(self.agents.keys())) or "(none)"
|
||||
raise ManifestError(
|
||||
f"agent '{agent_name}' not defined. Available: {available}"
|
||||
)
|
||||
agent = self.agents[agent_name]
|
||||
raw_bottle = self.bottles[agent.bottle]
|
||||
merged = _merge_git_user(agent.git_user, raw_bottle.git_user)
|
||||
bottle = raw_bottle if merged == raw_bottle.git_user else replace(raw_bottle, git_user=merged)
|
||||
return Manifest(agent=agent, bottle=bottle)
|
||||
|
||||
from .manifest_loader import load_bottle_chain_from_dir, scan_agent_names
|
||||
from .manifest_schema import validate_agent_frontmatter_keys
|
||||
from .yaml_subset import YamlSubsetError, parse_frontmatter
|
||||
|
||||
# Locate the agent file; cwd wins over home on name collision.
|
||||
home_agents = scan_agent_names(self.home_md / "agents")
|
||||
cwd_agents: dict[str, Path] = {}
|
||||
if self.cwd_md is not None:
|
||||
cwd_agents = scan_agent_names(self.cwd_md / "agents")
|
||||
merged_agents = {**home_agents, **cwd_agents}
|
||||
|
||||
if agent_name not in merged_agents:
|
||||
available = ", ".join(sorted(merged_agents.keys())) or "(none)"
|
||||
raise ManifestError(
|
||||
f"agent '{agent_name}' not defined. Available: {available}"
|
||||
)
|
||||
|
||||
agent_path = merged_agents[agent_name]
|
||||
try:
|
||||
fm, body = parse_frontmatter(agent_path.read_text())
|
||||
except OSError as e:
|
||||
raise ManifestError(f"could not read {agent_path}: {e}") from e
|
||||
except YamlSubsetError as e:
|
||||
raise ManifestError(f"{agent_path}: {e}") from e
|
||||
|
||||
validate_agent_frontmatter_keys(agent_path, fm.keys())
|
||||
|
||||
bottle_name = fm.get("bottle")
|
||||
if not isinstance(bottle_name, str) or not bottle_name:
|
||||
raise ManifestError(
|
||||
f"agent '{agent_name}' must declare a 'bottle' field "
|
||||
f"naming a defined bottle"
|
||||
)
|
||||
|
||||
# Load the bottle chain (may raise ManifestError).
|
||||
bottles_dir = self.home_md / "bottles"
|
||||
raw_bottle = load_bottle_chain_from_dir(bottle_name, bottles_dir)
|
||||
|
||||
# Build and validate the full ManifestAgent.
|
||||
agent_dict: dict[str, object] = {
|
||||
"bottle": bottle_name,
|
||||
"skills": fm.get("skills", []),
|
||||
"prompt": body.strip(),
|
||||
}
|
||||
if "git-gate" in fm:
|
||||
agent_dict["git-gate"] = fm["git-gate"]
|
||||
agent = ManifestAgent.from_dict(agent_name, agent_dict, {bottle_name})
|
||||
|
||||
merged_user = _merge_git_user(agent.git_user, raw_bottle.git_user)
|
||||
bottle = raw_bottle if merged_user == raw_bottle.git_user else replace(raw_bottle, git_user=merged_user)
|
||||
return Manifest(agent=agent, bottle=bottle)
|
||||
|
||||
def has_agent(self, name: str) -> bool:
|
||||
return name in self.agents
|
||||
|
||||
def require_agent(self, name: str) -> None:
|
||||
"""Check that `name` is a discoverable agent. In names-only mode
|
||||
this checks whether the .md file exists; in eager mode it checks
|
||||
the pre-parsed agents dict. Does NOT parse file content."""
|
||||
if self.has_agent(name):
|
||||
return
|
||||
available = ", ".join(self.agents.keys())
|
||||
if available:
|
||||
msg = f"agent '{name}' not defined in bot-bottle.json. Available: {available}"
|
||||
raise ManifestError(msg)
|
||||
raise ManifestError(
|
||||
f"agent '{name}' not defined in bot-bottle.json (manifest is empty)."
|
||||
)
|
||||
|
||||
def has_bottle(self, name: str) -> bool:
|
||||
return name in self.bottles
|
||||
|
||||
def require_bottle(self, name: str) -> None:
|
||||
if self.has_bottle(name):
|
||||
return
|
||||
available = ", ".join(self.bottles.keys())
|
||||
if available:
|
||||
raise ManifestError(
|
||||
f"bottle '{name}' not defined in bot-bottle.json. "
|
||||
f"Available bottles: {available}"
|
||||
if self.home_md is not None:
|
||||
# Names-only mode: check file existence without parsing.
|
||||
home_path = self.home_md / "agents" / f"{name}.md"
|
||||
cwd_path = (
|
||||
self.cwd_md / "agents" / f"{name}.md"
|
||||
if self.cwd_md else None
|
||||
)
|
||||
raise ManifestError(f"bottle '{name}' not defined in bot-bottle.json (no bottles defined).")
|
||||
|
||||
def _effective_git_user(self, agent_name: str) -> GitUser:
|
||||
"""Merge the agent's git.user over the referenced bottle's,
|
||||
per-field, agent-wins-on-non-empty (issue #94). Same overlay
|
||||
the `extends:` resolver applies between bottles
|
||||
(`_merge_bottles`)."""
|
||||
agent = self.agents[agent_name]
|
||||
base = self.bottles[agent.bottle].git_user
|
||||
over = agent.git_user
|
||||
if over.is_empty():
|
||||
return base
|
||||
return GitUser(
|
||||
name=over.name or base.name,
|
||||
email=over.email or base.email,
|
||||
if home_path.is_file() or (cwd_path and cwd_path.is_file()):
|
||||
return
|
||||
available = ", ".join(self.all_agent_names) or "(none)"
|
||||
raise ManifestError(
|
||||
f"agent '{name}' not defined. Available: {available}"
|
||||
)
|
||||
|
||||
def bottle_for(self, agent_name: str) -> Bottle:
|
||||
"""Resolve the Bottle the named agent references, with the
|
||||
agent's git.user overlaid on top. The validator guarantees both
|
||||
lookups succeed for a manifest built via from_json_obj.
|
||||
|
||||
The overlay lives here, the single point both backends call to
|
||||
resolve an agent's bottle, so the docker / smolmachines git
|
||||
provisioners pick up the merged identity unchanged."""
|
||||
bottle = self.bottles[self.agents[agent_name].bottle]
|
||||
merged = self._effective_git_user(agent_name)
|
||||
if merged == bottle.git_user:
|
||||
return bottle
|
||||
return replace(bottle, git_user=merged)
|
||||
|
||||
def git_identity_summary(self, agent_name: str) -> str | None:
|
||||
"""One-line effective git identity with per-field provenance
|
||||
for launch summaries, e.g.
|
||||
`name=claude (agent), email=eric@dideric.is (bottle)`.
|
||||
Returns None when neither agent nor bottle sets an identity."""
|
||||
over = self.agents[agent_name].git_user
|
||||
merged = self._effective_git_user(agent_name)
|
||||
if merged.is_empty():
|
||||
return None
|
||||
parts: list[str] = []
|
||||
if merged.name:
|
||||
parts.append(f"name={merged.name} ({'agent' if over.name else 'bottle'})")
|
||||
if merged.email:
|
||||
parts.append(f"email={merged.email} ({'agent' if over.email else 'bottle'})")
|
||||
return ", ".join(parts)
|
||||
|
||||
+105
-11
@@ -2,17 +2,17 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from dataclasses import dataclass, field
|
||||
from typing import cast
|
||||
|
||||
from .agent_provider import PROVIDER_TEMPLATES
|
||||
from .manifest_util import ManifestError, as_json_object
|
||||
from .manifest_git import GitUser
|
||||
from .manifest_git import ManifestGitUser
|
||||
from .manifest_schema import AGENT_MODEL_KEYS
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AgentProvider:
|
||||
class ManifestAgentProvider:
|
||||
"""Provider/template for the agent process inside a bottle.
|
||||
|
||||
`template` selects a built-in launch/runtime contract. `dockerfile`
|
||||
@@ -33,15 +33,23 @@ class AgentProvider:
|
||||
dockerfile: str = ""
|
||||
auth_token: str = ""
|
||||
forward_host_credentials: bool = False
|
||||
settings: dict[str, object] = field(default_factory=dict)
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, bottle_name: str, raw: object) -> "AgentProvider":
|
||||
def from_dict(cls, bottle_name: str, raw: object) -> "ManifestAgentProvider":
|
||||
d = as_json_object(raw, f"bottle '{bottle_name}' agent_provider")
|
||||
for k in d:
|
||||
if k not in {"template", "dockerfile", "auth_token", "forward_host_credentials"}:
|
||||
if k not in {
|
||||
"template",
|
||||
"dockerfile",
|
||||
"auth_token",
|
||||
"forward_host_credentials",
|
||||
"settings",
|
||||
}:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' agent_provider has unknown key {k!r}; "
|
||||
f"allowed: template, dockerfile, auth_token, forward_host_credentials"
|
||||
"allowed: template, dockerfile, auth_token, "
|
||||
"forward_host_credentials, settings"
|
||||
)
|
||||
template = d.get("template", "claude")
|
||||
if not isinstance(template, str) or not template:
|
||||
@@ -89,16 +97,18 @@ class AgentProvider:
|
||||
f"bottle '{bottle_name}' agent_provider.forward_host_credentials "
|
||||
"is currently only supported for template 'codex'"
|
||||
)
|
||||
settings = _parse_provider_settings(bottle_name, template, d.get("settings"))
|
||||
return cls(
|
||||
template=template,
|
||||
dockerfile=dockerfile,
|
||||
auth_token=auth_token,
|
||||
forward_host_credentials=forward_host_credentials,
|
||||
settings=settings,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Agent:
|
||||
class ManifestAgent:
|
||||
bottle: str
|
||||
skills: tuple[str, ...] = ()
|
||||
prompt: str = ""
|
||||
@@ -106,10 +116,10 @@ class Agent:
|
||||
# bottle's git-gate.user per-field at `Manifest.bottle_for`. Only
|
||||
# `user` is allowed at the agent level; `repos` stays bottle-only
|
||||
# because it carries credentials and host trust.
|
||||
git_user: GitUser = GitUser()
|
||||
git_user: ManifestGitUser = ManifestGitUser()
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, name: str, raw: object, bottle_names: set[str]) -> "Agent":
|
||||
def from_dict(cls, name: str, raw: object, bottle_names: set[str]) -> "ManifestAgent":
|
||||
d = as_json_object(raw, f"agent '{name}'")
|
||||
unknown = set(d.keys()) - AGENT_MODEL_KEYS
|
||||
if unknown:
|
||||
@@ -164,7 +174,7 @@ class Agent:
|
||||
|
||||
# git-gate: agents may declare only `git-gate.user` (name/email).
|
||||
# `git-gate.repos` is bottle-only — it carries credentials and host trust.
|
||||
git_user = GitUser()
|
||||
git_user = ManifestGitUser()
|
||||
git_raw = d.get("git-gate")
|
||||
if git_raw is not None:
|
||||
gd = as_json_object(git_raw, f"agent '{name}' git-gate")
|
||||
@@ -177,6 +187,90 @@ class Agent:
|
||||
f"(it carries credentials and host trust)."
|
||||
)
|
||||
if "user" in gd:
|
||||
git_user = GitUser.from_dict(name, gd["user"])
|
||||
git_user = ManifestGitUser.from_dict(name, gd["user"])
|
||||
|
||||
return cls(bottle=bottle, skills=skills, prompt=prompt, git_user=git_user)
|
||||
|
||||
|
||||
def _parse_provider_settings(
|
||||
bottle_name: str,
|
||||
template: str,
|
||||
raw: object,
|
||||
) -> dict[str, object]:
|
||||
if raw is None:
|
||||
return {}
|
||||
if template != "pi":
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' agent_provider.settings is only "
|
||||
"supported for template 'pi'"
|
||||
)
|
||||
settings = as_json_object(raw, f"bottle '{bottle_name}' agent_provider.settings")
|
||||
allowed = {
|
||||
"provider",
|
||||
"base_url",
|
||||
"api",
|
||||
"api_key",
|
||||
"api_key_env",
|
||||
"models",
|
||||
"context_window",
|
||||
"max_tokens_field",
|
||||
"max_tokens",
|
||||
"supports_developer_role",
|
||||
"supports_reasoning_effort",
|
||||
}
|
||||
for key in settings:
|
||||
if key not in allowed:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' agent_provider.settings has unknown "
|
||||
f"key {key!r}; allowed: {', '.join(sorted(allowed))}"
|
||||
)
|
||||
for key in ("provider", "base_url", "api", "api_key", "api_key_env"):
|
||||
value = settings.get(key)
|
||||
if value is not None and (not isinstance(value, str) or not value):
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' agent_provider.settings.{key} must "
|
||||
"be a non-empty string"
|
||||
)
|
||||
max_tokens_field = settings.get("max_tokens_field")
|
||||
if max_tokens_field is not None and max_tokens_field not in (
|
||||
"max_tokens", "max_completion_tokens",
|
||||
):
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' agent_provider.settings.max_tokens_field "
|
||||
"must be 'max_tokens' or 'max_completion_tokens'"
|
||||
)
|
||||
if settings.get("api_key") is not None and settings.get("api_key_env") is not None:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' agent_provider.settings may set either "
|
||||
"api_key or api_key_env, not both"
|
||||
)
|
||||
models = settings.get("models")
|
||||
if models is not None:
|
||||
if not isinstance(models, list) or not models:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' agent_provider.settings.models must "
|
||||
"be a non-empty array of strings"
|
||||
)
|
||||
for i, model in enumerate(models):
|
||||
if not isinstance(model, str) or not model:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' agent_provider.settings.models[{i}] "
|
||||
"must be a non-empty string"
|
||||
)
|
||||
for key in ("supports_developer_role", "supports_reasoning_effort"):
|
||||
value = settings.get(key)
|
||||
if value is not None and not isinstance(value, bool):
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' agent_provider.settings.{key} must "
|
||||
f"be a boolean (was {type(value).__name__})"
|
||||
)
|
||||
for key in ("context_window", "max_tokens"):
|
||||
value = settings.get(key)
|
||||
if value is not None and (
|
||||
not isinstance(value, int) or isinstance(value, bool) or value <= 0
|
||||
):
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' agent_provider.settings.{key} must "
|
||||
f"be a positive integer (was {type(value).__name__})"
|
||||
)
|
||||
return dict(settings)
|
||||
|
||||
@@ -24,7 +24,7 @@ INBOUND_DETECTOR_NAMES = frozenset({"naive_injection_detection"})
|
||||
|
||||
def validate_egress_routes(
|
||||
bottle_name: str,
|
||||
routes: tuple[EgressRoute, ...],
|
||||
routes: tuple[ManifestEgressRoute, ...],
|
||||
) -> None:
|
||||
seen_hosts: dict[str, None] = {}
|
||||
for r in routes:
|
||||
@@ -38,37 +38,38 @@ def validate_egress_routes(
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PathMatch:
|
||||
class ManifestPathMatch:
|
||||
Type: str = "prefix"
|
||||
Value: str = ""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class HeaderMatch:
|
||||
class ManifestHeaderMatch:
|
||||
Name: str = ""
|
||||
Value: str = ""
|
||||
Type: str = "exact"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class MatchEntry:
|
||||
Paths: tuple[PathMatch, ...] = ()
|
||||
class ManifestMatchEntry:
|
||||
Paths: tuple[ManifestPathMatch, ...] = ()
|
||||
Methods: tuple[str, ...] = ()
|
||||
Headers: tuple[HeaderMatch, ...] = ()
|
||||
Headers: tuple[ManifestHeaderMatch, ...] = ()
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class EgressRoute:
|
||||
class ManifestEgressRoute:
|
||||
Host: str
|
||||
Matches: tuple[MatchEntry, ...] = ()
|
||||
Matches: tuple[ManifestMatchEntry, ...] = ()
|
||||
AuthScheme: str = ""
|
||||
TokenRef: str = ""
|
||||
Role: tuple[str, ...] = ()
|
||||
GitFetch: bool = False
|
||||
OutboundDetectors: tuple[str, ...] | None = None
|
||||
InboundDetectors: tuple[str, ...] | None = None
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, bottle_name: str, idx: int, raw: object) -> "EgressRoute":
|
||||
def from_dict(cls, bottle_name: str, idx: int, raw: object) -> "ManifestEgressRoute":
|
||||
label = f"bottle '{bottle_name}' egress.routes[{idx}]"
|
||||
d = as_json_object(raw, label)
|
||||
host = d.get("host")
|
||||
@@ -76,7 +77,7 @@ class EgressRoute:
|
||||
raise ManifestError(f"{label} missing required string field 'host'")
|
||||
|
||||
# --- matches ---
|
||||
matches: tuple[MatchEntry, ...] = ()
|
||||
matches: tuple[ManifestMatchEntry, ...] = ()
|
||||
matches_raw = d.get("matches")
|
||||
if matches_raw is not None:
|
||||
if not isinstance(matches_raw, list):
|
||||
@@ -85,7 +86,7 @@ class EgressRoute:
|
||||
f"(was {type(matches_raw).__name__})"
|
||||
)
|
||||
matches_list = cast(list[object], matches_raw)
|
||||
entries: list[MatchEntry] = []
|
||||
entries: list[ManifestMatchEntry] = []
|
||||
for k, entry_raw in enumerate(matches_list):
|
||||
entries.append(
|
||||
_parse_match_entry(label, k, entry_raw)
|
||||
@@ -165,11 +166,30 @@ class EgressRoute:
|
||||
label, d.get("dlp"),
|
||||
)
|
||||
|
||||
# --- git-over-HTTPS policy ---
|
||||
git_fetch = False
|
||||
if "git" in d:
|
||||
git_d = as_json_object(d.get("git"), f"{label} git")
|
||||
raw_fetch = git_d.get("fetch", False)
|
||||
if isinstance(raw_fetch, bool):
|
||||
git_fetch = raw_fetch
|
||||
else:
|
||||
raise ManifestError(
|
||||
f"{label} git.fetch must be a boolean "
|
||||
f"(was {type(raw_fetch).__name__})"
|
||||
)
|
||||
for k in git_d:
|
||||
if k != "fetch":
|
||||
raise ManifestError(
|
||||
f"{label} git has unknown key {k!r}; "
|
||||
f"only 'fetch' is accepted"
|
||||
)
|
||||
|
||||
for k in d:
|
||||
if k not in ("host", "matches", "auth", "role", "dlp"):
|
||||
if k not in ("host", "matches", "auth", "role", "dlp", "git"):
|
||||
raise ManifestError(
|
||||
f"{label} has unknown key {k!r}; accepted keys are "
|
||||
f"'host', 'matches', 'auth', 'role', 'dlp'"
|
||||
f"'host', 'matches', 'auth', 'role', 'dlp', 'git'"
|
||||
)
|
||||
|
||||
return cls(
|
||||
@@ -178,6 +198,7 @@ class EgressRoute:
|
||||
AuthScheme=auth_scheme,
|
||||
TokenRef=token_ref,
|
||||
Role=roles,
|
||||
GitFetch=git_fetch,
|
||||
OutboundDetectors=outbound_detectors,
|
||||
InboundDetectors=inbound_detectors,
|
||||
)
|
||||
@@ -185,17 +206,17 @@ class EgressRoute:
|
||||
|
||||
def _parse_match_entry(
|
||||
route_label: str, k: int, raw: object,
|
||||
) -> MatchEntry:
|
||||
) -> ManifestMatchEntry:
|
||||
label = f"{route_label} matches[{k}]"
|
||||
d = as_json_object(raw, label)
|
||||
|
||||
paths: tuple[PathMatch, ...] = ()
|
||||
paths: tuple[ManifestPathMatch, ...] = ()
|
||||
paths_raw = d.get("paths")
|
||||
if paths_raw is not None:
|
||||
if not isinstance(paths_raw, list):
|
||||
raise ManifestError(f"{label} paths must be an array")
|
||||
paths_list = cast(list[object], paths_raw)
|
||||
parsed_paths: list[PathMatch] = []
|
||||
parsed_paths: list[ManifestPathMatch] = []
|
||||
for j, p_raw in enumerate(paths_list):
|
||||
parsed_paths.append(_parse_path_match(label, j, p_raw))
|
||||
paths = tuple(parsed_paths)
|
||||
@@ -220,13 +241,13 @@ def _parse_match_entry(
|
||||
normalised.append(upper)
|
||||
methods = tuple(normalised)
|
||||
|
||||
headers: tuple[HeaderMatch, ...] = ()
|
||||
headers: tuple[ManifestHeaderMatch, ...] = ()
|
||||
headers_raw = d.get("headers")
|
||||
if headers_raw is not None:
|
||||
if not isinstance(headers_raw, list):
|
||||
raise ManifestError(f"{label} headers must be an array")
|
||||
headers_list = cast(list[object], headers_raw)
|
||||
parsed_headers: list[HeaderMatch] = []
|
||||
parsed_headers: list[ManifestHeaderMatch] = []
|
||||
for j, h_raw in enumerate(headers_list):
|
||||
parsed_headers.append(_parse_header_match(label, j, h_raw))
|
||||
headers = tuple(parsed_headers)
|
||||
@@ -235,12 +256,12 @@ def _parse_match_entry(
|
||||
if key not in ("paths", "methods", "headers"):
|
||||
raise ManifestError(f"{label} has unknown key {key!r}")
|
||||
|
||||
return MatchEntry(Paths=paths, Methods=methods, Headers=headers)
|
||||
return ManifestMatchEntry(Paths=paths, Methods=methods, Headers=headers)
|
||||
|
||||
|
||||
def _parse_path_match(
|
||||
entry_label: str, j: int, raw: object,
|
||||
) -> PathMatch:
|
||||
) -> ManifestPathMatch:
|
||||
label = f"{entry_label} paths[{j}]"
|
||||
d = as_json_object(raw, label)
|
||||
ptype = d.get("type", "prefix")
|
||||
@@ -266,12 +287,12 @@ def _parse_path_match(
|
||||
for k in d:
|
||||
if k not in ("type", "value"):
|
||||
raise ManifestError(f"{label} has unknown key {k!r}")
|
||||
return PathMatch(Type=ptype, Value=value)
|
||||
return ManifestPathMatch(Type=ptype, Value=value)
|
||||
|
||||
|
||||
def _parse_header_match(
|
||||
entry_label: str, j: int, raw: object,
|
||||
) -> HeaderMatch:
|
||||
) -> ManifestHeaderMatch:
|
||||
label = f"{entry_label} headers[{j}]"
|
||||
d = as_json_object(raw, label)
|
||||
name = d.get("name")
|
||||
@@ -296,7 +317,7 @@ def _parse_header_match(
|
||||
for k in d:
|
||||
if k not in ("name", "value", "type"):
|
||||
raise ManifestError(f"{label} has unknown key {k!r}")
|
||||
return HeaderMatch(Name=name, Value=value, Type=htype)
|
||||
return ManifestHeaderMatch(Name=name, Value=value, Type=htype)
|
||||
|
||||
|
||||
def _parse_dlp_block(
|
||||
@@ -350,15 +371,15 @@ LOG_LEVELS = frozenset({0, 1, 2})
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class EgressConfig:
|
||||
routes: tuple[EgressRoute, ...] = ()
|
||||
class ManifestEgressConfig:
|
||||
routes: tuple[ManifestEgressRoute, ...] = ()
|
||||
Log: int = 0
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, bottle_name: str, raw: object) -> "EgressConfig":
|
||||
def from_dict(cls, bottle_name: str, raw: object) -> "ManifestEgressConfig":
|
||||
d = as_json_object(raw, f"bottle '{bottle_name}' egress")
|
||||
routes_raw = d.get("routes")
|
||||
routes: tuple[EgressRoute, ...] = ()
|
||||
routes: tuple[ManifestEgressRoute, ...] = ()
|
||||
if routes_raw is not None:
|
||||
if not isinstance(routes_raw, list):
|
||||
raise ManifestError(
|
||||
@@ -367,7 +388,7 @@ class EgressConfig:
|
||||
)
|
||||
routes_list = cast(list[object], routes_raw)
|
||||
routes = tuple(
|
||||
EgressRoute.from_dict(bottle_name, i, entry)
|
||||
ManifestEgressRoute.from_dict(bottle_name, i, entry)
|
||||
for i, entry in enumerate(routes_list)
|
||||
)
|
||||
validate_egress_routes(bottle_name, routes)
|
||||
|
||||
+106
-36
@@ -5,25 +5,31 @@ from __future__ import annotations
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from .manifest import Bottle, GitEntry
|
||||
from .manifest import ManifestBottle
|
||||
from .manifest_egress import ManifestEgressConfig
|
||||
|
||||
|
||||
def resolve_bottles(raws: dict[str, dict[str, object]]) -> dict[str, Bottle]:
|
||||
"""Apply `extends:` chains and return resolved Bottle objects."""
|
||||
cache: dict[str, Bottle] = {}
|
||||
def resolve_bottles(raws: dict[str, dict[str, object]]) -> dict[str, ManifestBottle]:
|
||||
"""Apply `extends:` chains and return resolved ManifestBottle objects."""
|
||||
cache: dict[str, ManifestBottle] = {}
|
||||
# Per-bottle effective git-gate.repos, as raw dicts keyed by repo name.
|
||||
# Threaded alongside `cache` so a child can field-merge against its
|
||||
# parent's repos without reconstructing them from parsed entries.
|
||||
repos_cache: dict[str, dict[str, object]] = {}
|
||||
for name in raws:
|
||||
if name not in cache:
|
||||
_resolve_one_bottle(name, raws, cache, ())
|
||||
_resolve_one_bottle(name, raws, cache, repos_cache, ())
|
||||
return cache
|
||||
|
||||
|
||||
def _resolve_one_bottle(
|
||||
name: str,
|
||||
raws: dict[str, dict[str, object]],
|
||||
cache: dict[str, Bottle],
|
||||
cache: dict[str, ManifestBottle],
|
||||
repos_cache: dict[str, dict[str, object]],
|
||||
seen: tuple[str, ...],
|
||||
) -> Bottle:
|
||||
from .manifest import Bottle, ManifestError
|
||||
) -> ManifestBottle:
|
||||
from .manifest import ManifestBottle, ManifestError
|
||||
|
||||
if name in cache:
|
||||
return cache[name]
|
||||
@@ -32,14 +38,15 @@ def _resolve_one_bottle(
|
||||
raise ManifestError(f"bottle '{name}' is in an extends cycle: {chain}")
|
||||
raw = raws[name]
|
||||
parent_name_raw = raw.get("extends")
|
||||
# Strip `extends:` before passing to Bottle.from_dict so it
|
||||
# is not accidentally treated as a real Bottle field by future
|
||||
# Strip `extends:` before passing to ManifestBottle.from_dict so it
|
||||
# is not accidentally treated as a real ManifestBottle field by future
|
||||
# schema additions. It is only meaningful here.
|
||||
child_raw = {k: v for k, v in raw.items() if k != "extends"}
|
||||
|
||||
if parent_name_raw is None:
|
||||
bottle = Bottle.from_dict(name, child_raw)
|
||||
bottle = ManifestBottle.from_dict(name, child_raw)
|
||||
cache[name] = bottle
|
||||
repos_cache[name] = _resolve_repos_raw({}, child_raw)
|
||||
return bottle
|
||||
|
||||
if not isinstance(parent_name_raw, str):
|
||||
@@ -59,49 +66,69 @@ def _resolve_one_bottle(
|
||||
f"bottle '{name}' extends '{parent_name}' which is not "
|
||||
f"defined. Available bottles: {avail}"
|
||||
)
|
||||
parent = _resolve_one_bottle(parent_name, raws, cache, seen + (name,))
|
||||
bottle = _merge_bottles(parent, child_raw, name)
|
||||
parent = _resolve_one_bottle(
|
||||
parent_name, raws, cache, repos_cache, seen + (name,)
|
||||
)
|
||||
merged_repos_raw = _resolve_repos_raw(repos_cache[parent_name], child_raw)
|
||||
bottle = _merge_bottles(parent, child_raw, merged_repos_raw, name)
|
||||
cache[name] = bottle
|
||||
repos_cache[name] = merged_repos_raw
|
||||
return bottle
|
||||
|
||||
|
||||
def _merge_bottles(
|
||||
parent: Bottle,
|
||||
parent: ManifestBottle,
|
||||
child_raw: dict[str, object],
|
||||
merged_repos_raw: dict[str, object],
|
||||
name: str,
|
||||
) -> Bottle:
|
||||
) -> ManifestBottle:
|
||||
"""Apply PRD 0025 merge rules."""
|
||||
from .manifest import Bottle, GitUser
|
||||
from .manifest import ManifestBottle, ManifestGitUser
|
||||
from .manifest_egress import validate_egress_routes
|
||||
from .manifest_util import as_json_object
|
||||
|
||||
# Parse the child's declared fields into a Bottle (with the
|
||||
# git-gate.repos: when the child declares repos, inject the already
|
||||
# name-merged repo set (computed by _resolve_repos_raw) so the child
|
||||
# parses with the full inherited+overridden list (issue #237).
|
||||
if _child_declares_git_gate_repos(child_raw):
|
||||
git_raw = as_json_object(child_raw.get("git-gate", {}), "child git-gate")
|
||||
child_raw = {**child_raw, "git-gate": {**git_raw, "repos": merged_repos_raw}}
|
||||
|
||||
# Parse the child's declared fields into a ManifestBottle (with the
|
||||
# usual defaults for anything missing). Validation runs the same
|
||||
# way it would for a leaf bottle: typos / wrong types die here.
|
||||
child = Bottle.from_dict(name, child_raw)
|
||||
child = ManifestBottle.from_dict(name, child_raw)
|
||||
|
||||
# env: dict merge, child wins on collision.
|
||||
merged_env = {**parent.env, **child.env}
|
||||
|
||||
# git-gate.user: per-field overlay. Each non-empty field on child
|
||||
# wins; empties fall through to parent. The default GitUser()
|
||||
# wins; empties fall through to parent. The default ManifestGitUser()
|
||||
# is two empty strings, so a child that omits git-gate.user
|
||||
# inherits the parent's user verbatim.
|
||||
merged_git_user = GitUser(
|
||||
merged_git_user = ManifestGitUser(
|
||||
name=child.git_user.name or parent.git_user.name,
|
||||
email=child.git_user.email or parent.git_user.email,
|
||||
)
|
||||
|
||||
# git-gate.repos: missing means inherit; an explicit empty object
|
||||
# clears; otherwise parent and child merge by UpstreamHost with
|
||||
# child entries replacing duplicate hosts.
|
||||
# git-gate.repos: when declared, child.git already holds the merged
|
||||
# set (an explicit empty dict clears parent, leaving child.git empty).
|
||||
# When omitted, the parent's entries are inherited verbatim.
|
||||
if _child_declares_git_gate_repos(child_raw):
|
||||
merged_git = _merge_git_remotes(parent.git, child.git) if child.git else ()
|
||||
merged_git = child.git
|
||||
else:
|
||||
merged_git = parent.git
|
||||
|
||||
# Presence-driven full-replace for the remaining list-valued +
|
||||
# scalar fields.
|
||||
merged_egress = child.egress if "egress" in child_raw else parent.egress
|
||||
# egress.routes: missing means inherit; otherwise parent and child
|
||||
# route lists concatenate. Other egress scalar fields remain
|
||||
# presence-driven overlays.
|
||||
merged_egress = (
|
||||
_merge_egress(parent.egress, child.egress, child_raw)
|
||||
if "egress" in child_raw
|
||||
else parent.egress
|
||||
)
|
||||
|
||||
# Presence-driven full-replace for the remaining scalar fields.
|
||||
merged_agent_provider = (
|
||||
child.agent_provider
|
||||
if "agent_provider" in child_raw
|
||||
@@ -112,7 +139,7 @@ def _merge_bottles(
|
||||
)
|
||||
validate_egress_routes(name, merged_egress.routes)
|
||||
|
||||
return Bottle(
|
||||
return ManifestBottle(
|
||||
env=merged_env,
|
||||
agent_provider=merged_agent_provider,
|
||||
git=merged_git,
|
||||
@@ -122,6 +149,45 @@ def _merge_bottles(
|
||||
)
|
||||
|
||||
|
||||
def _resolve_repos_raw(
|
||||
parent_repos: dict[str, object],
|
||||
child_raw: dict[str, object],
|
||||
) -> dict[str, object]:
|
||||
"""Compute a bottle's effective git-gate.repos as raw dicts.
|
||||
|
||||
Repos are keyed by name. When the child omits git-gate.repos it
|
||||
inherits the parent's set verbatim; an explicit empty dict clears it.
|
||||
Otherwise parent and child unite by name, with same-name entries
|
||||
field-merged (parent fields are defaults, child fields win)."""
|
||||
from .manifest_util import as_json_object
|
||||
|
||||
if not _child_declares_git_gate_repos(child_raw):
|
||||
return parent_repos
|
||||
child_repos = _declared_repos_raw(child_raw)
|
||||
if not child_repos:
|
||||
return {}
|
||||
# Parent entries keep their order; child-only names are appended.
|
||||
names = list(parent_repos) + [n for n in child_repos if n not in parent_repos]
|
||||
return {
|
||||
name: {
|
||||
**as_json_object(parent_repos.get(name, {}), "parent git-gate repo"),
|
||||
**as_json_object(child_repos.get(name, {}), "child git-gate repo"),
|
||||
}
|
||||
for name in names
|
||||
}
|
||||
|
||||
|
||||
def _declared_repos_raw(child_raw: dict[str, object]) -> dict[str, object]:
|
||||
"""Return the child's explicitly declared git-gate.repos as raw dicts,
|
||||
or an empty dict when none are declared."""
|
||||
from .manifest_util import as_json_object
|
||||
|
||||
if not _child_declares_git_gate_repos(child_raw):
|
||||
return {}
|
||||
git_raw = as_json_object(child_raw.get("git-gate", {}), "child git-gate")
|
||||
return as_json_object(git_raw.get("repos", {}), "child git-gate.repos")
|
||||
|
||||
|
||||
def _child_declares_git_gate_repos(child_raw: dict[str, object]) -> bool:
|
||||
from .manifest_util import as_json_object
|
||||
|
||||
@@ -132,11 +198,15 @@ def _child_declares_git_gate_repos(child_raw: dict[str, object]) -> bool:
|
||||
return "repos" in git_obj
|
||||
|
||||
|
||||
def _merge_git_remotes(
|
||||
parent: tuple[GitEntry, ...],
|
||||
child: tuple[GitEntry, ...],
|
||||
) -> tuple[GitEntry, ...]:
|
||||
by_host = {entry.UpstreamHost: entry for entry in parent}
|
||||
for entry in child:
|
||||
by_host[entry.UpstreamHost] = entry
|
||||
return tuple(by_host.values())
|
||||
def _merge_egress(
|
||||
parent: ManifestEgressConfig,
|
||||
child: ManifestEgressConfig,
|
||||
child_raw: dict[str, object],
|
||||
) -> ManifestEgressConfig:
|
||||
from .manifest_egress import ManifestEgressConfig
|
||||
from .manifest_util import as_json_object
|
||||
|
||||
child_egress_raw = as_json_object(child_raw.get("egress"), "child egress")
|
||||
routes = parent.routes + child.routes
|
||||
log = child.Log if "log" in child_egress_raw else parent.Log
|
||||
return ManifestEgressConfig(routes=routes, Log=log)
|
||||
|
||||
+83
-76
@@ -4,7 +4,6 @@ from __future__ import annotations
|
||||
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from typing import Optional
|
||||
|
||||
from .manifest_util import ManifestError, as_json_object
|
||||
|
||||
@@ -13,6 +12,8 @@ from .manifest_util import ManifestError, as_json_object
|
||||
# defence; this regex is belt-and-suspenders and documents intent).
|
||||
_GIT_NAME_RE = re.compile(r"^[A-Za-z0-9._-]+$")
|
||||
|
||||
_KEY_PROVIDERS = {"static", "gitea"}
|
||||
|
||||
|
||||
def _opt_str(value: object, label: str) -> str:
|
||||
if value is None:
|
||||
@@ -57,7 +58,7 @@ def parse_git_upstream(url: str, label: str) -> tuple[str, str, str, str]:
|
||||
return (user, host, port, path)
|
||||
|
||||
|
||||
def validate_unique_git_names(bottle_name: str, git: tuple[GitEntry, ...]) -> None:
|
||||
def validate_unique_git_names(bottle_name: str, git: tuple[ManifestGitEntry, ...]) -> None:
|
||||
seen: dict[str, None] = {}
|
||||
for g in git:
|
||||
if g.Name in seen:
|
||||
@@ -69,25 +70,27 @@ def validate_unique_git_names(bottle_name: str, git: tuple[GitEntry, ...]) -> No
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ProvisionedKeyConfig:
|
||||
"""Configuration for automatic deploy-key lifecycle management
|
||||
(PRD 0048). Used when a git-gate.repos entry opts out of a
|
||||
static identity file and instead wants a fresh SSH keypair
|
||||
generated at spin-up and revoked at teardown.
|
||||
class ManifestKeyConfig:
|
||||
"""Configuration for a repo's SSH key in git-gate.repos.
|
||||
|
||||
`provider` names the contrib sub-package to load (e.g. `gitea`).
|
||||
`token_env` is the name of a host-side env var carrying the API
|
||||
token; the value is read at provision time, never stored on the
|
||||
plan. `api_url` is the forge's HTTP API root; if empty, it is
|
||||
derived from the upstream URL's host at provision time."""
|
||||
`provider` is either `"static"` (a pre-existing key on the host) or
|
||||
`"gitea"` (automatic deploy-key lifecycle via the Gitea API).
|
||||
|
||||
For `static`: `path` is the host-side absolute path to the SSH private key.
|
||||
|
||||
For `gitea`: `forge_token_env` is the name of a host-side env var
|
||||
carrying the Gitea API token; the value is read at provision time,
|
||||
never stored on the plan. `api_url` is the forge's HTTP API root; if
|
||||
empty, it is derived from the upstream URL's host at provision time."""
|
||||
|
||||
provider: str
|
||||
token_env: str
|
||||
path: str = ""
|
||||
forge_token_env: str = ""
|
||||
api_url: str = ""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class GitEntry:
|
||||
class ManifestGitEntry:
|
||||
"""One upstream the per-agent git-gate (PRD 0008) is allowed to
|
||||
talk to. `Upstream` is the real remote URL the agent would push to
|
||||
if there were no gate; the gate hosts a bare repo at /git/<Name>.git
|
||||
@@ -99,15 +102,16 @@ class GitEntry:
|
||||
stashed in the `Upstream*` fields so the git-gate render step
|
||||
doesn't have to re-parse.
|
||||
|
||||
Manifest source: `git-gate.repos.<Name>` (PRD 0047/0048). Exactly
|
||||
one of `identity` (static key path) or `provisioned_key` (automatic
|
||||
lifecycle) must be present. The internal field names are stable."""
|
||||
Manifest source: `git-gate.repos.<Name>` (PRD 0047/0048). A `key`
|
||||
block is required; `key.provider` is `"static"` or `"gitea"`. For
|
||||
`static`, `IdentityFile` is populated at parse time from `key.path`.
|
||||
For `gitea`, `IdentityFile` is populated at provision time."""
|
||||
|
||||
Name: str
|
||||
Upstream: str
|
||||
Key: ManifestKeyConfig = ManifestKeyConfig(provider="")
|
||||
IdentityFile: str = ""
|
||||
KnownHostKey: str = ""
|
||||
ProvisionedKey: Optional[ProvisionedKeyConfig] = None
|
||||
RemoteKey: str = ""
|
||||
UpstreamUser: str = ""
|
||||
UpstreamHost: str = ""
|
||||
@@ -117,11 +121,11 @@ class GitEntry:
|
||||
@classmethod
|
||||
def from_repos_entry(
|
||||
cls, bottle_name: str, repo_name: str, raw: object
|
||||
) -> "GitEntry":
|
||||
) -> "ManifestGitEntry":
|
||||
"""Parse one entry from `git-gate.repos.<repo_name>`.
|
||||
|
||||
YAML keys: `url` (required), exactly one of `identity` or
|
||||
`provisioned_key` (required), `host_key` (optional).
|
||||
YAML keys: `url` (required), `key` (required object with
|
||||
`provider`, and provider-specific fields), `host_key` (optional).
|
||||
The repo_name becomes `Name`."""
|
||||
if not repo_name:
|
||||
raise ManifestError(
|
||||
@@ -135,10 +139,10 @@ class GitEntry:
|
||||
label = f"git-gate.repos[{repo_name!r}]"
|
||||
d = as_json_object(raw, f"bottle '{bottle_name}' {label}")
|
||||
for k in d:
|
||||
if k not in {"url", "identity", "provisioned_key", "host_key"}:
|
||||
if k not in {"url", "key", "host_key"}:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label} has unknown key {k!r}; "
|
||||
f"allowed: url, identity, provisioned_key, host_key"
|
||||
f"allowed: url, key, host_key"
|
||||
)
|
||||
upstream = d.get("url")
|
||||
if not isinstance(upstream, str) or not upstream:
|
||||
@@ -146,32 +150,13 @@ class GitEntry:
|
||||
f"bottle '{bottle_name}' {label} missing required string field 'url'"
|
||||
)
|
||||
|
||||
has_identity = "identity" in d
|
||||
has_provisioned = "provisioned_key" in d
|
||||
if has_identity and has_provisioned:
|
||||
if "key" not in d:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label} must set exactly one of "
|
||||
f"'identity' or 'provisioned_key'; got both."
|
||||
)
|
||||
if not has_identity and not has_provisioned:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label} must set exactly one of "
|
||||
f"'identity' or 'provisioned_key'; got neither."
|
||||
f"bottle '{bottle_name}' {label} missing required 'key' block"
|
||||
)
|
||||
key_config = _parse_key_config(bottle_name, label, d["key"])
|
||||
|
||||
ident = ""
|
||||
provisioned_key: Optional[ProvisionedKeyConfig] = None
|
||||
if has_identity:
|
||||
raw_ident = d.get("identity")
|
||||
if not isinstance(raw_ident, str) or not raw_ident:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label} 'identity' must be a non-empty string"
|
||||
)
|
||||
ident = raw_ident
|
||||
else:
|
||||
provisioned_key = _parse_provisioned_key_config(
|
||||
bottle_name, label, d["provisioned_key"]
|
||||
)
|
||||
ident = key_config.path if key_config.provider == "static" else ""
|
||||
|
||||
khk = _opt_str(
|
||||
d.get("host_key"),
|
||||
@@ -183,9 +168,9 @@ class GitEntry:
|
||||
return cls(
|
||||
Name=repo_name,
|
||||
Upstream=upstream,
|
||||
Key=key_config,
|
||||
IdentityFile=ident,
|
||||
KnownHostKey=khk,
|
||||
ProvisionedKey=provisioned_key,
|
||||
RemoteKey=host,
|
||||
UpstreamUser=user,
|
||||
UpstreamHost=host,
|
||||
@@ -194,42 +179,64 @@ class GitEntry:
|
||||
)
|
||||
|
||||
|
||||
def _parse_provisioned_key_config(
|
||||
def _parse_key_config(
|
||||
bottle_name: str, label: str, raw: object
|
||||
) -> ProvisionedKeyConfig:
|
||||
d = as_json_object(raw, f"bottle '{bottle_name}' {label}.provisioned_key")
|
||||
for k in d:
|
||||
if k not in {"provider", "token_env", "api_url"}:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label}.provisioned_key has unknown key {k!r}; "
|
||||
f"allowed: provider, token_env, api_url"
|
||||
)
|
||||
) -> ManifestKeyConfig:
|
||||
d = as_json_object(raw, f"bottle '{bottle_name}' {label}.key")
|
||||
provider = d.get("provider")
|
||||
if not isinstance(provider, str) or not provider:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label}.provisioned_key missing required "
|
||||
f"bottle '{bottle_name}' {label}.key missing required "
|
||||
f"string field 'provider'"
|
||||
)
|
||||
token_env = d.get("token_env")
|
||||
if not isinstance(token_env, str) or not token_env:
|
||||
if provider not in _KEY_PROVIDERS:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label}.provisioned_key missing required "
|
||||
f"string field 'token_env'"
|
||||
f"bottle '{bottle_name}' {label}.key provider {provider!r} is unknown; "
|
||||
f"allowed: {', '.join(sorted(_KEY_PROVIDERS))}"
|
||||
)
|
||||
api_url_raw = d.get("api_url", "")
|
||||
if not isinstance(api_url_raw, str):
|
||||
|
||||
if provider == "gitea":
|
||||
for k in d:
|
||||
if k not in {"provider", "forge_token_env", "api_url"}:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label}.key has unknown key {k!r} "
|
||||
f"for provider 'gitea'; allowed: provider, forge_token_env, api_url"
|
||||
)
|
||||
forge_token_env = d.get("forge_token_env")
|
||||
if not isinstance(forge_token_env, str) or not forge_token_env:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label}.key missing required "
|
||||
f"string field 'forge_token_env' for provider 'gitea'"
|
||||
)
|
||||
api_url_raw = d.get("api_url", "")
|
||||
if not isinstance(api_url_raw, str):
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label}.key 'api_url' must be a string"
|
||||
)
|
||||
return ManifestKeyConfig(
|
||||
provider=provider,
|
||||
forge_token_env=forge_token_env,
|
||||
api_url=api_url_raw,
|
||||
)
|
||||
|
||||
# provider == "static"
|
||||
for k in d:
|
||||
if k not in {"provider", "path"}:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label}.key has unknown key {k!r} "
|
||||
f"for provider 'static'; allowed: provider, path"
|
||||
)
|
||||
path = d.get("path")
|
||||
if not isinstance(path, str) or not path:
|
||||
raise ManifestError(
|
||||
f"bottle '{bottle_name}' {label}.provisioned_key 'api_url' must be a string"
|
||||
f"bottle '{bottle_name}' {label}.key missing required "
|
||||
f"string field 'path' for provider 'static'"
|
||||
)
|
||||
return ProvisionedKeyConfig(
|
||||
provider=provider,
|
||||
token_env=token_env,
|
||||
api_url=api_url_raw,
|
||||
)
|
||||
return ManifestKeyConfig(provider=provider, path=path)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class GitUser:
|
||||
class ManifestGitUser:
|
||||
"""Per-bottle `git config --global user.name` / `user.email`
|
||||
pair (issue #86). The agent's commits inside the bottle are
|
||||
attributed to this identity rather than the agent image's
|
||||
@@ -244,7 +251,7 @@ class GitUser:
|
||||
email: str = ""
|
||||
|
||||
@classmethod
|
||||
def from_dict(cls, bottle_name: str, raw: object) -> "GitUser":
|
||||
def from_dict(cls, bottle_name: str, raw: object) -> "ManifestGitUser":
|
||||
d = as_json_object(raw, f"bottle '{bottle_name}' git-gate.user")
|
||||
for k in d:
|
||||
if k not in {"name", "email"}:
|
||||
@@ -279,7 +286,7 @@ class GitUser:
|
||||
def parse_git_gate_config(
|
||||
bottle_name: str,
|
||||
raw: object,
|
||||
) -> tuple[tuple[GitEntry, ...], GitUser]:
|
||||
) -> tuple[tuple[ManifestGitEntry, ...], ManifestGitUser]:
|
||||
d = as_json_object(raw, f"bottle '{bottle_name}' git-gate")
|
||||
for k in d:
|
||||
if k not in {"user", "repos"}:
|
||||
@@ -289,17 +296,17 @@ def parse_git_gate_config(
|
||||
)
|
||||
|
||||
git_user = (
|
||||
GitUser.from_dict(bottle_name, d["user"])
|
||||
ManifestGitUser.from_dict(bottle_name, d["user"])
|
||||
if "user" in d
|
||||
else GitUser()
|
||||
else ManifestGitUser()
|
||||
)
|
||||
|
||||
git: tuple[GitEntry, ...] = ()
|
||||
git: tuple[ManifestGitEntry, ...] = ()
|
||||
repos_raw = d.get("repos")
|
||||
if repos_raw is not None:
|
||||
repos = as_json_object(repos_raw, f"bottle '{bottle_name}' git-gate.repos")
|
||||
git = tuple(
|
||||
GitEntry.from_repos_entry(bottle_name, name, entry)
|
||||
ManifestGitEntry.from_repos_entry(bottle_name, name, entry)
|
||||
for name, entry in repos.items()
|
||||
)
|
||||
validate_unique_git_names(bottle_name, git)
|
||||
|
||||
@@ -8,21 +8,19 @@ from typing import TYPE_CHECKING
|
||||
from .log import warn
|
||||
from .manifest_schema import (
|
||||
entity_name_from_path,
|
||||
validate_agent_frontmatter_keys,
|
||||
validate_bottle_frontmatter_keys,
|
||||
)
|
||||
from .manifest_util import ManifestError
|
||||
from .yaml_subset import YamlSubsetError, parse_frontmatter
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from .manifest import Agent, Bottle
|
||||
from .manifest import ManifestBottle
|
||||
|
||||
|
||||
def check_stale_json(dir_path: Path, md_dir: Path, label: str) -> None:
|
||||
"""Die if `<dir_path>/bot-bottle.json` exists but `md_dir` does
|
||||
not. The manifest format changed in PRD 0011 and we do not want
|
||||
to silently leave the JSON content unused."""
|
||||
from .manifest import ManifestError
|
||||
|
||||
legacy = dir_path / "bot-bottle.json"
|
||||
if legacy.is_file() and not md_dir.exists():
|
||||
raise ManifestError(
|
||||
@@ -34,48 +32,13 @@ def check_stale_json(dir_path: Path, md_dir: Path, label: str) -> None:
|
||||
)
|
||||
|
||||
|
||||
def load_bottles_from_dir(bottles_dir: Path) -> dict[str, Bottle]:
|
||||
"""Walk `<bottles_dir>/*.md`, parse each as a bottle, and return
|
||||
`{name: Bottle}`. Missing dir returns an empty dict."""
|
||||
from .manifest import ManifestError
|
||||
from .manifest_extends import resolve_bottles
|
||||
def scan_agent_names(agents_dir: Path) -> dict[str, Path]:
|
||||
"""Scan `<agents_dir>/*.md` for valid filenames and return `{name: path}`.
|
||||
|
||||
raws: dict[str, dict[str, object]] = {}
|
||||
if not bottles_dir.is_dir():
|
||||
return {}
|
||||
for path in sorted(bottles_dir.glob("*.md")):
|
||||
name = entity_name_from_path(path)
|
||||
if name is None:
|
||||
warn(
|
||||
f"skipping {path}: filename must match "
|
||||
f"[a-z][a-z0-9-]*.md (got {path.name!r})"
|
||||
)
|
||||
continue
|
||||
try:
|
||||
fm, _body = parse_frontmatter(path.read_text())
|
||||
except OSError as e:
|
||||
raise ManifestError(f"could not read {path}: {e}") from e
|
||||
except YamlSubsetError as e:
|
||||
raise ManifestError(f"{path}: {e}") from e
|
||||
validate_bottle_frontmatter_keys(path, fm.keys())
|
||||
raws[name] = fm
|
||||
return resolve_bottles(raws)
|
||||
|
||||
|
||||
def load_agents_from_dir(
|
||||
agents_dir: Path,
|
||||
bottle_names: set[str],
|
||||
*,
|
||||
source: str, # noqa: F841 — unused, but required by interface
|
||||
) -> dict[str, Agent]:
|
||||
"""Walk `<agents_dir>/*.md`, parse each as an agent, and return
|
||||
`{name: Agent}`. The Markdown body becomes the agent's prompt.
|
||||
Missing dir returns an empty dict."""
|
||||
from .manifest import Agent, ManifestError
|
||||
|
||||
out: dict[str, Agent] = {}
|
||||
No file content is read. Invalid filenames are skipped with a warning."""
|
||||
result: dict[str, Path] = {}
|
||||
if not agents_dir.is_dir():
|
||||
return out
|
||||
return result
|
||||
for path in sorted(agents_dir.glob("*.md")):
|
||||
name = entity_name_from_path(path)
|
||||
if name is None:
|
||||
@@ -84,22 +47,45 @@ def load_agents_from_dir(
|
||||
f"[a-z][a-z0-9-]*.md (got {path.name!r})"
|
||||
)
|
||||
continue
|
||||
result[name] = path
|
||||
return result
|
||||
|
||||
|
||||
def load_bottle_chain_from_dir(
|
||||
bottle_name: str, bottles_dir: Path
|
||||
) -> ManifestBottle:
|
||||
"""Load `bottle_name` and its full `extends:` chain from `bottles_dir`,
|
||||
returning the resolved ManifestBottle.
|
||||
|
||||
Only the files in the extends chain are read — unrelated bottle files
|
||||
are never touched. Raises ManifestError on parse or validation failure."""
|
||||
from .manifest_extends import resolve_bottles
|
||||
|
||||
raws: dict[str, dict[str, object]] = {}
|
||||
to_load = [bottle_name]
|
||||
while to_load:
|
||||
name = to_load.pop()
|
||||
if name in raws:
|
||||
continue
|
||||
path = bottles_dir / f"{name}.md"
|
||||
if not path.is_file():
|
||||
avail = ", ".join(
|
||||
p.stem for p in sorted(bottles_dir.glob("*.md")) if p.is_file()
|
||||
) or "(none)"
|
||||
raise ManifestError(
|
||||
f"bottle '{name}' not found at {path}. "
|
||||
f"Available: {avail}"
|
||||
)
|
||||
try:
|
||||
fm, body = parse_frontmatter(path.read_text())
|
||||
fm, _body = parse_frontmatter(path.read_text())
|
||||
except OSError as e:
|
||||
raise ManifestError(f"could not read {path}: {e}") from e
|
||||
except YamlSubsetError as e:
|
||||
raise ManifestError(f"{path}: {e}") from e
|
||||
validate_agent_frontmatter_keys(path, fm.keys())
|
||||
# Build the dict Agent.from_dict expects. The body becomes
|
||||
# prompt; Claude Code passthrough fields stay in fm and get
|
||||
# ignored by Agent.from_dict (reads bottle/skills/git-gate/prompt).
|
||||
agent_dict: dict[str, object] = {
|
||||
"bottle": fm.get("bottle"),
|
||||
"skills": fm.get("skills", []),
|
||||
"prompt": body.strip(),
|
||||
}
|
||||
if "git-gate" in fm:
|
||||
agent_dict["git-gate"] = fm["git-gate"]
|
||||
out[name] = Agent.from_dict(name, agent_dict, bottle_names)
|
||||
return out
|
||||
validate_bottle_frontmatter_keys(path, fm.keys())
|
||||
raws[name] = dict(fm)
|
||||
parent = fm.get("extends")
|
||||
if isinstance(parent, str):
|
||||
to_load.append(parent)
|
||||
|
||||
return resolve_bottles(raws)[bottle_name]
|
||||
|
||||
@@ -59,6 +59,7 @@ class _DaemonSpec:
|
||||
# reads to inject `Authorization` headers on configured routes;
|
||||
# no other daemon in the bundle should see these values.
|
||||
_EGRESS_ONLY_ENV_PREFIXES: tuple[str, ...] = ("EGRESS_TOKEN_",)
|
||||
_READY_GATED_DAEMONS: tuple[str, ...] = ("git-gate", "git-http")
|
||||
|
||||
|
||||
def _env_for_daemon(name: str, base_env: dict[str, str]) -> dict[str, str]:
|
||||
@@ -82,6 +83,22 @@ _DAEMONS: tuple[_DaemonSpec, ...] = (
|
||||
)
|
||||
|
||||
|
||||
def _argv_for_daemon(name: str, argv: Sequence[str], env: dict[str, str]) -> list[str]:
|
||||
ready_file = env.get("BOT_BOTTLE_GIT_GATE_READY_FILE", "").strip()
|
||||
if name not in _READY_GATED_DAEMONS or not ready_file:
|
||||
return list(argv)
|
||||
return [
|
||||
"/bin/sh",
|
||||
"-c",
|
||||
"while [ ! -f \"$BOT_BOTTLE_GIT_GATE_READY_FILE\" ]; do "
|
||||
"sleep 0.1; "
|
||||
"done; "
|
||||
"exec \"$@\"",
|
||||
name,
|
||||
*argv,
|
||||
]
|
||||
|
||||
|
||||
def _selected_daemons(
|
||||
env: dict[str, str],
|
||||
all_daemons: Sequence[_DaemonSpec] | None = None,
|
||||
@@ -118,12 +135,13 @@ def _pump(name: str, stream: IO[bytes]) -> None:
|
||||
|
||||
|
||||
def _spawn(spec: _DaemonSpec) -> subprocess.Popen[bytes]:
|
||||
env = _env_for_daemon(spec.name, dict(os.environ))
|
||||
proc = subprocess.Popen(
|
||||
list(spec.argv),
|
||||
_argv_for_daemon(spec.name, spec.argv, env),
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT,
|
||||
bufsize=0,
|
||||
env=_env_for_daemon(spec.name, dict(os.environ)),
|
||||
env=env,
|
||||
)
|
||||
threading.Thread(
|
||||
target=_pump, args=(spec.name, proc.stdout), daemon=True
|
||||
|
||||
+22
-17
@@ -5,7 +5,7 @@ queue/audit support. The sidecar (bot_bottle.supervise_server)
|
||||
sits on the bottle's internal network and exposes three MCP tools the
|
||||
agent calls when it hits a stuck-recovery category:
|
||||
|
||||
* egress-block — agent proposes a new routes.yaml
|
||||
* egress-block / allow — agent proposes a new routes.yaml
|
||||
* capability-block — agent proposes a new agent Dockerfile
|
||||
|
||||
Each tool call: the agent passes the full proposed file plus a
|
||||
@@ -49,27 +49,36 @@ SUPERVISE_HOSTNAME = "supervise"
|
||||
SUPERVISE_PORT = 9100
|
||||
|
||||
TOOL_CAPABILITY_BLOCK = "capability-block"
|
||||
TOOL_EGRESS_BLOCK = "egress-block"
|
||||
TOOL_ALLOW = "allow"
|
||||
TOOL_GITLEAKS_ALLOW = "gitleaks-allow"
|
||||
TOOL_LIST_EGRESS_ROUTES = "list-egress-routes"
|
||||
TOOLS: tuple[str, ...] = (
|
||||
TOOL_ALLOW,
|
||||
TOOL_CAPABILITY_BLOCK,
|
||||
TOOL_EGRESS_BLOCK,
|
||||
TOOL_GITLEAKS_ALLOW,
|
||||
TOOL_LIST_EGRESS_ROUTES,
|
||||
)
|
||||
|
||||
# The supervise sidecar uses these to query egress's
|
||||
# introspection endpoint for the `list-egress-routes` MCP
|
||||
# tool. The hostname + port match egress's docker network
|
||||
# alias + listen port (see bot_bottle.egress.EGRESS_HOSTNAME
|
||||
# and backend.docker.egress.EGRESS_PORT — the values
|
||||
# are inlined here so the in-container supervise_server doesn't
|
||||
# need to import the egress package).
|
||||
EGRESS_FORWARD_PROXY = "http://egress:9099"
|
||||
# listen port (see backend.docker.egress.EGRESS_PORT). The supervise
|
||||
# daemon runs inside the sidecar bundle alongside egress, so loopback
|
||||
# is the stable address across docker, smolmachines, and Apple
|
||||
# Container backends.
|
||||
EGRESS_FORWARD_PROXY = "http://127.0.0.1:9099"
|
||||
EGRESS_INTROSPECT_URL = "http://_egress.local/allowlist"
|
||||
|
||||
# capability-block has no on-disk config the operator edits in place
|
||||
# (the Dockerfile is rebuilt, not patched), so it has no audit log
|
||||
# here — those changes are captured by git history + the rebuild
|
||||
# record laid down in PRD 0016. egress-block was removed in issue #198.
|
||||
COMPONENT_FOR_TOOL: dict[str, str] = {}
|
||||
# here — those changes are captured by git history + the rebuild record
|
||||
# laid down in PRD 0016.
|
||||
COMPONENT_FOR_TOOL: dict[str, str] = {
|
||||
TOOL_ALLOW: "egress",
|
||||
TOOL_EGRESS_BLOCK: "egress",
|
||||
}
|
||||
|
||||
STATUS_APPROVED = "approved"
|
||||
STATUS_MODIFIED = "modified"
|
||||
@@ -431,9 +440,9 @@ def sha256_hex(content: str) -> str:
|
||||
# Dockerfile and propose modifications.
|
||||
#
|
||||
# routes.yaml + allowlist used to live here too; PRD 0017 chunk 3
|
||||
# moved them behind the `list-egress-routes` MCP tool (live
|
||||
# state from egress's introspection endpoint) so the agent
|
||||
# always sees current data rather than a launch-time snapshot.
|
||||
# moved them behind the `list-egress-routes` MCP tool (live state
|
||||
# from egress's introspection endpoint) so the agent always sees
|
||||
# current data rather than a launch-time snapshot.
|
||||
CURRENT_CONFIG_DOCKERFILE = "Dockerfile"
|
||||
|
||||
|
||||
@@ -465,8 +474,6 @@ class Supervise(ABC):
|
||||
self,
|
||||
slug: str,
|
||||
stage_dir: Path,
|
||||
*,
|
||||
dockerfile_content: str = "",
|
||||
) -> SupervisePlan:
|
||||
"""Stage the per-bottle queue dir on the host and the
|
||||
current-config dir under `stage_dir`. Returns the plan;
|
||||
@@ -476,9 +483,6 @@ class Supervise(ABC):
|
||||
queue_dir.mkdir(parents=True, exist_ok=True)
|
||||
current_config_dir = stage_dir / "current-config"
|
||||
current_config_dir.mkdir(parents=True, exist_ok=True)
|
||||
dockerfile_path = current_config_dir / CURRENT_CONFIG_DOCKERFILE
|
||||
dockerfile_path.write_text(dockerfile_content)
|
||||
dockerfile_path.chmod(0o644)
|
||||
return SupervisePlan(
|
||||
slug=slug,
|
||||
queue_dir=queue_dir,
|
||||
@@ -551,6 +555,7 @@ __all__ = [
|
||||
"EGRESS_FORWARD_PROXY",
|
||||
"EGRESS_INTROSPECT_URL",
|
||||
"TOOL_CAPABILITY_BLOCK",
|
||||
"TOOL_GITLEAKS_ALLOW",
|
||||
"TOOL_LIST_EGRESS_ROUTES",
|
||||
"archive_proposal",
|
||||
"audit_dir",
|
||||
|
||||
+108
-10
@@ -1,8 +1,8 @@
|
||||
"""Supervise sidecar HTTP server (PRD 0013).
|
||||
|
||||
Per-bottle MCP server exposing tools the agent calls to propose config
|
||||
changes when stuck. The egress-block tool was removed in issue #198;
|
||||
the remaining tools are `capability-block` and `list-egress-routes`.
|
||||
changes when stuck. The tools are `allow`, `egress-block`,
|
||||
`capability-block`, and `list-egress-routes`.
|
||||
|
||||
Each queued tool call:
|
||||
|
||||
@@ -44,9 +44,15 @@ import urllib.request
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
|
||||
# Same-directory import inside the bundle container; `supervise.py`
|
||||
# is COPYed alongside this file by Dockerfile.sidecars.
|
||||
import supervise as _sv
|
||||
try:
|
||||
# Same-directory imports inside the bundle container; these files are
|
||||
# COPYed flat under /app by Dockerfile.sidecars.
|
||||
from egress_addon_core import load_routes
|
||||
import supervise as _sv
|
||||
except ModuleNotFoundError:
|
||||
# Package imports for host-side tests and tooling.
|
||||
from .egress_addon_core import load_routes
|
||||
from . import supervise as _sv
|
||||
|
||||
|
||||
# --- JSON-RPC / MCP plumbing ----------------------------------------------
|
||||
@@ -142,8 +148,9 @@ TOOL_DEFINITIONS: list[dict[str, object]] = [
|
||||
"allowlist. Returns JSON with one entry per allowed host, "
|
||||
"each carrying its matches rules (if any) and whether "
|
||||
"the proxy injects Authorization for the route. Use this "
|
||||
"before composing an `egress-block` proposal so the new "
|
||||
"routes file extends the live one rather than replacing it."
|
||||
"before composing an `allow` or `egress-block` proposal so "
|
||||
"the new routes file extends the live one rather than "
|
||||
"replacing it."
|
||||
),
|
||||
"inputSchema": {
|
||||
"type": "object",
|
||||
@@ -151,6 +158,88 @@ TOOL_DEFINITIONS: list[dict[str, object]] = [
|
||||
"additionalProperties": False,
|
||||
},
|
||||
},
|
||||
{
|
||||
"name": _sv.TOOL_ALLOW,
|
||||
"description": (
|
||||
"Request operator approval to change the bottle's egress "
|
||||
"allowlist. Pass the full proposed routes.yaml content, not "
|
||||
"just the new host, plus a justification. Use "
|
||||
"`list-egress-routes` first so the proposal preserves existing "
|
||||
"routes."
|
||||
),
|
||||
"inputSchema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"routes_yaml": {
|
||||
"type": "string",
|
||||
"description": (
|
||||
"Full proposed /etc/egress/routes.yaml content. "
|
||||
"Each route entry accepts these keys:\n"
|
||||
" host: <hostname> (required)\n"
|
||||
" auth_scheme: Bearer|token (must pair with token_env)\n"
|
||||
" token_env: <ENV_VAR_NAME> (must pair with auth_scheme)\n"
|
||||
" matches: (optional list of match entries)\n"
|
||||
" - paths: [{type: prefix|exact|regex, value: /...}]\n"
|
||||
" methods: [GET, POST, ...]\n"
|
||||
" headers: [{name: X-Hdr, value: val, type: exact|regex}]\n"
|
||||
" git: (optional; omit to block git clone/fetch)\n"
|
||||
" fetch: true\n"
|
||||
" dlp: (optional DLP scanner overrides)\n"
|
||||
" outbound_detectors: [token_patterns, known_secrets]\n"
|
||||
" inbound_detectors: [naive_injection_detection]\n"
|
||||
"Omit any key that should use its default. "
|
||||
"`list-egress-routes` returns routes in this same format."
|
||||
),
|
||||
},
|
||||
"justification": {
|
||||
"type": "string",
|
||||
"description": "Why this egress route is needed.",
|
||||
},
|
||||
},
|
||||
"required": ["routes_yaml", "justification"],
|
||||
},
|
||||
},
|
||||
{
|
||||
"name": _sv.TOOL_EGRESS_BLOCK,
|
||||
"description": (
|
||||
"Request operator approval to change the bottle's egress "
|
||||
"allowlist after a blocked outbound request. Pass the full "
|
||||
"proposed routes.yaml content plus a justification. Use "
|
||||
"`list-egress-routes` first so the proposal preserves existing "
|
||||
"routes."
|
||||
),
|
||||
"inputSchema": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"routes_yaml": {
|
||||
"type": "string",
|
||||
"description": (
|
||||
"Full proposed /etc/egress/routes.yaml content. "
|
||||
"Each route entry accepts these keys:\n"
|
||||
" host: <hostname> (required)\n"
|
||||
" auth_scheme: Bearer|token (must pair with token_env)\n"
|
||||
" token_env: <ENV_VAR_NAME> (must pair with auth_scheme)\n"
|
||||
" matches: (optional list of match entries)\n"
|
||||
" - paths: [{type: prefix|exact|regex, value: /...}]\n"
|
||||
" methods: [GET, POST, ...]\n"
|
||||
" headers: [{name: X-Hdr, value: val, type: exact|regex}]\n"
|
||||
" git: (optional; omit to block git clone/fetch)\n"
|
||||
" fetch: true\n"
|
||||
" dlp: (optional DLP scanner overrides)\n"
|
||||
" outbound_detectors: [token_patterns, known_secrets]\n"
|
||||
" inbound_detectors: [naive_injection_detection]\n"
|
||||
"Omit any key that should use its default. "
|
||||
"`list-egress-routes` returns routes in this same format."
|
||||
),
|
||||
},
|
||||
"justification": {
|
||||
"type": "string",
|
||||
"description": "Why this egress route is needed.",
|
||||
},
|
||||
},
|
||||
"required": ["routes_yaml", "justification"],
|
||||
},
|
||||
},
|
||||
{
|
||||
"name": _sv.TOOL_CAPABILITY_BLOCK,
|
||||
"description": (
|
||||
@@ -182,11 +271,12 @@ TOOL_DEFINITIONS: list[dict[str, object]] = [
|
||||
]
|
||||
|
||||
|
||||
# Map each non-egress tool to the input field that carries the agent's
|
||||
# payload (stored in Proposal.proposed_file). egress-block builds its
|
||||
# payload from structured input fields in `handle_egress_block`.
|
||||
# Map each proposal tool to the input field that carries the agent's
|
||||
# payload (stored in Proposal.proposed_file).
|
||||
PROPOSED_FILE_FIELD: dict[str, str] = {
|
||||
_sv.TOOL_ALLOW: "routes_yaml",
|
||||
_sv.TOOL_CAPABILITY_BLOCK: "dockerfile",
|
||||
_sv.TOOL_EGRESS_BLOCK: "routes_yaml",
|
||||
}
|
||||
|
||||
|
||||
@@ -203,6 +293,14 @@ def validate_proposed_file(tool: str, content: str) -> None:
|
||||
# Dockerfiles are too varied to validate syntactically beyond
|
||||
# non-empty. The operator reads the diff in the TUI.
|
||||
pass
|
||||
elif tool in (_sv.TOOL_ALLOW, _sv.TOOL_EGRESS_BLOCK):
|
||||
try:
|
||||
load_routes(content)
|
||||
except ValueError as e:
|
||||
raise _RpcError(
|
||||
ERR_INVALID_PARAMS,
|
||||
f"{tool}: proposed routes.yaml is not valid: {e}",
|
||||
) from e
|
||||
else:
|
||||
raise _RpcError(ERR_INVALID_PARAMS, f"unknown tool {tool!r}")
|
||||
|
||||
|
||||
+30
-36
@@ -69,12 +69,6 @@ class YamlSubsetError(ValueError):
|
||||
egress sidecar's addon) handle it as a normal exception."""
|
||||
|
||||
|
||||
def die(msg: str) -> None:
|
||||
"""Module-local helper so the parser body reads cleanly. Just
|
||||
raises YamlSubsetError — the `bot-bottle: error: ` prefix
|
||||
is added by the boundary `die` in `bot_bottle.log`."""
|
||||
raise YamlSubsetError(msg)
|
||||
|
||||
|
||||
# --- Tokenizer / line preprocessing ----------------------------------------
|
||||
|
||||
@@ -119,7 +113,7 @@ def _tokenize(text: str) -> list[_Line]:
|
||||
# editors render them differently and the spec says spaces.
|
||||
leading = len(raw) - len(raw.lstrip(" \t"))
|
||||
if "\t" in raw[:leading]:
|
||||
die(f"yaml-subset: tab character in indent on line {n}")
|
||||
raise YamlSubsetError(f"yaml-subset: tab character in indent on line {n}")
|
||||
stripped = raw.strip()
|
||||
if not stripped:
|
||||
continue
|
||||
@@ -169,14 +163,14 @@ def _parse_scalar(s: str, lineno: int) -> object:
|
||||
s.startswith("'") and s.endswith("'")
|
||||
):
|
||||
if len(s) < 2:
|
||||
die(f"yaml-subset: unterminated quoted string on line {lineno}")
|
||||
raise YamlSubsetError(f"yaml-subset: unterminated quoted string on line {lineno}")
|
||||
body = s[1:-1]
|
||||
if s.startswith('"'):
|
||||
# JSON-style escapes for double quotes.
|
||||
try:
|
||||
return body.encode("utf-8").decode("unicode_escape")
|
||||
except UnicodeDecodeError as e:
|
||||
die(f"yaml-subset: bad escape on line {lineno}: {e}")
|
||||
raise YamlSubsetError(f"yaml-subset: bad escape on line {lineno}: {e}")
|
||||
else:
|
||||
# Single quotes: only '' → ' (standard YAML); no other escapes.
|
||||
return body.replace("''", "'")
|
||||
@@ -186,7 +180,7 @@ def _parse_scalar(s: str, lineno: int) -> object:
|
||||
if s in _RESERVED_BOOL_LIKE:
|
||||
if s in ("true", "false"):
|
||||
return s == "true"
|
||||
die(
|
||||
raise YamlSubsetError(
|
||||
f"yaml-subset: bare {s!r} on line {lineno} is ambiguous "
|
||||
f"(use literal `true` / `false`, or quote it as a string)"
|
||||
)
|
||||
@@ -203,22 +197,22 @@ def _parse_scalar(s: str, lineno: int) -> object:
|
||||
|
||||
# Look-alikes that we reject to keep the user in control.
|
||||
if _DATE_RX.match(s):
|
||||
die(
|
||||
raise YamlSubsetError(
|
||||
f"yaml-subset: bare {s!r} on line {lineno} looks like a "
|
||||
f"date — quote it as a string or use an explicit int"
|
||||
)
|
||||
if _OCTAL_RX.match(s):
|
||||
die(
|
||||
raise YamlSubsetError(
|
||||
f"yaml-subset: bare {s!r} on line {lineno} looks like an "
|
||||
f"octal/0-prefixed integer — quote it as a string"
|
||||
)
|
||||
if _HEX_RX.match(s):
|
||||
die(
|
||||
raise YamlSubsetError(
|
||||
f"yaml-subset: bare {s!r} on line {lineno} looks like a "
|
||||
f"hex integer — quote it as a string"
|
||||
)
|
||||
if _FLOAT_RX.match(s):
|
||||
die(
|
||||
raise YamlSubsetError(
|
||||
f"yaml-subset: floats not supported (line {lineno}, "
|
||||
f"value {s!r}); use an int or quote as a string"
|
||||
)
|
||||
@@ -241,7 +235,7 @@ def _parse_inline(s: str, lineno: int) -> object:
|
||||
s = s.strip()
|
||||
if s.startswith("["):
|
||||
if not s.endswith("]"):
|
||||
die(f"yaml-subset: unterminated `[` on line {lineno}")
|
||||
raise YamlSubsetError(f"yaml-subset: unterminated `[` on line {lineno}")
|
||||
body = s[1:-1].strip()
|
||||
if not body:
|
||||
return []
|
||||
@@ -252,21 +246,21 @@ def _parse_inline(s: str, lineno: int) -> object:
|
||||
return items
|
||||
if s.startswith("{"):
|
||||
if not s.endswith("}"):
|
||||
die(f"yaml-subset: unterminated `{{` on line {lineno}")
|
||||
raise YamlSubsetError(f"yaml-subset: unterminated `{{` on line {lineno}")
|
||||
body = s[1:-1].strip()
|
||||
if not body:
|
||||
return {}
|
||||
out: dict[str, object] = {}
|
||||
for raw in _split_flow(body, lineno, "dict"):
|
||||
if ":" not in raw:
|
||||
die(
|
||||
raise YamlSubsetError(
|
||||
f"yaml-subset: inline dict entry on line {lineno} "
|
||||
f"missing `:` ({raw!r})"
|
||||
)
|
||||
k, _, v = raw.partition(":")
|
||||
k = k.strip()
|
||||
if not _BARE_RX.match(k):
|
||||
die(
|
||||
raise YamlSubsetError(
|
||||
f"yaml-subset: inline dict key on line {lineno} "
|
||||
f"must be a bare identifier ({k!r})"
|
||||
)
|
||||
@@ -296,7 +290,7 @@ def _split_flow(body: str, lineno: int, kind: str) -> list[str]:
|
||||
elif ch in "]}":
|
||||
depth_b -= 1
|
||||
if depth_b > 0:
|
||||
die(
|
||||
raise YamlSubsetError(
|
||||
f"yaml-subset: nested flow {kind} on line "
|
||||
f"{lineno} (only one level of flow allowed)"
|
||||
)
|
||||
@@ -330,7 +324,7 @@ def _split_key_value(content: str, lineno: int) -> tuple[str, str]:
|
||||
# ambiguous with URLs etc.).
|
||||
if i + 1 >= len(content) or content[i + 1] in (" ", "\t"):
|
||||
return content[:i].strip(), content[i + 1:].lstrip()
|
||||
die(f"yaml-subset: line {lineno} missing `: ` separator: {content!r}")
|
||||
raise YamlSubsetError(f"yaml-subset: line {lineno} missing `: ` separator: {content!r}")
|
||||
return "", "" # unreachable, but needed for type checker
|
||||
|
||||
|
||||
@@ -341,15 +335,15 @@ def _parse_block(
|
||||
to live at `base_indent`. Returns (value, new_idx) where
|
||||
`new_idx` is the index of the first unconsumed line."""
|
||||
if idx >= len(lines):
|
||||
die("yaml-subset: unexpected end of document")
|
||||
raise YamlSubsetError("yaml-subset: unexpected end of document")
|
||||
first = lines[idx]
|
||||
if first.indent < base_indent:
|
||||
die(
|
||||
raise YamlSubsetError(
|
||||
f"yaml-subset: line {first.lineno} indented less than "
|
||||
f"expected (got {first.indent}, expected >= {base_indent})"
|
||||
)
|
||||
if first.indent > base_indent:
|
||||
die(
|
||||
raise YamlSubsetError(
|
||||
f"yaml-subset: line {first.lineno} indented more than "
|
||||
f"expected (got {first.indent}, expected {base_indent})"
|
||||
)
|
||||
@@ -366,18 +360,18 @@ def _parse_block_mapping(
|
||||
while idx < len(lines) and lines[idx].indent == base_indent:
|
||||
line = lines[idx]
|
||||
if line.content.startswith("- "):
|
||||
die(
|
||||
raise YamlSubsetError(
|
||||
f"yaml-subset: line {line.lineno} unexpected list "
|
||||
f"item at mapping indent (got `-`, expected `key:`)"
|
||||
)
|
||||
key, value_text = _split_key_value(line.content, line.lineno)
|
||||
if not _BARE_RX.match(key):
|
||||
die(
|
||||
raise YamlSubsetError(
|
||||
f"yaml-subset: line {line.lineno} key {key!r} is not "
|
||||
f"a bare identifier"
|
||||
)
|
||||
if key in out:
|
||||
die(
|
||||
raise YamlSubsetError(
|
||||
f"yaml-subset: line {line.lineno} duplicate key {key!r}"
|
||||
)
|
||||
if value_text:
|
||||
@@ -417,7 +411,7 @@ def _parse_block_list(
|
||||
content_col = base_indent + 2
|
||||
first_key, first_value_text = _split_key_value(rest, line.lineno)
|
||||
if not _BARE_RX.match(first_key):
|
||||
die(
|
||||
raise YamlSubsetError(
|
||||
f"yaml-subset: line {line.lineno} key {first_key!r} "
|
||||
f"is not a bare identifier"
|
||||
)
|
||||
@@ -440,12 +434,12 @@ def _parse_block_list(
|
||||
break # next list item, not a sibling key
|
||||
k, v_text = _split_key_value(ln.content, ln.lineno)
|
||||
if not _BARE_RX.match(k):
|
||||
die(
|
||||
raise YamlSubsetError(
|
||||
f"yaml-subset: line {ln.lineno} key {k!r} is "
|
||||
f"not a bare identifier"
|
||||
)
|
||||
if k in item:
|
||||
die(f"yaml-subset: line {ln.lineno} duplicate key {k!r}")
|
||||
raise YamlSubsetError(f"yaml-subset: line {ln.lineno} duplicate key {k!r}")
|
||||
if v_text:
|
||||
item[k] = _parse_inline(v_text, ln.lineno)
|
||||
idx += 1
|
||||
@@ -501,7 +495,7 @@ def parse_yaml_subset(text: str) -> dict[str, object]:
|
||||
for n, raw in enumerate(text.splitlines(), start=1):
|
||||
s = raw.strip()
|
||||
if s.startswith("|") or s.startswith(">") or s.startswith("- |") or s.startswith("- >"):
|
||||
die(
|
||||
raise YamlSubsetError(
|
||||
f"yaml-subset: line {n} uses a multi-line block "
|
||||
f"scalar (`|` / `>`) — not supported. Use a quoted "
|
||||
f"single-line string instead."
|
||||
@@ -511,12 +505,12 @@ def parse_yaml_subset(text: str) -> dict[str, object]:
|
||||
# not when it's inside a quoted string. Cheap check: any
|
||||
# bare `&foo:` / `*foo` at the start of a value position.
|
||||
if re.search(r"(^|\s)[&*][A-Za-z0-9_]+", s):
|
||||
die(
|
||||
raise YamlSubsetError(
|
||||
f"yaml-subset: line {n} uses anchors / aliases "
|
||||
f"(`&` / `*`) — not supported."
|
||||
)
|
||||
if "!!" in s and not (s.count("'") % 2 or s.count('"') % 2):
|
||||
die(
|
||||
raise YamlSubsetError(
|
||||
f"yaml-subset: line {n} uses a YAML tag (`!!`) — not "
|
||||
f"supported."
|
||||
)
|
||||
@@ -526,18 +520,18 @@ def parse_yaml_subset(text: str) -> dict[str, object]:
|
||||
return {}
|
||||
base_indent = lines[0].indent
|
||||
if base_indent != 0:
|
||||
die(
|
||||
raise YamlSubsetError(
|
||||
f"yaml-subset: top-level content must start in column 0 "
|
||||
f"(got column {base_indent} on line {lines[0].lineno})"
|
||||
)
|
||||
value, consumed = _parse_block(lines, 0, 0)
|
||||
if consumed < len(lines):
|
||||
die(
|
||||
raise YamlSubsetError(
|
||||
f"yaml-subset: trailing content starting on line "
|
||||
f"{lines[consumed].lineno}"
|
||||
)
|
||||
if not isinstance(value, dict):
|
||||
die("yaml-subset: top-level value must be a mapping")
|
||||
raise YamlSubsetError("yaml-subset: top-level value must be a mapping")
|
||||
return cast(dict[str, object], value)
|
||||
|
||||
|
||||
@@ -576,7 +570,7 @@ def parse_frontmatter(text: str) -> tuple[dict[str, object], str]:
|
||||
fm_end_lineno = line_idx
|
||||
break
|
||||
if body_start < 0:
|
||||
die("frontmatter: opening `---` has no matching closing `---`")
|
||||
raise YamlSubsetError("frontmatter: opening `---` has no matching closing `---`")
|
||||
|
||||
fm_text = text[line_starts[1]:line_starts[fm_end_lineno]] if fm_end_lineno > 1 else ""
|
||||
fm = parse_yaml_subset(fm_text)
|
||||
|
||||
@@ -13,13 +13,13 @@ Add Content-Length validation and a body-size cap to `git_http_backend.py` so ma
|
||||
|
||||
`bot_bottle/git_http_backend.py` calls `int(self.headers.get("Content-Length", 0))` without catching `ValueError`. A request with a non-numeric Content-Length raises an unhandled exception in the request handler.
|
||||
|
||||
The handler reads the full declared length into memory before passing the body to `git http-backend` with no upper bound. A local or compromised client can force arbitrarily high memory use. For comparison, `supervise_server.py` caps request bodies at 1 MiB.
|
||||
The handler reads the full declared length into memory before passing the body to `git http-backend` with no upper bound. A local or compromised client can force arbitrarily high memory use.
|
||||
|
||||
## Goals / Success Criteria
|
||||
|
||||
- A missing or non-numeric Content-Length returns HTTP 400.
|
||||
- A negative Content-Length returns HTTP 400.
|
||||
- A body larger than the cap (1 MiB, matching `supervise_server.py`) returns HTTP 413.
|
||||
- A body larger than the cap (100 MiB) returns HTTP 413.
|
||||
- Valid Git smart-HTTP pushes and fetches continue to work.
|
||||
- Unit tests cover: missing length, non-numeric length, negative length, over-cap length, and a valid push/fetch passthrough.
|
||||
|
||||
@@ -43,12 +43,12 @@ Out of scope:
|
||||
|
||||
## Design
|
||||
|
||||
Wrap the Content-Length parse in a try/except and return 400 on `ValueError`. Add an explicit check for negative values. After parsing, compare the declared length against a module-level `MAX_BODY_BYTES` constant (default 1 MiB) and return 413 if exceeded. Read exactly `min(content_length, MAX_BODY_BYTES)` bytes.
|
||||
Wrap the Content-Length parse in a try/except and return 400 on `ValueError`. Add an explicit check for negative values. After parsing, compare the declared length against a module-level `MAX_BODY_BYTES` constant (default 100 MiB) and return 413 if exceeded. Read exactly `min(content_length, MAX_BODY_BYTES)` bytes.
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
- Unit tests using `unittest.mock` to drive the handler with crafted headers.
|
||||
- Test cases: no Content-Length header, `Content-Length: abc`, `Content-Length: -1`, `Content-Length: 2097152` (over cap), and a normal small POST body.
|
||||
- Test cases: no Content-Length header, `Content-Length: abc`, `Content-Length: -1`, a declared length above `MAX_BODY_BYTES`, and a normal small POST body.
|
||||
|
||||
Run:
|
||||
|
||||
|
||||
@@ -199,6 +199,25 @@ Named inbound detectors: `naive_injection_detection`.
|
||||
The manifest parser (`manifest_egress.py`) validates the `dlp` block and
|
||||
rejects unknown detector names.
|
||||
|
||||
### Manifest schema — `git` block
|
||||
|
||||
HTTPS Git clone/fetch traffic is not implied by a host-level egress route.
|
||||
Smart HTTP Git fetch uses `git-upload-pack`, which can transfer large repo
|
||||
packfiles and bypass the git-gate mirror path. It is therefore blocked by
|
||||
default and must be explicitly enabled per route:
|
||||
|
||||
```yaml
|
||||
egress:
|
||||
routes:
|
||||
- host: github.com
|
||||
git:
|
||||
fetch: true
|
||||
```
|
||||
|
||||
`git.fetch: true` permits read-only smart HTTP clone/fetch requests
|
||||
(`git-upload-pack`) after the normal host and `matches` checks pass. HTTPS
|
||||
Git push (`git-receive-pack`) remains blocked by the egress addon.
|
||||
|
||||
### `EgressRoute` changes
|
||||
|
||||
`EgressRoute` replaces `PathAllowlist` with `Matches` and gains two new
|
||||
@@ -232,6 +251,7 @@ class EgressRoute:
|
||||
AuthScheme: str = ""
|
||||
TokenRef: str = ""
|
||||
Role: tuple[str, ...] = ()
|
||||
GitFetch: bool = False
|
||||
OutboundDetectors: tuple[str, ...] | None = None # None = all enabled
|
||||
InboundDetectors: tuple[str, ...] | None = None # None = all enabled
|
||||
```
|
||||
@@ -252,6 +272,7 @@ class Route:
|
||||
matches: tuple[MatchEntry, ...] = ()
|
||||
auth_scheme: str = ""
|
||||
token_env: str = ""
|
||||
git_fetch: bool = False
|
||||
outbound_detectors: tuple[str, ...] | None = None
|
||||
inbound_detectors: tuple[str, ...] | None = None
|
||||
```
|
||||
|
||||
@@ -0,0 +1,79 @@
|
||||
# PRD 0057: Promote smolmachines to default backend; convert Docker to example-only
|
||||
|
||||
- **Status:** Active
|
||||
- **Author:** didericis
|
||||
- **Created:** 2026-06-06
|
||||
- **Issue:** #206
|
||||
|
||||
## Summary
|
||||
|
||||
Make smolmachines the default bot-bottle backend and demote Docker to an example-only configuration. This closes the DNS sinkhole gap that exists in the Docker backend: the mitmproxy egress addon intercepts HTTP(S) but cannot see raw UDP port-53 DNS queries, so an agent can exfiltrate data via DNS tunnelling without the egress guard seeing it. The smolmachines backend eliminates this gap at the VMM layer — DNS filtering is built in and the agent container cannot bypass it.
|
||||
|
||||
## Problem
|
||||
|
||||
The current default backend is Docker. The egress addon (PRDs 0052/0053) intercepts HTTPS and scans request/response surfaces, but it is an HTTP proxy: raw UDP/TCP port-53 DNS queries go to the OS resolver and never pass through it. An agent can encode secrets as base32 or hex subdomains in a DNS query (`<encoded>.attacker.com`) and exfiltrate them silently.
|
||||
|
||||
The smolmachines backend already solves this: its Transport Socket Interface (TSI) enforces a CIDR allowlist at the VMM layer, and DNS is handled via vsock port 6002 — the guest's `/etc/resolv.conf` points at `127.0.0.1`, and a guest-side DNS proxy tunnels queries over vsock to the host, which returns NXDOMAIN for anything not on the allowlist. The agent cannot bypass this by hardcoding IPs or by configuring an alternate resolver, because both mechanisms are enforced below the guest OS.
|
||||
|
||||
Docker has no equivalent. Adding dnsmasq to the Docker backend would close the gap at some cost (dnsmasq sidecar, iptables `NET_ADMIN`, per-launch config generation), but it is the wrong direction if smolmachines supersedes Docker anyway.
|
||||
|
||||
## Goals / Success Criteria
|
||||
|
||||
- `BOT_BOTTLE_BACKEND` defaults to `smolmachines` when not set.
|
||||
- The existing Docker backend remains functional (not removed) but is no longer the default and is documented as legacy/example-only.
|
||||
- Example bottles (`examples/bottles/`) reference smolmachines, not Docker.
|
||||
- `AGENTS.md` documents the backend choice and the DNS gap closure.
|
||||
- Existing Docker-backed integration tests continue to pass; they select Docker explicitly via `BOT_BOTTLE_BACKEND=docker` rather than relying on the default.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Removing the Docker backend or its tests.
|
||||
- Implementing a dnsmasq layer for the Docker backend (closed by this change; not needed on the default path).
|
||||
- Iptables / `NET_ADMIN` work for Docker (deferred).
|
||||
- Subdomain-depth filtering for allowlisted zones (documented residual gap; tracked separately per the issue).
|
||||
|
||||
## Design
|
||||
|
||||
### Default backend change
|
||||
|
||||
`bot_bottle/backend/__init__.py`, line ~440:
|
||||
|
||||
```python
|
||||
# Before
|
||||
resolved = name or os.environ.get("BOT_BOTTLE_BACKEND") or "docker"
|
||||
|
||||
# After
|
||||
resolved = name or os.environ.get("BOT_BOTTLE_BACKEND") or "smolmachines"
|
||||
```
|
||||
|
||||
### DNS gap closure (how smolmachines handles it)
|
||||
|
||||
When the smolmachines backend launches an agent VM:
|
||||
|
||||
1. The VM's network device uses TSI (`--allow-host` / `--allow-cidr` flags), which enforces a CIDR allowlist at the VMM layer. The guest cannot dial IPs outside the allowlist even with raw sockets.
|
||||
2. The guest's `/etc/resolv.conf` is set to `127.0.0.1`; a guest-side DNS proxy relays queries over vsock port 6002 to the host.
|
||||
3. The host-side DNS filter returns NXDOMAIN for any hostname not in the allowlist derived from `egress.routes` in the bottle manifest.
|
||||
|
||||
This means DNS exfiltration via unknown subdomains is blocked by NXDOMAIN before the query leaves the host, and even if the agent hardcoded the IP of an attacker-controlled server, TSI would drop the packet at the VMM layer.
|
||||
|
||||
**Residual gap:** if the attacker controls a subdomain of an allowlisted zone (e.g., a legitimate zone like `api.anthropic.com` that the attacker can inject into via a separate compromise), DNS queries for that subdomain would be forwarded. This is accepted and documented.
|
||||
|
||||
### Example bottles
|
||||
|
||||
Update `examples/bottles/dev.md` and `examples/bottles/claude.md` to remove Docker-specific notes and reference smolmachines as the runtime.
|
||||
|
||||
### Integration test migration
|
||||
|
||||
Tests that exercise the Docker backend explicitly should set `BOT_BOTTLE_BACKEND=docker` rather than relying on the default. Tests that are backend-agnostic continue to use whatever `BOT_BOTTLE_BACKEND` is set to (defaulting to smolmachines in the test environment if available).
|
||||
|
||||
## Resolved questions
|
||||
|
||||
- **TSI + egress proxy loopback.** The implementation uses a per-bottle loopback alias rather than broad `127.0.0.1` passthrough. The smolmachines launch integration test now asserts that the guest receives proxy env vars on a `127.x` alias, can reach an allowlisted host through the proxy, cannot reach the same host directly with proxy vars unset, and cannot reach a non-allowlisted host through the proxy.
|
||||
- **smolmachines availability check.** The smolmachines preflight error points operators at the smolvm installer and explicitly suggests `BOT_BOTTLE_BACKEND=docker` / `--backend=docker` for legacy Docker-backed runs.
|
||||
|
||||
## References
|
||||
|
||||
- `docs/research/smolmachines-as-vm-backend.md` — smolmachines evaluation
|
||||
- `docs/research/network-egress-guard.md` — Approach 4 (DNS-based egress control)
|
||||
- `docs/research/secret-exfil-tripwire-encodings.md` — DNS exfil discussion
|
||||
- PRD 0052, PRD 0053 — egress DLP addon (HTTP-level; partial mitigation only)
|
||||
@@ -0,0 +1,123 @@
|
||||
# PRD 0058: Add built-in Pi agent provider
|
||||
|
||||
- **Status:** Active
|
||||
- **Author:** codex
|
||||
- **Created:** 2026-06-09
|
||||
- **Issue:** #221
|
||||
|
||||
## Summary
|
||||
|
||||
Add `pi` as a built-in `agent_provider.template`. The provider runs the Pi
|
||||
coding-agent CLI, provisions its agent config under `~/.pi/agent`, and writes a
|
||||
provider settings file that targets an unauthenticated Ollama-compatible server.
|
||||
|
||||
The default settings assume an Ollama server at `http://ollama:11434/v1`, using
|
||||
the `openai-completions` API with a dummy API key because Ollama ignores it.
|
||||
Users can override the provider id, base URL, model list, API key, API-key env
|
||||
reference, API type, and compatibility flags through a new
|
||||
`agent_provider.settings` object.
|
||||
|
||||
## Problem
|
||||
|
||||
bot-bottle currently ships Claude and Codex as built-in agent providers. Pi is a
|
||||
useful third harness, but using it today requires a custom provider plugin and a
|
||||
custom image. That repeats boilerplate for prompt copying, skill copying,
|
||||
provider config, and runtime registration.
|
||||
|
||||
Pi's local-model path is also easy to misconfigure: its custom-model docs require
|
||||
`~/.pi/agent/models.json`, an API entry, at least one model id, and a dummy
|
||||
`apiKey` for Ollama even though the server does not authenticate. bot-bottle
|
||||
should generate that shape consistently.
|
||||
|
||||
## Goals / Success Criteria
|
||||
|
||||
- `agent_provider.template: pi` is accepted as a built-in provider.
|
||||
- `bot_bottle/contrib/pi/` provides a Pi image and `PiAgentProvider`.
|
||||
- Pi receives the bot-bottle prompt at `~/.bot-bottle-prompt.txt` and starts in
|
||||
print-mode prompt delivery like Codex.
|
||||
- Pi skills are copied into `~/.pi/agent/skills/<name>/`.
|
||||
- Pi provider settings are configurable from the bottle manifest via
|
||||
`agent_provider.settings`.
|
||||
- The default Pi provider settings configure an unauthenticated Ollama-compatible
|
||||
server.
|
||||
- Unit tests cover manifest parsing, runtime selection, plan generation, prompt,
|
||||
skills, and provider provisioning.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Managing or launching an Ollama server.
|
||||
- Authenticating to Ollama or any remote Pi provider.
|
||||
- Forwarding host Pi credentials.
|
||||
- Implementing Pi extensions or MCP registration.
|
||||
- Changing Claude or Codex provider behavior.
|
||||
|
||||
## Design
|
||||
|
||||
### Manifest
|
||||
|
||||
Extend `agent_provider` with an optional `settings` object. It is currently only
|
||||
supported for built-in `pi`.
|
||||
|
||||
Supported keys:
|
||||
|
||||
- `base_url`: string, defaults to `http://ollama:11434/v1`
|
||||
- `provider`: string, defaults to `ollama`
|
||||
- `api`: string, defaults to `openai-completions`
|
||||
- `api_key`: string, defaults to `ollama`
|
||||
- `api_key_env`: string, optional host env var name for egress auth injection
|
||||
- `models`: non-empty array of strings, defaults to `["qwen2.5-coder:7b"]`
|
||||
- `context_window`: positive integer, defaults to `4096`; this is the Ollama
|
||||
runtime context, and bot-bottle subtracts `max_tokens` before writing Pi's
|
||||
`contextWindow` so output space is reserved
|
||||
- `max_tokens`: positive integer, defaults to `1024`
|
||||
- `max_tokens_field`: `max_tokens` or `max_completion_tokens`, defaults to
|
||||
`max_tokens`
|
||||
- `supports_developer_role`: boolean, defaults to `false`
|
||||
- `supports_reasoning_effort`: boolean, defaults to `false`
|
||||
|
||||
The snake-case manifest keys are converted into Pi's JSON field names:
|
||||
`baseUrl`, `apiKey`, `contextWindow`, `maxTokens`,
|
||||
`supportsDeveloperRole`, and `supportsReasoningEffort`. `context_window`
|
||||
describes the server's total context; Pi's `contextWindow` receives
|
||||
`context_window - max_tokens` because Pi uses it as an input compaction target.
|
||||
|
||||
`api_key` and `api_key_env` are mutually exclusive. When targeting a hosted
|
||||
provider through bot-bottle's egress sidecar, omit `api_key` and set
|
||||
`api_key_env` to the host env var that holds the API key. The generated
|
||||
`models.json` receives only an `egress-placeholder` API key, and the egress
|
||||
route injects the real `Authorization` header from the sidecar env. For example,
|
||||
OpenRouter can use provider id `openrouter` with
|
||||
`api_key_env: OPENROUTER_API_KEY`, keeping the key out of the agent env and
|
||||
`models.json`.
|
||||
|
||||
### Provider
|
||||
|
||||
`PiAgentProvider.provision_plan` writes `models.json` into the per-launch state
|
||||
directory and returns an `AgentProvisionPlan` that copies it to
|
||||
`~/.pi/agent/models.json`. The provider also declares an unauthenticated egress
|
||||
route for the configured base URL host so the egress layer can allow the Ollama
|
||||
endpoint.
|
||||
|
||||
The Pi runtime uses:
|
||||
|
||||
- `command="pi"`
|
||||
- `prompt_mode="append_system_prompt"`
|
||||
- `image="bot-bottle-pi:latest"`
|
||||
- `bypass_args=()`
|
||||
- `resume_args=()`
|
||||
- `remote_control_args=()`
|
||||
|
||||
The Dockerfile installs `@earendil-works/pi-coding-agent` globally from npm and
|
||||
keeps the same Debian/node base shape as the existing provider images.
|
||||
|
||||
### Supervise MCP
|
||||
|
||||
Pi does not have built-in MCP support in the current public docs, so
|
||||
`provision_supervise_mcp` is a no-op. This keeps Pi bottles launchable with
|
||||
`supervise: true` while preserving the explicit non-goal of implementing Pi
|
||||
extensions.
|
||||
|
||||
## Merge rule(s)
|
||||
|
||||
This PR can merge when the focused unit tests pass and the PRD status is flipped
|
||||
from Draft to Active in the final implementation commit.
|
||||
@@ -0,0 +1,190 @@
|
||||
# PRD 0059: macOS Container backend
|
||||
|
||||
- **Status:** Active
|
||||
- **Author:** Codex
|
||||
- **Created:** 2026-06-10
|
||||
- **Issue:** #220
|
||||
|
||||
## Summary
|
||||
|
||||
Add a `macos-container` backend that integrates Apple's `container`
|
||||
CLI as a host runtime on macOS. The shipped slices register the
|
||||
backend, implement reusable host primitives (`build`, `exec`, `cp`,
|
||||
image inspection, cleanup, active enumeration), make launch runnable
|
||||
with the proven two-network sidecar topology, and add real-runtime
|
||||
coverage without weakening bot-bottle's sidecar egress model.
|
||||
|
||||
## Problem
|
||||
|
||||
bot-bottle currently has two local execution paths:
|
||||
|
||||
- `docker`, which runs the whole bottle topology through Docker
|
||||
Compose.
|
||||
- `smolmachines`, which runs the agent in smolvm but still depends on
|
||||
Docker for the sidecar bundle and image-building pipeline.
|
||||
|
||||
Issue #220 explored removing Docker as a host dependency. A follow-up
|
||||
review comment verified that smolvm can publish guest ports back to
|
||||
host loopback and that another smolvm guest can reach that service
|
||||
through the existing per-bottle loopback alias plus `--allow-cidr`
|
||||
path. That keeps the VM-contained sidecar direction viable and rejects
|
||||
the host-process sidecar fallback.
|
||||
|
||||
Apple's `container` CLI is another macOS-native way to run OCI images
|
||||
as lightweight Linux VMs. Its current command surface includes
|
||||
Docker-like `build`, `run`, `exec`, `cp`, port publishing, image
|
||||
inspection, and user-defined networks. That makes it a plausible local
|
||||
backend, but it does not remove the need to preserve bot-bottle's
|
||||
sidecar enforcement property: the agent must not have a direct egress
|
||||
path around the egress sidecar.
|
||||
|
||||
## Goals / Success Criteria
|
||||
|
||||
- `--backend=macos-container` and
|
||||
`BOT_BOTTLE_BACKEND=macos-container` are accepted by the existing
|
||||
backend selector.
|
||||
- Compatible macOS hosts default to `macos-container` when
|
||||
`BOT_BOTTLE_BACKEND` and `--backend` are both unset.
|
||||
- Backend availability is true only on macOS hosts with `container` on
|
||||
`PATH`.
|
||||
- The backend has tested wrappers for Apple Container image build,
|
||||
image inspection, container `exec`, container `cp`, cleanup, and
|
||||
active-agent enumeration.
|
||||
- Full launch uses a host-only internal network for the agent and a
|
||||
separate NAT egress network for the sidecar bundle.
|
||||
- The agent container does not attach to the egress network. It reaches
|
||||
allowed outbound hosts through HTTP(S)_PROXY pointing at the
|
||||
sidecar's internal-network IP.
|
||||
- `bottle.git` / git-gate bottles fail loudly on this backend until a
|
||||
safe Apple Container key-delivery path exists.
|
||||
- Real-runtime integration coverage is present and guarded by macOS and
|
||||
Apple Container availability.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Do not remove or deprecate the Docker backend.
|
||||
- Do not remove or deprecate the smolmachines backend.
|
||||
- Do not run sidecar daemons as host processes.
|
||||
- Do not launch a degraded backend where the agent can bypass the
|
||||
egress sidecar through direct network access.
|
||||
- Do not require Docker Desktop as part of the macOS Container backend.
|
||||
|
||||
## Design
|
||||
|
||||
### Backend name
|
||||
|
||||
The selectable backend name is `macos-container`. The Python package
|
||||
uses `bot_bottle.backend.macos_container` because module names cannot
|
||||
contain hyphens.
|
||||
|
||||
### Availability and preflight
|
||||
|
||||
`MacosContainerBottleBackend.is_available()` returns true only when:
|
||||
|
||||
- `platform.system() == "Darwin"`
|
||||
- `container` is discoverable on `PATH`
|
||||
|
||||
`prepare()` calls `require_container()`, which produces a concrete
|
||||
install pointer and rejects non-macOS hosts.
|
||||
|
||||
### Implemented primitives
|
||||
|
||||
The backend owns an Apple Container wrapper module instead of reusing
|
||||
Docker wrappers. The wrapper maps bot-bottle's backend needs to
|
||||
Apple's CLI:
|
||||
|
||||
| bot-bottle need | Apple Container command |
|
||||
|---|---|
|
||||
| Build provider image | `container build -t <ref> [-f Dockerfile] <context>` |
|
||||
| Run agent commands | `container exec [--interactive --tty] <id> ...` |
|
||||
| Copy files into guest | `container cp <host> <id>:<path>` |
|
||||
| Inspect image identity | `container image inspect <ref>` |
|
||||
| Cleanup stale containers | `container delete --force <id>` |
|
||||
| Cleanup stale networks | `container network delete <name>` |
|
||||
| Active enumeration | `container list --quiet` |
|
||||
|
||||
The bottle handle mirrors `DockerBottle`: it builds a host argv for
|
||||
foreground agent execution, pipes shell snippets through stdin for
|
||||
`Bottle.exec`, and exposes `cp_in` for provisioning.
|
||||
|
||||
### Launch topology
|
||||
|
||||
`launch()` uses Apple Container's two-network topology:
|
||||
|
||||
- create a host-only internal network for the bottle;
|
||||
- create a normal NAT egress network for the sidecar bundle;
|
||||
- start the sidecar bundle attached to the egress network first and the
|
||||
internal network second;
|
||||
- discover the sidecar's internal-network IPv4 address from
|
||||
`container inspect`;
|
||||
- start the agent attached only to the internal network, with
|
||||
HTTP_PROXY / HTTPS_PROXY / lowercase proxy vars pointing at the
|
||||
sidecar IP and egress port.
|
||||
|
||||
This keeps the agent off the outbound network while preserving the
|
||||
proxy-env contract that existing agent tooling already honors. The
|
||||
integration smoke also removes the proxy env in-guest and confirms
|
||||
direct egress fails.
|
||||
|
||||
### Deferred git-gate support
|
||||
|
||||
Apple Container currently rejects single-file bind mounts, and
|
||||
`container cp` into a stopped container is not available. Starting the
|
||||
container earlier would allow `container cp` into a running container,
|
||||
but it would also mean delivering SSH private key material into a live
|
||||
sidecar before the git-gate daemon is ready to own it. Mounting broad
|
||||
host SSH directories is not acceptable.
|
||||
|
||||
For this PRD, `bottle.git` / git-gate support is explicitly deferred on
|
||||
the `macos-container` backend. Bottles with git-gate upstreams fail
|
||||
loudly and should use `docker` or `smolmachines` until a narrower key
|
||||
delivery design lands.
|
||||
|
||||
## Implementation chunks
|
||||
|
||||
1. Register `macos-container`, add availability/preflight, bottle
|
||||
handle, utility wrappers, cleanup, active enumeration, unit tests,
|
||||
and this PRD.
|
||||
2. Spike Apple Container networking against real macOS 26 hosts:
|
||||
repeated `--network`, internal network egress behavior, published
|
||||
loopback reachability from another container, DNS behavior, and
|
||||
labels/JSON output stability.
|
||||
3. Implement launch once the enforcement shape is proven. Reuse the
|
||||
existing sidecar bundle image and daemon subset env contract where
|
||||
possible.
|
||||
4. Add real-runtime integration tests guarded by `container` presence
|
||||
and macOS version.
|
||||
5. Consider moving smolmachines sidecar/image-building work to
|
||||
VM-contained or Apple Container-backed execution only after the
|
||||
`macos-container` launch path is trustworthy.
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
- Unit tests cover backend registration through `known_backend_names`.
|
||||
- Unit tests cover availability/preflight behavior without requiring
|
||||
macOS.
|
||||
- Unit tests cover `MacosContainerBottle` command construction and
|
||||
stdin-based shell execution.
|
||||
- Unit tests cover cleanup and active enumeration parsing.
|
||||
- Unit tests cover launch argv/env construction, sidecar mount
|
||||
staging, sidecar IP parsing, and git-gate rejection.
|
||||
- Integration tests run on macOS hosts with Apple Container installed
|
||||
and verify that egress cannot bypass the sidecar. They also preflight
|
||||
Apple Container BuildKit DNS because image builds must resolve
|
||||
package mirrors before a launch smoke can be meaningful. The backend
|
||||
probes the running builder before image builds and leaves it alone
|
||||
when its current resolver works. If the probe fails, or if the
|
||||
operator explicitly sets `BOT_BOTTLE_MACOS_CONTAINER_DNS`, the backend
|
||||
restarts the Apple Container builder with the configured DNS server.
|
||||
Without an explicit override, that server is discovered from the
|
||||
host's directly reachable IPv4 resolver before falling back to a
|
||||
public resolver.
|
||||
|
||||
## References
|
||||
|
||||
- [Issue #220 review comment](https://gitea.dideric.is/didericis/bot-bottle/issues/220#issuecomment-1980):
|
||||
smolvm `--port/-p` can expose a guest service to host loopback, and
|
||||
another smolvm guest can reach it through the existing per-bottle
|
||||
loopback alias path.
|
||||
- Apple Container command reference: `container run`, `build`, `exec`,
|
||||
port publishing, and network commands.
|
||||
@@ -0,0 +1,159 @@
|
||||
# PRD 0060: Commit bottle state to an image
|
||||
|
||||
- **Status:** Active
|
||||
- **Author:** Claude
|
||||
- **Created:** 2026-06-20
|
||||
- **Issue:** #194
|
||||
|
||||
## Summary
|
||||
|
||||
Add a `commit` CLI command that freezes a running bottle's state to a
|
||||
resumable local artifact. Docker bottles are stored as Docker images;
|
||||
smolmachines bottles are stored as `.smolmachine` artifacts. Operators
|
||||
can then resume the bottle from that exact filesystem snapshot, or
|
||||
export the artifact to migrate work to a different host.
|
||||
|
||||
## Problem
|
||||
|
||||
When a long-running agent session is interrupted — by a host reboot, a
|
||||
network failure, or a planned infrastructure migration — the in-progress
|
||||
container state is lost. `cli.py resume` rebuilds the agent image from
|
||||
the Dockerfile and reprovi-sions the bottle, but that returns the guest
|
||||
to its initial state, not to wherever the agent was mid-task.
|
||||
|
||||
There is no mechanism today to capture "what's installed / configured
|
||||
inside the running container right now" and make it reproducible. The
|
||||
`capability-block` flow writes a new Dockerfile and marks the bottle for
|
||||
resume, but that only applies when the agent itself has requested a
|
||||
capability change; it doesn't help the operator who wants to take a
|
||||
snapshot before a planned host reboot or hardware migration.
|
||||
|
||||
## Goals / Success Criteria
|
||||
|
||||
- `./cli.py commit [<slug>]` takes a snapshot of the running agent and
|
||||
stores it as a local artifact.
|
||||
- Without a slug argument the command shows the same interactive picker
|
||||
as `start` (the list of active slugs).
|
||||
- The committed artifact reference is stored in per-bottle state so
|
||||
that the next `./cli.py resume <slug>` automatically uses the
|
||||
snapshot instead of rebuilding from the Dockerfile.
|
||||
- `mark_preserved` is called so the state dir survives the normal
|
||||
session-end cleanup.
|
||||
- A backend-specific export hint is printed so operators know how to
|
||||
migrate the snapshot.
|
||||
- The command errors clearly on unsupported backends.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- macOS-container backend support.
|
||||
- Automatic commit on agent exit.
|
||||
- Image push to a remote registry.
|
||||
- Storing the image tag in the manifest or sharing it between operators.
|
||||
|
||||
## Design
|
||||
|
||||
### Docker image tag
|
||||
|
||||
`bot-bottle-committed-<slug>:latest` — namespaced under `bot-bottle-`
|
||||
to match existing image naming conventions; `committed` distinguishes it
|
||||
from the build-time image (`bot-bottle-claude:latest`) and the
|
||||
capability-block rebuild image (`bot-bottle-rebuilt-<identity>:latest`).
|
||||
|
||||
### State storage
|
||||
|
||||
A new plain-text file `committed-image` is added to the per-bottle state
|
||||
directory:
|
||||
|
||||
```
|
||||
~/.bot-bottle/state/<identity>/
|
||||
metadata.json
|
||||
Dockerfile (capability-block override; optional)
|
||||
committed-image (committed artifact reference; optional)
|
||||
transcript/
|
||||
```
|
||||
|
||||
`bottle_state.committed_image_path(identity)` returns the path.
|
||||
`write_committed_image` / `read_committed_image` are the read/write
|
||||
helpers, matching the existing `per_bottle_dockerfile` pattern. Docker
|
||||
stores a Docker tag in this file; smolmachines stores the absolute path
|
||||
to the committed `.smolmachine` artifact.
|
||||
|
||||
### `commit` command
|
||||
|
||||
```
|
||||
./cli.py commit [<slug>]
|
||||
```
|
||||
|
||||
1. Resolve slug (arg or interactive picker from `enumerate_active_agents`).
|
||||
2. Check metadata and branch by backend.
|
||||
3. For Docker, derive container name `bot-bottle-<slug>` and run
|
||||
`docker commit <container> bot-bottle-committed-<slug>:latest`.
|
||||
4. For smolmachines, derive machine name `bot-bottle-<slug>` and run
|
||||
`smolvm pack create --from-vm <machine> -o ~/.bot-bottle/state/<slug>/committed-smolmachine`.
|
||||
5. Write the Docker image tag or smolmachine artifact path to
|
||||
`~/.bot-bottle/state/<slug>/committed-image`.
|
||||
6. Call `mark_preserved(<slug>)` so the state dir survives session-end.
|
||||
7. Print the resume hint and a backend-specific export example.
|
||||
|
||||
### Resume from committed image
|
||||
|
||||
`bot_bottle/backend/docker/launch.py` already rebuilds the agent image
|
||||
at the top of the `launch` context manager. The change is a check
|
||||
immediately before that step:
|
||||
|
||||
```python
|
||||
committed = read_committed_image(plan.slug)
|
||||
if committed and docker_mod.image_exists(committed):
|
||||
info(f"using committed image {committed!r}")
|
||||
plan = dataclasses.replace(
|
||||
plan,
|
||||
agent_provision=dataclasses.replace(plan.agent_provision, image=committed),
|
||||
)
|
||||
else:
|
||||
docker_mod.build_image(plan.image, _REPO_DIR, dockerfile=plan.dockerfile_path)
|
||||
```
|
||||
|
||||
Replacing `agent_provision.image` propagates to `plan.image` (a
|
||||
property) and from there to the Compose spec renderer's `_agent_service`
|
||||
→ `image:` field, so the container boots from the committed snapshot.
|
||||
The build step is skipped entirely when a committed image is found and
|
||||
exists locally.
|
||||
|
||||
If the committed image has been deleted from the local daemon (e.g.
|
||||
after `docker rmi` or a `docker system prune`), the launch falls back
|
||||
to a normal Dockerfile build, matching the pre-commit behavior.
|
||||
|
||||
### Resume from committed smolmachine
|
||||
|
||||
`bot_bottle/backend/smolmachines/launch.py` checks the committed
|
||||
reference before the normal Docker build -> pack cache path:
|
||||
|
||||
```python
|
||||
committed = read_committed_image(plan.slug)
|
||||
if committed and Path(committed).is_file():
|
||||
return Path(committed)
|
||||
return _ensure_smolmachine(plan.agent_image, dockerfile=plan.agent_dockerfile_path)
|
||||
```
|
||||
|
||||
The returned path is passed to `smolvm machine create --from`, so the
|
||||
resumed VM boots from the committed snapshot. If the artifact has been
|
||||
deleted, launch falls back to the normal build and pack flow.
|
||||
|
||||
## Testing strategy
|
||||
|
||||
- Unit tests for `write_committed_image` / `read_committed_image` in
|
||||
`tests/unit/test_bottle_state.py`, using the existing `_FakeHomeMixin`
|
||||
pattern.
|
||||
- Unit tests for `commit_container` in `tests/unit/test_docker_util_image.py`,
|
||||
mocking `subprocess.run` and asserting on the `docker commit` argv.
|
||||
- Unit tests for `cmd_commit` argument parsing, Docker commit,
|
||||
smolmachines pack, and the unsupported backend error path, mocking
|
||||
`enumerate_active_agents`, `commit_container`, and
|
||||
`pack_create_from_vm`.
|
||||
- Unit tests for the launch-step committed-image branch: patch
|
||||
`read_committed_image` to return a tag, patch `image_exists` to return
|
||||
True, and assert that `build_image` is not called and `plan.image` is
|
||||
overridden.
|
||||
- Unit tests for the smolmachines launch-step committed-artifact branch:
|
||||
patch `read_committed_image` to return an existing path and assert the
|
||||
normal `_ensure_smolmachine` path is skipped.
|
||||
@@ -0,0 +1,101 @@
|
||||
# PRD 0061: Gitleaks inline suppression supervision
|
||||
|
||||
- **Status:** Active
|
||||
- **Author:** codex
|
||||
- **Created:** 2026-06-10
|
||||
- **Issue:** #208
|
||||
|
||||
## Summary
|
||||
|
||||
Route every `# gitleaks:allow` inline suppression through the existing
|
||||
supervisor approval queue before git-gate forwards a push upstream. The
|
||||
annotation remains available for test fixtures and confirmed false positives,
|
||||
but it no longer lets an agent silently bypass the gate's secret scan.
|
||||
|
||||
## Problem
|
||||
|
||||
Gitleaks supports inline suppression comments. With the default behaviour,
|
||||
a pushed line that contains a secret-like value and `# gitleaks:allow` is
|
||||
ignored by gitleaks and accepted by git-gate. That is convenient for dummy
|
||||
credentials in tests, but it also gives an agent a one-line bypass for a real
|
||||
secret. Once the commit lands, the diff alone cannot prove whether the
|
||||
suppression was legitimate.
|
||||
|
||||
## Goals / Success Criteria
|
||||
|
||||
1. Git-gate continues to run the normal gitleaks scan for every incoming ref.
|
||||
2. After the normal scan passes, git-gate runs a second scan with
|
||||
`--ignore-gitleaks-allow` and a JSON report so suppressed findings become
|
||||
visible.
|
||||
3. If that second scan reports no suppressed findings, the push proceeds
|
||||
unchanged.
|
||||
4. If it reports suppressed findings, git-gate creates a `gitleaks-allow`
|
||||
supervisor proposal containing the ref, file path, line number, rule,
|
||||
commit, and flagged line for each finding.
|
||||
5. The push proceeds only when the supervisor explicitly approves the
|
||||
proposal; rejection, malformed responses, missing supervisor configuration,
|
||||
and timeout all refuse the push.
|
||||
6. The supervisor TUI requires a reason when approving a `gitleaks-allow`
|
||||
proposal, so the audit trail records whether the approval was for a test
|
||||
fixture or a false positive.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Replacing gitleaks or changing the main secret-detection rule set.
|
||||
- Removing support for `# gitleaks:allow`.
|
||||
- Automatically classifying fixture files or false positives.
|
||||
- Adding new supervisor transport or authentication mechanisms.
|
||||
|
||||
## Design
|
||||
|
||||
### Git-gate flow
|
||||
|
||||
`git_gate_render_hook()` emits a `supervise_gitleaks_allow` shell helper.
|
||||
For each incoming ref, git-gate first runs the existing gitleaks command. If
|
||||
that scan passes, it runs:
|
||||
|
||||
```sh
|
||||
gitleaks git \
|
||||
--log-opts="$log_opts" \
|
||||
--no-banner \
|
||||
--redact \
|
||||
--ignore-gitleaks-allow \
|
||||
--report-format=json \
|
||||
--report-path="$report_file" \
|
||||
--exit-code 0
|
||||
```
|
||||
|
||||
The second pass keeps the push path non-interactive while producing a report
|
||||
of findings that would otherwise have been hidden by inline suppression.
|
||||
|
||||
### Supervisor proposal
|
||||
|
||||
When the JSON report contains findings, an embedded Python helper writes a
|
||||
proposal into `SUPERVISE_QUEUE_DIR` using the existing proposal schema. The
|
||||
proposal uses:
|
||||
|
||||
- `tool: "gitleaks-allow"`
|
||||
- a text payload with the ref and each finding's file, line, rule, commit,
|
||||
and redacted code line
|
||||
- a justification that tells the operator to approve only dummy test fixtures
|
||||
or confirmed false positives
|
||||
|
||||
Git-gate then waits for `<proposal-id>.response.json` for
|
||||
`SUPERVISE_GITLEAKS_ALLOW_TIMEOUT_SECONDS`, defaulting to 300 seconds.
|
||||
`approved` and `modified` responses allow the push; `rejected`, invalid
|
||||
responses, invalid timeout configuration, or timeout refuse it.
|
||||
|
||||
### Supervisor UI
|
||||
|
||||
`TOOL_GITLEAKS_ALLOW` is added to the supervisor tool registry. The curses
|
||||
supervisor renders the proposal as text and allows approval or rejection.
|
||||
Modification is unavailable for this proposal type because there is no file
|
||||
patch to apply. Approval from the TUI prompts for a non-empty reason and
|
||||
writes that reason to the response/audit path.
|
||||
|
||||
### Tests
|
||||
|
||||
Unit tests assert that the rendered git-gate hook includes the second gitleaks
|
||||
pass, supervisor queue fields, and fail-closed messages. Supervisor tests cover
|
||||
the new tool constant, proposal archiving, and the required TUI approval
|
||||
reason.
|
||||
@@ -0,0 +1,75 @@
|
||||
# PRD prd-new: Install script
|
||||
|
||||
- **Status:** Active
|
||||
- **Author:** didericis
|
||||
- **Created:** 2026-06-06
|
||||
- **Issue:** #197
|
||||
|
||||
## Summary
|
||||
|
||||
Add a proper Python package distribution and a thin `install.sh` bootstrapper so users can install bot-bottle with a single command without cloning the repo.
|
||||
|
||||
## Problem
|
||||
|
||||
There is currently no install path for new users. The only way to run bot-bottle is to clone the repo and invoke `cli.py` directly. This blocks any HN-style public demo: readers want `curl | sh` or `pipx install`, not a manual clone-and-configure flow.
|
||||
|
||||
## Goals / Success Criteria
|
||||
|
||||
- `curl -fsSL <url>/install.sh | sh` (or equivalent) leaves a working `bot-bottle` command on PATH.
|
||||
- Python-native users can install with `pipx install bot-bottle` or `uv tool install bot-bottle`.
|
||||
- `install.sh` validates prerequisites (Python ≥ 3.11, Docker) and exits with a clear message if they are missing. It does not silently install Docker.
|
||||
- `install.sh` runs `bot-bottle doctor` (or equivalent diagnostic) after install to confirm the environment is ready.
|
||||
- The package has no runtime pip dependencies (stdlib-only, matching the existing constraint).
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Bundling a Python runtime or producing a standalone binary.
|
||||
- Automatic Docker installation.
|
||||
- Plugin architecture changes (out of scope; see issue #197 for future direction).
|
||||
- Publishing to PyPI in this PR — the package structure is the deliverable; publishing is a separate step.
|
||||
|
||||
## Design
|
||||
|
||||
### Package structure
|
||||
|
||||
Add a minimal `pyproject.toml` at the repo root:
|
||||
|
||||
```toml
|
||||
[project]
|
||||
name = "bot-bottle"
|
||||
version = "0.1.0"
|
||||
requires-python = ">=3.11"
|
||||
dependencies = []
|
||||
|
||||
[project.scripts]
|
||||
bot-bottle = "bot_bottle.cli:main"
|
||||
```
|
||||
|
||||
The existing `bot_bottle/` package and `cli.py` entry point already contain the logic; this just wires up the standard entry point. `cli.py` may need a small refactor to expose a `main()` callable if it uses `if __name__ == "__main__"` only.
|
||||
|
||||
### `install.sh`
|
||||
|
||||
A thin bootstrapper that:
|
||||
|
||||
1. Checks `python3 --version` ≥ 3.11; exits with instructions if not met.
|
||||
2. Checks `docker info` exits 0; exits with instructions if Docker is not running.
|
||||
3. Installs via `pipx` if available, otherwise falls back to `pip install --user`.
|
||||
4. Runs `bot-bottle doctor` to verify the install.
|
||||
|
||||
The script must be idempotent (safe to re-run) and must not require `sudo`.
|
||||
|
||||
### `bot-bottle doctor`
|
||||
|
||||
A new subcommand that checks and reports:
|
||||
|
||||
- Python version.
|
||||
- Docker daemon reachability.
|
||||
- Whether `~/.bot-bottle/` config directory exists.
|
||||
|
||||
Exits 0 if all checks pass, non-zero otherwise.
|
||||
|
||||
## Decisions
|
||||
|
||||
- `install.sh` is hosted from the repo's raw Gitea URL for now:
|
||||
`https://gitea.dideric.is/didericis/bot-bottle/raw/branch/main/install.sh`.
|
||||
- Should `version` in `pyproject.toml` be driven by a git tag at build time (e.g. via `hatch-vcs`) or kept as a static string? Static is simpler for now.
|
||||
@@ -0,0 +1,360 @@
|
||||
# Apple Container networking spike
|
||||
|
||||
Issue: https://gitea.dideric.is/didericis/bot-bottle/issues/230
|
||||
|
||||
## Summary
|
||||
|
||||
Apple Container 1.0.0 on macOS 26 can support the core two-network
|
||||
sidecar shape, but not as a drop-in Docker Compose clone.
|
||||
|
||||
The viable shape is:
|
||||
|
||||
- agent container on one `--internal` host-only network;
|
||||
- sidecar bundle container on both the NAT egress network and the
|
||||
host-only agent network;
|
||||
- sidecar network flags ordered with the NAT network first, because
|
||||
Apple Container chooses the first network as the default route;
|
||||
- explicit DNS on the sidecar, because the tested NAT gateway routed
|
||||
packets but did not resolve DNS;
|
||||
- agent talks to sidecar by the sidecar's host-only-network IP, not by
|
||||
container name or host-published loopback alias.
|
||||
|
||||
This is enough to unblock a cautious `macos-container` launch spike if
|
||||
the backend records inspect-derived IPs and avoids depending on Docker
|
||||
Compose-style aliases. It is not enough to reuse the Docker backend's
|
||||
service-name assumptions unchanged.
|
||||
|
||||
## Local Environment
|
||||
|
||||
Tested on 2026-06-10:
|
||||
|
||||
```console
|
||||
$ sw_vers
|
||||
ProductName: macOS
|
||||
ProductVersion: 26.5.1
|
||||
BuildVersion: 25F80
|
||||
|
||||
$ uname -m
|
||||
arm64
|
||||
|
||||
$ container --version
|
||||
container CLI version 1.0.0 (build: release, commit: ee848e3)
|
||||
|
||||
$ container system version --format json
|
||||
[
|
||||
{
|
||||
"appName": "container",
|
||||
"buildType": "release",
|
||||
"commit": "ee848e3ebfd7c73b04dd419683be54fb450b8779",
|
||||
"version": "1.0.0"
|
||||
},
|
||||
{
|
||||
"appName": "container-apiserver",
|
||||
"buildType": "release",
|
||||
"commit": "ee848e3ebfd7c73b04dd419683be54fb450b8779",
|
||||
"version": "container-apiserver version 1.0.0 (build: release, commit: ee848e3)"
|
||||
}
|
||||
]
|
||||
|
||||
$ container system status --format json
|
||||
{
|
||||
"apiServerAppName": "container-apiserver",
|
||||
"apiServerBuild": "release",
|
||||
"apiServerCommit": "ee848e3ebfd7c73b04dd419683be54fb450b8779",
|
||||
"apiServerVersion": "container-apiserver version 1.0.0 (build: release, commit: ee848e3)",
|
||||
"appRoot": "/Users/didericis/Library/Application Support/com.apple.container/",
|
||||
"installRoot": "/usr/local/",
|
||||
"status": "running"
|
||||
}
|
||||
```
|
||||
|
||||
Apple Container was installed from the official signed 1.0.0 GitHub
|
||||
release package, `container-1.0.0-installer-signed.pkg`. The package was
|
||||
signed by `Developer ID Installer: Apple Inc. - Containerization
|
||||
(UPBK2H6LZM)` and notarized by Apple.
|
||||
|
||||
## Commands Run
|
||||
|
||||
Create the networks:
|
||||
|
||||
```bash
|
||||
container network create bb-spike-230-agent \
|
||||
--internal \
|
||||
--label bot-bottle.spike=apple-container-networking
|
||||
|
||||
container network create bb-spike-230-egress \
|
||||
--label bot-bottle.spike=apple-container-networking
|
||||
```
|
||||
|
||||
`container network inspect bb-spike-230-agent bb-spike-230-egress`
|
||||
showed:
|
||||
|
||||
```json
|
||||
[
|
||||
{
|
||||
"configuration": {
|
||||
"labels": {"bot-bottle.spike": "apple-container-networking"},
|
||||
"mode": "hostOnly",
|
||||
"name": "bb-spike-230-agent",
|
||||
"plugin": "container-network-vmnet"
|
||||
},
|
||||
"id": "bb-spike-230-agent",
|
||||
"status": {
|
||||
"ipv4Gateway": "192.168.128.1",
|
||||
"ipv4Subnet": "192.168.128.0/24"
|
||||
}
|
||||
},
|
||||
{
|
||||
"configuration": {
|
||||
"labels": {"bot-bottle.spike": "apple-container-networking"},
|
||||
"mode": "nat",
|
||||
"name": "bb-spike-230-egress",
|
||||
"plugin": "container-network-vmnet"
|
||||
},
|
||||
"id": "bb-spike-230-egress",
|
||||
"status": {
|
||||
"ipv4Gateway": "192.168.66.1",
|
||||
"ipv4Subnet": "192.168.66.0/24"
|
||||
}
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
Repeated `--network` flags are accepted. With the agent network first,
|
||||
the sidecar got two interfaces but the default route pointed at the
|
||||
host-only gateway, so egress failed:
|
||||
|
||||
```bash
|
||||
container run --name bb-spike-230-sidecar \
|
||||
--label bot-bottle.spike=apple-container-networking \
|
||||
--network bb-spike-230-agent \
|
||||
--network bb-spike-230-egress \
|
||||
--detach --rm docker.io/python:alpine \
|
||||
sh -c 'mkdir -p /srv && printf ok >/srv/index.html && cd /srv && python3 -m http.server 80 --bind 0.0.0.0'
|
||||
|
||||
container exec bb-spike-230-sidecar sh -c 'ip route && cat /etc/resolv.conf'
|
||||
```
|
||||
|
||||
Observed:
|
||||
|
||||
```console
|
||||
default via 192.168.128.1 dev eth0
|
||||
192.168.66.0/24 dev eth1 scope link src 192.168.66.3
|
||||
192.168.128.0/24 dev eth0 scope link src 192.168.128.3
|
||||
nameserver 192.168.128.1
|
||||
```
|
||||
|
||||
With the NAT network first and explicit DNS, the sidecar can egress:
|
||||
|
||||
```bash
|
||||
container run --name bb-spike-230-sidecar \
|
||||
--label bot-bottle.spike=apple-container-networking \
|
||||
--network bb-spike-230-egress \
|
||||
--network bb-spike-230-agent \
|
||||
--dns 1.1.1.1 \
|
||||
--detach docker.io/python:alpine \
|
||||
sh -c 'mkdir -p /srv && printf ok >/srv/index.html && cd /srv && python3 -m http.server 80 --bind 0.0.0.0'
|
||||
|
||||
container exec bb-spike-230-sidecar sh -c 'ip route; cat /etc/resolv.conf; wget -T 8 -O- https://example.com'
|
||||
```
|
||||
|
||||
Observed:
|
||||
|
||||
```console
|
||||
default via 192.168.66.1 dev eth0
|
||||
192.168.66.0/24 dev eth0 scope link src 192.168.66.5
|
||||
192.168.128.0/24 dev eth1 scope link src 192.168.128.7
|
||||
nameserver 1.1.1.1
|
||||
Connecting to example.com (172.66.147.243:443)
|
||||
... 100%
|
||||
```
|
||||
|
||||
Start an agent only on the host-only network:
|
||||
|
||||
```bash
|
||||
container run --name bb-spike-230-agent \
|
||||
--label bot-bottle.spike=apple-container-networking \
|
||||
--network bb-spike-230-agent \
|
||||
--detach docker.io/alpine:latest sleep 600
|
||||
```
|
||||
|
||||
Agent network probes:
|
||||
|
||||
```bash
|
||||
container exec bb-spike-230-agent sh -c '
|
||||
ip route
|
||||
cat /etc/resolv.conf
|
||||
wget -T 5 -O- http://192.168.128.7
|
||||
wget -T 5 -O- http://bb-spike-230-sidecar || true
|
||||
ping -c 2 1.1.1.1 || true
|
||||
wget -T 5 -O- https://example.com || true
|
||||
'
|
||||
```
|
||||
|
||||
Observed:
|
||||
|
||||
```console
|
||||
default via 192.168.128.1 dev eth0
|
||||
192.168.128.0/24 dev eth0 scope link src 192.168.128.8
|
||||
nameserver 192.168.128.1
|
||||
Connecting to 192.168.128.7 (192.168.128.7:80)
|
||||
ok
|
||||
wget: bad address 'bb-spike-230-sidecar'
|
||||
2 packets transmitted, 0 packets received, 100% packet loss
|
||||
wget: bad address 'example.com'
|
||||
```
|
||||
|
||||
Host-published loopback aliases work and are constrained to the bound
|
||||
alias on the host:
|
||||
|
||||
```bash
|
||||
container run --name bb-spike-230-sidecar-alias \
|
||||
--label bot-bottle.spike=apple-container-networking \
|
||||
--network bb-spike-230-egress \
|
||||
--network bb-spike-230-agent \
|
||||
--dns 1.1.1.1 \
|
||||
--publish 127.0.0.31:18080:80 \
|
||||
--detach docker.io/python:alpine \
|
||||
sh -c 'mkdir -p /srv && printf ok >/srv/index.html && cd /srv && python3 -m http.server 80 --bind 0.0.0.0'
|
||||
|
||||
curl -fsS --max-time 5 http://127.0.0.31:18080
|
||||
curl -fsS --max-time 5 http://127.0.0.1:18080
|
||||
lsof -nP -iTCP:18080 -sTCP:LISTEN
|
||||
```
|
||||
|
||||
Observed:
|
||||
|
||||
```console
|
||||
$ curl -fsS --max-time 5 http://127.0.0.31:18080
|
||||
ok
|
||||
|
||||
$ curl -fsS --max-time 5 http://127.0.0.1:18080
|
||||
curl: (7) Failed to connect to 127.0.0.1 port 18080 after 0 ms: Couldn't connect to server
|
||||
|
||||
$ lsof -nP -iTCP:18080 -sTCP:LISTEN
|
||||
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
|
||||
container 17908 didericis 25u IPv4 ... 0t0 TCP 127.0.0.31:18080 (LISTEN)
|
||||
```
|
||||
|
||||
The guest cannot reach that host loopback-published listener through
|
||||
the host-only gateway or through its own loopback address:
|
||||
|
||||
```bash
|
||||
container exec bb-spike-230-agent sh -c '
|
||||
wget -T 5 -O- http://192.168.128.10
|
||||
wget -T 5 -O- http://192.168.128.1:18080 || true
|
||||
wget -T 5 -O- http://127.0.0.31:18080 || true
|
||||
wget -T 5 -O- http://bb-spike-230-sidecar-alias || true
|
||||
'
|
||||
```
|
||||
|
||||
Observed:
|
||||
|
||||
```console
|
||||
Connecting to 192.168.128.10 (192.168.128.10:80)
|
||||
ok
|
||||
Connecting to 192.168.128.1:18080 (192.168.128.1:18080)
|
||||
wget: can't connect to remote host (192.168.128.1): Connection refused
|
||||
Connecting to 127.0.0.31:18080 (127.0.0.31:18080)
|
||||
wget: can't connect to remote host (127.0.0.31): Connection refused
|
||||
wget: bad address 'bb-spike-230-sidecar-alias'
|
||||
```
|
||||
|
||||
## Answers
|
||||
|
||||
### 1. Does `container network create --internal` prevent outbound internet access?
|
||||
|
||||
Yes in this run. `--internal` produced a `hostOnly` network. An
|
||||
internal-only agent had a default route to the host-only gateway, but
|
||||
could not ping `1.1.1.1` and could not resolve or fetch
|
||||
`https://example.com`.
|
||||
|
||||
### 2. Can `container run` attach one container to multiple networks?
|
||||
|
||||
Yes. Repeated `--network` flags produced multiple interfaces and the
|
||||
inspect JSON preserved both network attachments.
|
||||
|
||||
Important caveat: network order matters. The first network became
|
||||
`eth0`, supplied the default route, and supplied `/etc/resolv.conf`.
|
||||
For a sidecar that needs internet egress, put the NAT network first and
|
||||
the internal agent network second.
|
||||
|
||||
### 3. Can the sidecar bundle sit on both an internal agent network and an egress-capable network?
|
||||
|
||||
Yes. The sidecar had a NAT interface and a host-only interface. With the
|
||||
NAT network first and explicit DNS, it could fetch `https://example.com`
|
||||
while the agent on only the host-only network could not.
|
||||
|
||||
### 4. Can Apple Container provide stable network aliases or service discovery equivalent to Docker Compose aliases?
|
||||
|
||||
Not by default in this run. The agent could not resolve
|
||||
`bb-spike-230-sidecar` or `bb-spike-230-sidecar-alias`, even though
|
||||
those were the container names and hostnames in inspect output. The
|
||||
agent could reach the sidecar by the sidecar's host-only-network IP.
|
||||
|
||||
The backend should not assume Docker Compose-style aliases. It should
|
||||
read the sidecar's host-only IP from `container inspect` and inject
|
||||
that concrete endpoint into the agent environment/config, or run a
|
||||
small internal DNS/hosts-file setup as an explicit backend feature.
|
||||
|
||||
### 5. Can a published sidecar port bound to a per-bottle loopback alias be reached from another Apple Container guest and constrained to that alias?
|
||||
|
||||
Host-side alias binding works and is constrained on the host:
|
||||
`127.0.0.31:18080` reached the sidecar, while `127.0.0.1:18080` failed.
|
||||
|
||||
Guest-to-host-published-loopback did not work. From the agent,
|
||||
`192.168.128.1:18080` and `127.0.0.31:18080` both failed. For
|
||||
agent-to-sidecar traffic, use the sidecar's internal network IP rather
|
||||
than a host-published loopback alias.
|
||||
|
||||
### 6. What structured output is available for robust enumeration and cleanup?
|
||||
|
||||
Confirmed structured output:
|
||||
|
||||
- `container list --all --format json`
|
||||
- `container inspect <container...>` as JSON
|
||||
- `container image inspect <image...>` as JSON
|
||||
- `container network list --format json`
|
||||
- `container network inspect <network...>` as JSON
|
||||
- `container system status --format json`
|
||||
- `container system version --format json`
|
||||
|
||||
Useful fields observed:
|
||||
|
||||
- containers: `id`, `configuration.labels`,
|
||||
`configuration.networks`, `configuration.publishedPorts`,
|
||||
`status.state`, `status.networks[].network`,
|
||||
`status.networks[].ipv4Address`, `status.networks[].ipv4Gateway`;
|
||||
- networks: `id`, `configuration.name`, `configuration.labels`,
|
||||
`configuration.mode`, `status.ipv4Gateway`, `status.ipv4Subnet`;
|
||||
- images: `id`, `configuration.name`, `configuration.descriptor`,
|
||||
`variants[].platform`, `variants[].size`.
|
||||
|
||||
### 7. Are labels supported on containers and networks enough to replace prefix-only discovery?
|
||||
|
||||
Labels are present in container and network inspect/list JSON, so they
|
||||
are sufficient as metadata if the backend lists resources and filters
|
||||
client-side. I did not find or validate a server-side label filter for
|
||||
`container list` or `container network list`.
|
||||
|
||||
## Recommendation
|
||||
|
||||
Proceed with a narrow `macos-container` launch prototype, but encode
|
||||
the Apple Container-specific constraints directly:
|
||||
|
||||
- create one host-only agent network and one NAT egress network per
|
||||
bottle;
|
||||
- start the sidecar bundle with `--network <egress>` before
|
||||
`--network <agent>`;
|
||||
- set sidecar DNS explicitly, ideally from the bottle/host policy
|
||||
rather than hardcoding a public resolver;
|
||||
- start the agent only on the host-only network;
|
||||
- discover the sidecar's host-only IP from `container inspect` and pass
|
||||
concrete URLs to the agent;
|
||||
- use host loopback publishing only for host-to-sidecar access, not
|
||||
guest-to-sidecar access;
|
||||
- enumerate and clean up by labels plus name prefixes until/unless the
|
||||
CLI adds label filters.
|
||||
|
||||
Do not implement the backend as a direct clone of Docker Compose
|
||||
service aliases. That assumption failed in this run.
|
||||
@@ -0,0 +1,476 @@
|
||||
# Apple Container transparent egress spike
|
||||
|
||||
Issue: https://gitea.dideric.is/didericis/bot-bottle/issues/230#issuecomment-1994
|
||||
|
||||
## Summary
|
||||
|
||||
Transparent egress is mechanically possible on Apple Container 1.0.0,
|
||||
but it is not a free property of the platform and it is not a drop-in
|
||||
replacement for `HTTP_PROXY` yet.
|
||||
|
||||
The spike proved two separate things:
|
||||
|
||||
- Plain routing/NAT works if the sidecar has `CAP_NET_ADMIN`, IP
|
||||
forwarding, and masquerade rules, and if the agent default route is
|
||||
changed to the sidecar's host-only-network IP.
|
||||
- Transparent mitmproxy interception works if the sidecar redirects
|
||||
agent-facing TCP 80/443 traffic to `mitmdump --mode transparent`.
|
||||
Direct HTTP was logged by mitmproxy. Direct HTTPS reached mitmproxy;
|
||||
it failed with normal certificate verification until the client
|
||||
skipped verification, which is consistent with bot-bottle's existing
|
||||
requirement that agents trust the sidecar CA.
|
||||
- Running DNS on the sidecar and pointing the agent at the sidecar's
|
||||
host-only IP also works. This is cleaner than relying on forwarded
|
||||
UDP DNS to a public resolver and gives the backend a natural place to
|
||||
enforce or observe DNS policy.
|
||||
|
||||
The hard blocker is agent routing. Apple Container 1.0.0 exposes no
|
||||
documented `--network` gateway option. An ordinary agent container
|
||||
cannot replace its default route:
|
||||
|
||||
```console
|
||||
$ container exec bb-spike-230t-agent sh -c \
|
||||
'ip route replace default via 192.168.128.2 dev eth0; ip route'
|
||||
default via 192.168.128.1 dev eth0
|
||||
192.168.128.0/24 dev eth0 scope link src 192.168.128.3
|
||||
ip: RTNETLINK answers: Operation not permitted
|
||||
```
|
||||
|
||||
The successful route-through-sidecar tests used `--cap-add
|
||||
CAP_NET_ADMIN` on the agent so the route could be changed after start.
|
||||
That is not an acceptable final design by itself: it expands the
|
||||
agent's kernel-facing privilege and lets the agent mutate its own
|
||||
network namespace. A production design needs either a backend-owned
|
||||
init/shim that sets the route then drops privilege in a way the agent
|
||||
cannot regain, a platform-supported gateway option, or a different
|
||||
network attachment layer.
|
||||
|
||||
## Environment
|
||||
|
||||
Tested on 2026-06-10:
|
||||
|
||||
```console
|
||||
$ sw_vers
|
||||
ProductName: macOS
|
||||
ProductVersion: 26.5.1
|
||||
BuildVersion: 25F80
|
||||
|
||||
$ uname -m
|
||||
arm64
|
||||
|
||||
$ container --version
|
||||
container CLI version 1.0.0 (build: release, commit: ee848e3)
|
||||
```
|
||||
|
||||
Apple Container system status:
|
||||
|
||||
```json
|
||||
{
|
||||
"apiServerAppName": "container-apiserver",
|
||||
"apiServerBuild": "release",
|
||||
"apiServerCommit": "ee848e3ebfd7c73b04dd419683be54fb450b8779",
|
||||
"apiServerVersion": "container-apiserver version 1.0.0 (build: release, commit: ee848e3)",
|
||||
"appRoot": "/Users/didericis/Library/Application Support/com.apple.container/",
|
||||
"installRoot": "/usr/local/",
|
||||
"status": "running"
|
||||
}
|
||||
```
|
||||
|
||||
## Baseline
|
||||
|
||||
Networks:
|
||||
|
||||
```bash
|
||||
container network create bb-spike-230t-agent \
|
||||
--internal \
|
||||
--label bot-bottle.spike=transparent-egress
|
||||
|
||||
container network create bb-spike-230t-egress \
|
||||
--label bot-bottle.spike=transparent-egress
|
||||
```
|
||||
|
||||
Sidecar, dual-homed with NAT first:
|
||||
|
||||
```bash
|
||||
container run --name bb-spike-230t-sidecar \
|
||||
--label bot-bottle.spike=transparent-egress \
|
||||
--network bb-spike-230t-egress \
|
||||
--network bb-spike-230t-agent \
|
||||
--dns 1.1.1.1 \
|
||||
--detach docker.io/alpine:latest sleep 1800
|
||||
```
|
||||
|
||||
Agent, host-only network:
|
||||
|
||||
```bash
|
||||
container run --name bb-spike-230t-agent \
|
||||
--label bot-bottle.spike=transparent-egress \
|
||||
--network bb-spike-230t-agent \
|
||||
--detach docker.io/alpine:latest sleep 1800
|
||||
```
|
||||
|
||||
Observed sidecar addresses:
|
||||
|
||||
```console
|
||||
eth0 192.168.66.2/24 # NAT egress network
|
||||
eth1 192.168.128.2/24 # host-only agent network
|
||||
default via 192.168.66.1 dev eth0
|
||||
nameserver 1.1.1.1
|
||||
```
|
||||
|
||||
Observed agent baseline:
|
||||
|
||||
```console
|
||||
eth0 192.168.128.3/24
|
||||
default via 192.168.128.1 dev eth0
|
||||
nameserver 192.168.128.1
|
||||
wget: bad address 'pypi.org'
|
||||
```
|
||||
|
||||
That confirms the previous spike's baseline: sidecar can egress, agent
|
||||
cannot egress directly.
|
||||
|
||||
## Plain NAT Test
|
||||
|
||||
Relaunch sidecar and agent with `CAP_NET_ADMIN`:
|
||||
|
||||
```bash
|
||||
container run --name bb-spike-230t-sidecar \
|
||||
--label bot-bottle.spike=transparent-egress \
|
||||
--network bb-spike-230t-egress \
|
||||
--network bb-spike-230t-agent \
|
||||
--dns 1.1.1.1 \
|
||||
--cap-add CAP_NET_ADMIN \
|
||||
--detach docker.io/alpine:latest sleep 1800
|
||||
|
||||
container run --name bb-spike-230t-agent \
|
||||
--label bot-bottle.spike=transparent-egress \
|
||||
--network bb-spike-230t-agent \
|
||||
--cap-add CAP_NET_ADMIN \
|
||||
--detach docker.io/alpine:latest sleep 1800
|
||||
```
|
||||
|
||||
Configure sidecar forwarding:
|
||||
|
||||
```bash
|
||||
container exec bb-spike-230t-sidecar sh -c '
|
||||
apk add --no-cache iptables iproute2
|
||||
sysctl -w net.ipv4.ip_forward=1
|
||||
iptables -t nat -A POSTROUTING -s 192.168.128.0/24 -o eth0 -j MASQUERADE
|
||||
iptables -A FORWARD -i eth1 -o eth0 -j ACCEPT
|
||||
iptables -A FORWARD -i eth0 -o eth1 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
|
||||
'
|
||||
```
|
||||
|
||||
Point the agent at the sidecar:
|
||||
|
||||
```bash
|
||||
container exec bb-spike-230t-agent sh -c '
|
||||
ip route replace default via 192.168.128.4 dev eth0
|
||||
printf "nameserver 1.1.1.1\n" > /etc/resolv.conf
|
||||
'
|
||||
```
|
||||
|
||||
Normal direct PyPI fetch from the agent, with no proxy variables set:
|
||||
|
||||
```bash
|
||||
container exec bb-spike-230t-agent sh -c '
|
||||
for v in HTTP_PROXY HTTPS_PROXY http_proxy https_proxy ALL_PROXY all_proxy; do
|
||||
if [ -n "$(printenv "$v")" ]; then echo "$v=SET"; fi
|
||||
done
|
||||
wget -T 10 -O- https://pypi.org/simple/pip/ | head -c 120
|
||||
'
|
||||
```
|
||||
|
||||
Observed:
|
||||
|
||||
```console
|
||||
Connecting to pypi.org (151.101.0.223:443)
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta name="pypi:repository-version" content="1.4">
|
||||
```
|
||||
|
||||
Sidecar NAT counters increased:
|
||||
|
||||
```console
|
||||
POSTROUTING MASQUERADE 3 packets / 168 bytes
|
||||
FORWARD eth1 -> eth0 22 packets / 2806 bytes
|
||||
FORWARD eth0 -> eth1 29 packets / 54781 bytes
|
||||
```
|
||||
|
||||
Verdict: plain transparent routing through the sidecar works, but this
|
||||
is only NAT. It does not apply bot-bottle's existing route allowlist,
|
||||
authorization stripping/injection, or DLP logic.
|
||||
|
||||
## Transparent Mitmproxy Test
|
||||
|
||||
The current sidecar launcher uses explicit proxy mode:
|
||||
|
||||
```sh
|
||||
MODE="--mode regular@9099"
|
||||
exec mitmdump $CONFDIR_FLAG $MODE $LISTEN_HOST_FLAG $TRUST_FLAG -s /app/egress_addon.py
|
||||
```
|
||||
|
||||
So transparent egress needs a launcher mode change plus iptables
|
||||
redirects.
|
||||
|
||||
Run a test mitmproxy container:
|
||||
|
||||
```bash
|
||||
container run --name bb-spike-230t-mitm \
|
||||
--label bot-bottle.spike=transparent-egress \
|
||||
--network bb-spike-230t-egress \
|
||||
--network bb-spike-230t-agent \
|
||||
--dns 1.1.1.1 \
|
||||
--cap-add CAP_NET_ADMIN \
|
||||
--detach mitmproxy/mitmproxy:11.1.3 \
|
||||
sh -c 'apt-get update >/tmp/apt.log &&
|
||||
apt-get install -y --no-install-recommends iptables iproute2 >>/tmp/apt.log &&
|
||||
echo 1 > /proc/sys/net/ipv4/ip_forward &&
|
||||
iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 80 -j REDIRECT --to-port 8080 &&
|
||||
iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 443 -j REDIRECT --to-port 8080 &&
|
||||
mitmdump --mode transparent@8080 --set showhost=true --set ssl_insecure=true --set confdir=/tmp/mitm -v'
|
||||
```
|
||||
|
||||
The container listened successfully:
|
||||
|
||||
```console
|
||||
Transparent Proxy listening at *:8080.
|
||||
```
|
||||
|
||||
It had an agent-facing address of `192.168.128.7`. Point the agent at
|
||||
it and set DNS:
|
||||
|
||||
```bash
|
||||
container exec bb-spike-230t-agent sh -c '
|
||||
ip route replace default via 192.168.128.7 dev eth0
|
||||
printf "nameserver 1.1.1.1\n" > /etc/resolv.conf
|
||||
'
|
||||
```
|
||||
|
||||
DNS also needs NAT/forwarding because only TCP 80/443 is redirected:
|
||||
|
||||
```bash
|
||||
container exec bb-spike-230t-mitm sh -c '
|
||||
iptables -t nat -A POSTROUTING -s 192.168.128.0/24 -o eth0 -j MASQUERADE
|
||||
iptables -A FORWARD -i eth1 -o eth0 -j ACCEPT
|
||||
iptables -A FORWARD -i eth0 -o eth1 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
|
||||
'
|
||||
```
|
||||
|
||||
An alternative, and likely better, DNS shape is to run a DNS forwarder on
|
||||
the sidecar's host-only IP and point the agent at it. This was tested
|
||||
with `dnsmasq`:
|
||||
|
||||
```bash
|
||||
container exec bb-spike-230t-mitm sh -c '
|
||||
apt-get install -y --no-install-recommends dnsmasq
|
||||
cat >/tmp/dnsmasq.conf <<EOF
|
||||
no-daemon
|
||||
listen-address=192.168.128.7
|
||||
bind-interfaces
|
||||
server=1.1.1.1
|
||||
log-queries
|
||||
log-facility=-
|
||||
EOF
|
||||
(dnsmasq --conf-file=/tmp/dnsmasq.conf >/tmp/dnsmasq.log 2>&1 &)
|
||||
sleep 1
|
||||
ss -lunp | grep :53
|
||||
'
|
||||
```
|
||||
|
||||
Observed:
|
||||
|
||||
```console
|
||||
UNCONN 0 0 192.168.128.7:53 0.0.0.0:* users:(("dnsmasq",pid=515,fd=4))
|
||||
```
|
||||
|
||||
Point the agent to sidecar DNS:
|
||||
|
||||
```bash
|
||||
container exec bb-spike-230t-agent sh -c '
|
||||
printf "nameserver 192.168.128.7\n" > /etc/resolv.conf
|
||||
nslookup pypi.org
|
||||
'
|
||||
```
|
||||
|
||||
Observed:
|
||||
|
||||
```console
|
||||
Server: 192.168.128.7
|
||||
Address: 192.168.128.7:53
|
||||
|
||||
Non-authoritative answer:
|
||||
Name: pypi.org
|
||||
Address: 151.101.128.223
|
||||
Name: pypi.org
|
||||
Address: 151.101.192.223
|
||||
Name: pypi.org
|
||||
Address: 151.101.64.223
|
||||
Name: pypi.org
|
||||
Address: 151.101.0.223
|
||||
```
|
||||
|
||||
Direct HTTP from the agent worked and mitmproxy logged the request:
|
||||
|
||||
```console
|
||||
$ container exec bb-spike-230t-agent sh -c \
|
||||
'wget -T 10 -O- http://example.com | head -c 100'
|
||||
Connecting to example.com (172.66.147.243:80)
|
||||
<!doctype html><html lang="en"><head><title>Example Domain</title>
|
||||
```
|
||||
|
||||
Mitmproxy log:
|
||||
|
||||
```console
|
||||
192.168.128.5:39742: GET http://example.com/
|
||||
Host: example.com
|
||||
User-Agent: Wget
|
||||
<< 200 OK 559b
|
||||
```
|
||||
|
||||
After switching the agent to sidecar DNS, direct HTTP still hit
|
||||
mitmproxy:
|
||||
|
||||
```console
|
||||
192.168.128.5:50784: GET http://example.com/
|
||||
Host: example.com
|
||||
User-Agent: Wget
|
||||
<< 200 OK 559b
|
||||
```
|
||||
|
||||
Direct HTTPS from the agent reached mitmproxy but failed certificate
|
||||
verification, as expected when the client does not trust the mitmproxy
|
||||
CA:
|
||||
|
||||
```console
|
||||
$ container exec bb-spike-230t-agent sh -c \
|
||||
'wget -T 10 -O- https://pypi.org/simple/pip/ | head -c 100'
|
||||
Connecting to pypi.org (151.101.128.223:443)
|
||||
... certificate verify failed ...
|
||||
```
|
||||
|
||||
Mitmproxy log:
|
||||
|
||||
```console
|
||||
Client TLS handshake failed. The client does not trust the proxy's
|
||||
certificate for pypi.org (tlsv1 alert unknown ca)
|
||||
```
|
||||
|
||||
With verification disabled, the same direct URL succeeded and mitmproxy
|
||||
logged the full HTTPS request:
|
||||
|
||||
```console
|
||||
$ container exec bb-spike-230t-agent sh -c \
|
||||
'wget --no-check-certificate -T 10 -O- https://pypi.org/simple/pip/ | head -c 100'
|
||||
Connecting to pypi.org (151.101.128.223:443)
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta name="pypi:repository-version" content="1.4">
|
||||
```
|
||||
|
||||
Mitmproxy log:
|
||||
|
||||
```console
|
||||
192.168.128.5:32802: GET https://pypi.org/simple/pip/
|
||||
Host: pypi.org
|
||||
User-Agent: Wget
|
||||
<< 200 OK 103k
|
||||
```
|
||||
|
||||
After switching the agent to sidecar DNS, direct HTTPS still hit
|
||||
mitmproxy:
|
||||
|
||||
```console
|
||||
192.168.128.5:50254: GET https://pypi.org/simple/pip/
|
||||
Host: pypi.org
|
||||
User-Agent: Wget
|
||||
<< 200 OK 103k
|
||||
```
|
||||
|
||||
Verdict: transparent mitmproxy mode works in this topology. The bot
|
||||
agent would still need the egress CA installed, which bot-bottle already
|
||||
does for explicit proxy mode.
|
||||
|
||||
## Answers
|
||||
|
||||
### Can the sidecar become the agent network's default gateway?
|
||||
|
||||
Not directly through Apple Container's documented CLI. The installed
|
||||
`container run --help` documents `--network
|
||||
<name>[,mac=XX:XX:XX:XX:XX:XX][,mtu=VALUE]`; it does not document a
|
||||
gateway option.
|
||||
|
||||
The route can be changed after container start only if the agent has
|
||||
`CAP_NET_ADMIN`. Without it, `ip route replace default via <sidecar>`
|
||||
fails with `Operation not permitted`.
|
||||
|
||||
### Can Apple Container support sidecar forwarding/NAT/transparent proxying?
|
||||
|
||||
Yes. A dual-homed sidecar with `CAP_NET_ADMIN` can enable IP forwarding,
|
||||
set iptables NAT/forwarding rules, and route agent traffic out through
|
||||
the NAT network.
|
||||
|
||||
Transparent mitmproxy interception also works with `PREROUTING`
|
||||
redirects to `mitmdump --mode transparent`.
|
||||
|
||||
### What capabilities/custom image are required?
|
||||
|
||||
At minimum:
|
||||
|
||||
- sidecar needs `CAP_NET_ADMIN`;
|
||||
- sidecar image needs `iptables`/`iproute2` or equivalent nftables
|
||||
tooling;
|
||||
- sidecar should run a DNS listener on its host-only IP, or otherwise
|
||||
provide a controlled resolver path for the agent;
|
||||
- sidecar launcher needs a transparent mode variant;
|
||||
- agent route must be changed to the sidecar's host-only IP;
|
||||
- agent DNS should point to the sidecar DNS listener;
|
||||
- agent must trust the sidecar CA for HTTPS interception.
|
||||
|
||||
The tested agent route mutation required agent `CAP_NET_ADMIN`, which
|
||||
should not be accepted as the final design without a privilege-dropping
|
||||
init/shim story.
|
||||
|
||||
### Can host-level `pf` or vmnet rules replace agent route mutation?
|
||||
|
||||
Not tested. The successful transparent paths did not use host `pf`;
|
||||
they used container-local routing and iptables. Host-level `pf` remains
|
||||
a possible escape hatch if Apple Container cannot set a custom gateway
|
||||
and we reject agent `CAP_NET_ADMIN`.
|
||||
|
||||
### Can existing route policy and DLP semantics be preserved?
|
||||
|
||||
Likely, but not fully validated in this spike. Mitmproxy transparent
|
||||
mode produced normal HTTP flows with correct `Host` values for both
|
||||
HTTP and HTTPS. The existing `egress_addon.py` hooks should still see
|
||||
`flow.request.pretty_host`, method, path, headers, and response bodies.
|
||||
|
||||
But the current sidecar entrypoint only starts `mitmdump` in regular
|
||||
explicit-proxy mode. A real implementation must add a transparent mode
|
||||
launcher and then run the existing egress addon test suite against
|
||||
transparent flows.
|
||||
|
||||
## Recommendation
|
||||
|
||||
Do not switch `macos-container` to transparent egress yet, but keep it
|
||||
as a plausible implementation path.
|
||||
|
||||
The next implementation spike should focus on removing the agent
|
||||
`CAP_NET_ADMIN` requirement. Acceptable options:
|
||||
|
||||
- find or add an Apple Container-supported default-gateway setting;
|
||||
- start the agent through a tiny root init that sets route/DNS, drops
|
||||
capabilities, and then execs the agent as the normal user;
|
||||
- include a sidecar DNS service and set the agent resolver to the
|
||||
sidecar's host-only IP as part of that init/setup path;
|
||||
- avoid routing mutation by using host/vmnet-level packet redirection;
|
||||
- explicitly decide that route mutation is only a convenience layer and
|
||||
keep explicit proxy env vars for v1.
|
||||
|
||||
Bluntly: transparent egress is feasible, but not production-ready until
|
||||
the agent route can be controlled without leaving network-admin power in
|
||||
the agent runtime.
|
||||
@@ -1,16 +1,18 @@
|
||||
---
|
||||
agent_provider:
|
||||
template: claude
|
||||
|
||||
egress:
|
||||
routes:
|
||||
- host: api.anthropic.com
|
||||
role: claude_code_oauth
|
||||
auth:
|
||||
scheme: Bearer
|
||||
token_ref: BOT_BOTTLE_CLAUDE_OAUTH_TOKEN
|
||||
# auth_token names the host env var holding the Claude OAuth token. The
|
||||
# provider injects a provider-owned api.anthropic.com egress route that
|
||||
# re-injects this token as the Bearer header; the agent only ever sees a
|
||||
# placeholder CLAUDE_CODE_OAUTH_TOKEN. DLP defaults (token_patterns,
|
||||
# known_secrets outbound; naive_injection_detection inbound) apply to
|
||||
# that route. To scan additional hosts, declare them under egress.routes
|
||||
# with per-route matches/dlp (see README "Egress route fields").
|
||||
auth_token: BOT_BOTTLE_CLAUDE_OAUTH_TOKEN
|
||||
---
|
||||
|
||||
Common Claude provider boundary. Drop this file into
|
||||
`~/.bot-bottle/bottles/claude.md`, then extend it from task-specific
|
||||
bottles.
|
||||
bottles. The default smolmachines backend keeps DNS resolution under
|
||||
the VM-layer egress policy; use `BOT_BOTTLE_BACKEND=docker` only for
|
||||
legacy Docker-backed runs.
|
||||
|
||||
@@ -10,4 +10,5 @@ The `dev` bottle — backs a generic development workflow.
|
||||
|
||||
Inherits the Claude provider boundary from `claude`. Drop this file
|
||||
into `~/.bot-bottle/bottles/dev.md` and any agent referencing
|
||||
`bottle: dev` will launch against this infrastructure.
|
||||
`bottle: dev` will launch against this infrastructure. By default,
|
||||
bot-bottle runs this bottle on the smolmachines backend.
|
||||
|
||||
Executable
+50
@@ -0,0 +1,50 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
PACKAGE_SPEC="${BOT_BOTTLE_INSTALL_SPEC:-git+https://gitea.dideric.is/didericis/bot-bottle.git}"
|
||||
MIN_PYTHON="3.11"
|
||||
|
||||
say() {
|
||||
printf 'bot-bottle install: %s\n' "$*" >&2
|
||||
}
|
||||
|
||||
die() {
|
||||
say "error: $*"
|
||||
exit 1
|
||||
}
|
||||
|
||||
command -v python3 >/dev/null 2>&1 || die "python3 is required (version ${MIN_PYTHON} or newer)"
|
||||
|
||||
python3 - <<'PY' || die "python3 3.11 or newer is required"
|
||||
import sys
|
||||
|
||||
raise SystemExit(0 if sys.version_info >= (3, 11) else 1)
|
||||
PY
|
||||
|
||||
command -v docker >/dev/null 2>&1 || die "Docker is required; install Docker and start the daemon, then re-run this script"
|
||||
docker info >/dev/null 2>&1 || die "Docker is installed but the daemon is not reachable; start Docker and re-run this script"
|
||||
|
||||
mkdir -p \
|
||||
"${HOME}/.bot-bottle/agents" \
|
||||
"${HOME}/.bot-bottle/bottles" \
|
||||
"${HOME}/.bot-bottle/contrib"
|
||||
|
||||
if command -v pipx >/dev/null 2>&1; then
|
||||
say "installing with pipx"
|
||||
pipx install --force "${PACKAGE_SPEC}"
|
||||
else
|
||||
say "pipx not found; installing with python3 -m pip --user"
|
||||
python3 -m pip install --user --upgrade "${PACKAGE_SPEC}"
|
||||
fi
|
||||
|
||||
if command -v bot-bottle >/dev/null 2>&1; then
|
||||
BOT_BOTTLE_BIN="bot-bottle"
|
||||
elif [ -x "${HOME}/.local/bin/bot-bottle" ]; then
|
||||
BOT_BOTTLE_BIN="${HOME}/.local/bin/bot-bottle"
|
||||
say "using ${BOT_BOTTLE_BIN}; add ${HOME}/.local/bin to PATH for future shells"
|
||||
else
|
||||
die "bot-bottle was installed but is not on PATH"
|
||||
fi
|
||||
|
||||
say "running bot-bottle doctor"
|
||||
"${BOT_BOTTLE_BIN}" doctor
|
||||
@@ -0,0 +1,27 @@
|
||||
[build-system]
|
||||
requires = ["setuptools>=68"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "bot-bottle"
|
||||
version = "0.1.0"
|
||||
description = "Self-hosted sandbox for AI coding agents with egress controls"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.11"
|
||||
license = { text = "Apache-2.0" }
|
||||
dependencies = []
|
||||
|
||||
[project.scripts]
|
||||
bot-bottle = "bot_bottle.cli:main"
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
include = ["bot_bottle*"]
|
||||
|
||||
[tool.setuptools.package-data]
|
||||
bot_bottle = [
|
||||
"Dockerfile.sidecars",
|
||||
"egress_entrypoint.sh",
|
||||
"contrib/claude/Dockerfile",
|
||||
"contrib/codex/Dockerfile",
|
||||
"contrib/pi/Dockerfile",
|
||||
]
|
||||
@@ -35,5 +35,5 @@ chmod 600 "$fake_key_dir/fake-key"
|
||||
|
||||
# Build the image graph quietly so the recorded run shows only the
|
||||
# bottle launch and the four `!` probes, not BuildKit progress.
|
||||
docker build -q -f Dockerfile.claude -t bot-bottle-claude:latest . >/dev/null 2>&1 || true
|
||||
docker build -q -f bot_bottle/contrib/claude/Dockerfile -t bot-bottle-claude:latest . >/dev/null 2>&1 || true
|
||||
docker build -q -f Dockerfile.git-gate -t bot-bottle-git-gate:latest . >/dev/null 2>&1 || true
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user