Compare commits
22 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 650f3aa93e | |||
| 8a038dcceb | |||
| 3b96de95ab | |||
| 962f8144fc | |||
| 200a113cce | |||
| 2eb6e02ee1 | |||
| 86cfd94b72 | |||
| 0233b481b1 | |||
| b79b49090f | |||
| 1fceaae8e6 | |||
| 12f8c37af2 | |||
| 10c009c37b | |||
| a8b2237964 | |||
| 1d2dfeefa4 | |||
| 3942b51184 | |||
| 3060dc553c | |||
| 8e5262b539 | |||
| 237afce844 | |||
| 91d6f5d8d2 | |||
| 55117c5ff4 | |||
| 1a5ea3b713 | |||
| 6e7f0756cf |
@@ -1,76 +0,0 @@
|
|||||||
---
|
|
||||||
name: quality-eval
|
|
||||||
description: Use when the user asks to objectively evaluate, score, rate, audit, or quality-gate code, codebases, files, pull requests, or snippets using a strict 5-dimension engineering rubric with scores and refactoring steps.
|
|
||||||
metadata:
|
|
||||||
short-description: Score code quality with a strict rubric
|
|
||||||
---
|
|
||||||
|
|
||||||
# Quality Eval
|
|
||||||
|
|
||||||
## Role
|
|
||||||
|
|
||||||
Act as a Staff Software Engineer and automated quality gate. Evaluate code objectively against the rubric below, surface hidden anti-patterns, and provide a mathematical grade with atomic refactoring steps.
|
|
||||||
|
|
||||||
## Evaluation Rules
|
|
||||||
|
|
||||||
- Evaluate only against the five rubric dimensions.
|
|
||||||
- Be candid. Do not inflate scores for politeness.
|
|
||||||
- Avoid generic advice. Every recommendation must name a specific code location, behavior, or pattern and include a concrete improvement direction.
|
|
||||||
- Inspect the code before scoring. For codebases, read enough representative files, tests, and architecture boundaries to justify the scope.
|
|
||||||
- When exact line numbers are available, cite them.
|
|
||||||
- Do not reveal private chain-of-thought. In the required `Chain of Thought Analysis` section, provide a concise, step-by-step audit rationale with observable findings and score justifications.
|
|
||||||
|
|
||||||
## Rubric
|
|
||||||
|
|
||||||
Score each dimension from 1 to 5 using these anchors:
|
|
||||||
|
|
||||||
| Dimension | Score 1 (Fail) | Score 3 (Pass) | Score 5 (Exemplary) |
|
|
||||||
| :--- | :--- | :--- | :--- |
|
|
||||||
| **Architecture** | Spaghettified; tight coupling; violated separation of concerns. | Modular but relies on leaky abstractions or mixed domains. | Strict domain isolation; follows SOLID; clear dependency inversion. |
|
|
||||||
| **Readability** | Cryptic naming; deep nesting (>3 levels); widespread DRY violations. | Idiomatic but features over-complex functions or sparse documentation. | Self-documenting; expressive naming; high cohesion; flat structure. |
|
|
||||||
| **Resilience** | Swallows errors blindly; lacks contextual logging; fragile to bad input. | Basic try/catch blocks present but lacks granular, typed error handling. | Explicit error boundaries; contextual logging; structured failure modes. |
|
|
||||||
| **Testability** | Hardcoded dependencies make mocking or isolated testing impossible. | Pure functions are testable, but side-effect heavy logic lacks test hooks. | Decoupled IO; deterministic execution; structured for unit and integration tests. |
|
|
||||||
| **SecOps** | Hardcoded secrets; O(n^2) bottlenecks; zero input sanitization. | Safe from obvious flaws but lacks deep defensive optimization. | Validated inputs; optimized algorithmic complexity; zero security debt. |
|
|
||||||
|
|
||||||
## Scoring Method
|
|
||||||
|
|
||||||
1. Determine the evaluated scope and primary language.
|
|
||||||
2. Identify concrete evidence for each dimension.
|
|
||||||
3. Assign integer dimension scores from 1 to 5.
|
|
||||||
4. Compute `composite_score` as the arithmetic mean of the five dimension scores, rounded to one decimal place.
|
|
||||||
5. Include code snippets only when they make a refactoring step more actionable.
|
|
||||||
|
|
||||||
## Required Output
|
|
||||||
|
|
||||||
Structure every response into exactly these three Markdown sections:
|
|
||||||
|
|
||||||
### 1. Chain of Thought Analysis
|
|
||||||
|
|
||||||
Provide a concise step-by-step audit rationale. Name specific files, functions, patterns, anti-patterns, and rubric anchors. Keep it evidence-based and do not include hidden private reasoning.
|
|
||||||
|
|
||||||
### 2. Normalized Score Report
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"evaluation_metadata": {
|
|
||||||
"target_scope": "string",
|
|
||||||
"primary_language": "string"
|
|
||||||
},
|
|
||||||
"metrics": {
|
|
||||||
"architecture_and_modularity": 0,
|
|
||||||
"readability_and_maintainability": 0,
|
|
||||||
"error_handling_and_resilience": 0,
|
|
||||||
"testability_and_mocking": 0,
|
|
||||||
"security_and_performance": 0
|
|
||||||
},
|
|
||||||
"composite_score": 0.0
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. Atomic Refactoring Playbook
|
|
||||||
|
|
||||||
* **High Priority (To lift Score 1/2 to 3):**
|
|
||||||
- [ ] Actionable, specific refactoring step with file/line/context reference.
|
|
||||||
* **Medium Priority (To lift Score 3 to 4/5):**
|
|
||||||
- [ ] Optimization or architectural pattern implementation step.
|
|
||||||
|
|
||||||
@@ -1,3 +0,0 @@
|
|||||||
display_name: Quality Eval
|
|
||||||
short_description: Scores code quality with a strict five-dimension rubric and refactoring playbook.
|
|
||||||
default_prompt: Evaluate this code objectively using the quality-eval rubric and return the three-section score report.
|
|
||||||
+1
-1
@@ -23,7 +23,7 @@ FROM node:22-slim
|
|||||||
# tool (curl itself, plus anything that shells out to it) works
|
# tool (curl itself, plus anything that shells out to it) works
|
||||||
# against pipelock's bumped TLS without the agent needing local DNS.
|
# against pipelock's bumped TLS without the agent needing local DNS.
|
||||||
RUN apt-get update \
|
RUN apt-get update \
|
||||||
&& apt-get install -y --no-install-recommends git ca-certificates openssh-client socat curl dnsutils python3 python3-pip python3-venv \
|
&& apt-get install -y --no-install-recommends git ca-certificates openssh-client socat curl dnsutils \
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
# Install claude-code globally. Pinned to the version verified in the v1
|
# Install claude-code globally. Pinned to the version verified in the v1
|
||||||
|
|||||||
+2
-2
@@ -6,10 +6,10 @@
|
|||||||
FROM node:22-slim
|
FROM node:22-slim
|
||||||
|
|
||||||
RUN apt-get update \
|
RUN apt-get update \
|
||||||
&& apt-get install -y --no-install-recommends git ca-certificates openssh-client socat curl dnsutils python3 python3-pip python3-venv \
|
&& apt-get install -y --no-install-recommends git ca-certificates openssh-client socat curl dnsutils \
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
RUN npm install -g --no-fund --no-audit @openai/codex@0.136.0 \
|
RUN npm install -g --no-fund --no-audit @openai/codex@0.134.0 \
|
||||||
&& npm cache clean --force
|
&& npm cache clean --force
|
||||||
|
|
||||||
USER node
|
USER node
|
||||||
|
|||||||
@@ -6,26 +6,96 @@
|
|||||||
|
|
||||||
[](https://gitea.dideric.is/didericis/bot-bottle/actions?workflow=test.yml)
|
[](https://gitea.dideric.is/didericis/bot-bottle/actions?workflow=test.yml)
|
||||||
|
|
||||||
**Problem:** Developer wants to run a coding agent without supervision, but they don't want a prompt injected or misbehaving agent wrecking their environment or exfiltrating sensitive data.
|
Run multiple Claude Code agents on your own machine, each scoped to its own secrets, skills, and egress allowlist.
|
||||||
|
|
||||||
**Solution:** Ephemeral, per agent "bottles" the agent cannot modify that scan all traffic for data exfiltration and limit capabilities and egress to only what the agent needs.
|

|
||||||
|
|
||||||
## Features
|
Four prompts to the agent inside a real bottle:
|
||||||
|
claude replies to `hello there` — proof api.anthropic.com routes
|
||||||
|
through pipelock's bumped TLS end-to-end;
|
||||||
|
asked to GET a non-allowlisted host, the agent's curl gets 403 back
|
||||||
|
from pipelock;
|
||||||
|
asked to POST a credential-shaped body to an allowlisted host, the
|
||||||
|
same 403 — pipelock's DLP body scanner caught it;
|
||||||
|
asked to commit and push an AKIA-shaped key, git-gate's gitleaks
|
||||||
|
pre-receive hook rejects the ref.
|
||||||
|
Run it yourself with `bash scripts/demo.sh`.
|
||||||
|
|
||||||
- **Per-bottle egress allowlist** — TLS-bumped HTTP/HTTPS chokepoint with a per-manifest host allowlist and request-body DLP scanner; DoH and arbitrary hosts blocked by default.
|
## Why "bot-bottle"?
|
||||||
- **Tokens the agent never sees** — host secrets live in a sidecar; the agent dials `http://sidecar:9099/<path>` and the proxy strips inbound `Authorization` and injects the real token before forwarding. `printenv` in the agent shows proxy URLs only.
|
|
||||||
- **Gitleaks-scanned push (git-gate)** — `bottle.git` remotes route through a per-bottle `git daemon` that gitleaks-scans incoming refs pre-receive and forwards clean refs upstream over SSH. The agent never holds the upstream credential.
|
Each container is a bottle; Claude is the genie inside. The genie's
|
||||||
- **Manifest-scoped skills + secrets** — each bottle declares its skills, env, git identity, remotes, and egress routes; unknown keys die at load.
|
powers are exactly what the manifest grants it — a specific set of
|
||||||
- **Trust boundary at `$HOME`** — bottles (credentials, egress, remotes) live only under `~/.bot-bottle/bottles/`. Repos may ship agents but not bottles, so a cloned repo can't redirect an env var to an attacker host.
|
skills, a specific set of secrets, and a specific set of hosts it can
|
||||||
- **Composable bottles (`extends:`)** — keep provider/runtime policy in one base bottle (e.g. `claude.md`) and overlay task bottles on top.
|
reach — nothing more. You uncork one bottle per agent
|
||||||
- **Parallel, isolated bottles** — each bottle is its own per-agent Docker `--internal` network; bottles don't share state or talk to each other.
|
(`./cli.py start <agent>`), many bottles run in parallel, and each is
|
||||||
- **Provider templates (Claude, Codex)** — `Dockerfile.claude` / `Dockerfile.codex`, or a bottle-supplied Dockerfile. Claude auth via long-lived OAuth token; Codex via opt-in host device-auth forwarding.
|
scoped to its task. When the session ends the bottle is destroyed and
|
||||||
- **gVisor auto-detect** — on Linux hosts where `runsc` is registered with Docker, every bottle launches under it for a userspace syscall barrier; no manifest config required.
|
the genie does not persist.
|
||||||
- **Smolmachines backend (macOS)** — opt-in `BOT_BOTTLE_BACKEND=smolmachines` runs the agent in a libkrun micro-VM with the sidecar bundle still in Docker.
|
|
||||||
|
## Goals
|
||||||
|
|
||||||
|
- Scope each agent to the minimum credentials and network egress its task actually needs
|
||||||
|
- Run multiple agents in parallel, isolated from each other
|
||||||
|
- Keep code, credentials, and agent activity on infrastructure I control — no third-party agent runtime
|
||||||
|
|
||||||
|
## Project status
|
||||||
|
|
||||||
|
bot-bottle is a self-hosted secure runtime for AI coding agents.
|
||||||
|
Each agent runs in an isolated container or micro-VM-backed bottle with
|
||||||
|
scoped secrets, allowlisted egress, TLS-aware proxying, DLP checks, and
|
||||||
|
a git-gate that withholds upstream credentials and scans pushes before
|
||||||
|
forwarding. The project includes a documented threat model, PRD-driven
|
||||||
|
development history, Docker and smolmachines backends, dashboard and
|
||||||
|
remediation flows, and unit/integration tests covering exfiltration and
|
||||||
|
sandbox escape scenarios.
|
||||||
|
|
||||||
|
## Security model
|
||||||
|
|
||||||
|
Each agent runs in its own bottle: its own container, its own internal
|
||||||
|
Docker network, and its own pipelock sidecar. Bottles don't share
|
||||||
|
state, don't talk to each other, and only get the env vars, skills,
|
||||||
|
SSH identities, and egress hosts the manifest grants them — nothing
|
||||||
|
more. Any one agent only has the access it needs to do its job.
|
||||||
|
|
||||||
|
The bottle limits both what an agent can see and where it can send
|
||||||
|
it. Each bottle gets only the secrets and SSH identities the manifest
|
||||||
|
grants it — a Gitea token but not a GitHub token, a deploy key but
|
||||||
|
not a personal SSH key — so even a compromised or misbehaving agent
|
||||||
|
only handles credentials it was already trusted with for its job.
|
||||||
|
Egress flows through pipelock, which constrains where those
|
||||||
|
credentials can travel: an agent with a Gitea token can reach
|
||||||
|
`gitea.dideric.is`, not arbitrary attacker-controlled hosts. The same
|
||||||
|
constraint blocks DNS-over-HTTPS as an exfil channel — a DoH resolver
|
||||||
|
like `cloudflare-dns.com` would have to be on the allowlist for the
|
||||||
|
agent to reach it at all. The container itself adds a layer between
|
||||||
|
the agent and the host, but the v1 design leans more on secret
|
||||||
|
minimization and egress allowlisting than on the container as a
|
||||||
|
hardened boundary. On Linux hosts where [gVisor](https://gvisor.dev/)
|
||||||
|
is registered with Docker, bot-bottle auto-detects it and launches
|
||||||
|
every bottle under `runsc` for a userspace syscall barrier — no
|
||||||
|
manifest configuration required. The broader v2 discussion lives in
|
||||||
|
`docs/research/stronger-isolation-alternatives.md`.
|
||||||
|
|
||||||
|
The egress proxy and OAuth-token handling below are the load-bearing
|
||||||
|
pieces of v1.
|
||||||
|
|
||||||
## Architecture
|
## Architecture
|
||||||
|
|
||||||
A bottle is two containers per agent: an `agent` container, and a `sidecars` container that bundles pipelock + cred-proxy + git-gate + supervise behind a Python init supervisor. They share a per-agent Docker `--internal` network; the agent has no default route off-box.
|
A bottle is two containers per agent: an `agent` container, and a
|
||||||
|
`sidecars` container that bundles pipelock + egress + git-gate +
|
||||||
|
supervise behind a Python init supervisor (PRD 0024). They share a
|
||||||
|
per-agent Docker `--internal` network; the agent has no default
|
||||||
|
route off-box. All HTTP and HTTPS egress funnels through pipelock,
|
||||||
|
where the egress allowlist, TLS interception, and request-body DLP
|
||||||
|
scanner enforce the manifest before any byte leaves the host. The
|
||||||
|
only egress that doesn't traverse pipelock is git-gate's SSH
|
||||||
|
push/fetch to `bottle.git` upstreams — pipelock can't proxy SSH,
|
||||||
|
so git-gate is its own L4-style egress path with gitleaks doing
|
||||||
|
the pre-receive scan.
|
||||||
|
|
||||||
|
The agent dials the bundle by the legacy short names (`pipelock`,
|
||||||
|
`egress`, `git-gate`, `supervise`); the renderer registers those as
|
||||||
|
docker-network aliases on the bundle so existing HTTPS_PROXY URLs
|
||||||
|
and MCP endpoints resolve without an agent-side change.
|
||||||
|
|
||||||
```
|
```
|
||||||
host ( ./cli.py )
|
host ( ./cli.py )
|
||||||
@@ -34,21 +104,26 @@ A bottle is two containers per agent: an `agent` container, and a `sidecars` con
|
|||||||
▼
|
▼
|
||||||
┌─────────────────────────── bottle ──────────────────────────────────┐
|
┌─────────────────────────── bottle ──────────────────────────────────┐
|
||||||
│ │
|
│ │
|
||||||
│ ┌──────────────────┐ ┌──────────────┐ │
|
│ ┌──────────────────┐ │
|
||||||
│ │ agent image │ HTTP(S) proxy │ cred-proxy │ │
|
│ │ agent image │ HTTPS_PROXY │
|
||||||
│ │ (claude-code, │ ─────────────────►│ (strips/inj │ │
|
│ │ (claude-code, │ ────────────────────────┐ │
|
||||||
│ │ codex, etc) │ │ Authoriz.) │ │
|
│ │ built locally) │ │ │
|
||||||
│ │ │ └──────┬───────┘ │
|
│ │ │ plain HTTP │ │
|
||||||
│ │ environ: URLs │ │ │
|
│ │ skills, env, │ (token injection) ┌────▼─────────┐ │
|
||||||
│ │ only, no real │ ▼ │
|
│ │ ~/.gitconfig, │ ──────────────────►│ cred-proxy │ │
|
||||||
│ │ tokens │ ┌────────────────┐ │ HTTPS to
|
│ │ ~/.npmrc, tea │ │ (strips/inj │ │
|
||||||
|
│ │ │ │ Authoriz.) │ │
|
||||||
|
│ │ environ: URLs │ └─────┬────────┘ │
|
||||||
|
│ │ only, no real │ HTTPS_PROXY │ │
|
||||||
|
│ │ tokens │ ▼ │
|
||||||
|
│ │ │ ┌────────────────┐ │ HTTPS to
|
||||||
│ │ │ │ pipelock image │──────────┼──► allowlisted
|
│ │ │ │ pipelock image │──────────┼──► allowlisted
|
||||||
│ │ │ │ (TLS bump, DLP │ │ hosts (incl.
|
│ │ │ │ (TLS bump, DLP │ │ hosts (incl.
|
||||||
│ │ │ │ body scan, │ │ cred-proxy
|
│ │ │ │ body scan, │ │ cred-proxy
|
||||||
│ │ │ │ allowlist) │ │ upstreams)
|
│ │ │ │ allowlist) │ │ upstreams)
|
||||||
│ │ │ └────────────────┘ │
|
│ │ │ └────────────────┘ │
|
||||||
│ │ │ │
|
│ │ │ │
|
||||||
│ │ │ git proxy ┌────────────────┐ │ SSH push/fetch
|
│ │ │ git:// ┌────────────────┐ │ SSH push/fetch
|
||||||
│ │ │ ────────────────►│ git-gate image │──────────┼──► to bottle.git
|
│ │ │ ────────────────►│ git-gate image │──────────┼──► to bottle.git
|
||||||
│ │ │ │ (gitleaks + │ │ upstreams
|
│ │ │ │ (gitleaks + │ │ upstreams
|
||||||
│ └──────────────────┘ │ git daemon) │ │ (direct — not
|
│ └──────────────────┘ │ git daemon) │ │ (direct — not
|
||||||
@@ -62,25 +137,198 @@ A bottle is two containers per agent: an `agent` container, and a `sidecars` con
|
|||||||
└─────────────────────────────────────────────────────────────────────┘
|
└─────────────────────────────────────────────────────────────────────┘
|
||||||
```
|
```
|
||||||
|
|
||||||
When the agent exits, `cli.py` tears down every sidecar and both networks; nothing about a bottle persists between runs.
|
- **agent image** — built from the provider template Dockerfile
|
||||||
|
(`Dockerfile.claude` for Claude, `Dockerfile.codex` for Codex, or
|
||||||
|
`agent_provider.dockerfile`) on first run; runs the selected agent
|
||||||
|
CLI with the manifest-granted skills, env vars, and `~/.gitconfig`
|
||||||
|
(the latter for the git-gate's `insteadOf` rules when `bottle.git`
|
||||||
|
is set).
|
||||||
|
- **pipelock image** — per-agent sidecar. Terminates the agent's
|
||||||
|
outbound HTTP/HTTPS, enforces the resolved allowlist, runs DLP
|
||||||
|
scanning. Design in `docs/prds/0001-per-agent-egress-proxy-via-pipelock.md`
|
||||||
|
and `docs/prds/0006-pipelock-tls-interception.md`.
|
||||||
|
- **git-gate image** — per-agent sidecar built on `zricethezav/gitleaks`
|
||||||
|
(alpine + gitleaks + git-daemon + openssh-client). Runs
|
||||||
|
`git daemon` over `git://` as a bidirectional mirror of each
|
||||||
|
declared upstream. A pre-receive hook gitleaks-scans incoming
|
||||||
|
refs and forwards clean refs to the real upstream over SSH; an
|
||||||
|
access-hook runs `git fetch origin --prune` against the upstream
|
||||||
|
before every upload-pack so an agent fetch returns whatever the
|
||||||
|
upstream has *now* (fail-closed if unreachable). The agent's
|
||||||
|
`~/.gitconfig` rewrites the real URL to the gate via `insteadOf`,
|
||||||
|
so push, fetch, clone, and pull all route through. The agent
|
||||||
|
never sees the upstream credential. If the upstream's hostname
|
||||||
|
isn't resolvable from the gate container (e.g. a Tailscale-only
|
||||||
|
host whose public DNS points elsewhere), pin its IP via
|
||||||
|
`ExtraHosts: { "<hostname>": "<ip>" }` on the `bottle.git` entry —
|
||||||
|
the gate's `/etc/hosts` gets the override while the agent's
|
||||||
|
`insteadOf` rewrite still keys off the original hostname. Brought
|
||||||
|
up only when `bottle.git` has entries. Design in
|
||||||
|
`docs/prds/0008-git-gate.md`.
|
||||||
|
- **cred-proxy image** — per-bottle sidecar (`python:3.13-alpine`
|
||||||
|
base, stdlib-only) that holds API tokens declared in
|
||||||
|
`bottle.cred_proxy.routes`. Each route names a `path`,
|
||||||
|
`upstream`, `auth_scheme`, and `token_ref` (host env var); the
|
||||||
|
agent dials `http://cred-proxy:9099<path>...` over plain HTTP
|
||||||
|
and the proxy strips any inbound `Authorization`, injects
|
||||||
|
`<auth_scheme> <token>` using the value held only in its own
|
||||||
|
container's environ, and forwards to the real upstream over
|
||||||
|
HTTPS. SSE responses stream back unbuffered. The cred-proxy's
|
||||||
|
outbound HTTPS routes through pipelock (it trusts pipelock's
|
||||||
|
per-bottle CA), so pipelock's egress allowlist + body scanner
|
||||||
|
apply to cred-proxy traffic the same way they apply to direct
|
||||||
|
agent traffic. Smart-HTTP push paths (`/git-receive-pack`,
|
||||||
|
`/info/refs?service=git-receive-pack`) are refused at the
|
||||||
|
proxy — push must go through `bottle.git` / git-gate where
|
||||||
|
gitleaks runs. Optional per-route `role` tags drive agent-side
|
||||||
|
rewrites: `anthropic-base-url`, `npm-registry`, `git-insteadof`,
|
||||||
|
`tea-login`. The agent's `printenv` shows only proxy URLs —
|
||||||
|
none of the real token values. Design in
|
||||||
|
`docs/prds/0010-cred-proxy.md`.
|
||||||
|
|
||||||
|
When the agent exits, `cli.py` tears down every sidecar that was
|
||||||
|
brought up and the two networks; nothing about a bottle persists
|
||||||
|
between runs.
|
||||||
|
|
||||||
## Quickstart
|
## Quickstart
|
||||||
|
|
||||||
Requires Docker on the host and a long-lived Claude Code OAuth token (`claude setup-token`) exported as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`.
|
Requires Docker on the host and a long-lived Claude Code OAuth token in
|
||||||
|
your shell env.
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
./cli.py start <agent> # builds the image on first run, drops you into claude
|
./cli.py start <agent> # builds the image on first run, drops you into claude
|
||||||
```
|
```
|
||||||
|
|
||||||
|
The container is removed automatically when the session ends. If the script
|
||||||
|
is killed with SIGKILL the exit trap won't fire and the container may be
|
||||||
|
left running; remove it with `docker rm -f <container-name>`.
|
||||||
|
|
||||||
|
### Smolmachines backend (experimental, macOS-only)
|
||||||
|
|
||||||
|
A second backend runs the agent in a smolvm micro-VM (libkrun) with the
|
||||||
|
sidecar bundle still in Docker. Selected via
|
||||||
|
`BOT_BOTTLE_BACKEND=smolmachines ./cli.py start <agent>`. Requires
|
||||||
|
`smolvm` on PATH (`curl -sSL https://smolmachines.com/install.sh | sh`).
|
||||||
|
|
||||||
|
The integration tests run against whichever backend the env var
|
||||||
|
selects and skip cleanly when its prerequisites are missing.
|
||||||
|
|
||||||
|
**One-time sudo on first launch (macOS):** smolmachines bottles
|
||||||
|
each reserve a loopback alias from a pool (`127.0.0.16` ..
|
||||||
|
`127.0.0.31`) and bind their bundle's port-forwards to it; the
|
||||||
|
first `./cli.py start` after each reboot prompts for sudo to add
|
||||||
|
missing aliases via `ifconfig lo0 alias`. Aliases persist until
|
||||||
|
reboot; subsequent launches don't prompt. The agent's TSI
|
||||||
|
allowlist is the alias's `/32`, so each bottle can only reach
|
||||||
|
its own bundle's published ports — not other bottles' ports,
|
||||||
|
not other host loopback services (postgres, dev servers, etc.).
|
||||||
|
|
||||||
|
This enforcement requires a workaround for a smolvm 0.8.0 bug:
|
||||||
|
the CLI's `--allow-cidr` flag is silently dropped when combined
|
||||||
|
with `--from <smolmachine>`. The launcher patches smolvm's
|
||||||
|
persistent state DB
|
||||||
|
(`~/Library/Application Support/smolvm/server/smolvm.db`)
|
||||||
|
directly between `machine create` and `machine start` to set
|
||||||
|
the allowlist. The hack falls away automatically when smolvm
|
||||||
|
honors the flag upstream — see the `loopback_alias` module's
|
||||||
|
docstring for the investigation trail.
|
||||||
|
|
||||||
## Manifest
|
## Manifest
|
||||||
|
|
||||||
Bottles and agents are Markdown files with YAML frontmatter under `~/.bot-bottle/`. The Markdown body is the system prompt. Bottles live in `~/.bot-bottle/bottles/`; agents may also be shipped by a repo at `<repo>/.bot-bottle/agents/<name>.md`.
|
Bottles and agents live as Markdown files with YAML frontmatter under
|
||||||
|
`~/.bot-bottle/`. Each bottle is one file in `bottles/`, each agent
|
||||||
|
is one file in `agents/`:
|
||||||
|
|
||||||
**Bottle** (`~/.bot-bottle/bottles/gitea-dev.md`):
|
```
|
||||||
|
~/.bot-bottle/
|
||||||
|
├── bottles/
|
||||||
|
│ ├── dev.md
|
||||||
|
│ └── gitea-dev.md
|
||||||
|
└── agents/
|
||||||
|
├── implementer.md
|
||||||
|
└── researcher.md
|
||||||
|
```
|
||||||
|
|
||||||
|
The filename (without `.md`) is the entity's name. Filenames must
|
||||||
|
match `[a-z][a-z0-9-]*`; files that don't are skipped with a warning.
|
||||||
|
|
||||||
|
A repo can ship its own agent files alongside its code at
|
||||||
|
`<repo>/.bot-bottle/agents/<name>.md`. Those agents reference
|
||||||
|
bottles defined in `~/.bot-bottle/bottles/` (the only place
|
||||||
|
bottles can come from); a `bottles/` subdir in a repo is ignored
|
||||||
|
with a warning. **This is the trust boundary**: bottle infrastructure
|
||||||
|
— credentials, egress allowlists, git remotes — comes from your home
|
||||||
|
directory only. A cloned repo cannot redirect a host env var to an
|
||||||
|
attacker-named upstream because it has no way to declare a bottle.
|
||||||
|
|
||||||
|
### Bottle composition with `extends:`
|
||||||
|
|
||||||
|
A bottle can inherit from another via `extends: <bottle-name>` so
|
||||||
|
operators don't have to duplicate a whole bottle file to vary one
|
||||||
|
field (PRD 0025). The parent's resolved config is the base; the
|
||||||
|
child's declared fields overlay. Merge rules:
|
||||||
|
|
||||||
|
- `env:` — dict merge, child wins on key collision.
|
||||||
|
- `git.user:` — per-field overlay (child's non-empty `name` /
|
||||||
|
`email` wins; empty falls through to parent).
|
||||||
|
- `git.remotes:` — dict merge by host, child wins on host collision.
|
||||||
|
An explicit `git.remotes: {}` clears the parent's remotes; omitting
|
||||||
|
`git.remotes` inherits the parent's remotes.
|
||||||
|
- `agent_provider:`, `egress:`, `supervise:` — full replace when the
|
||||||
|
child declares the field.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
---
|
||||||
|
extends: dev # inherit everything from bottles/dev.md
|
||||||
|
egress:
|
||||||
|
routes:
|
||||||
|
- host: staging.example.com
|
||||||
|
auth:
|
||||||
|
scheme: Bearer
|
||||||
|
token_ref: STAGING_TOKEN
|
||||||
|
---
|
||||||
|
```
|
||||||
|
|
||||||
|
Cycles (`A extends B extends A`), self-references, and missing
|
||||||
|
parents die at parse with a clear pointer. Bottles remain
|
||||||
|
`$HOME`-only — `extends:` preserves the trust boundary above.
|
||||||
|
|
||||||
|
### Provider base bottles
|
||||||
|
|
||||||
|
Keep provider/runtime policy in one home-owned base bottle, then have
|
||||||
|
task bottles extend it. That keeps provider egress/auth in one place
|
||||||
|
without hiding security-relevant routes behind `agent_provider.template`.
|
||||||
|
|
||||||
|
For example, `~/.bot-bottle/bottles/claude.md` can hold the Claude
|
||||||
|
provider selection and Anthropic API egress:
|
||||||
|
|
||||||
````markdown
|
````markdown
|
||||||
---
|
---
|
||||||
extends: claude # inherit the Claude provider boundary
|
agent_provider:
|
||||||
|
template: claude
|
||||||
|
|
||||||
|
egress:
|
||||||
|
routes:
|
||||||
|
- host: api.anthropic.com
|
||||||
|
role: claude_code_oauth
|
||||||
|
auth:
|
||||||
|
scheme: Bearer
|
||||||
|
token_ref: BOT_BOTTLE_CLAUDE_OAUTH_TOKEN
|
||||||
|
pipelock:
|
||||||
|
tls_passthrough: true
|
||||||
|
---
|
||||||
|
|
||||||
|
Common Claude provider boundary.
|
||||||
|
````
|
||||||
|
|
||||||
|
Task bottles can then inherit that provider boundary and add their own
|
||||||
|
env/git configuration without repeating the Claude route.
|
||||||
|
|
||||||
|
### Example bottle (`~/.bot-bottle/bottles/gitea-dev.md`)
|
||||||
|
|
||||||
|
````markdown
|
||||||
|
---
|
||||||
|
extends: claude
|
||||||
|
|
||||||
env:
|
env:
|
||||||
GIT_AUTHOR_NAME: didericis
|
GIT_AUTHOR_NAME: didericis
|
||||||
@@ -95,7 +343,148 @@ git:
|
|||||||
Upstream: ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git
|
Upstream: ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git
|
||||||
IdentityFile: /Users/didericis/.ssh/id_ed25519_gitea
|
IdentityFile: /Users/didericis/.ssh/id_ed25519_gitea
|
||||||
KnownHostKey: ssh-ed25519 AAAA...
|
KnownHostKey: ssh-ed25519 AAAA...
|
||||||
|
---
|
||||||
|
|
||||||
|
The `gitea-dev` bottle. Backs my work on personal projects: provider
|
||||||
|
auth through egress and gitea.dideric.is over SSH.
|
||||||
|
````
|
||||||
|
|
||||||
|
For a Codex-backed base bottle, set `agent_provider.template: codex`.
|
||||||
|
The Codex template expects ChatGPT/device login state instead of an
|
||||||
|
`OPENAI_API_KEY` env var; no API-key placeholder is forwarded into the
|
||||||
|
agent. To let bot-bottle read the host's current Codex ChatGPT access
|
||||||
|
token and inject it from egress only for Codex's API calls, opt in
|
||||||
|
explicitly:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
agent_provider:
|
||||||
|
template: codex
|
||||||
|
forward_host_credentials: true
|
||||||
|
|
||||||
|
egress:
|
||||||
|
routes:
|
||||||
|
- host: auth.openai.com
|
||||||
|
path_allowlist:
|
||||||
|
- /api/accounts/deviceauth/
|
||||||
|
```
|
||||||
|
|
||||||
|
Run `codex login --device-auth` on the host before launch. The
|
||||||
|
launcher reads `tokens.access_token` from the host's
|
||||||
|
`~/.codex/auth.json`, verifies it is fresh user/device auth, and passes
|
||||||
|
it to the sidecar's `EGRESS_TOKEN_N` env slot. The agent container gets
|
||||||
|
a dummy `~/.codex/auth.json` that preserves the host auth-mode shape
|
||||||
|
but replaces credential values with placeholders. It keeps the selected
|
||||||
|
ChatGPT account id so Codex sends requests for the same account while
|
||||||
|
egress owns the real bearer token. The agent never receives real access
|
||||||
|
tokens, refresh tokens, or `OPENAI_API_KEY`. The effective egress table
|
||||||
|
automatically adds or upgrades `api.openai.com` and `chatgpt.com` to
|
||||||
|
authenticated routes when `forward_host_credentials` is true.
|
||||||
|
|
||||||
|
The built-in Codex template uses `Dockerfile.codex`; set
|
||||||
|
`agent_provider.dockerfile` to build the agent from a custom Dockerfile
|
||||||
|
while keeping the bot-bottle sidecars in place.
|
||||||
|
|
||||||
|
### Example agent (`~/.bot-bottle/agents/gitea-helper.md`)
|
||||||
|
|
||||||
|
````markdown
|
||||||
|
---
|
||||||
|
bottle: gitea-dev
|
||||||
|
skills:
|
||||||
|
- init-prd
|
||||||
|
git:
|
||||||
|
user:
|
||||||
|
name: gitea-helper
|
||||||
|
email: eric+gitea-helper@dideric.is
|
||||||
|
---
|
||||||
|
|
||||||
|
You help maintain Gitea-hosted projects.
|
||||||
|
````
|
||||||
|
|
||||||
|
The agent's Markdown body is its system prompt (whitespace
|
||||||
|
stripped). The frontmatter declares the bottle to launch in and any
|
||||||
|
skills to mount. You can also include Claude Code subagent fields
|
||||||
|
(`name`, `description`, `model`, `color`, `memory`) in the
|
||||||
|
frontmatter — bot-bottle ignores them at launch but doesn't
|
||||||
|
reject them, so the same file can drop into `~/.claude/agents/` as a
|
||||||
|
Claude Code subagent.
|
||||||
|
|
||||||
|
An agent may also declare `git.user` (`name` / `email`). It overlays
|
||||||
|
the referenced bottle's `git.user` per-field — the agent's non-empty
|
||||||
|
fields win, the rest fall through to the bottle — so two agents can
|
||||||
|
share one bottle and still commit under distinct identities without
|
||||||
|
an identity-only bottle (PRD 0027). Only `git.user` is allowed at the
|
||||||
|
agent level; `git.remotes` stays bottle-only because it carries
|
||||||
|
credentials and host trust. The launch preflight and `cli.py info`
|
||||||
|
print the effective identity annotated `(agent)` / `(bottle)` so you
|
||||||
|
can see where each field came from. Git authorship is not a
|
||||||
|
credential — push auth is the bottle's remote key/token — so a
|
||||||
|
repo-shipped agent setting its own identity grants no access; treat
|
||||||
|
an agent identity as *claimed, not vouched*.
|
||||||
|
|
||||||
|
Unknown top-level frontmatter keys die at load with a "did you mean"
|
||||||
|
pointer; typos don't silently ghost into an empty config.
|
||||||
|
|
||||||
|
The YAML subset the frontmatter accepts is bounded (flat keys,
|
||||||
|
strings / ints / true-or-false bools / null / lists / one-level
|
||||||
|
nested dicts). Anchors, multi-line block scalars, tags, and
|
||||||
|
ambiguous bare strings (`yes` / `NO` / `2026-05-24` /
|
||||||
|
`0x...`) all die with a clear pointer at the spec — quote your
|
||||||
|
strings when in doubt. The full schema lives in
|
||||||
|
`bot_bottle/yaml_subset.py` (~450 lines, stdlib-only, no PyYAML).
|
||||||
|
|
||||||
|
Working examples live under `examples/`. Pipelock's design lives in
|
||||||
|
`docs/prds/0001-per-agent-egress-proxy-via-pipelock.md` and the
|
||||||
|
rationale in `docs/research/pipelock-assessment.md`. The trust
|
||||||
|
boundary rationale lives in `docs/prds/0011-per-file-md-manifest.md`.
|
||||||
|
|
||||||
|
## Auth: Claude OAuth token, not API key
|
||||||
|
|
||||||
|
Bottles that use `agent_provider.template: claude` authenticate
|
||||||
|
`claude` inside the container with the same Pro/Max subscription you
|
||||||
|
already use on the host, via a long-lived OAuth token. No
|
||||||
|
`ANTHROPIC_API_KEY` is needed.
|
||||||
|
|
||||||
|
**Why a token instead of mounting `~/.claude.json`:** on macOS, Claude
|
||||||
|
Code stores OAuth credentials in the encrypted Keychain, not in
|
||||||
|
`~/.claude.json`. Mounting that file into a Linux container does not
|
||||||
|
carry the credentials with it. Linux hosts keep credentials in
|
||||||
|
`~/.claude/.credentials.json`, but to keep the launcher portable
|
||||||
|
bot-bottle uses the env-var path on every host.
|
||||||
|
|
||||||
|
**One-time setup on the host:**
|
||||||
|
|
||||||
|
```sh
|
||||||
|
claude setup-token # browser login, prints a ~1-year OAuth token
|
||||||
|
```
|
||||||
|
|
||||||
|
Stash the token in your shell env (e.g. `~/.zshrc` or a secret manager)
|
||||||
|
as `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
export BOT_BOTTLE_CLAUDE_OAUTH_TOKEN="<token>"
|
||||||
|
```
|
||||||
|
|
||||||
|
The Claude bottle reaches the Anthropic API only through the cred-proxy
|
||||||
|
sidecar. To let `claude` authenticate, declare an egress route with
|
||||||
|
`role: claude_code_oauth` and
|
||||||
|
`token_ref: BOT_BOTTLE_CLAUDE_OAUTH_TOKEN`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
egress:
|
||||||
|
routes:
|
||||||
|
- host: api.anthropic.com
|
||||||
|
role: claude_code_oauth
|
||||||
|
auth:
|
||||||
|
scheme: Bearer
|
||||||
|
token_ref: BOT_BOTTLE_CLAUDE_OAUTH_TOKEN
|
||||||
|
pipelock:
|
||||||
|
tls_passthrough: true
|
||||||
|
```
|
||||||
|
|
||||||
|
Routes that resolve to private or Tailscale addresses can opt into
|
||||||
|
pipelock's SSRF destination allowlist explicitly:
|
||||||
|
|
||||||
|
```yaml
|
||||||
egress:
|
egress:
|
||||||
routes:
|
routes:
|
||||||
- host: gitea.dideric.is
|
- host: gitea.dideric.is
|
||||||
@@ -103,31 +492,38 @@ egress:
|
|||||||
scheme: token
|
scheme: token
|
||||||
token_ref: BOT_BOTTLE_GITEA_TOKEN
|
token_ref: BOT_BOTTLE_GITEA_TOKEN
|
||||||
pipelock:
|
pipelock:
|
||||||
ssrf_ip_allowlist: [100.78.141.42/32]
|
ssrf_ip_allowlist:
|
||||||
---
|
- 100.78.141.42/32
|
||||||
|
```
|
||||||
|
|
||||||
The `gitea-dev` bottle. Provider auth via the inherited Claude route;
|
At launch, `cli.py` reads `BOT_BOTTLE_CLAUDE_OAUTH_TOKEN` from the host
|
||||||
gitea over SSH for push, token over HTTPS for the API.
|
env and forwards it into the cred-proxy container's environ — never
|
||||||
````
|
into the agent's. The agent receives `ANTHROPIC_BASE_URL` pointing at
|
||||||
|
`http://cred-proxy:9099/anthropic` and a non-secret placeholder for
|
||||||
|
`CLAUDE_CODE_OAUTH_TOKEN` (claude-code refuses to start without one;
|
||||||
|
the proxy strips and replaces the header on every request). `printenv`
|
||||||
|
inside the agent does not surface the real token, and the value is
|
||||||
|
never written to disk or placed on argv on the host.
|
||||||
|
|
||||||
**Agent** (`~/.bot-bottle/agents/gitea-helper.md`):
|
A Claude bottle without a `claude_code_oauth` route has no path to the
|
||||||
|
Anthropic API — there is no fallback that forwards the token directly
|
||||||
````markdown
|
to the agent. Caveats: the token is bound to your subscription tier
|
||||||
---
|
(Pro/Max/Team/Enterprise), it does not work with `claude --bare`
|
||||||
bottle: gitea-dev
|
(which only reads `ANTHROPIC_API_KEY`), and if it leaks, regenerate
|
||||||
skills:
|
via `claude setup-token` again. Reference:
|
||||||
- init-prd
|
<https://code.claude.com/docs/en/authentication>.
|
||||||
---
|
|
||||||
|
|
||||||
You help maintain Gitea-hosted projects.
|
|
||||||
````
|
|
||||||
|
|
||||||
More examples in `examples/`. Full design lives under `docs/prds/`; the trust-boundary rationale is in `docs/prds/0011-per-file-md-manifest.md`.
|
|
||||||
|
|
||||||
## Trademarks
|
## Trademarks
|
||||||
|
|
||||||
bot-bottle is an independent project and is not affiliated with, endorsed by, or sponsored by Anthropic, PBC. "Claude" and "Claude Code" are trademarks of Anthropic, PBC; the project name uses "claude" descriptively to indicate that the tool runs Claude Code inside a sandbox.
|
bot-bottle is an independent project and is not affiliated with,
|
||||||
|
endorsed by, or sponsored by Anthropic, PBC. "Claude" and "Claude
|
||||||
|
Code" are trademarks of Anthropic, PBC; the project name uses
|
||||||
|
"claude" descriptively to indicate that the tool runs Claude Code
|
||||||
|
inside a sandbox.
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
Copyright 2026 Eric Bauerfeld. Licensed under the Apache License, Version 2.0. See [LICENSE](LICENSE) for the full text.
|
Copyright 2026 Eric Bauerfeld
|
||||||
|
|
||||||
|
Licensed under the Apache License, Version 2.0. See [LICENSE](LICENSE)
|
||||||
|
for the full text.
|
||||||
|
|||||||
@@ -4,15 +4,14 @@
|
|||||||
"env": {
|
"env": {
|
||||||
"FAKE_TOKEN": "ghp_aB3cD4eF5gH6iJ7kL8mN9oP0qR1sT2uV3wX4yZ"
|
"FAKE_TOKEN": "ghp_aB3cD4eF5gH6iJ7kL8mN9oP0qR1sT2uV3wX4yZ"
|
||||||
},
|
},
|
||||||
"git-gate": {
|
"git": [
|
||||||
"repos": {
|
{
|
||||||
"foo": {
|
"Name": "foo",
|
||||||
"url": "ssh://git@upstream.invalid/path.git",
|
"Upstream": "ssh://git@upstream.invalid/path.git",
|
||||||
"identity": "~/.cache/bot-bottle-demo/fake-key",
|
"IdentityFile": "~/.cache/bot-bottle-demo/fake-key",
|
||||||
"host_key": "ssh-ed25519 AAAAEXAMPLE"
|
"KnownHostKey": "ssh-ed25519 AAAAEXAMPLE"
|
||||||
}
|
|
||||||
}
|
}
|
||||||
}
|
]
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
|
||||||
|
|||||||
+112
-109
@@ -3,32 +3,17 @@
|
|||||||
The manifest owns the user-facing AgentProvider shape. This module is
|
The manifest owns the user-facing AgentProvider shape. This module is
|
||||||
the launch-time table that turns a provider template into an executable
|
the launch-time table that turns a provider template into an executable
|
||||||
command, default image, and prompt/auth behavior.
|
command, default image, and prompt/auth behavior.
|
||||||
|
|
||||||
Per PRD 0050 the per-provider implementations live under
|
|
||||||
`bot_bottle/contrib/<template>/agent_provider.py`. This module exposes:
|
|
||||||
|
|
||||||
- `AgentProvider` (ABC) — the contract each plugin implements.
|
|
||||||
- `get_provider(template)` — lazy-imported registry; the analogue
|
|
||||||
of `bot_bottle/deploy_key_provisioner.get_provisioner`.
|
|
||||||
- `AgentProvisionPlan` (+ helper dataclasses) — declarative shape
|
|
||||||
each provider produces and the backends consume unchanged.
|
|
||||||
- `agent_provision_plan` / `runtime_for` — thin wrappers around the
|
|
||||||
registry kept so existing callers keep working without per-call
|
|
||||||
edits.
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from abc import ABC, abstractmethod
|
import os
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import TYPE_CHECKING, Literal
|
from typing import Literal
|
||||||
|
|
||||||
from .egress import EgressRoute
|
from .codex_auth import write_codex_dummy_auth_file
|
||||||
|
from .egress import CODEX_HOST_CREDENTIAL_TOKEN_REF, EgressRoute
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
|
||||||
from .backend import Bottle, BottlePlan
|
|
||||||
|
|
||||||
|
|
||||||
PROVIDER_CLAUDE = "claude"
|
PROVIDER_CLAUDE = "claude"
|
||||||
@@ -107,91 +92,37 @@ class AgentProvisionPlan:
|
|||||||
verify: tuple[AgentProvisionCommand, ...] = ()
|
verify: tuple[AgentProvisionCommand, ...] = ()
|
||||||
egress_routes: tuple[EgressRoute, ...] = ()
|
egress_routes: tuple[EgressRoute, ...] = ()
|
||||||
hidden_env_names: frozenset[str] = field(default_factory=frozenset)
|
hidden_env_names: frozenset[str] = field(default_factory=frozenset)
|
||||||
provisioned_env: dict[str, str] = field(default_factory=dict)
|
|
||||||
|
|
||||||
|
|
||||||
class AgentProvider(ABC):
|
_REPO_ROOT = Path(__file__).resolve().parent.parent
|
||||||
"""Per-template plugin: produces the provision plan and applies
|
|
||||||
the provider-specific in-guest setup steps (skills, prompt, the
|
|
||||||
declarative `dirs`/`files`/`pre_copy`/`verify` apply loop, and
|
|
||||||
supervise MCP registration). Concrete subclasses live under
|
|
||||||
`bot_bottle/contrib/<template>/agent_provider.py`."""
|
|
||||||
|
|
||||||
@property
|
|
||||||
@abstractmethod
|
|
||||||
def runtime(self) -> AgentProviderRuntime:
|
|
||||||
"""The static command / image / prompt-mode table for this
|
|
||||||
template."""
|
|
||||||
|
|
||||||
@abstractmethod
|
|
||||||
def provision_plan(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
dockerfile: str,
|
|
||||||
state_dir: Path,
|
|
||||||
guest_home: str,
|
|
||||||
guest_env: dict[str, str] | None = None,
|
|
||||||
auth_token: str = "",
|
|
||||||
forward_host_credentials: bool = False,
|
|
||||||
host_env: dict[str, str] | None = None,
|
|
||||||
trusted_project_path: str = "",
|
|
||||||
) -> AgentProvisionPlan:
|
|
||||||
"""Build the declarative AgentProvisionPlan for one launch.
|
|
||||||
Backends call this during `prepare` and consume the result as
|
|
||||||
before."""
|
|
||||||
|
|
||||||
@abstractmethod
|
|
||||||
def provision_skills(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
|
||||||
"""Copy each of the agent's named skills from the host into
|
|
||||||
the guest. No-op when the agent has no skills. The in-guest
|
|
||||||
layout is provider-specific (claude-code's
|
|
||||||
`~/.claude/skills/` today; future providers may differ)."""
|
|
||||||
|
|
||||||
@abstractmethod
|
|
||||||
def provision_prompt(self, plan: "BottlePlan", bottle: "Bottle") -> str | None:
|
|
||||||
"""Copy the prompt file into the guest, fix ownership/mode,
|
|
||||||
and return the in-guest path iff the agent has a non-empty
|
|
||||||
prompt (drives the `--append-system-prompt-file` flag).
|
|
||||||
|
|
||||||
The file is copied either way so the path always exists."""
|
|
||||||
|
|
||||||
@abstractmethod
|
|
||||||
def provision(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
|
||||||
"""Apply the provider's declarative
|
|
||||||
`dirs`/`pre_copy`/`files`/`verify` steps from
|
|
||||||
`plan.agent_provision`. Was called `provision_provider_auth`
|
|
||||||
on `BottleBackend` before PRD 0050."""
|
|
||||||
|
|
||||||
@abstractmethod
|
|
||||||
def provision_supervise_mcp(
|
|
||||||
self,
|
|
||||||
plan: "BottlePlan",
|
|
||||||
bottle: "Bottle",
|
|
||||||
supervise_url: str,
|
|
||||||
) -> None:
|
|
||||||
"""Register the per-bottle supervise sidecar as an MCP server
|
|
||||||
in the provider's in-guest config. Called by the backend after
|
|
||||||
the supervise sidecar is reachable. No-op when
|
|
||||||
`plan.supervise_plan is None`."""
|
|
||||||
|
|
||||||
|
|
||||||
def get_provider(template: str) -> AgentProvider:
|
_RUNTIMES = {
|
||||||
"""Resolve a provider template name to its plugin instance.
|
PROVIDER_CLAUDE: AgentProviderRuntime(
|
||||||
|
template=PROVIDER_CLAUDE,
|
||||||
Lazy-imports the contrib module so importing this module doesn't
|
command="claude",
|
||||||
pull provider-specific code paths in. Mirrors the contrib
|
image="bot-bottle-claude:latest",
|
||||||
convention PRD 0048 established for deploy key provisioners."""
|
dockerfile=str(_REPO_ROOT / "Dockerfile.claude"),
|
||||||
if template == PROVIDER_CLAUDE:
|
prompt_mode="append_file",
|
||||||
from .contrib.claude.agent_provider import ClaudeAgentProvider
|
bypass_args=("--dangerously-skip-permissions",),
|
||||||
return ClaudeAgentProvider()
|
resume_args=("--continue",),
|
||||||
if template == PROVIDER_CODEX:
|
remote_control_args=("--remote-control",),
|
||||||
from .contrib.codex.agent_provider import CodexAgentProvider
|
),
|
||||||
return CodexAgentProvider()
|
PROVIDER_CODEX: AgentProviderRuntime(
|
||||||
raise ValueError(f"unknown agent provider template: {template!r}")
|
template=PROVIDER_CODEX,
|
||||||
|
command="codex",
|
||||||
|
image="bot-bottle-codex:latest",
|
||||||
|
dockerfile=str(_REPO_ROOT / "Dockerfile.codex"),
|
||||||
|
prompt_mode="read_prompt_file",
|
||||||
|
bypass_args=("--dangerously-bypass-approvals-and-sandbox",),
|
||||||
|
resume_args=("resume", "--last"),
|
||||||
|
remote_control_args=(),
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
def runtime_for(template: str) -> AgentProviderRuntime:
|
def runtime_for(template: str) -> AgentProviderRuntime:
|
||||||
return get_provider(template).runtime
|
return _RUNTIMES[template]
|
||||||
|
|
||||||
|
|
||||||
def agent_provision_plan(
|
def agent_provision_plan(
|
||||||
@@ -199,24 +130,96 @@ def agent_provision_plan(
|
|||||||
template: str,
|
template: str,
|
||||||
dockerfile: str,
|
dockerfile: str,
|
||||||
state_dir: Path,
|
state_dir: Path,
|
||||||
guest_home: str,
|
guest_home: str = "/home/node",
|
||||||
guest_env: dict[str, str] | None = None,
|
guest_env: dict[str, str] | None = None,
|
||||||
auth_token: str = "",
|
auth_token: str = "",
|
||||||
forward_host_credentials: bool = False,
|
forward_host_credentials: bool = False,
|
||||||
host_env: dict[str, str] | None = None,
|
host_env: dict[str, str] | None = None,
|
||||||
trusted_project_path: str = "",
|
|
||||||
) -> AgentProvisionPlan:
|
) -> AgentProvisionPlan:
|
||||||
"""Back-compat shim — `prepare` callers stay the same; the work
|
runtime = runtime_for(template)
|
||||||
now lives on the provider plugin."""
|
resolved_guest_env = dict(guest_env or {})
|
||||||
return get_provider(template).provision_plan(
|
env_vars: dict[str, str] = {}
|
||||||
|
dirs: list[AgentProvisionDir] = []
|
||||||
|
files: list[AgentProvisionFile] = []
|
||||||
|
pre_copy: list[AgentProvisionCommand] = []
|
||||||
|
verify: list[AgentProvisionCommand] = []
|
||||||
|
egress_routes: list[EgressRoute] = []
|
||||||
|
hidden_env_names: frozenset[str] = frozenset()
|
||||||
|
|
||||||
|
if template == PROVIDER_CODEX:
|
||||||
|
env_vars["CODEX_CA_CERTIFICATE"] = "/etc/ssl/certs/ca-certificates.crt"
|
||||||
|
auth_dir = resolved_guest_env.get("CODEX_HOME", f"{guest_home}/.codex")
|
||||||
|
if forward_host_credentials:
|
||||||
|
env_vars["CODEX_HOME"] = auth_dir
|
||||||
|
dirs.append(AgentProvisionDir(auth_dir))
|
||||||
|
config_path = f"{auth_dir}/config.toml"
|
||||||
|
config_file = state_dir / "codex-config.toml"
|
||||||
|
config_file.write_text(
|
||||||
|
f'[projects."{guest_home}"]\n'
|
||||||
|
'trust_level = "trusted"\n'
|
||||||
|
)
|
||||||
|
config_file.chmod(0o600)
|
||||||
|
files.append(AgentProvisionFile(config_file, config_path))
|
||||||
|
|
||||||
|
for host in CODEX_HOST_CREDENTIAL_HOSTS:
|
||||||
|
egress_routes.append(EgressRoute(
|
||||||
|
host=host,
|
||||||
|
auth_scheme="Bearer" if forward_host_credentials else "",
|
||||||
|
token_ref=CODEX_HOST_CREDENTIAL_TOKEN_REF if forward_host_credentials else "",
|
||||||
|
tls_passthrough=True,
|
||||||
|
))
|
||||||
|
if forward_host_credentials:
|
||||||
|
auth_file = state_dir / "codex-auth.json"
|
||||||
|
write_codex_dummy_auth_file(auth_file, host_env or dict(os.environ))
|
||||||
|
files.append(AgentProvisionFile(auth_file, f"{auth_dir}/auth.json"))
|
||||||
|
pre_copy.append(AgentProvisionCommand((
|
||||||
|
"find", auth_dir,
|
||||||
|
"-maxdepth", "1",
|
||||||
|
"-type", "f",
|
||||||
|
"(",
|
||||||
|
"-name", "*.sqlite",
|
||||||
|
"-o", "-name", "*.sqlite-*",
|
||||||
|
"-o", "-name", "*.codex-repair-*.bak",
|
||||||
|
")",
|
||||||
|
"-delete",
|
||||||
|
), "codex host credentials: could not reset runtime db files"))
|
||||||
|
verify.append(AgentProvisionCommand((
|
||||||
|
"runuser", "-u", "node", "--",
|
||||||
|
"env",
|
||||||
|
f"HOME={guest_home}",
|
||||||
|
f"CODEX_HOME={auth_dir}",
|
||||||
|
"codex", "login", "status",
|
||||||
|
), (
|
||||||
|
"codex host credentials: dummy auth was copied into the "
|
||||||
|
"guest, but Codex did not accept it"
|
||||||
|
)))
|
||||||
|
if template == PROVIDER_CLAUDE:
|
||||||
|
env_vars["CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC"] = "1"
|
||||||
|
env_vars["DISABLE_ERROR_REPORTING"] = "1"
|
||||||
|
egress_routes.append(EgressRoute(
|
||||||
|
host="api.anthropic.com",
|
||||||
|
auth_scheme="Bearer" if auth_token else "",
|
||||||
|
token_ref=auth_token,
|
||||||
|
tls_passthrough=True,
|
||||||
|
))
|
||||||
|
if auth_token:
|
||||||
|
env_vars["CLAUDE_CODE_OAUTH_TOKEN"] = "egress-placeholder"
|
||||||
|
hidden_env_names = frozenset({"CLAUDE_CODE_OAUTH_TOKEN"})
|
||||||
|
|
||||||
|
return AgentProvisionPlan(
|
||||||
|
template=template,
|
||||||
|
command=runtime.command,
|
||||||
|
prompt_mode=runtime.prompt_mode,
|
||||||
|
image=runtime.image,
|
||||||
dockerfile=dockerfile,
|
dockerfile=dockerfile,
|
||||||
state_dir=state_dir,
|
env_vars=env_vars,
|
||||||
guest_home=guest_home,
|
guest_env=resolved_guest_env,
|
||||||
guest_env=guest_env,
|
dirs=tuple(dirs),
|
||||||
auth_token=auth_token,
|
files=tuple(files),
|
||||||
forward_host_credentials=forward_host_credentials,
|
pre_copy=tuple(pre_copy),
|
||||||
host_env=host_env,
|
verify=tuple(verify),
|
||||||
trusted_project_path=trusted_project_path,
|
egress_routes=tuple(egress_routes),
|
||||||
|
hidden_env_names=hidden_env_names,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
+49
-105
@@ -32,22 +32,15 @@ manifest does not carry a backend field; the host picks.
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import os
|
import os
|
||||||
import sys
|
|
||||||
from abc import ABC, abstractmethod
|
from abc import ABC, abstractmethod
|
||||||
from contextlib import AbstractContextManager
|
from contextlib import AbstractContextManager
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Generic, Sequence, TypeVar
|
from typing import Any, Generic, Sequence, TypeVar
|
||||||
|
|
||||||
from ..agent_provider import AgentProvisionPlan, get_provider
|
from ..log import die
|
||||||
from ..egress import EgressPlan
|
|
||||||
from ..git_gate import GitGatePlan
|
|
||||||
from ..log import die, info
|
|
||||||
from ..manifest import GitEntry, Manifest
|
from ..manifest import GitEntry, Manifest
|
||||||
from ..supervise import SupervisePlan
|
|
||||||
from ..util import expand_tilde
|
from ..util import expand_tilde
|
||||||
from ..workspace import WorkspacePlan
|
|
||||||
from .print_util import print_multi, visible_agent_env_names
|
|
||||||
from .util import host_skill_dir
|
from .util import host_skill_dir
|
||||||
|
|
||||||
|
|
||||||
@@ -72,58 +65,15 @@ class BottleSpec:
|
|||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class BottlePlan(ABC):
|
class BottlePlan(ABC):
|
||||||
"""Base output of a backend's prepare step. Concrete subclasses
|
"""Base output of a backend's prepare step. Concrete subclasses
|
||||||
(e.g. DockerBottlePlan) add backend-specific resolved fields."""
|
(e.g. DockerBottlePlan) add backend-specific resolved fields and
|
||||||
|
implement `print`."""
|
||||||
|
|
||||||
spec: BottleSpec
|
spec: BottleSpec
|
||||||
stage_dir: Path
|
stage_dir: Path
|
||||||
guest_home: str
|
|
||||||
git_gate_plan: GitGatePlan
|
|
||||||
egress_plan: EgressPlan
|
|
||||||
supervise_plan: SupervisePlan | None
|
|
||||||
agent_provision: AgentProvisionPlan
|
|
||||||
workspace_plan: WorkspacePlan
|
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
def print(self, *, remote_control: bool) -> None:
|
def print(self, *, remote_control: bool) -> None:
|
||||||
"""Render the y/N preflight summary to stderr."""
|
"""Render the y/N preflight summary to stderr."""
|
||||||
del remote_control
|
|
||||||
spec = self.spec
|
|
||||||
manifest = spec.manifest
|
|
||||||
agent = manifest.agents[spec.agent_name]
|
|
||||||
bottle = manifest.bottle_for(spec.agent_name)
|
|
||||||
|
|
||||||
env_names = visible_agent_env_names(
|
|
||||||
sorted(
|
|
||||||
set(bottle.env.keys())
|
|
||||||
| set(self.agent_provision.guest_env.keys())
|
|
||||||
),
|
|
||||||
hidden_env_names=self.agent_provision.hidden_env_names,
|
|
||||||
)
|
|
||||||
|
|
||||||
print(file=sys.stderr)
|
|
||||||
info(f"agent : {spec.agent_name}")
|
|
||||||
info(f"provider : {self.agent_provision.template}")
|
|
||||||
print_multi("env ", env_names)
|
|
||||||
print_multi("skills ", list(agent.skills))
|
|
||||||
info(f"bottle : {agent.bottle}")
|
|
||||||
|
|
||||||
identity = manifest.git_identity_summary(spec.agent_name)
|
|
||||||
if identity:
|
|
||||||
info(f" git identity : {identity}")
|
|
||||||
|
|
||||||
git_lines = [
|
|
||||||
f"{u.name} → {u.upstream_host}:{u.upstream_port}"
|
|
||||||
for u in self.git_gate_plan.upstreams
|
|
||||||
]
|
|
||||||
if git_lines:
|
|
||||||
print_multi(" git gate ", git_lines)
|
|
||||||
|
|
||||||
if self.egress_plan.routes:
|
|
||||||
egress_lines = []
|
|
||||||
for r in self.egress_plan.routes:
|
|
||||||
auth = f" [auth:{r.auth_scheme}]" if r.auth_scheme else ""
|
|
||||||
egress_lines.append(f"{r.host}{auth}")
|
|
||||||
print_multi(" egress ", egress_lines)
|
|
||||||
print(file=sys.stderr)
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
@@ -313,44 +263,36 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
|
|||||||
def launch(self, plan: PlanT) -> AbstractContextManager[Bottle]:
|
def launch(self, plan: PlanT) -> AbstractContextManager[Bottle]:
|
||||||
"""Build/run the bottle and yield a handle; tear down on exit."""
|
"""Build/run the bottle and yield a handle; tear down on exit."""
|
||||||
|
|
||||||
def provision(self, plan: PlanT, bottle: "Bottle") -> str | None:
|
def provision(self, plan: PlanT, target: str) -> str | None:
|
||||||
"""Copy host-side files (CA cert, prompt, skills, .git) into
|
"""Copy host-side files (CA cert, prompt, skills, .git) into
|
||||||
the running bottle. Called from `launch` after the container
|
the running bottle. Called from `launch` after the container
|
||||||
/ machine is up. Returns the in-container prompt path if a
|
/ machine is up. `target` identifies the running instance in
|
||||||
prompt was provisioned, else None — the Bottle handle uses it
|
backend-specific terms (Docker: resolved container name; fly:
|
||||||
to decide whether to add provider-specific prompt args to the
|
machine id). Returns the in-container prompt path if a prompt
|
||||||
agent's argv.
|
was provisioned, else None — the Bottle handle uses it to
|
||||||
|
decide whether to add provider-specific prompt args to the agent's
|
||||||
|
argv.
|
||||||
|
|
||||||
Default orchestration: ca → prompt → provider apply → skills
|
Default orchestration: ca → prompt → skills → git →
|
||||||
→ workspace → git → supervise-mcp. CA install runs first so
|
supervise. CA install runs first so the agent's trust store
|
||||||
the agent's trust store is rebuilt before anything inside the
|
is rebuilt before anything inside the agent makes a TLS call.
|
||||||
agent makes a TLS call.
|
Subclasses typically don't override this; they implement the
|
||||||
|
sub-methods below.
|
||||||
Per PRD 0050 the per-provider steps (prompt, skills,
|
|
||||||
declarative provision-plan apply, supervise MCP registration)
|
|
||||||
live on the `AgentProvider` plugin. The backend only owns the
|
|
||||||
steps that are about backend infrastructure (CA, workspace,
|
|
||||||
git) and surfaces the supervise sidecar URL its launch step
|
|
||||||
knows about via `supervise_mcp_url`.
|
|
||||||
|
|
||||||
PRD 0017: cred-proxy's agent-side dotfile rewrites (~/.npmrc,
|
PRD 0017: cred-proxy's agent-side dotfile rewrites (~/.npmrc,
|
||||||
~/.gitconfig insteadOf, tea config) are gone. Egress-proxy is
|
~/.gitconfig insteadOf, tea config) are gone. Egress-proxy is
|
||||||
on the agent's HTTP_PROXY path so every tool that respects
|
on the agent's HTTP_PROXY path so every tool that respects
|
||||||
HTTPS_PROXY (claude-code, git over HTTPS, npm, curl) is
|
HTTPS_PROXY (claude-code, git over HTTPS, npm, curl) is
|
||||||
intercepted without per-tool reconfiguration."""
|
intercepted without per-tool reconfiguration."""
|
||||||
provider = get_provider(plan.agent_provision.template)
|
self.provision_ca(plan, target)
|
||||||
self.provision_ca(plan, bottle)
|
prompt_path = self.provision_prompt(plan, target)
|
||||||
prompt_path = provider.provision_prompt(plan, bottle)
|
self.provision_provider_auth(plan, target)
|
||||||
provider.provision(plan, bottle)
|
self.provision_skills(plan, target)
|
||||||
provider.provision_skills(plan, bottle)
|
self.provision_git(plan, target)
|
||||||
self.provision_workspace(plan, bottle)
|
self.provision_supervise(plan, target)
|
||||||
self.provision_git(plan, bottle)
|
|
||||||
provider.provision_supervise_mcp(
|
|
||||||
plan, bottle, self.supervise_mcp_url(plan),
|
|
||||||
)
|
|
||||||
return prompt_path
|
return prompt_path
|
||||||
|
|
||||||
def provision_ca(self, plan: PlanT, bottle: "Bottle") -> None:
|
def provision_ca(self, plan: PlanT, target: str) -> None:
|
||||||
"""Install the per-bottle CA into the agent's trust store so
|
"""Install the per-bottle CA into the agent's trust store so
|
||||||
the agent trusts the bumped CONNECT cert egress (was
|
the agent trusts the bumped CONNECT cert egress (was
|
||||||
pipelock, pre-PRD-0017) presents. Default impl is a no-op so
|
pipelock, pre-PRD-0017) presents. Default impl is a no-op so
|
||||||
@@ -359,26 +301,34 @@ class BottleBackend(ABC, Generic[PlanT, CleanupT]):
|
|||||||
backend overrides to docker-cp the cert in and run
|
backend overrides to docker-cp the cert in and run
|
||||||
`update-ca-certificates`."""
|
`update-ca-certificates`."""
|
||||||
|
|
||||||
def provision_workspace(self, plan: PlanT, bottle: "Bottle") -> None:
|
def provision_provider_auth(self, plan: PlanT, target: str) -> None:
|
||||||
"""Copy the operator workspace into the running bottle when
|
"""Install non-secret provider auth marker files into the agent
|
||||||
the backend cannot bake it into the agent image. Default is
|
home when a provider needs them to select the right auth mode.
|
||||||
no-op for backends like Docker that handle this before launch."""
|
The default is no-op."""
|
||||||
|
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
def provision_git(self, plan: PlanT, bottle: "Bottle") -> None:
|
def provision_prompt(self, plan: PlanT, target: str) -> str | None:
|
||||||
|
"""Copy the prompt file into the running bottle. Returns the
|
||||||
|
in-container path iff the agent has a non-empty prompt;
|
||||||
|
callers use the return value to decide whether to add
|
||||||
|
provider-specific prompt args to the agent's argv."""
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def provision_skills(self, plan: PlanT, target: str) -> None:
|
||||||
|
"""Copy the agent's named skills from the host into the
|
||||||
|
running bottle. No-op when the agent has no skills."""
|
||||||
|
|
||||||
|
@abstractmethod
|
||||||
|
def provision_git(self, plan: PlanT, target: str) -> None:
|
||||||
"""Copy the host's cwd `.git` directory into the running
|
"""Copy the host's cwd `.git` directory into the running
|
||||||
bottle if the user requested --cwd. No-op otherwise."""
|
bottle if the user requested --cwd. No-op otherwise."""
|
||||||
|
|
||||||
def supervise_mcp_url(self, plan: PlanT) -> str:
|
def provision_supervise(self, plan: PlanT, target: str) -> None:
|
||||||
"""Return the agent-side URL of the per-bottle supervise
|
"""Write the in-bottle Claude Code MCP config so the agent
|
||||||
sidecar, or "" when this bottle has no sidecar. The provider
|
discovers the per-bottle supervise sidecar (PRD 0013).
|
||||||
plugin's `provision_supervise_mcp` uses it to register the
|
No-op when bottle.supervise is False or the backend doesn't
|
||||||
MCP entry inside the guest.
|
support the supervise sidecar yet. The Docker backend
|
||||||
|
overrides."""
|
||||||
Default returns "" so backends without supervise support
|
|
||||||
don't have to implement it. Docker and smolmachines override."""
|
|
||||||
del plan
|
|
||||||
return ""
|
|
||||||
|
|
||||||
@abstractmethod
|
@abstractmethod
|
||||||
def prepare_cleanup(self) -> CleanupT:
|
def prepare_cleanup(self) -> CleanupT:
|
||||||
@@ -469,20 +419,14 @@ def enumerate_active_agents() -> list[ActiveAgent]:
|
|||||||
"""All currently-running agents, across every available
|
"""All currently-running agents, across every available
|
||||||
backend. Used by CLI `list active` and the dashboard's agents
|
backend. Used by CLI `list active` and the dashboard's agents
|
||||||
pane so neither has to know which backends exist. Skips
|
pane so neither has to know which backends exist. Skips
|
||||||
backends whose `is_available()` reports False.
|
backends whose `is_available()` reports False. Ordered by
|
||||||
|
backend name, then by whatever each backend's
|
||||||
Sorted by `(started_at, slug)` so the list is stable across
|
`enumerate_active` returns."""
|
||||||
dashboard refresh ticks — agents don't shift position while
|
|
||||||
the operator navigates with arrow keys. ISO 8601 timestamps
|
|
||||||
sort lexicographically in chronological order; `slug` is the
|
|
||||||
deterministic tiebreaker. Agents with missing metadata
|
|
||||||
(`started_at == ""`) sort first."""
|
|
||||||
out: list[ActiveAgent] = []
|
out: list[ActiveAgent] = []
|
||||||
for name in known_backend_names():
|
for name in known_backend_names():
|
||||||
if not has_backend(name):
|
if not has_backend(name):
|
||||||
continue
|
continue
|
||||||
out.extend(_BACKENDS[name].enumerate_active())
|
out.extend(_BACKENDS[name].enumerate_active())
|
||||||
out.sort(key=lambda a: (a.started_at, a.slug))
|
|
||||||
return out
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -9,12 +9,6 @@ This module is a thin façade. The real work lives in four siblings:
|
|||||||
|
|
||||||
The base class's `prepare` template runs cross-backend host-side
|
The base class's `prepare` template runs cross-backend host-side
|
||||||
validation before calling `_resolve_plan` here.
|
validation before calling `_resolve_plan` here.
|
||||||
|
|
||||||
Per PRD 0050 the per-provider provisioning steps (prompt, skills,
|
|
||||||
the declarative provision-plan apply, supervise MCP registration)
|
|
||||||
live on the `AgentProvider` plugin under `bot_bottle/contrib/`. The
|
|
||||||
Docker backend only owns the steps that are about backend
|
|
||||||
infrastructure: CA install and git copy-in.
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
@@ -24,8 +18,7 @@ from contextlib import contextmanager
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Generator, Sequence
|
from typing import Generator, Sequence
|
||||||
|
|
||||||
from ...supervise import SUPERVISE_HOSTNAME, SUPERVISE_PORT
|
from .. import ActiveAgent, BottleBackend, BottleSpec
|
||||||
from .. import ActiveAgent, Bottle, BottleBackend, BottleSpec
|
|
||||||
from . import cleanup as _cleanup
|
from . import cleanup as _cleanup
|
||||||
from . import enumerate as _enumerate
|
from . import enumerate as _enumerate
|
||||||
from . import launch as _launch
|
from . import launch as _launch
|
||||||
@@ -35,6 +28,10 @@ from .bottle_cleanup_plan import DockerBottleCleanupPlan
|
|||||||
from .bottle_plan import DockerBottlePlan
|
from .bottle_plan import DockerBottlePlan
|
||||||
from .provision import ca as _ca
|
from .provision import ca as _ca
|
||||||
from .provision import git as _git
|
from .provision import git as _git
|
||||||
|
from .provision import prompt as _prompt
|
||||||
|
from .provision import provider_auth as _provider_auth
|
||||||
|
from .provision import skills as _skills
|
||||||
|
from .provision import supervise as _supervise_prov
|
||||||
|
|
||||||
|
|
||||||
class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanupPlan"]):
|
class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanupPlan"]):
|
||||||
@@ -60,19 +57,23 @@ class DockerBottleBackend(BottleBackend["DockerBottlePlan", "DockerBottleCleanup
|
|||||||
with _launch.launch(plan, provision=self.provision) as bottle:
|
with _launch.launch(plan, provision=self.provision) as bottle:
|
||||||
yield bottle
|
yield bottle
|
||||||
|
|
||||||
def provision_ca(self, plan: DockerBottlePlan, bottle: Bottle) -> None:
|
def provision_ca(self, plan: DockerBottlePlan, target: str) -> None:
|
||||||
_ca.provision_ca(plan, bottle)
|
_ca.provision_ca(plan, target)
|
||||||
|
|
||||||
def provision_git(self, plan: DockerBottlePlan, bottle: Bottle) -> None:
|
def provision_prompt(self, plan: DockerBottlePlan, target: str) -> str | None:
|
||||||
_git.provision_git(plan, bottle)
|
return _prompt.provision_prompt(plan, target)
|
||||||
|
|
||||||
def supervise_mcp_url(self, plan: DockerBottlePlan) -> str:
|
def provision_provider_auth(self, plan: DockerBottlePlan, target: str) -> None:
|
||||||
"""Docker bottles reach the supervise sidecar via the
|
_provider_auth.provision_provider_auth(plan, target)
|
||||||
compose-network alias `supervise:9100`. No per-bottle URL
|
|
||||||
plumbing needed; the alias resolves inside the bridge."""
|
def provision_skills(self, plan: DockerBottlePlan, target: str) -> None:
|
||||||
if plan.supervise_plan is None:
|
_skills.provision_skills(plan, target)
|
||||||
return ""
|
|
||||||
return f"http://{SUPERVISE_HOSTNAME}:{SUPERVISE_PORT}/"
|
def provision_git(self, plan: DockerBottlePlan, target: str) -> None:
|
||||||
|
_git.provision_git(plan, target)
|
||||||
|
|
||||||
|
def provision_supervise(self, plan: DockerBottlePlan, target: str) -> None:
|
||||||
|
_supervise_prov.provision_supervise(plan, target)
|
||||||
|
|
||||||
def prepare_cleanup(self) -> DockerBottleCleanupPlan:
|
def prepare_cleanup(self) -> DockerBottleCleanupPlan:
|
||||||
return _cleanup.prepare_cleanup()
|
return _cleanup.prepare_cleanup()
|
||||||
|
|||||||
@@ -2,25 +2,30 @@
|
|||||||
|
|
||||||
Carries the Docker-specific resolved fields produced by
|
Carries the Docker-specific resolved fields produced by
|
||||||
DockerBottleBackend.prepare. The launch step consumes it without
|
DockerBottleBackend.prepare. The launch step consumes it without
|
||||||
further resolution; preflight rendering is inherited from BottlePlan.
|
further resolution; show_plan-style rendering is the `print` method.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import sys
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from ...agent_provider import PromptMode
|
from ...agent_provider import AgentProvisionPlan, PromptMode
|
||||||
|
from ...egress import EgressPlan
|
||||||
|
from ...git_gate import GitGatePlan
|
||||||
|
from ...log import info
|
||||||
from ...pipelock import PipelockProxyPlan
|
from ...pipelock import PipelockProxyPlan
|
||||||
|
from ...supervise import SupervisePlan
|
||||||
from .. import BottlePlan
|
from .. import BottlePlan
|
||||||
|
from ..print_util import print_multi, visible_agent_env_names
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class DockerBottlePlan(BottlePlan):
|
class DockerBottlePlan(BottlePlan):
|
||||||
"""Docker-specific resolved fields produced by
|
"""Docker-specific resolved fields produced by
|
||||||
DockerBottleBackend.prepare. Inherits `spec`, `stage_dir`,
|
DockerBottleBackend.prepare. Inherits `spec` and `stage_dir` from
|
||||||
`git_gate_plan`, `egress_plan`, `supervise_plan`, and
|
BottlePlan."""
|
||||||
`agent_provision` from BottlePlan."""
|
|
||||||
|
|
||||||
slug: str
|
slug: str
|
||||||
container_name: str
|
container_name: str
|
||||||
@@ -41,7 +46,13 @@ class DockerBottlePlan(BottlePlan):
|
|||||||
forwarded_env: dict[str, str] = field(repr=False)
|
forwarded_env: dict[str, str] = field(repr=False)
|
||||||
prompt_file: Path
|
prompt_file: Path
|
||||||
proxy_plan: PipelockProxyPlan
|
proxy_plan: PipelockProxyPlan
|
||||||
|
git_gate_plan: GitGatePlan
|
||||||
|
egress_plan: EgressPlan
|
||||||
|
# None when bottle.supervise is False. PRD 0013 supervise sidecar
|
||||||
|
# is opt-in via the manifest's bottle.supervise field.
|
||||||
|
supervise_plan: SupervisePlan | None
|
||||||
use_runsc: bool
|
use_runsc: bool
|
||||||
|
agent_provision: AgentProvisionPlan
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def agent_command(self) -> str:
|
def agent_command(self) -> str:
|
||||||
@@ -54,3 +65,55 @@ class DockerBottlePlan(BottlePlan):
|
|||||||
@property
|
@property
|
||||||
def agent_provider_template(self) -> str:
|
def agent_provider_template(self) -> str:
|
||||||
return self.agent_provision.template
|
return self.agent_provision.template
|
||||||
|
|
||||||
|
def print(self, *, remote_control: bool) -> None:
|
||||||
|
"""Render the y/N preflight summary to stderr — compact form
|
||||||
|
intended to fit on screen without scrolling. The full
|
||||||
|
structured shape (image, container, runtime, etc.) lives on
|
||||||
|
this dataclass for tooling that wants to introspect it."""
|
||||||
|
del remote_control # not surfaced in the compact summary
|
||||||
|
spec = self.spec
|
||||||
|
manifest = spec.manifest
|
||||||
|
agent = manifest.agents[spec.agent_name]
|
||||||
|
bottle = manifest.bottle_for(spec.agent_name)
|
||||||
|
# The agent sees the union of literal env names (rendered into
|
||||||
|
# --env-file) and forwarded env names (`-e NAME` with the
|
||||||
|
# value arriving via subprocess env). The forwarded set holds
|
||||||
|
# the OAuth token (CLAUDE_CODE_OAUTH_TOKEN) and any host-env
|
||||||
|
# interpolations from the manifest; egress holds
|
||||||
|
# upstream tokens in its own environ, so no token forwarding
|
||||||
|
# from the agent to the proxy is needed.
|
||||||
|
env_names = visible_agent_env_names(
|
||||||
|
sorted(
|
||||||
|
set(bottle.env.keys())
|
||||||
|
| set(self.forwarded_env.keys())
|
||||||
|
| set(self.agent_provision.guest_env.keys())
|
||||||
|
),
|
||||||
|
hidden_env_names=self.agent_provision.hidden_env_names,
|
||||||
|
)
|
||||||
|
|
||||||
|
print(file=sys.stderr)
|
||||||
|
info(f"agent : {spec.agent_name}")
|
||||||
|
info(f"provider : {self.agent_provider_template}")
|
||||||
|
print_multi("env ", env_names)
|
||||||
|
print_multi("skills ", list(agent.skills))
|
||||||
|
info(f"bottle : {agent.bottle}")
|
||||||
|
|
||||||
|
identity = manifest.git_identity_summary(spec.agent_name)
|
||||||
|
if identity:
|
||||||
|
info(f" git identity : {identity}")
|
||||||
|
|
||||||
|
git_lines = [
|
||||||
|
f"{u.upstream_host}:{u.upstream_port}"
|
||||||
|
for u in self.git_gate_plan.upstreams
|
||||||
|
]
|
||||||
|
if git_lines:
|
||||||
|
print_multi(" git gate ", git_lines)
|
||||||
|
|
||||||
|
if self.egress_plan.routes:
|
||||||
|
egress_lines = []
|
||||||
|
for r in self.egress_plan.routes:
|
||||||
|
auth = f" [auth:{r.auth_scheme}]" if r.auth_scheme else ""
|
||||||
|
egress_lines.append(f"{r.host}{auth}")
|
||||||
|
print_multi(" egress ", egress_lines)
|
||||||
|
print(file=sys.stderr)
|
||||||
|
|||||||
@@ -105,10 +105,6 @@ class BottleMetadata:
|
|||||||
# written before chunk 3 (resume / inspect should fall back to
|
# written before chunk 3 (resume / inspect should fall back to
|
||||||
# deriving from identity in that case).
|
# deriving from identity in that case).
|
||||||
compose_project: str = ""
|
compose_project: str = ""
|
||||||
# PRD 0040: backend name ("docker" or "smolmachines"). Empty string
|
|
||||||
# for state dirs written before PRD 0040; callers default to "docker"
|
|
||||||
# for backward compatibility.
|
|
||||||
backend: str = ""
|
|
||||||
|
|
||||||
|
|
||||||
def metadata_path(identity: str) -> Path:
|
def metadata_path(identity: str) -> Path:
|
||||||
@@ -142,7 +138,6 @@ def read_metadata(identity: str) -> BottleMetadata | None:
|
|||||||
copy_cwd=bool(raw.get("copy_cwd", False)),
|
copy_cwd=bool(raw.get("copy_cwd", False)),
|
||||||
started_at=str(raw.get("started_at", "")),
|
started_at=str(raw.get("started_at", "")),
|
||||||
compose_project=str(raw.get("compose_project", "")),
|
compose_project=str(raw.get("compose_project", "")),
|
||||||
backend=str(raw.get("backend", "")),
|
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -49,7 +49,7 @@ from ...egress import (
|
|||||||
EGRESS_HOSTNAME,
|
EGRESS_HOSTNAME,
|
||||||
EGRESS_ROUTES_IN_CONTAINER,
|
EGRESS_ROUTES_IN_CONTAINER,
|
||||||
)
|
)
|
||||||
from ...git_gate import GIT_GATE_HOSTNAME
|
from ...git_gate import GIT_GATE_HOSTNAME, git_gate_aggregate_extra_hosts
|
||||||
from ...log import die, warn
|
from ...log import die, warn
|
||||||
from ...pipelock import PIPELOCK_HOSTNAME
|
from ...pipelock import PIPELOCK_HOSTNAME
|
||||||
from ...supervise import (
|
from ...supervise import (
|
||||||
@@ -198,6 +198,7 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
|||||||
env.append(token_env)
|
env.append(token_env)
|
||||||
|
|
||||||
# --- git-gate ----------------------------------------------------
|
# --- git-gate ----------------------------------------------------
|
||||||
|
extra_hosts: list[str] = []
|
||||||
gp = plan.git_gate_plan
|
gp = plan.git_gate_plan
|
||||||
if gp.upstreams:
|
if gp.upstreams:
|
||||||
volumes += [
|
volumes += [
|
||||||
@@ -216,6 +217,8 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
|||||||
u.known_hosts_file,
|
u.known_hosts_file,
|
||||||
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{u.name}-known_hosts",
|
f"{GIT_GATE_CREDS_DIR_IN_CONTAINER}/{u.name}-known_hosts",
|
||||||
))
|
))
|
||||||
|
extra_map = git_gate_aggregate_extra_hosts(gp.upstreams)
|
||||||
|
extra_hosts = [f"{host}:{ip}" for host, ip in sorted(extra_map.items())]
|
||||||
|
|
||||||
# --- supervise ---------------------------------------------------
|
# --- supervise ---------------------------------------------------
|
||||||
sp = plan.supervise_plan
|
sp = plan.supervise_plan
|
||||||
@@ -258,6 +261,8 @@ def _sidecar_bundle_service(plan: DockerBottlePlan) -> dict[str, Any]:
|
|||||||
"environment": env,
|
"environment": env,
|
||||||
"volumes": volumes,
|
"volumes": volumes,
|
||||||
}
|
}
|
||||||
|
if extra_hosts:
|
||||||
|
service["extra_hosts"] = extra_hosts
|
||||||
return service
|
return service
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -42,9 +42,12 @@ from contextlib import ExitStack, contextmanager
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Callable, Generator
|
from typing import Callable, Generator
|
||||||
|
|
||||||
from ...egress import egress_resolve_token_values
|
from ...codex_auth import codex_host_access_token
|
||||||
from ...git_gate import revoke_git_gate_provisioned_keys
|
from ...egress import (
|
||||||
from ...log import info, warn
|
CODEX_HOST_CREDENTIAL_TOKEN_REF,
|
||||||
|
egress_resolve_token_values,
|
||||||
|
)
|
||||||
|
from ...log import info
|
||||||
from . import network as network_mod
|
from . import network as network_mod
|
||||||
from . import util as docker_mod
|
from . import util as docker_mod
|
||||||
from .bottle import DockerBottle
|
from .bottle import DockerBottle
|
||||||
@@ -52,7 +55,6 @@ from .bottle_plan import DockerBottlePlan
|
|||||||
from .bottle_state import (
|
from .bottle_state import (
|
||||||
bottle_state_dir,
|
bottle_state_dir,
|
||||||
egress_state_dir,
|
egress_state_dir,
|
||||||
git_gate_state_dir,
|
|
||||||
pipelock_state_dir,
|
pipelock_state_dir,
|
||||||
)
|
)
|
||||||
from .compose import (
|
from .compose import (
|
||||||
@@ -86,20 +88,13 @@ def launch(
|
|||||||
Teardown on exit."""
|
Teardown on exit."""
|
||||||
stack = ExitStack()
|
stack = ExitStack()
|
||||||
|
|
||||||
_bottle_for_revoke = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
|
||||||
_git_gate_dir_for_revoke = git_gate_state_dir(plan.slug)
|
|
||||||
|
|
||||||
def teardown() -> None:
|
def teardown() -> None:
|
||||||
try:
|
try:
|
||||||
stack.close()
|
stack.close()
|
||||||
except BaseException as exc:
|
except BaseException:
|
||||||
warn(
|
# Teardown must not raise; swallow so the caller's
|
||||||
f"teardown failed for container {plan.container_name}"
|
# __exit__ path can still propagate the original error.
|
||||||
f" (compose-down): {exc!r}"
|
pass
|
||||||
)
|
|
||||||
revoke_git_gate_provisioned_keys(
|
|
||||||
_bottle_for_revoke, _git_gate_dir_for_revoke
|
|
||||||
)
|
|
||||||
|
|
||||||
try:
|
try:
|
||||||
# Step 1: agent image build. Sidecar images get built lazily by
|
# Step 1: agent image build. Sidecar images get built lazily by
|
||||||
@@ -110,7 +105,7 @@ def launch(
|
|||||||
)
|
)
|
||||||
if plan.derived_image:
|
if plan.derived_image:
|
||||||
docker_mod.build_image_with_cwd(
|
docker_mod.build_image_with_cwd(
|
||||||
plan.derived_image, plan.image, plan.workspace_plan
|
plan.derived_image, plan.image, plan.spec.user_cwd
|
||||||
)
|
)
|
||||||
|
|
||||||
# Networks: compose-managed. The names are derived
|
# Networks: compose-managed. The names are derived
|
||||||
@@ -185,10 +180,18 @@ def launch(
|
|||||||
# Step 7: compose up. Token values + the OAuth placeholder
|
# Step 7: compose up. Token values + the OAuth placeholder
|
||||||
# flow through subprocess env; the compose file holds only
|
# flow through subprocess env; the compose file holds only
|
||||||
# bare names for the secret-carrying entries.
|
# bare names for the secret-carrying entries.
|
||||||
effective_env = {**dict(os.environ), **plan.agent_provision.provisioned_env}
|
token_values: dict[str, str] = {}
|
||||||
token_values = egress_resolve_token_values(
|
if plan.egress_plan.routes:
|
||||||
plan.egress_plan.token_env_map, effective_env,
|
token_values = egress_resolve_token_values(
|
||||||
)
|
plan.egress_plan.token_env_map, dict(os.environ),
|
||||||
|
)
|
||||||
|
if plan.spec.manifest.bottle_for(
|
||||||
|
plan.spec.agent_name,
|
||||||
|
).agent_provider.forward_host_credentials:
|
||||||
|
access_token = codex_host_access_token(dict(os.environ))
|
||||||
|
for token_env, token_ref in plan.egress_plan.token_env_map.items():
|
||||||
|
if token_ref == CODEX_HOST_CREDENTIAL_TOKEN_REF:
|
||||||
|
token_values[token_env] = access_token
|
||||||
compose_env: dict[str, str] = {
|
compose_env: dict[str, str] = {
|
||||||
**os.environ,
|
**os.environ,
|
||||||
**plan.forwarded_env,
|
**plan.forwarded_env,
|
||||||
@@ -208,21 +211,19 @@ def launch(
|
|||||||
compose_dump_logs, project, compose_file, compose_log_path(state_dir),
|
compose_dump_logs, project, compose_file, compose_log_path(state_dir),
|
||||||
)
|
)
|
||||||
|
|
||||||
# Step 8: provision. Create the bottle first so provisioners
|
# Step 8: provision. Unchanged — uses `docker exec` against
|
||||||
# can use bottle.exec / bottle.cp_in; set the prompt path
|
# the agent container by its known name.
|
||||||
# returned by provision_prompt after the fact.
|
prompt_path = provision(plan, plan.container_name)
|
||||||
bottle = DockerBottle(
|
|
||||||
plan.container_name,
|
|
||||||
teardown,
|
|
||||||
None,
|
|
||||||
agent_command=plan.agent_command,
|
|
||||||
agent_prompt_mode=plan.agent_prompt_mode,
|
|
||||||
)
|
|
||||||
bottle._prompt_path = provision(plan, bottle)
|
|
||||||
|
|
||||||
# Step 9: yield. exec_agent continues to use `docker exec -it`
|
# Step 9: yield. exec_agent continues to use `docker exec -it`
|
||||||
# — the agent runs `sleep infinity` per the renderer's
|
# — the agent runs `sleep infinity` per the renderer's
|
||||||
# service spec.
|
# service spec.
|
||||||
yield bottle
|
yield DockerBottle(
|
||||||
|
plan.container_name,
|
||||||
|
teardown,
|
||||||
|
prompt_path,
|
||||||
|
agent_command=plan.agent_command,
|
||||||
|
agent_prompt_mode=plan.agent_prompt_mode,
|
||||||
|
)
|
||||||
finally:
|
finally:
|
||||||
teardown()
|
teardown()
|
||||||
|
|||||||
@@ -22,7 +22,6 @@ from ...git_gate import GitGate
|
|||||||
from ...log import die
|
from ...log import die
|
||||||
from ...pipelock import PipelockProxy
|
from ...pipelock import PipelockProxy
|
||||||
from ...supervise import Supervise
|
from ...supervise import Supervise
|
||||||
from ...workspace import workspace_plan as resolve_workspace_plan
|
|
||||||
from .. import BottleSpec
|
from .. import BottleSpec
|
||||||
from . import util as docker_mod
|
from . import util as docker_mod
|
||||||
from .bottle_plan import DockerBottlePlan
|
from .bottle_plan import DockerBottlePlan
|
||||||
@@ -63,8 +62,6 @@ def resolve_plan(
|
|||||||
bottle = manifest.bottle_for(spec.agent_name)
|
bottle = manifest.bottle_for(spec.agent_name)
|
||||||
provider = bottle.agent_provider
|
provider = bottle.agent_provider
|
||||||
provider_runtime = runtime_for(provider.template)
|
provider_runtime = runtime_for(provider.template)
|
||||||
guest_home = "/home/node"
|
|
||||||
workspace_plan = resolve_workspace_plan(spec, guest_home=guest_home)
|
|
||||||
|
|
||||||
# PRD 0016 follow-up: identity, not bare slug. A fresh `start`
|
# PRD 0016 follow-up: identity, not bare slug. A fresh `start`
|
||||||
# mints a random-suffixed identity (so parallel runs of the same
|
# mints a random-suffixed identity (so parallel runs of the same
|
||||||
@@ -82,7 +79,6 @@ def resolve_plan(
|
|||||||
copy_cwd=spec.copy_cwd,
|
copy_cwd=spec.copy_cwd,
|
||||||
started_at=datetime.now(timezone.utc).isoformat(),
|
started_at=datetime.now(timezone.utc).isoformat(),
|
||||||
compose_project=f"bot-bottle-{slug}",
|
compose_project=f"bot-bottle-{slug}",
|
||||||
backend="docker",
|
|
||||||
))
|
))
|
||||||
# Clear any leftover preserve marker from a prior capability-block
|
# Clear any leftover preserve marker from a prior capability-block
|
||||||
# so this fresh launch can be cleaned up at session-end unless
|
# so this fresh launch can be cleaned up at session-end unless
|
||||||
@@ -180,11 +176,10 @@ def resolve_plan(
|
|||||||
template=provider.template,
|
template=provider.template,
|
||||||
dockerfile=dockerfile_path,
|
dockerfile=dockerfile_path,
|
||||||
state_dir=agent_dir,
|
state_dir=agent_dir,
|
||||||
guest_home=guest_home,
|
guest_home=os.environ.get("BOT_BOTTLE_CONTAINER_HOME", "/home/node"),
|
||||||
forward_host_credentials=provider.forward_host_credentials,
|
forward_host_credentials=provider.forward_host_credentials,
|
||||||
auth_token=provider.auth_token,
|
auth_token=provider.auth_token,
|
||||||
host_env=dict(os.environ),
|
host_env=dict(os.environ),
|
||||||
trusted_project_path=workspace_plan.workdir,
|
|
||||||
)
|
)
|
||||||
guest_env = dict(agent_provision.guest_env)
|
guest_env = dict(agent_provision.guest_env)
|
||||||
for key, val in agent_provision.env_vars.items():
|
for key, val in agent_provision.env_vars.items():
|
||||||
@@ -233,7 +228,6 @@ def resolve_plan(
|
|||||||
return DockerBottlePlan(
|
return DockerBottlePlan(
|
||||||
spec=spec,
|
spec=spec,
|
||||||
stage_dir=stage_dir,
|
stage_dir=stage_dir,
|
||||||
guest_home=guest_home,
|
|
||||||
slug=slug,
|
slug=slug,
|
||||||
container_name=container_name,
|
container_name=container_name,
|
||||||
container_name_pinned=container_name_pinned,
|
container_name_pinned=container_name_pinned,
|
||||||
@@ -250,7 +244,6 @@ def resolve_plan(
|
|||||||
supervise_plan=supervise_plan,
|
supervise_plan=supervise_plan,
|
||||||
use_runsc=use_runsc,
|
use_runsc=use_runsc,
|
||||||
agent_provision=agent_provision,
|
agent_provision=agent_provision,
|
||||||
workspace_plan=workspace_plan,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -1,11 +1,8 @@
|
|||||||
"""Backend-infrastructure provisioners for the Docker backend.
|
"""Per-provisioner modules for the Docker backend.
|
||||||
|
|
||||||
Per PRD 0050 the per-provider provisioning steps (prompt, skills,
|
Each module exports one top-level function:
|
||||||
declarative provision-plan apply, supervise MCP registration) live on
|
provision_<thing>(plan: DockerBottlePlan, target: str) -> ...
|
||||||
the `AgentProvider` plugin under `bot_bottle/contrib/`. The modules
|
|
||||||
left in this subpackage handle only the steps that are
|
|
||||||
backend-specific:
|
|
||||||
|
|
||||||
- ca.py — install per-bottle CA bundle into the guest trust store
|
`DockerBottleBackend.provision_*` methods delegate to these. The
|
||||||
- git.py — copy host cwd `.git` into the guest when --cwd is used
|
abstract `BottleBackend.provision_*` surface is unchanged; this
|
||||||
"""
|
subpackage exists only to keep `backend.py` from being a god-file."""
|
||||||
|
|||||||
@@ -31,21 +31,33 @@ stage dir; nothing in the agent ever sees it."""
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from ... import Bottle
|
import subprocess
|
||||||
|
|
||||||
from ...util import AGENT_CA_PATH, log_ca_fingerprint, select_ca_cert
|
from ...util import AGENT_CA_PATH, log_ca_fingerprint, select_ca_cert
|
||||||
from ..bottle_plan import DockerBottlePlan
|
from ..bottle_plan import DockerBottlePlan
|
||||||
|
|
||||||
|
|
||||||
def provision_ca(plan: DockerBottlePlan, bottle: Bottle) -> None:
|
def provision_ca(plan: DockerBottlePlan, target: str) -> None:
|
||||||
"""Copy the agent-facing CA cert into the agent, rebuild the
|
"""Copy the agent-facing CA cert into the agent, rebuild the
|
||||||
trust bundle, emit a one-line fingerprint log. Called from
|
trust bundle, emit a one-line fingerprint log. Called from
|
||||||
`BottleBackend.provision` after the agent container is up."""
|
`BottleBackend.provision` after the agent container is up."""
|
||||||
|
container = target
|
||||||
cert_host_path, label = select_ca_cert(plan.egress_plan, plan.proxy_plan)
|
cert_host_path, label = select_ca_cert(plan.egress_plan, plan.proxy_plan)
|
||||||
|
|
||||||
bottle.cp_in(str(cert_host_path), AGENT_CA_PATH)
|
subprocess.run(
|
||||||
bottle.exec(
|
["docker", "cp", str(cert_host_path), f"{container}:{AGENT_CA_PATH}"],
|
||||||
f"chmod 644 {AGENT_CA_PATH} && update-ca-certificates",
|
stdout=subprocess.DEVNULL,
|
||||||
user="root",
|
check=True,
|
||||||
|
)
|
||||||
|
subprocess.run(
|
||||||
|
["docker", "exec", "-u", "0", container, "chmod", "644", AGENT_CA_PATH],
|
||||||
|
stdout=subprocess.DEVNULL,
|
||||||
|
check=True,
|
||||||
|
)
|
||||||
|
subprocess.run(
|
||||||
|
["docker", "exec", "-u", "0", container, "update-ca-certificates"],
|
||||||
|
stdout=subprocess.DEVNULL,
|
||||||
|
check=True,
|
||||||
)
|
)
|
||||||
|
|
||||||
log_ca_fingerprint(cert_host_path, label)
|
log_ca_fingerprint(cert_host_path, label)
|
||||||
|
|||||||
@@ -3,7 +3,7 @@
|
|||||||
Three concerns, all about git in the agent:
|
Three concerns, all about git in the agent:
|
||||||
|
|
||||||
1. If --cwd was passed AND the host cwd has a .git, copy that .git
|
1. If --cwd was passed AND the host cwd has a .git, copy that .git
|
||||||
into the planned guest workspace so the agent operates on the
|
into /home/node/workspace/.git so the agent operates on the
|
||||||
user's repo.
|
user's repo.
|
||||||
2. If the bottle declares `git` entries (PRD 0008), write a
|
2. If the bottle declares `git` entries (PRD 0008), write a
|
||||||
~/.gitconfig with insteadOf rules so every git operation
|
~/.gitconfig with insteadOf rules so every git operation
|
||||||
@@ -18,62 +18,73 @@ Three concerns, all about git in the agent:
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import shlex
|
import os
|
||||||
|
import subprocess
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
from ....git_gate import GIT_GATE_HOSTNAME, git_gate_render_gitconfig
|
from ....git_gate import GIT_GATE_HOSTNAME, git_gate_render_gitconfig
|
||||||
from ....log import info
|
from ....log import info
|
||||||
from ... import Bottle
|
from .. import util as docker_mod
|
||||||
from ..bottle_plan import DockerBottlePlan
|
from ..bottle_plan import DockerBottlePlan
|
||||||
|
|
||||||
|
|
||||||
def provision_git(plan: DockerBottlePlan, bottle: Bottle) -> None:
|
def provision_git(plan: DockerBottlePlan, target: str) -> None:
|
||||||
"""Set up git inside the bottle. Runs all three subcases; each
|
"""Set up git inside the bottle. Runs all three subcases; each
|
||||||
no-ops when its condition isn't met."""
|
no-ops when its condition isn't met."""
|
||||||
_provision_cwd_git(plan, bottle)
|
_provision_cwd_git(plan, target)
|
||||||
_provision_git_gate_config(plan, bottle)
|
_provision_git_gate_config(plan, target)
|
||||||
_provision_git_user(plan, bottle)
|
_provision_git_user(plan, target)
|
||||||
|
|
||||||
|
|
||||||
def _provision_cwd_git(plan: DockerBottlePlan, bottle: Bottle) -> None:
|
def _provision_cwd_git(plan: DockerBottlePlan, target: str) -> None:
|
||||||
"""If --cwd was set and the host cwd has a .git directory, copy
|
"""If --cwd was set and the host cwd has a .git directory, copy
|
||||||
it into /home/node/workspace/.git and fix ownership. No-op
|
it into /home/node/workspace/.git and fix ownership. No-op
|
||||||
otherwise."""
|
otherwise."""
|
||||||
workspace = plan.workspace_plan
|
if not (plan.spec.copy_cwd and Path(plan.spec.user_cwd, ".git").is_dir()):
|
||||||
if not (workspace.enabled and workspace.copy_git and workspace.has_host_git_dir):
|
|
||||||
return
|
return
|
||||||
guest_workspace_git = f"{workspace.guest_path}/.git"
|
container = target
|
||||||
host_git = str(workspace.host_path / ".git")
|
info(f"copying {plan.spec.user_cwd}/.git -> {container}:/home/node/workspace/.git")
|
||||||
info(f"copying {host_git} -> {bottle.name}:{guest_workspace_git}")
|
subprocess.run(
|
||||||
bottle.cp_in(host_git, guest_workspace_git)
|
["docker", "cp", f"{plan.spec.user_cwd}/.git", f"{container}:/home/node/workspace/.git"],
|
||||||
bottle.exec(
|
stdout=subprocess.DEVNULL,
|
||||||
f"chown -R {shlex.quote(workspace.owner)} {shlex.quote(guest_workspace_git)}",
|
check=True,
|
||||||
user="root",
|
)
|
||||||
|
subprocess.run(
|
||||||
|
[
|
||||||
|
"docker", "exec", "-u", "0", container,
|
||||||
|
"chown", "-R", "node:node", "/home/node/workspace/.git",
|
||||||
|
],
|
||||||
|
stdout=subprocess.DEVNULL,
|
||||||
|
check=True,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def _provision_git_gate_config(plan: DockerBottlePlan, bottle: Bottle) -> None:
|
def _provision_git_gate_config(plan: DockerBottlePlan, target: str) -> None:
|
||||||
"""Write ~/.gitconfig in the bottle with the git-gate
|
"""Write ~/.gitconfig in the bottle with the git-gate
|
||||||
insteadOf rules. No-op when the bottle has no `git` entries."""
|
insteadOf rules. No-op when the bottle has no `git` entries."""
|
||||||
manifest_bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
||||||
if not manifest_bottle.git:
|
if not bottle.git:
|
||||||
return
|
return
|
||||||
container_gitconfig = f"{plan.guest_home}/.gitconfig"
|
container = target
|
||||||
|
container_home = os.environ.get("BOT_BOTTLE_CONTAINER_HOME", "/home/node")
|
||||||
|
container_gitconfig = f"{container_home}/.gitconfig"
|
||||||
|
|
||||||
content = git_gate_render_gitconfig(manifest_bottle.git, GIT_GATE_HOSTNAME)
|
content = git_gate_render_gitconfig(bottle.git, GIT_GATE_HOSTNAME)
|
||||||
config_file = plan.stage_dir / "agent_gitconfig"
|
config_file = plan.stage_dir / "agent_gitconfig"
|
||||||
config_file.write_text(content)
|
config_file.write_text(content)
|
||||||
config_file.chmod(0o600)
|
config_file.chmod(0o600)
|
||||||
|
|
||||||
info(f"writing {container_gitconfig} with {len(manifest_bottle.git)} insteadOf rule(s)")
|
info(f"writing {container_gitconfig} with {len(bottle.git)} insteadOf rule(s)")
|
||||||
bottle.cp_in(str(config_file), container_gitconfig)
|
subprocess.run(
|
||||||
bottle.exec(
|
["docker", "cp", str(config_file), f"{container}:{container_gitconfig}"],
|
||||||
f"chown node:node {shlex.quote(container_gitconfig)} && "
|
stdout=subprocess.DEVNULL,
|
||||||
f"chmod 644 {shlex.quote(container_gitconfig)}",
|
check=True,
|
||||||
user="root",
|
|
||||||
)
|
)
|
||||||
|
docker_mod.docker_exec_root(container, ["chown", "node:node", container_gitconfig])
|
||||||
|
docker_mod.docker_exec_root(container, ["chmod", "644", container_gitconfig])
|
||||||
|
|
||||||
|
|
||||||
def _provision_git_user(plan: DockerBottlePlan, bottle: Bottle) -> None:
|
def _provision_git_user(plan: DockerBottlePlan, target: str) -> None:
|
||||||
"""Apply `git config --global user.{name,email}` inside the
|
"""Apply `git config --global user.{name,email}` inside the
|
||||||
bottle so the agent's commits are attributed to the operator-
|
bottle so the agent's commits are attributed to the operator-
|
||||||
chosen identity instead of the agent image's default
|
chosen identity instead of the agent image's default
|
||||||
@@ -88,19 +99,23 @@ def _provision_git_user(plan: DockerBottlePlan, bottle: Bottle) -> None:
|
|||||||
Each field set independently — name-only or email-only
|
Each field set independently — name-only or email-only
|
||||||
configs only run the `git config` line for the field
|
configs only run the `git config` line for the field
|
||||||
present."""
|
present."""
|
||||||
manifest_bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
||||||
gu = manifest_bottle.git_user
|
gu = bottle.git_user
|
||||||
if gu.is_empty():
|
if gu.is_empty():
|
||||||
return
|
return
|
||||||
if gu.name:
|
if gu.name:
|
||||||
info(f"git config --global user.name = {gu.name!r}")
|
info(f"git config --global user.name = {gu.name!r}")
|
||||||
bottle.exec(
|
subprocess.run(
|
||||||
f"git config --global user.name {shlex.quote(gu.name)}",
|
["docker", "exec", "-u", "node", target,
|
||||||
user="node",
|
"git", "config", "--global", "user.name", gu.name],
|
||||||
|
stdout=subprocess.DEVNULL,
|
||||||
|
check=True,
|
||||||
)
|
)
|
||||||
if gu.email:
|
if gu.email:
|
||||||
info(f"git config --global user.email = {gu.email!r}")
|
info(f"git config --global user.email = {gu.email!r}")
|
||||||
bottle.exec(
|
subprocess.run(
|
||||||
f"git config --global user.email {shlex.quote(gu.email)}",
|
["docker", "exec", "-u", "node", target,
|
||||||
user="node",
|
"git", "config", "--global", "user.email", gu.email],
|
||||||
|
stdout=subprocess.DEVNULL,
|
||||||
|
check=True,
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -0,0 +1,43 @@
|
|||||||
|
"""Copy the agent prompt into a running Docker bottle.
|
||||||
|
|
||||||
|
The prompt file is always copied (so the in-container path always
|
||||||
|
exists) but `--append-system-prompt-file` only fires when the agent
|
||||||
|
actually has a prompt — the return value signals which case."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
|
||||||
|
from ..bottle_plan import DockerBottlePlan
|
||||||
|
|
||||||
|
|
||||||
|
def provision_prompt(plan: DockerBottlePlan, target: str) -> str | None:
|
||||||
|
"""Copy the prompt file into the container, fix ownership/mode.
|
||||||
|
Returns the in-container path if the agent has a non-empty
|
||||||
|
prompt (drives --append-system-prompt-file), else None. The
|
||||||
|
file is copied either way so the path always exists."""
|
||||||
|
container = target
|
||||||
|
container_home = os.environ.get("BOT_BOTTLE_CONTAINER_HOME", "/home/node")
|
||||||
|
in_container_prompt_path = f"{container_home}/.bot-bottle-prompt.txt"
|
||||||
|
|
||||||
|
subprocess.run(
|
||||||
|
["docker", "cp", str(plan.prompt_file), f"{container}:{in_container_prompt_path}"],
|
||||||
|
stdout=subprocess.DEVNULL,
|
||||||
|
check=True,
|
||||||
|
)
|
||||||
|
# `docker cp` preserves host UID; re-own/mode as root so node
|
||||||
|
# can read its own mode-600 prompt regardless of host UID.
|
||||||
|
subprocess.run(
|
||||||
|
["docker", "exec", "-u", "0", container, "chown", "node:node", in_container_prompt_path],
|
||||||
|
stdout=subprocess.DEVNULL,
|
||||||
|
check=True,
|
||||||
|
)
|
||||||
|
subprocess.run(
|
||||||
|
["docker", "exec", "-u", "0", container, "chmod", "600", in_container_prompt_path],
|
||||||
|
stdout=subprocess.DEVNULL,
|
||||||
|
check=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||||
|
return in_container_prompt_path if agent.prompt else None
|
||||||
@@ -0,0 +1,36 @@
|
|||||||
|
"""Provision non-secret provider auth markers into a Docker bottle."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import subprocess
|
||||||
|
|
||||||
|
from ..bottle_plan import DockerBottlePlan
|
||||||
|
|
||||||
|
|
||||||
|
def provision_provider_auth(plan: DockerBottlePlan, target: str) -> None:
|
||||||
|
"""Apply provider-owned guest setup through Docker primitives."""
|
||||||
|
provision = plan.agent_provision
|
||||||
|
for d in provision.dirs:
|
||||||
|
_exec(target, ["mkdir", "-p", d.guest_path])
|
||||||
|
_exec(target, ["chown", d.owner, d.guest_path])
|
||||||
|
_exec(target, ["chmod", d.mode, d.guest_path])
|
||||||
|
for command in provision.pre_copy:
|
||||||
|
_exec(target, list(command.argv))
|
||||||
|
for f in provision.files:
|
||||||
|
subprocess.run(
|
||||||
|
["docker", "cp", str(f.host_path), f"{target}:{f.guest_path}"],
|
||||||
|
stdout=subprocess.DEVNULL,
|
||||||
|
check=True,
|
||||||
|
)
|
||||||
|
_exec(target, ["chown", f.owner, f.guest_path])
|
||||||
|
_exec(target, ["chmod", f.mode, f.guest_path])
|
||||||
|
for command in provision.verify:
|
||||||
|
_exec(target, list(command.argv))
|
||||||
|
|
||||||
|
|
||||||
|
def _exec(target: str, argv: list[str]) -> None:
|
||||||
|
subprocess.run(
|
||||||
|
["docker", "exec", "-u", "0", target, *argv],
|
||||||
|
stdout=subprocess.DEVNULL,
|
||||||
|
check=True,
|
||||||
|
)
|
||||||
@@ -0,0 +1,62 @@
|
|||||||
|
"""Copy host-side skill directories into a running Docker bottle.
|
||||||
|
|
||||||
|
Skills are validated on the host before launch by the base class's
|
||||||
|
`BottleBackend._validate_skills` (called from `prepare`); this module
|
||||||
|
assumes that validation has already run. A skill disappearing between
|
||||||
|
validation and copy still dies loudly rather than silently producing
|
||||||
|
a partial container."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
import subprocess
|
||||||
|
|
||||||
|
from ....log import die, info
|
||||||
|
from ...util import host_skill_dir
|
||||||
|
from ..bottle_plan import DockerBottlePlan
|
||||||
|
|
||||||
|
|
||||||
|
def provision_skills(plan: DockerBottlePlan, target: str) -> None:
|
||||||
|
"""Copy each of the agent's named skills from the host's
|
||||||
|
~/.claude/skills/<name>/ into the container's equivalent path.
|
||||||
|
For each skill: ensure parent dir, wipe any prior copy, then
|
||||||
|
`docker cp <host>/. <container>:<dst>/` so the contents are
|
||||||
|
copied into a freshly-created destination dir. No-op when the
|
||||||
|
agent has no skills."""
|
||||||
|
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||||
|
if not agent.skills:
|
||||||
|
return
|
||||||
|
|
||||||
|
container = target
|
||||||
|
container_home = os.environ.get("BOT_BOTTLE_CONTAINER_HOME", "/home/node")
|
||||||
|
skills_dir = os.environ.get(
|
||||||
|
"BOT_BOTTLE_CONTAINER_SKILLS_DIR", f"{container_home}/.claude/skills"
|
||||||
|
)
|
||||||
|
|
||||||
|
subprocess.run(
|
||||||
|
["docker", "exec", container, "mkdir", "-p", skills_dir],
|
||||||
|
stdout=subprocess.DEVNULL,
|
||||||
|
check=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
for n in agent.skills:
|
||||||
|
src = host_skill_dir(n)
|
||||||
|
if not os.path.isdir(src):
|
||||||
|
die(f"skill '{n}' disappeared from host between validation and copy at {src}.")
|
||||||
|
dst = f"{skills_dir}/{n}"
|
||||||
|
info(f"copying skill {n} into {container}:{dst}")
|
||||||
|
subprocess.run(
|
||||||
|
["docker", "exec", container, "rm", "-rf", dst],
|
||||||
|
stdout=subprocess.DEVNULL,
|
||||||
|
check=True,
|
||||||
|
)
|
||||||
|
subprocess.run(
|
||||||
|
["docker", "exec", container, "mkdir", "-p", dst],
|
||||||
|
stdout=subprocess.DEVNULL,
|
||||||
|
check=True,
|
||||||
|
)
|
||||||
|
subprocess.run(
|
||||||
|
["docker", "cp", f"{src}/.", f"{container}:{dst}/"],
|
||||||
|
stdout=subprocess.DEVNULL,
|
||||||
|
check=True,
|
||||||
|
)
|
||||||
@@ -0,0 +1,65 @@
|
|||||||
|
"""Supervise sidecar provisioning inside a running Docker bottle
|
||||||
|
(PRD 0013).
|
||||||
|
|
||||||
|
Registers the per-bottle supervise sidecar as an HTTP MCP server in
|
||||||
|
the agent's claude-code config so the agent discovers the three
|
||||||
|
stuck-recovery MCP tools (cred-proxy-block, pipelock-block,
|
||||||
|
capability-block) at startup.
|
||||||
|
|
||||||
|
Uses `claude mcp add` rather than writing JSON directly. claude-code
|
||||||
|
owns the on-disk config format (`~/.claude.json` `mcpServers` shape,
|
||||||
|
field names, scope semantics) and changes it between versions; the
|
||||||
|
official command handles whatever the installed version expects.
|
||||||
|
|
||||||
|
No-op when bottle.supervise is False — bottles that haven't opted
|
||||||
|
into the supervise sidecar shouldn't get an MCP entry pointing at a
|
||||||
|
sidecar that isn't running.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import subprocess
|
||||||
|
|
||||||
|
from ....log import info, warn
|
||||||
|
from ....supervise import SUPERVISE_HOSTNAME, SUPERVISE_PORT
|
||||||
|
from ..bottle_plan import DockerBottlePlan
|
||||||
|
|
||||||
|
|
||||||
|
_SUPERVISE_MCP_NAME = "supervise"
|
||||||
|
|
||||||
|
|
||||||
|
def supervise_mcp_url() -> str:
|
||||||
|
return f"http://{SUPERVISE_HOSTNAME}:{SUPERVISE_PORT}/"
|
||||||
|
|
||||||
|
|
||||||
|
def provision_supervise(plan: DockerBottlePlan, target: str) -> None:
|
||||||
|
"""Run `claude mcp add` inside the agent container to register
|
||||||
|
the supervise sidecar in claude-code's user config. No-op when
|
||||||
|
bottle.supervise is False.
|
||||||
|
|
||||||
|
Failure is logged but not fatal: the bottle still works (you
|
||||||
|
just can't call supervise tools from the agent until the entry
|
||||||
|
is added manually). The operator sees the warning at launch."""
|
||||||
|
if plan.supervise_plan is None:
|
||||||
|
return
|
||||||
|
url = supervise_mcp_url()
|
||||||
|
argv = [
|
||||||
|
"docker", "exec", "-u", "node", target,
|
||||||
|
"claude", "mcp", "add",
|
||||||
|
"--scope", "user",
|
||||||
|
"--transport", "http",
|
||||||
|
_SUPERVISE_MCP_NAME,
|
||||||
|
url,
|
||||||
|
]
|
||||||
|
info(f"registering supervise MCP server in agent claude config → {url}")
|
||||||
|
r = subprocess.run(argv, capture_output=True, text=True, check=False)
|
||||||
|
if r.returncode != 0:
|
||||||
|
warn(
|
||||||
|
f"`claude mcp add supervise` failed (exit {r.returncode}): "
|
||||||
|
f"{(r.stderr or r.stdout or '').strip()}. Inside the bottle, "
|
||||||
|
f"register manually with: "
|
||||||
|
f"claude mcp add --scope user --transport http supervise {url}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = ["provision_supervise", "supervise_mcp_url"]
|
||||||
@@ -7,11 +7,9 @@ from __future__ import annotations
|
|||||||
import re
|
import re
|
||||||
import shutil
|
import shutil
|
||||||
import subprocess
|
import subprocess
|
||||||
import tempfile
|
|
||||||
from typing import Iterable, Iterator
|
from typing import Iterable, Iterator
|
||||||
|
|
||||||
from ...log import die, info
|
from ...log import die, info
|
||||||
from ...workspace import WorkspacePlan
|
|
||||||
|
|
||||||
|
|
||||||
# Cap on the suffix the container-name conflict logic will try before
|
# Cap on the suffix the container-name conflict logic will try before
|
||||||
@@ -118,39 +116,35 @@ def build_image(ref: str, context: str, *, dockerfile: str = "") -> None:
|
|||||||
subprocess.run(args, check=True)
|
subprocess.run(args, check=True)
|
||||||
|
|
||||||
|
|
||||||
def build_image_with_cwd(
|
_TRUST_DIALOG_NODE_SCRIPT = (
|
||||||
derived: str,
|
'const fs=require("fs"),p=process.env.HOME+"/.claude.json",'
|
||||||
base: str,
|
'c=JSON.parse(fs.readFileSync(p,"utf8"));'
|
||||||
workspace: WorkspacePlan,
|
'c.projects=c.projects||{};'
|
||||||
) -> None:
|
'c.projects[process.env.HOME+"/workspace"]={hasTrustDialogAccepted:true};'
|
||||||
"""Build a thin derived image that copies the workspace into
|
'fs.writeFileSync(p,JSON.stringify(c,null,2));'
|
||||||
the plan's guest path and sets the plan's workdir."""
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def build_image_with_cwd(derived: str, base: str, cwd: str) -> None:
|
||||||
|
"""Build a thin derived image that copies <cwd> into
|
||||||
|
/home/node/workspace and adds a trust-dialog entry for it."""
|
||||||
import os
|
import os
|
||||||
|
|
||||||
cwd = str(workspace.host_path)
|
|
||||||
if not os.path.isdir(cwd):
|
if not os.path.isdir(cwd):
|
||||||
die(f"cwd not found at {cwd}")
|
die(f"cwd not found at {cwd}")
|
||||||
info(f"building image {derived} from {base} with {cwd} -> {workspace.guest_path}")
|
info(f"building image {derived} from {base} with {cwd} -> /home/node/workspace")
|
||||||
with tempfile.TemporaryDirectory(prefix="bot-bottle-cwd.") as tmp:
|
dockerfile = (
|
||||||
context_dir = os.path.join(tmp, "context")
|
f"FROM {base}\n"
|
||||||
staged_workspace = os.path.join(context_dir, "workspace")
|
f"COPY --chown=node:node . /home/node/workspace\n"
|
||||||
shutil.copytree(
|
f"RUN node -e '{_TRUST_DIALOG_NODE_SCRIPT}'\n"
|
||||||
cwd,
|
f"WORKDIR /home/node/workspace\n"
|
||||||
staged_workspace,
|
)
|
||||||
symlinks=True,
|
subprocess.run(
|
||||||
ignore=shutil.ignore_patterns(".git"),
|
["docker", "build", "-t", derived, "-f", "-", cwd],
|
||||||
)
|
input=dockerfile,
|
||||||
dockerfile = (
|
text=True,
|
||||||
f"FROM {base}\n"
|
check=True,
|
||||||
f"COPY --chown=node:node workspace/. {workspace.guest_path}\n"
|
)
|
||||||
f"WORKDIR {workspace.workdir}\n"
|
|
||||||
)
|
|
||||||
subprocess.run(
|
|
||||||
["docker", "build", "-t", derived, "-f", "-", context_dir],
|
|
||||||
input=dockerfile,
|
|
||||||
text=True,
|
|
||||||
check=True,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def image_id(ref: str) -> str:
|
def image_id(ref: str) -> str:
|
||||||
|
|||||||
@@ -1,11 +1,5 @@
|
|||||||
"""SmolmachinesBottleBackend — the smolmachines implementation of
|
"""SmolmachinesBottleBackend — the smolmachines implementation of
|
||||||
BottleBackend (PRD 0023).
|
BottleBackend (PRD 0023)."""
|
||||||
|
|
||||||
Per PRD 0050 the per-provider provisioning steps (prompt, skills,
|
|
||||||
the declarative provision-plan apply, supervise MCP registration)
|
|
||||||
live on the `AgentProvider` plugin under `bot_bottle/contrib/`. The
|
|
||||||
smolmachines backend only owns the steps that are about backend
|
|
||||||
infrastructure: CA install (no-op for now), workspace, git copy-in."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
@@ -13,7 +7,7 @@ from contextlib import contextmanager
|
|||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Generator, Sequence
|
from typing import Generator, Sequence
|
||||||
|
|
||||||
from .. import ActiveAgent, Bottle, BottleBackend, BottleSpec
|
from .. import ActiveAgent, BottleBackend, BottleSpec
|
||||||
from . import cleanup as _cleanup
|
from . import cleanup as _cleanup
|
||||||
from . import enumerate as _enumerate
|
from . import enumerate as _enumerate
|
||||||
from . import launch as _launch
|
from . import launch as _launch
|
||||||
@@ -24,7 +18,10 @@ from .bottle_cleanup_plan import SmolmachinesBottleCleanupPlan
|
|||||||
from .bottle_plan import SmolmachinesBottlePlan
|
from .bottle_plan import SmolmachinesBottlePlan
|
||||||
from .provision import ca as _ca
|
from .provision import ca as _ca
|
||||||
from .provision import git as _git
|
from .provision import git as _git
|
||||||
from .provision import workspace as _workspace
|
from .provision import prompt as _prompt
|
||||||
|
from .provision import provider_auth as _provider_auth
|
||||||
|
from .provision import skills as _skills
|
||||||
|
from .provision import supervise as _supervise
|
||||||
|
|
||||||
|
|
||||||
class SmolmachinesBottleBackend(
|
class SmolmachinesBottleBackend(
|
||||||
@@ -56,26 +53,34 @@ class SmolmachinesBottleBackend(
|
|||||||
yield bottle
|
yield bottle
|
||||||
|
|
||||||
def provision_ca(
|
def provision_ca(
|
||||||
self, plan: SmolmachinesBottlePlan, bottle: Bottle
|
self, plan: SmolmachinesBottlePlan, target: str
|
||||||
) -> None:
|
) -> None:
|
||||||
_ca.provision_ca(plan, bottle)
|
_ca.provision_ca(plan, target)
|
||||||
|
|
||||||
def provision_workspace(
|
def provision_prompt(
|
||||||
self, plan: SmolmachinesBottlePlan, bottle: Bottle
|
self, plan: SmolmachinesBottlePlan, target: str
|
||||||
|
) -> str | None:
|
||||||
|
return _prompt.provision_prompt(plan, target)
|
||||||
|
|
||||||
|
def provision_provider_auth(
|
||||||
|
self, plan: SmolmachinesBottlePlan, target: str
|
||||||
) -> None:
|
) -> None:
|
||||||
_workspace.provision_workspace(plan, bottle)
|
_provider_auth.provision_provider_auth(plan, target)
|
||||||
|
|
||||||
|
def provision_skills(
|
||||||
|
self, plan: SmolmachinesBottlePlan, target: str
|
||||||
|
) -> None:
|
||||||
|
_skills.provision_skills(plan, target)
|
||||||
|
|
||||||
def provision_git(
|
def provision_git(
|
||||||
self, plan: SmolmachinesBottlePlan, bottle: Bottle
|
self, plan: SmolmachinesBottlePlan, target: str
|
||||||
) -> None:
|
) -> None:
|
||||||
_git.provision_git(plan, bottle)
|
_git.provision_git(plan, target)
|
||||||
|
|
||||||
def supervise_mcp_url(self, plan: SmolmachinesBottlePlan) -> str:
|
def provision_supervise(
|
||||||
"""The smolmachines guest reaches the supervise sidecar via a
|
self, plan: SmolmachinesBottlePlan, target: str
|
||||||
host-published random port the launch step pinned earlier
|
) -> None:
|
||||||
(`http://<loopback_ip>:<random_port>/`). `agent_supervise_url`
|
_supervise.provision_supervise(plan, target)
|
||||||
on the plan is "" when the bottle has no sidecar."""
|
|
||||||
return plan.agent_supervise_url
|
|
||||||
|
|
||||||
def prepare_cleanup(self) -> SmolmachinesBottleCleanupPlan:
|
def prepare_cleanup(self) -> SmolmachinesBottleCleanupPlan:
|
||||||
return _cleanup.prepare_cleanup()
|
return _cleanup.prepare_cleanup()
|
||||||
|
|||||||
@@ -8,20 +8,25 @@ in chunk 4."""
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import sys
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from ...agent_provider import PromptMode
|
from ...agent_provider import AgentProvisionPlan, PromptMode
|
||||||
|
from ...egress import EgressPlan
|
||||||
|
from ...git_gate import GitGatePlan
|
||||||
|
from ...log import info
|
||||||
from ...pipelock import PipelockProxyPlan
|
from ...pipelock import PipelockProxyPlan
|
||||||
|
from ...supervise import SupervisePlan
|
||||||
from .. import BottlePlan
|
from .. import BottlePlan
|
||||||
|
from ..print_util import print_multi, visible_agent_env_names
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class SmolmachinesBottlePlan(BottlePlan):
|
class SmolmachinesBottlePlan(BottlePlan):
|
||||||
"""Resolved fields the launch step needs to bring up the bottle.
|
"""Resolved fields the launch step needs to bring up the bottle.
|
||||||
|
|
||||||
Inherits `spec`, `stage_dir`, `git_gate_plan`, `egress_plan`,
|
Inherits `spec` and `stage_dir` from BottlePlan."""
|
||||||
`supervise_plan`, and `agent_provision` from BottlePlan."""
|
|
||||||
|
|
||||||
slug: str
|
slug: str
|
||||||
# Per-bottle docker subnet for the sidecar bundle container.
|
# Per-bottle docker subnet for the sidecar bundle container.
|
||||||
@@ -72,6 +77,12 @@ class SmolmachinesBottlePlan(BottlePlan):
|
|||||||
# per-bottle bridge with a pinned IP. The unused fields stay
|
# per-bottle bridge with a pinned IP. The unused fields stay
|
||||||
# at their dataclass defaults.
|
# at their dataclass defaults.
|
||||||
proxy_plan: PipelockProxyPlan
|
proxy_plan: PipelockProxyPlan
|
||||||
|
git_gate_plan: GitGatePlan
|
||||||
|
egress_plan: EgressPlan
|
||||||
|
# None when bottle.supervise is False, matching the docker
|
||||||
|
# backend's convention.
|
||||||
|
supervise_plan: SupervisePlan | None
|
||||||
|
agent_provision: AgentProvisionPlan
|
||||||
# Agent-side endpoints. On Docker Desktop the docker bridge
|
# Agent-side endpoints. On Docker Desktop the docker bridge
|
||||||
# IPs aren't reachable from the smolvm guest (TSI uses macOS
|
# IPs aren't reachable from the smolvm guest (TSI uses macOS
|
||||||
# networking; docker container IPs live in the daemon's VM),
|
# networking; docker container IPs live in the daemon's VM),
|
||||||
@@ -99,3 +110,42 @@ class SmolmachinesBottlePlan(BottlePlan):
|
|||||||
@property
|
@property
|
||||||
def agent_dockerfile_path(self) -> str:
|
def agent_dockerfile_path(self) -> str:
|
||||||
return self.agent_provision.dockerfile
|
return self.agent_provision.dockerfile
|
||||||
|
|
||||||
|
def print(self, *, remote_control: bool) -> None:
|
||||||
|
"""Compact y/N preflight. Same shape as the Docker
|
||||||
|
backend's so operators see one format across backends."""
|
||||||
|
del remote_control # not surfaced in the compact summary
|
||||||
|
spec = self.spec
|
||||||
|
manifest = spec.manifest
|
||||||
|
agent = manifest.agents[spec.agent_name]
|
||||||
|
bottle = manifest.bottle_for(spec.agent_name)
|
||||||
|
|
||||||
|
env_names = visible_agent_env_names(
|
||||||
|
sorted(
|
||||||
|
set(bottle.env.keys())
|
||||||
|
| set(self.agent_provision.guest_env.keys())
|
||||||
|
),
|
||||||
|
hidden_env_names=self.agent_provision.hidden_env_names,
|
||||||
|
)
|
||||||
|
upstreams = [
|
||||||
|
f"{g.Name} → {g.Upstream}" for g in bottle.git
|
||||||
|
]
|
||||||
|
# Use the resolved egress_plan (lowercase `host` on the
|
||||||
|
# plan-level EgressRoute) rather than `bottle.egress.routes`,
|
||||||
|
# which is the manifest's capitalized-attr form.
|
||||||
|
routes = [r.host for r in self.egress_plan.routes]
|
||||||
|
|
||||||
|
print(file=sys.stderr)
|
||||||
|
info(f"agent : {spec.agent_name}")
|
||||||
|
info(f"provider : {self.agent_provider_template}")
|
||||||
|
print_multi("env ", env_names)
|
||||||
|
print_multi("skills ", list(agent.skills))
|
||||||
|
info(f"bottle : {agent.bottle}")
|
||||||
|
identity = manifest.git_identity_summary(spec.agent_name)
|
||||||
|
if identity:
|
||||||
|
info(f" git identity : {identity}")
|
||||||
|
if upstreams:
|
||||||
|
print_multi(" git gate ", upstreams)
|
||||||
|
if routes:
|
||||||
|
print_multi(" egress ", routes)
|
||||||
|
print(file=sys.stderr)
|
||||||
|
|||||||
@@ -21,11 +21,14 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import dataclasses
|
import dataclasses
|
||||||
import os
|
import os
|
||||||
|
import time
|
||||||
from contextlib import ExitStack, contextmanager
|
from contextlib import ExitStack, contextmanager
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Callable, Generator
|
from typing import Callable, Generator
|
||||||
|
|
||||||
|
from ...codex_auth import codex_host_access_token
|
||||||
from ...egress import (
|
from ...egress import (
|
||||||
|
CODEX_HOST_CREDENTIAL_TOKEN_REF,
|
||||||
EGRESS_ROUTES_IN_CONTAINER,
|
EGRESS_ROUTES_IN_CONTAINER,
|
||||||
egress_resolve_token_values,
|
egress_resolve_token_values,
|
||||||
)
|
)
|
||||||
@@ -53,9 +56,6 @@ from ..docker.pipelock import (
|
|||||||
PIPELOCK_PORT as _PIPELOCK_PORT_STR,
|
PIPELOCK_PORT as _PIPELOCK_PORT_STR,
|
||||||
pipelock_tls_init,
|
pipelock_tls_init,
|
||||||
)
|
)
|
||||||
from ...git_gate import revoke_git_gate_provisioned_keys
|
|
||||||
from ...log import warn
|
|
||||||
from ..docker.bottle_state import git_gate_state_dir
|
|
||||||
from . import loopback_alias as _loopback
|
from . import loopback_alias as _loopback
|
||||||
from . import sidecar_bundle as _bundle
|
from . import sidecar_bundle as _bundle
|
||||||
from . import smolvm as _smolvm
|
from . import smolvm as _smolvm
|
||||||
@@ -96,230 +96,211 @@ def launch(
|
|||||||
via the ExitStack."""
|
via the ExitStack."""
|
||||||
stack = ExitStack()
|
stack = ExitStack()
|
||||||
try:
|
try:
|
||||||
loopback_ip, network = _allocate_resources(plan, stack)
|
# 1. Reserve a loopback alias for this bottle. macOS only
|
||||||
plan = _mint_certs(plan)
|
# routes 127.0.0.1 by default; the per-bottle alias is
|
||||||
plan = _start_bundle(plan, network, loopback_ip, stack)
|
# what bundles the docker port-publishes and TSI allowlist
|
||||||
plan = _discover_urls(plan, loopback_ip)
|
# against, so this bottle can't reach other bottles' (or
|
||||||
|
# other host services') ports on the loopback. Lazy
|
||||||
|
# sudo-driven on first use per boot. No-op on Linux.
|
||||||
|
_loopback.ensure_pool()
|
||||||
|
loopback_ip = _loopback.allocate(plan.slug)
|
||||||
|
|
||||||
# Build the agent image and pack it into a `.smolmachine`
|
# 2. Per-bottle docker bridge.
|
||||||
# artifact (or hit the per-Dockerfile-digest cache). Runs
|
network = _bundle.bundle_network_name(plan.slug)
|
||||||
# here, not in prepare, so the docker-build output doesn't
|
_bundle.create_bundle_network(network, plan.bundle_subnet, plan.bundle_gateway)
|
||||||
# garble the dashboard's preflight modal.
|
stack.callback(_bundle.remove_bundle_network, network)
|
||||||
|
|
||||||
|
# 2. Mint per-bottle CAs and update the inner Plans with
|
||||||
|
# their launch-time paths. pipelock always runs in the
|
||||||
|
# bundle; egress's CA is only minted when the bottle
|
||||||
|
# declares routes (otherwise egress runs idle without
|
||||||
|
# MITM and the CA files would be unused).
|
||||||
|
ca_cert_host, ca_key_host = pipelock_tls_init(plan.proxy_plan.yaml_path.parent)
|
||||||
|
proxy_plan = dataclasses.replace(
|
||||||
|
plan.proxy_plan,
|
||||||
|
ca_cert_host_path=ca_cert_host,
|
||||||
|
ca_key_host_path=ca_key_host,
|
||||||
|
)
|
||||||
|
egress_plan = plan.egress_plan
|
||||||
|
if egress_plan.routes:
|
||||||
|
egress_ca_host, egress_ca_cert_only = egress_tls_init(
|
||||||
|
plan.egress_plan.routes_path.parent,
|
||||||
|
)
|
||||||
|
egress_plan = dataclasses.replace(
|
||||||
|
egress_plan,
|
||||||
|
mitmproxy_ca_host_path=egress_ca_host,
|
||||||
|
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
|
||||||
|
pipelock_ca_host_path=ca_cert_host,
|
||||||
|
# On smolmachines, egress's upstream is pipelock
|
||||||
|
# on the bundle's localhost — they're in the same
|
||||||
|
# container's network namespace.
|
||||||
|
pipelock_proxy_url=BUNDLE_LOCAL_PIPELOCK_URL,
|
||||||
|
)
|
||||||
|
plan = dataclasses.replace(
|
||||||
|
plan, proxy_plan=proxy_plan, egress_plan=egress_plan,
|
||||||
|
)
|
||||||
|
|
||||||
|
# 3. Build the BundleLaunchSpec from the (now-resolved)
|
||||||
|
# inner Plans: daemon subset, env, bind-mounts, and the
|
||||||
|
# loopback alias to bind published ports against. The
|
||||||
|
# spec's ports_to_publish list expands depending on which
|
||||||
|
# daemons the agent needs to reach from the smolvm guest.
|
||||||
|
bundle_spec = _bundle_launch_spec(plan, network, loopback_ip)
|
||||||
|
token_env = _resolve_token_env(plan, os.environ)
|
||||||
|
_bundle.ensure_bundle_image(bundle_spec.image)
|
||||||
|
_bundle.start_bundle(bundle_spec, env={**os.environ, **token_env})
|
||||||
|
stack.callback(_bundle.stop_bundle, plan.slug)
|
||||||
|
|
||||||
|
# 4. Discover the host-side ports docker assigned for the
|
||||||
|
# bundle's published container ports, and bind the
|
||||||
|
# agent's URLs to `<loopback_ip>:<host port>`. Docker
|
||||||
|
# container IPs (192.168.x.x in the daemon's bridge)
|
||||||
|
# aren't reachable from the smolvm guest on macOS — TSI
|
||||||
|
# uses macOS networking, and macOS sees the daemon's
|
||||||
|
# bridge via the published-port loopback forward only.
|
||||||
|
#
|
||||||
|
# Proxy hop order matches the docker backend: when the
|
||||||
|
# bottle declares egress routes, the agent's first hop is
|
||||||
|
# egress (for token injection), then pipelock. Without
|
||||||
|
# routes, the agent dials pipelock directly. Whichever
|
||||||
|
# one is "agent-facing" is the daemon whose port we
|
||||||
|
# publish on host loopback; the other stays bundle-
|
||||||
|
# internal as the upstream proxy.
|
||||||
|
if plan.egress_plan.routes:
|
||||||
|
agent_facing_port = _EGRESS_PORT
|
||||||
|
else:
|
||||||
|
agent_facing_port = _PIPELOCK_PORT
|
||||||
|
agent_facing_host_port = _bundle.bundle_host_port(
|
||||||
|
plan.slug, agent_facing_port, host_ip=loopback_ip,
|
||||||
|
)
|
||||||
|
agent_proxy_url = f"http://{loopback_ip}:{agent_facing_host_port}"
|
||||||
|
agent_git_gate_host = ""
|
||||||
|
if plan.git_gate_plan.upstreams:
|
||||||
|
git_gate_host_port = _bundle.bundle_host_port(
|
||||||
|
plan.slug, _GIT_HTTP_PORT, host_ip=loopback_ip,
|
||||||
|
)
|
||||||
|
agent_git_gate_host = f"{loopback_ip}:{git_gate_host_port}"
|
||||||
|
agent_supervise_url = ""
|
||||||
|
if plan.supervise_plan is not None:
|
||||||
|
supervise_host_port = _bundle.bundle_host_port(
|
||||||
|
plan.slug, _SUPERVISE_PORT, host_ip=loopback_ip,
|
||||||
|
)
|
||||||
|
agent_supervise_url = f"http://{loopback_ip}:{supervise_host_port}/"
|
||||||
|
|
||||||
|
# Stamp the URLs onto the plan + guest_env. provision_git
|
||||||
|
# and provision_supervise read the plan fields; the agent
|
||||||
|
# reads guest_env on every exec_agent.
|
||||||
|
#
|
||||||
|
# NO_PROXY has to include the per-bottle loopback alias —
|
||||||
|
# otherwise claude's HTTPS_PROXY catches direct calls to
|
||||||
|
# the supervise URL (`http://<alias>:<port>/`) and proxies
|
||||||
|
# them through egress, which has no route for the alias
|
||||||
|
# and rejects with "Failed to connect". The smolmachines
|
||||||
|
# git-gate URL uses smart HTTP, so it also has to bypass
|
||||||
|
# the agent's HTTP_PROXY and go straight to the host-
|
||||||
|
# published git HTTP endpoint. Append rather than overwrite
|
||||||
|
# so prepare.py's
|
||||||
|
# `localhost,127.0.0.1` baseline stays in place.
|
||||||
|
existing_no_proxy = plan.guest_env.get("NO_PROXY", "localhost,127.0.0.1")
|
||||||
|
guest_env = {
|
||||||
|
**plan.guest_env,
|
||||||
|
"HTTPS_PROXY": agent_proxy_url,
|
||||||
|
"HTTP_PROXY": agent_proxy_url,
|
||||||
|
"NO_PROXY": f"{existing_no_proxy},{loopback_ip}",
|
||||||
|
}
|
||||||
|
if agent_git_gate_host:
|
||||||
|
guest_env["GIT_GATE_URL"] = f"http://{agent_git_gate_host}"
|
||||||
|
if agent_supervise_url:
|
||||||
|
guest_env["MCP_SUPERVISE_URL"] = agent_supervise_url
|
||||||
|
plan = dataclasses.replace(
|
||||||
|
plan,
|
||||||
|
guest_env=guest_env,
|
||||||
|
agent_proxy_url=agent_proxy_url,
|
||||||
|
agent_git_gate_host=agent_git_gate_host,
|
||||||
|
agent_supervise_url=agent_supervise_url,
|
||||||
|
)
|
||||||
|
|
||||||
|
# 5. Build the agent image and pack it into a
|
||||||
|
# `.smolmachine` artifact (or hit the per-Dockerfile-digest
|
||||||
|
# cache). Runs here, not in prepare, so the docker-build
|
||||||
|
# output doesn't garble the dashboard's preflight modal:
|
||||||
|
# both the curses-endwin path and the tmux pane-routing
|
||||||
|
# path redirect stderr around `launch` already.
|
||||||
agent_from_path = _ensure_smolmachine(
|
agent_from_path = _ensure_smolmachine(
|
||||||
plan.agent_image_ref,
|
plan.agent_image_ref,
|
||||||
dockerfile=plan.agent_dockerfile_path,
|
dockerfile=plan.agent_dockerfile_path,
|
||||||
)
|
)
|
||||||
|
|
||||||
_launch_vm(plan, agent_from_path, loopback_ip, stack)
|
# smolvm VM. --from carries the pre-packed .smolmachine
|
||||||
_init_vm(plan)
|
# artifact; --allow-cidr + -e carry the per-bottle TSI
|
||||||
|
# allowlist + env. The allowlist is the per-bottle
|
||||||
bottle = SmolmachinesBottle(
|
# loopback alias — narrowing it to one /32 keeps the
|
||||||
|
# agent from reaching other host loopback services or
|
||||||
|
# other bottles' published ports. Smolfile isn't usable
|
||||||
|
# here — smolvm 0.8.0 makes `--from` and `--smolfile`
|
||||||
|
# mutually exclusive.
|
||||||
|
_smolvm.machine_create(
|
||||||
plan.machine_name,
|
plan.machine_name,
|
||||||
prompt_path=None,
|
from_path=agent_from_path,
|
||||||
|
allow_cidrs=[f"{loopback_ip}/32"],
|
||||||
|
env=plan.guest_env,
|
||||||
|
)
|
||||||
|
stack.callback(_smolvm.machine_delete, plan.machine_name)
|
||||||
|
# Workaround smolvm 0.8.0: `--allow-cidr` is silently
|
||||||
|
# dropped when combined with `--from`. Patch the persisted
|
||||||
|
# state DB to set the allowlist before start so the booted
|
||||||
|
# VM's TSI actually enforces. See loopback_alias's module
|
||||||
|
# docstring for the investigation that led here.
|
||||||
|
_loopback.force_allowlist(plan.machine_name, [f"{loopback_ip}/32"])
|
||||||
|
_smolvm.machine_start(plan.machine_name)
|
||||||
|
stack.callback(_smolvm.machine_stop, plan.machine_name)
|
||||||
|
|
||||||
|
# 6. Repair filesystem ownership + perms that smolvm's
|
||||||
|
# pack process remapped to the host invoker's uid (501
|
||||||
|
# on macOS) rather than preserving the image's expected
|
||||||
|
# ownership.
|
||||||
|
#
|
||||||
|
# - /home/node → node:node so the node user can write
|
||||||
|
# its own dotfiles (claude appendFileSync on
|
||||||
|
# ~/.claude.json otherwise bails with ENOENT/EPERM
|
||||||
|
# and the TUI hangs without surfacing the error).
|
||||||
|
# - /tmp + /var/tmp → root:root mode 1777 so non-root
|
||||||
|
# processes can create their per-uid scratch dirs
|
||||||
|
# (claude-code creates /tmp/claude-<uid>/ as soon as
|
||||||
|
# it spawns a Bash tool call).
|
||||||
|
#
|
||||||
|
# All folded into one sh -c so we only pay one
|
||||||
|
# machine_exec round trip — back-to-back exec calls
|
||||||
|
# right after machine_start hit a SIGKILL race in
|
||||||
|
# libkrun's exec channel (see provision_ca for the
|
||||||
|
# other half of this same workaround).
|
||||||
|
_smolvm.machine_exec(plan.machine_name, [
|
||||||
|
"sh", "-c",
|
||||||
|
"chown -R node:node /home/node && "
|
||||||
|
"chown root:root /tmp /var/tmp && "
|
||||||
|
"chmod 1777 /tmp /var/tmp",
|
||||||
|
])
|
||||||
|
|
||||||
|
# Wait briefly for the VM to settle. Back-to-back smolvm
|
||||||
|
# machine_exec calls immediately after machine_start
|
||||||
|
# occasionally SIGKILL the in-VM child at ~100ms (looks
|
||||||
|
# like a VM warm-up race in libkrun's exec channel).
|
||||||
|
# 1.5s is empirically enough to dodge it; provisioning
|
||||||
|
# already takes seconds so the wait is amortized.
|
||||||
|
time.sleep(1.5)
|
||||||
|
|
||||||
|
# 7. Provision (CA / prompt / skills / git / supervise).
|
||||||
|
prompt_path = provision(plan, plan.machine_name)
|
||||||
|
|
||||||
|
yield SmolmachinesBottle(
|
||||||
|
plan.machine_name,
|
||||||
|
prompt_path=prompt_path,
|
||||||
guest_env=plan.guest_env,
|
guest_env=plan.guest_env,
|
||||||
agent_command=plan.agent_command,
|
agent_command=plan.agent_command,
|
||||||
agent_prompt_mode=plan.agent_prompt_mode,
|
agent_prompt_mode=plan.agent_prompt_mode,
|
||||||
)
|
)
|
||||||
bottle._prompt_path = provision(plan, bottle)
|
|
||||||
|
|
||||||
yield bottle
|
|
||||||
finally:
|
finally:
|
||||||
_teardown_smolmachines(stack, plan)
|
|
||||||
|
|
||||||
|
|
||||||
def _teardown_smolmachines(
|
|
||||||
stack: ExitStack,
|
|
||||||
plan: SmolmachinesBottlePlan,
|
|
||||||
) -> None:
|
|
||||||
"""Unwind the ExitStack, then revoke any provisioned deploy keys.
|
|
||||||
|
|
||||||
ExitStack errors are caught and logged (non-fatal) so that key
|
|
||||||
revocation always runs. Revocation errors propagate — a stranded
|
|
||||||
deploy key is a security concern the operator must address."""
|
|
||||||
teardown_exc: BaseException | None = None
|
|
||||||
try:
|
|
||||||
stack.close()
|
stack.close()
|
||||||
except BaseException as exc:
|
|
||||||
teardown_exc = exc
|
|
||||||
warn(f"smolmachines teardown failed: {exc!r}")
|
|
||||||
bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
|
||||||
revoke_git_gate_provisioned_keys(bottle, git_gate_state_dir(plan.slug))
|
|
||||||
if teardown_exc is not None:
|
|
||||||
raise teardown_exc
|
|
||||||
|
|
||||||
|
|
||||||
def _allocate_resources(
|
|
||||||
plan: SmolmachinesBottlePlan,
|
|
||||||
stack: ExitStack,
|
|
||||||
) -> tuple[str, str]:
|
|
||||||
"""Reserve a loopback alias and create the per-bottle docker bridge.
|
|
||||||
|
|
||||||
macOS only routes 127.0.0.1 by default; the per-bottle alias
|
|
||||||
scopes TSI's allowlist to this bottle's published ports so the
|
|
||||||
agent can't reach other bottles' or host services' ports on
|
|
||||||
loopback. No-op on Linux."""
|
|
||||||
_loopback.ensure_pool()
|
|
||||||
loopback_ip = _loopback.allocate(plan.slug)
|
|
||||||
network = _bundle.bundle_network_name(plan.slug)
|
|
||||||
_bundle.create_bundle_network(network, plan.bundle_subnet, plan.bundle_gateway)
|
|
||||||
stack.callback(_bundle.remove_bundle_network, network)
|
|
||||||
return loopback_ip, network
|
|
||||||
|
|
||||||
|
|
||||||
def _mint_certs(plan: SmolmachinesBottlePlan) -> SmolmachinesBottlePlan:
|
|
||||||
"""Mint per-bottle CAs and return the plan with CA paths filled.
|
|
||||||
|
|
||||||
Pipelock always runs in the bundle. Egress's CA is only minted
|
|
||||||
when the bottle declares routes — otherwise egress runs idle
|
|
||||||
without MITM and the CA files would be unused."""
|
|
||||||
ca_cert_host, ca_key_host = pipelock_tls_init(plan.proxy_plan.yaml_path.parent)
|
|
||||||
proxy_plan = dataclasses.replace(
|
|
||||||
plan.proxy_plan,
|
|
||||||
ca_cert_host_path=ca_cert_host,
|
|
||||||
ca_key_host_path=ca_key_host,
|
|
||||||
)
|
|
||||||
egress_plan = plan.egress_plan
|
|
||||||
if egress_plan.routes:
|
|
||||||
egress_ca_host, egress_ca_cert_only = egress_tls_init(
|
|
||||||
plan.egress_plan.routes_path.parent,
|
|
||||||
)
|
|
||||||
egress_plan = dataclasses.replace(
|
|
||||||
egress_plan,
|
|
||||||
mitmproxy_ca_host_path=egress_ca_host,
|
|
||||||
mitmproxy_ca_cert_only_host_path=egress_ca_cert_only,
|
|
||||||
pipelock_ca_host_path=ca_cert_host,
|
|
||||||
# On smolmachines, egress's upstream is pipelock on the
|
|
||||||
# bundle's localhost — they're in the same container's
|
|
||||||
# network namespace.
|
|
||||||
pipelock_proxy_url=BUNDLE_LOCAL_PIPELOCK_URL,
|
|
||||||
)
|
|
||||||
return dataclasses.replace(plan, proxy_plan=proxy_plan, egress_plan=egress_plan)
|
|
||||||
|
|
||||||
|
|
||||||
def _start_bundle(
|
|
||||||
plan: SmolmachinesBottlePlan,
|
|
||||||
network: str,
|
|
||||||
loopback_ip: str,
|
|
||||||
stack: ExitStack,
|
|
||||||
) -> SmolmachinesBottlePlan:
|
|
||||||
"""Build the BundleLaunchSpec, resolve token env, start the
|
|
||||||
sidecar bundle container, and register teardown."""
|
|
||||||
bundle_spec = _bundle_launch_spec(plan, network, loopback_ip)
|
|
||||||
token_env = _resolve_token_env(plan, dict(os.environ))
|
|
||||||
_bundle.ensure_bundle_image(bundle_spec.image)
|
|
||||||
_bundle.start_bundle(bundle_spec, env={**os.environ, **token_env})
|
|
||||||
stack.callback(_bundle.stop_bundle, plan.slug)
|
|
||||||
return plan
|
|
||||||
|
|
||||||
|
|
||||||
def _discover_urls(
|
|
||||||
plan: SmolmachinesBottlePlan,
|
|
||||||
loopback_ip: str,
|
|
||||||
) -> SmolmachinesBottlePlan:
|
|
||||||
"""Discover host-side ports for published container ports and
|
|
||||||
return the plan with URLs + guest_env stamped in.
|
|
||||||
|
|
||||||
Docker container IPs (192.168.x.x in the daemon's bridge)
|
|
||||||
aren't reachable from the smolvm guest on macOS — TSI uses
|
|
||||||
macOS networking, and macOS sees the daemon's bridge via the
|
|
||||||
published-port loopback forward only.
|
|
||||||
|
|
||||||
Proxy hop order: when the bottle declares egress routes, the
|
|
||||||
agent's first hop is egress (for token injection), then
|
|
||||||
pipelock. Without routes, the agent dials pipelock directly.
|
|
||||||
NO_PROXY includes the per-bottle loopback alias so the
|
|
||||||
supervise + git-gate URLs bypass HTTPS_PROXY."""
|
|
||||||
if plan.egress_plan.routes:
|
|
||||||
agent_facing_port = _EGRESS_PORT
|
|
||||||
else:
|
|
||||||
agent_facing_port = _PIPELOCK_PORT
|
|
||||||
agent_facing_host_port = _bundle.bundle_host_port(
|
|
||||||
plan.slug, agent_facing_port, host_ip=loopback_ip,
|
|
||||||
)
|
|
||||||
agent_proxy_url = f"http://{loopback_ip}:{agent_facing_host_port}"
|
|
||||||
|
|
||||||
agent_git_gate_host = ""
|
|
||||||
if plan.git_gate_plan.upstreams:
|
|
||||||
git_gate_host_port = _bundle.bundle_host_port(
|
|
||||||
plan.slug, _GIT_HTTP_PORT, host_ip=loopback_ip,
|
|
||||||
)
|
|
||||||
agent_git_gate_host = f"{loopback_ip}:{git_gate_host_port}"
|
|
||||||
|
|
||||||
agent_supervise_url = ""
|
|
||||||
if plan.supervise_plan is not None:
|
|
||||||
supervise_host_port = _bundle.bundle_host_port(
|
|
||||||
plan.slug, _SUPERVISE_PORT, host_ip=loopback_ip,
|
|
||||||
)
|
|
||||||
agent_supervise_url = f"http://{loopback_ip}:{supervise_host_port}/"
|
|
||||||
|
|
||||||
existing_no_proxy = plan.guest_env.get("NO_PROXY", "localhost,127.0.0.1")
|
|
||||||
guest_env = {
|
|
||||||
**plan.guest_env,
|
|
||||||
"HTTPS_PROXY": agent_proxy_url,
|
|
||||||
"HTTP_PROXY": agent_proxy_url,
|
|
||||||
"NO_PROXY": f"{existing_no_proxy},{loopback_ip}",
|
|
||||||
}
|
|
||||||
if agent_git_gate_host:
|
|
||||||
guest_env["GIT_GATE_URL"] = f"http://{agent_git_gate_host}"
|
|
||||||
if agent_supervise_url:
|
|
||||||
guest_env["MCP_SUPERVISE_URL"] = agent_supervise_url
|
|
||||||
|
|
||||||
return dataclasses.replace(
|
|
||||||
plan,
|
|
||||||
guest_env=guest_env,
|
|
||||||
agent_proxy_url=agent_proxy_url,
|
|
||||||
agent_git_gate_host=agent_git_gate_host,
|
|
||||||
agent_supervise_url=agent_supervise_url,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _launch_vm(
|
|
||||||
plan: SmolmachinesBottlePlan,
|
|
||||||
agent_from_path: Path,
|
|
||||||
loopback_ip: str,
|
|
||||||
stack: ExitStack,
|
|
||||||
) -> None:
|
|
||||||
"""Create, patch, and start the smolvm VM; register teardown.
|
|
||||||
|
|
||||||
--allow-cidr is the per-bottle loopback alias so the guest can
|
|
||||||
only reach this bottle's bundle ports. force_allowlist patches
|
|
||||||
smolvm 0.8.0's silent-drop of --allow-cidr when combined with
|
|
||||||
--from. Smolfile isn't usable here — smolvm 0.8.0 makes --from
|
|
||||||
and --smolfile mutually exclusive."""
|
|
||||||
_smolvm.machine_create(
|
|
||||||
plan.machine_name,
|
|
||||||
from_path=agent_from_path,
|
|
||||||
allow_cidrs=[f"{loopback_ip}/32"],
|
|
||||||
env=plan.guest_env,
|
|
||||||
)
|
|
||||||
stack.callback(_smolvm.machine_delete, plan.machine_name)
|
|
||||||
# Workaround smolvm 0.8.0: `--allow-cidr` is silently dropped
|
|
||||||
# when combined with `--from`. Patch the persisted state DB
|
|
||||||
# before start so the booted VM's TSI actually enforces.
|
|
||||||
_loopback.force_allowlist(plan.machine_name, [f"{loopback_ip}/32"])
|
|
||||||
_smolvm.machine_start(plan.machine_name)
|
|
||||||
stack.callback(_smolvm.machine_stop, plan.machine_name)
|
|
||||||
|
|
||||||
|
|
||||||
def _init_vm(plan: SmolmachinesBottlePlan) -> None:
|
|
||||||
"""Repair filesystem ownership and wait for exec channel readiness.
|
|
||||||
|
|
||||||
Ownership repair: smolvm's pack process remaps files to the host
|
|
||||||
invoker's uid (501 on macOS). /home/node must be node:node so
|
|
||||||
Claude Code can write ~/.claude.json; /tmp + /var/tmp need root
|
|
||||||
mode 1777 so non-root processes can create per-uid scratch dirs.
|
|
||||||
All folded into one sh -c to avoid back-to-back exec calls
|
|
||||||
immediately after machine_start (libkrun exec-channel race).
|
|
||||||
|
|
||||||
wait_exec_ready polls until the exec channel is ready for the
|
|
||||||
subsequent provision calls, replacing the empirical sleep."""
|
|
||||||
_smolvm.machine_exec(plan.machine_name, [
|
|
||||||
"sh", "-c",
|
|
||||||
"chown -R node:node /home/node && "
|
|
||||||
"chown root:root /tmp /var/tmp && "
|
|
||||||
"chmod 1777 /tmp /var/tmp",
|
|
||||||
])
|
|
||||||
_smolvm.wait_exec_ready(plan.machine_name)
|
|
||||||
|
|
||||||
|
|
||||||
def _bundle_launch_spec(
|
def _bundle_launch_spec(
|
||||||
@@ -345,9 +326,10 @@ def _bundle_launch_spec(
|
|||||||
# is "agent-facing" gets its port published on the host
|
# is "agent-facing" gets its port published on the host
|
||||||
# loopback (see `_ensure_smolmachine`'s discovery loop) and the
|
# loopback (see `_ensure_smolmachine`'s discovery loop) and the
|
||||||
# other stays bundle-internal. The bundle is NOT reachable by
|
# other stays bundle-internal. The bundle is NOT reachable by
|
||||||
# bridge IP from the smolvm guest on macOS — TSI uses macOS
|
# bridge IP from the smolvm guest, so the
|
||||||
# networking, and macOS sees the daemon's bridge via the
|
# PRD-0023-chunk-3 EGRESS_LISTEN_HOST=127.0.0.1 mitigation
|
||||||
# published-port loopback forward only.
|
# isn't needed: the agent can only dial whatever daemon's
|
||||||
|
# host port we publish, period.
|
||||||
|
|
||||||
# --- pipelock ---------------------------------------------
|
# --- pipelock ---------------------------------------------
|
||||||
pp = plan.proxy_plan
|
pp = plan.proxy_plan
|
||||||
@@ -374,6 +356,7 @@ def _bundle_launch_spec(
|
|||||||
env.append(token_env)
|
env.append(token_env)
|
||||||
|
|
||||||
# --- git-gate ---------------------------------------------
|
# --- git-gate ---------------------------------------------
|
||||||
|
extra_hosts: list[str] = []
|
||||||
gp = plan.git_gate_plan
|
gp = plan.git_gate_plan
|
||||||
if gp.upstreams:
|
if gp.upstreams:
|
||||||
daemons += ["git-gate", "git-http"]
|
daemons += ["git-gate", "git-http"]
|
||||||
@@ -437,13 +420,24 @@ def _bundle_launch_spec(
|
|||||||
|
|
||||||
|
|
||||||
def _resolve_token_env(
|
def _resolve_token_env(
|
||||||
plan: SmolmachinesBottlePlan, host_env: dict[str, str],
|
plan: SmolmachinesBottlePlan, host_env: object
|
||||||
) -> dict[str, str]:
|
) -> dict[str, str]:
|
||||||
"""Resolve the egress token env-var values from the host's
|
"""Resolve the egress token env-var values from the host's
|
||||||
environ so they reach the bundle's process env via docker's
|
environ so they reach the bundle's process env via docker's
|
||||||
`-e NAME` inheritance. Empty when no routes declare auth."""
|
`-e NAME` inheritance. Empty when no routes declare auth."""
|
||||||
effective_env = {**host_env, **plan.agent_provision.provisioned_env}
|
ep = plan.egress_plan
|
||||||
return egress_resolve_token_values(plan.egress_plan.token_env_map, effective_env)
|
if not ep.routes:
|
||||||
|
return {}
|
||||||
|
env = dict(host_env)
|
||||||
|
token_values = egress_resolve_token_values(ep.token_env_map, env)
|
||||||
|
if plan.spec.manifest.bottle_for(
|
||||||
|
plan.spec.agent_name,
|
||||||
|
).agent_provider.forward_host_credentials:
|
||||||
|
access_token = codex_host_access_token(env)
|
||||||
|
for token_env, token_ref in ep.token_env_map.items():
|
||||||
|
if token_ref == CODEX_HOST_CREDENTIAL_TOKEN_REF:
|
||||||
|
token_values[token_env] = access_token
|
||||||
|
return token_values
|
||||||
|
|
||||||
|
|
||||||
def _ensure_smolmachine(image_ref: str, *, dockerfile: str = "") -> Path:
|
def _ensure_smolmachine(image_ref: str, *, dockerfile: str = "") -> Path:
|
||||||
|
|||||||
@@ -45,7 +45,6 @@ alias gets handed to a new bottle."""
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import fcntl
|
|
||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
import platform
|
import platform
|
||||||
@@ -84,14 +83,6 @@ _POOL_START = 16
|
|||||||
_POOL_END = 31 # inclusive
|
_POOL_END = 31 # inclusive
|
||||||
|
|
||||||
|
|
||||||
# File lock that serialises concurrent allocate() calls so two
|
|
||||||
# simultaneous launches can't read the same docker state and claim
|
|
||||||
# the same alias. Narrowed to the allocate() call itself; docker run
|
|
||||||
# runs after the lock is released. Once the container is running it
|
|
||||||
# appears in docker state and future allocate() calls will see it.
|
|
||||||
_ALLOC_LOCK_PATH = Path.home() / ".cache" / "bot-bottle" / "smolmachines.lock"
|
|
||||||
|
|
||||||
|
|
||||||
# Loopback aliases pool: 127.0.0.<start>..127.0.0.<end>.
|
# Loopback aliases pool: 127.0.0.<start>..127.0.0.<end>.
|
||||||
def _pool_addresses() -> list[str]:
|
def _pool_addresses() -> list[str]:
|
||||||
return [f"127.0.0.{i}" for i in range(_POOL_START, _POOL_END + 1)]
|
return [f"127.0.0.{i}" for i in range(_POOL_START, _POOL_END + 1)]
|
||||||
@@ -188,20 +179,9 @@ def allocate(slug: str) -> str:
|
|||||||
On non-macOS the whole `127.0.0.0/8` is loopback by default;
|
On non-macOS the whole `127.0.0.0/8` is loopback by default;
|
||||||
`127.0.0.1` is fine to share and we skip the alias dance.
|
`127.0.0.1` is fine to share and we skip the alias dance.
|
||||||
This still returns a deterministic address so launch.py's
|
This still returns a deterministic address so launch.py's
|
||||||
callers don't have to branch on platform.
|
callers don't have to branch on platform."""
|
||||||
|
|
||||||
An exclusive file lock serialises concurrent calls so two
|
|
||||||
simultaneous launches don't read the same docker state and
|
|
||||||
claim the same alias."""
|
|
||||||
if not _is_macos():
|
if not _is_macos():
|
||||||
return "127.0.0.1"
|
return "127.0.0.1"
|
||||||
_ALLOC_LOCK_PATH.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
with open(_ALLOC_LOCK_PATH, "w") as lf:
|
|
||||||
fcntl.flock(lf, fcntl.LOCK_EX)
|
|
||||||
return _allocate_locked()
|
|
||||||
|
|
||||||
|
|
||||||
def _allocate_locked() -> str:
|
|
||||||
in_use = _aliases_in_use()
|
in_use = _aliases_in_use()
|
||||||
for ip in _pool_addresses():
|
for ip in _pool_addresses():
|
||||||
if ip not in in_use:
|
if ip not in in_use:
|
||||||
|
|||||||
@@ -28,11 +28,9 @@ from ...backend.docker.bottle_state import (
|
|||||||
write_metadata,
|
write_metadata,
|
||||||
)
|
)
|
||||||
from ...egress import Egress
|
from ...egress import Egress
|
||||||
from ...env import resolve_env
|
|
||||||
from ...git_gate import GitGate
|
from ...git_gate import GitGate
|
||||||
from ...pipelock import PipelockProxy
|
from ...pipelock import PipelockProxy
|
||||||
from ...supervise import Supervise
|
from ...supervise import Supervise
|
||||||
from ...workspace import workspace_plan as resolve_workspace_plan
|
|
||||||
from .bottle_plan import SmolmachinesBottlePlan
|
from .bottle_plan import SmolmachinesBottlePlan
|
||||||
from .util import smolmachines_bundle_subnet, smolmachines_preflight
|
from .util import smolmachines_bundle_subnet, smolmachines_preflight
|
||||||
|
|
||||||
@@ -61,8 +59,6 @@ def resolve_plan(
|
|||||||
bottle = manifest.bottle_for(spec.agent_name)
|
bottle = manifest.bottle_for(spec.agent_name)
|
||||||
provider = bottle.agent_provider
|
provider = bottle.agent_provider
|
||||||
provider_runtime = runtime_for(provider.template)
|
provider_runtime = runtime_for(provider.template)
|
||||||
guest_home = "/home/node"
|
|
||||||
workspace_plan = resolve_workspace_plan(spec, guest_home=guest_home)
|
|
||||||
|
|
||||||
slug = spec.identity or bottle_identity(spec.agent_name)
|
slug = spec.identity or bottle_identity(spec.agent_name)
|
||||||
|
|
||||||
@@ -74,24 +70,25 @@ def resolve_plan(
|
|||||||
cwd=spec.user_cwd if spec.copy_cwd else "",
|
cwd=spec.user_cwd if spec.copy_cwd else "",
|
||||||
copy_cwd=spec.copy_cwd,
|
copy_cwd=spec.copy_cwd,
|
||||||
started_at=datetime.now(timezone.utc).isoformat(),
|
started_at=datetime.now(timezone.utc).isoformat(),
|
||||||
|
# No compose project for smolmachines bottles; chunk 4
|
||||||
|
# will give dashboard discovery a backend-specific path.
|
||||||
compose_project="",
|
compose_project="",
|
||||||
backend="smolmachines",
|
|
||||||
))
|
))
|
||||||
|
|
||||||
subnet, gateway, bundle_ip = smolmachines_bundle_subnet(slug)
|
subnet, gateway, bundle_ip = smolmachines_bundle_subnet(slug)
|
||||||
|
|
||||||
# Agent's env: resolve through resolve_env() so ?prompt entries
|
# Agent's env: the prepare-time view doesn't yet know the
|
||||||
# are prompted and ${HOST_VAR} entries are interpolated — matching
|
# host loopback ports the bundle's daemons get published on
|
||||||
# the Docker backend's contract. Forwarded (secret/interpolated)
|
# (those come from docker AFTER `docker run` returns), so
|
||||||
# values still reach the guest as -e K=V smolvm flags because
|
# HTTPS_PROXY / GIT_GATE_URL / MCP_SUPERVISE_URL are
|
||||||
# smolvm 0.8.0 has no env-file or stdin injection path; this is
|
# populated in launch.py and stamped onto guest_env there.
|
||||||
# the known argv-exposure gap documented in PRD 0038.
|
# What we set here is the part that doesn't depend on
|
||||||
# HTTPS_PROXY / GIT_GATE_URL / MCP_SUPERVISE_URL are populated
|
# bundle bringup — bottle.env literals, the empty-NO_PROXY
|
||||||
# in launch.py after bundle bringup.
|
# safe default, and the TLS trust env trio
|
||||||
resolved = resolve_env(manifest, spec.agent_name)
|
# (NODE_EXTRA_CA_CERTS / SSL_CERT_FILE / REQUESTS_CA_BUNDLE)
|
||||||
|
# pointing at Debian's update-ca-certificates output bundle.
|
||||||
guest_env: dict[str, str] = {
|
guest_env: dict[str, str] = {
|
||||||
**resolved.literals,
|
**bottle.env,
|
||||||
**resolved.forwarded,
|
|
||||||
"NO_PROXY": "localhost,127.0.0.1",
|
"NO_PROXY": "localhost,127.0.0.1",
|
||||||
"NODE_EXTRA_CA_CERTS": "/etc/ssl/certs/ca-certificates.crt",
|
"NODE_EXTRA_CA_CERTS": "/etc/ssl/certs/ca-certificates.crt",
|
||||||
"SSL_CERT_FILE": "/etc/ssl/certs/ca-certificates.crt",
|
"SSL_CERT_FILE": "/etc/ssl/certs/ca-certificates.crt",
|
||||||
@@ -133,12 +130,11 @@ def resolve_plan(
|
|||||||
template=provider.template,
|
template=provider.template,
|
||||||
dockerfile=agent_dockerfile_path,
|
dockerfile=agent_dockerfile_path,
|
||||||
state_dir=agent_dir,
|
state_dir=agent_dir,
|
||||||
guest_home=guest_home,
|
guest_home=os.environ.get("BOT_BOTTLE_GUEST_HOME", "/home/node"),
|
||||||
guest_env=guest_env,
|
guest_env=guest_env,
|
||||||
forward_host_credentials=provider.forward_host_credentials,
|
forward_host_credentials=provider.forward_host_credentials,
|
||||||
auth_token=provider.auth_token,
|
auth_token=provider.auth_token,
|
||||||
host_env=dict(os.environ),
|
host_env=dict(os.environ),
|
||||||
trusted_project_path=workspace_plan.workdir,
|
|
||||||
)
|
)
|
||||||
merged_guest_env = dict(agent_provision.guest_env)
|
merged_guest_env = dict(agent_provision.guest_env)
|
||||||
for key, val in agent_provision.env_vars.items():
|
for key, val in agent_provision.env_vars.items():
|
||||||
@@ -172,7 +168,6 @@ def resolve_plan(
|
|||||||
return SmolmachinesBottlePlan(
|
return SmolmachinesBottlePlan(
|
||||||
spec=spec,
|
spec=spec,
|
||||||
stage_dir=stage_dir,
|
stage_dir=stage_dir,
|
||||||
guest_home=guest_home,
|
|
||||||
slug=slug,
|
slug=slug,
|
||||||
bundle_subnet=subnet,
|
bundle_subnet=subnet,
|
||||||
bundle_gateway=gateway,
|
bundle_gateway=gateway,
|
||||||
@@ -186,7 +181,6 @@ def resolve_plan(
|
|||||||
egress_plan=egress_plan,
|
egress_plan=egress_plan,
|
||||||
supervise_plan=supervise_plan,
|
supervise_plan=supervise_plan,
|
||||||
agent_provision=agent_provision,
|
agent_provision=agent_provision,
|
||||||
workspace_plan=workspace_plan,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
@@ -1,12 +1,14 @@
|
|||||||
"""Backend-infrastructure provisioners for the smolmachines backend.
|
"""Provisioning helpers for the smolmachines backend (PRD 0023
|
||||||
|
chunk 4).
|
||||||
|
|
||||||
Per PRD 0050 the per-provider provisioning steps (prompt, skills,
|
Each method maps onto one of `BottleBackend`'s `provision_*`
|
||||||
declarative provision-plan apply, supervise MCP registration) live on
|
overrides. They run after the VM is up + the bundle is reachable
|
||||||
the `AgentProvider` plugin under `bot_bottle/contrib/`. The modules
|
and copy host-side state (prompt, skills, .git, CA cert,
|
||||||
left in this subpackage handle only the steps that are
|
supervise MCP config) into the guest via `smolvm machine cp` /
|
||||||
backend-specific:
|
`smolvm machine exec`.
|
||||||
|
|
||||||
- ca.py — install per-bottle CA bundle into the guest trust store
|
Chunk 4a ships `provision_prompt` and `provision_skills` — the
|
||||||
- git.py — copy host cwd `.git` into the guest when --cwd is used
|
two that don't depend on agent-image tooling (claude-code,
|
||||||
- workspace.py — copy the operator workspace into the guest
|
update-ca-certificates) beyond `cp` and `mkdir`. provision_ca /
|
||||||
"""
|
provision_git / provision_supervise land once the agent-image
|
||||||
|
gap is solved."""
|
||||||
|
|||||||
@@ -2,8 +2,8 @@
|
|||||||
trust store (PRD 0023 chunk 4d).
|
trust store (PRD 0023 chunk 4d).
|
||||||
|
|
||||||
Mirrors `backend.docker.provision.ca`: select the right CA (egress
|
Mirrors `backend.docker.provision.ca`: select the right CA (egress
|
||||||
when the bottle has routes, else pipelock), copy it to Debian's
|
when the bottle has routes, else pipelock), `smolvm machine cp` it
|
||||||
`/usr/local/share/ca-certificates/` path,
|
to Debian's `/usr/local/share/ca-certificates/` path,
|
||||||
`update-ca-certificates` to rebuild the trust bundle, and log the
|
`update-ca-certificates` to rebuild the trust bundle, and log the
|
||||||
fingerprint once. The selected cert depends on the agent's
|
fingerprint once. The selected cert depends on the agent's
|
||||||
HTTP_PROXY target — same logic as the docker backend, since the
|
HTTP_PROXY target — same logic as the docker backend, since the
|
||||||
@@ -15,8 +15,6 @@ flag exists; the VM init is root), so we don't need the explicit
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import time
|
|
||||||
|
|
||||||
from ....log import die
|
from ....log import die
|
||||||
from ...util import (
|
from ...util import (
|
||||||
AGENT_CA_BUNDLE,
|
AGENT_CA_BUNDLE,
|
||||||
@@ -24,20 +22,17 @@ from ...util import (
|
|||||||
log_ca_fingerprint,
|
log_ca_fingerprint,
|
||||||
select_ca_cert,
|
select_ca_cert,
|
||||||
)
|
)
|
||||||
from ... import Bottle, ExecResult
|
from .. import smolvm as _smolvm
|
||||||
from ..bottle_plan import SmolmachinesBottlePlan
|
from ..bottle_plan import SmolmachinesBottlePlan
|
||||||
|
|
||||||
|
|
||||||
_SIGKILL_EXIT = 128 + 9
|
def provision_ca(plan: SmolmachinesBottlePlan, target: str) -> None:
|
||||||
|
|
||||||
|
|
||||||
def provision_ca(plan: SmolmachinesBottlePlan, bottle: Bottle) -> None:
|
|
||||||
"""Copy the agent-facing CA cert into the guest, rebuild the
|
"""Copy the agent-facing CA cert into the guest, rebuild the
|
||||||
trust bundle, emit a one-line fingerprint log. Called from
|
trust bundle, emit a one-line fingerprint log. Called from
|
||||||
`BottleBackend.provision` after the smolvm guest is up."""
|
`BottleBackend.provision` after the smolvm guest is up."""
|
||||||
cert_host_path, label = select_ca_cert(plan.egress_plan, plan.proxy_plan)
|
cert_host_path, label = select_ca_cert(plan.egress_plan, plan.proxy_plan)
|
||||||
|
|
||||||
bottle.cp_in(str(cert_host_path), AGENT_CA_PATH)
|
_smolvm.machine_cp(str(cert_host_path), f"{target}:{AGENT_CA_PATH}")
|
||||||
# Mode 0644 — readable to non-root tools in the guest.
|
# Mode 0644 — readable to non-root tools in the guest.
|
||||||
# update-ca-certificates rebuilds the bundle at AGENT_CA_BUNDLE,
|
# update-ca-certificates rebuilds the bundle at AGENT_CA_BUNDLE,
|
||||||
# which is what curl / Python ssl / OpenSSL-based tools read by
|
# which is what curl / Python ssl / OpenSSL-based tools read by
|
||||||
@@ -45,21 +40,22 @@ def provision_ca(plan: SmolmachinesBottlePlan, bottle: Bottle) -> None:
|
|||||||
# REQUESTS_CA_BUNDLE) on the guest_env covers Node + Python
|
# REQUESTS_CA_BUNDLE) on the guest_env covers Node + Python
|
||||||
# `requests` / libraries that don't load the system bundle.
|
# `requests` / libraries that don't load the system bundle.
|
||||||
#
|
#
|
||||||
r = _install_ca(bottle)
|
# chown + chmod + update-ca-certificates run in one
|
||||||
if r.returncode == _SIGKILL_EXIT:
|
# `sh -c` so we only pay one machine_exec round trip; the
|
||||||
# smolvm/libkrun can SIGKILL an otherwise-normal exec
|
# `&&` chaining surfaces the first failure as the return
|
||||||
# during early-VM provisioning. `update-ca-certificates`
|
# code.
|
||||||
# is idempotent, so retry the same install once after a
|
r = _smolvm.machine_exec(target, [
|
||||||
# short settle delay before treating it as fatal.
|
"sh", "-c",
|
||||||
time.sleep(1.0)
|
f"chown root:root {AGENT_CA_PATH} && "
|
||||||
r = _install_ca(bottle)
|
f"chmod 644 {AGENT_CA_PATH} && "
|
||||||
|
f"update-ca-certificates",
|
||||||
if r.returncode != 0:
|
])
|
||||||
|
if r.returncode != 0 or "1 added" not in (r.stdout or ""):
|
||||||
# update-ca-certificates not adding our cert is fatal —
|
# update-ca-certificates not adding our cert is fatal —
|
||||||
# claude-code's TLS handshake against the egress-MITM'd
|
# claude-code's TLS handshake against the egress-MITM'd
|
||||||
# api.anthropic.com would fail downstream. Bail early
|
# api.anthropic.com would fail downstream. Bail early
|
||||||
# with what we can see (output is captured so we can
|
# with what we can see (output is captured by smolvm so
|
||||||
# surface it).
|
# we can surface it).
|
||||||
die(
|
die(
|
||||||
f"update-ca-certificates didn't add the agent CA "
|
f"update-ca-certificates didn't add the agent CA "
|
||||||
f"(exit {r.returncode}): "
|
f"(exit {r.returncode}): "
|
||||||
@@ -70,23 +66,6 @@ def provision_ca(plan: SmolmachinesBottlePlan, bottle: Bottle) -> None:
|
|||||||
log_ca_fingerprint(cert_host_path, label)
|
log_ca_fingerprint(cert_host_path, label)
|
||||||
|
|
||||||
|
|
||||||
def _install_ca(bottle: Bottle) -> ExecResult:
|
|
||||||
# chown + chmod + update-ca-certificates + bundle
|
|
||||||
# verification run in one exec so we only pay one
|
|
||||||
# round trip; the `&&` chaining surfaces the first failure
|
|
||||||
# as the return code. The verify check is more stable than
|
|
||||||
# requiring "1 added" in stdout: a retry after a
|
|
||||||
# partially-completed first run may legitimately report "0
|
|
||||||
# added" while the cert is already installed.
|
|
||||||
return bottle.exec(
|
|
||||||
f"chown root:root {AGENT_CA_PATH} && "
|
|
||||||
f"chmod 644 {AGENT_CA_PATH} && "
|
|
||||||
f"update-ca-certificates && "
|
|
||||||
f"openssl verify -CAfile {AGENT_CA_BUNDLE} {AGENT_CA_PATH}",
|
|
||||||
user="root",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# Re-exported for the launch/provision_ca caller + tests. The path
|
# Re-exported for the launch/provision_ca caller + tests. The path
|
||||||
# constants live in the shared `backend.util` (Debian's
|
# constants live in the shared `backend.util` (Debian's
|
||||||
# `update-ca-certificates` layout is the same in both backends).
|
# `update-ca-certificates` layout is the same in both backends).
|
||||||
|
|||||||
@@ -4,7 +4,7 @@
|
|||||||
Three concerns, all about git in the agent:
|
Three concerns, all about git in the agent:
|
||||||
|
|
||||||
1. If --cwd was passed AND the host cwd has a .git, copy that
|
1. If --cwd was passed AND the host cwd has a .git, copy that
|
||||||
.git into the planned guest workspace so the agent operates on
|
.git into /home/node/workspace/.git so the agent operates on
|
||||||
the user's repo.
|
the user's repo.
|
||||||
2. If the bottle declares `git` entries (PRD 0008), write a
|
2. If the bottle declares `git` entries (PRD 0008), write a
|
||||||
~/.gitconfig with insteadOf rules so every git operation
|
~/.gitconfig with insteadOf rules so every git operation
|
||||||
@@ -26,53 +26,60 @@ git_gate module."""
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import os
|
import os
|
||||||
import shlex
|
|
||||||
import tempfile
|
import tempfile
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from ....git_gate import git_gate_render_gitconfig
|
from ....git_gate import git_gate_render_gitconfig
|
||||||
from ....log import info
|
from ....log import info
|
||||||
from ... import Bottle
|
from .. import smolvm as _smolvm
|
||||||
from ..bottle_plan import SmolmachinesBottlePlan
|
from ..bottle_plan import SmolmachinesBottlePlan
|
||||||
|
|
||||||
|
|
||||||
def provision_git(plan: SmolmachinesBottlePlan, bottle: Bottle) -> None:
|
# `node` is the agent user from the repo Dockerfile. Override via
|
||||||
|
# BOT_BOTTLE_GUEST_HOME mirrors the docker backend's
|
||||||
|
# BOT_BOTTLE_CONTAINER_HOME knob — same purpose, different
|
||||||
|
# transport.
|
||||||
|
_DEFAULT_GUEST_HOME = "/home/node"
|
||||||
|
|
||||||
|
|
||||||
|
def _guest_home() -> str:
|
||||||
|
return os.environ.get("BOT_BOTTLE_GUEST_HOME", _DEFAULT_GUEST_HOME)
|
||||||
|
|
||||||
|
|
||||||
|
def provision_git(plan: SmolmachinesBottlePlan, target: str) -> None:
|
||||||
"""Set up git inside the guest. Runs all three subcases; each
|
"""Set up git inside the guest. Runs all three subcases; each
|
||||||
no-ops when its condition isn't met."""
|
no-ops when its condition isn't met."""
|
||||||
_provision_cwd_git(plan, bottle)
|
_provision_cwd_git(plan, target)
|
||||||
_provision_git_gate_config(plan, bottle)
|
_provision_git_gate_config(plan, target)
|
||||||
_provision_git_user(plan, bottle)
|
_provision_git_user(plan, target)
|
||||||
|
|
||||||
|
|
||||||
def _provision_cwd_git(plan: SmolmachinesBottlePlan, bottle: Bottle) -> None:
|
def _provision_cwd_git(plan: SmolmachinesBottlePlan, target: str) -> None:
|
||||||
"""If --cwd was set and the host cwd has a .git directory, copy
|
"""If --cwd was set and the host cwd has a .git directory, copy
|
||||||
it into <guest_home>/workspace/.git and fix ownership. No-op
|
it into <guest_home>/workspace/.git and fix ownership. No-op
|
||||||
otherwise."""
|
otherwise."""
|
||||||
workspace = plan.workspace_plan
|
if not (plan.spec.copy_cwd and Path(plan.spec.user_cwd, ".git").is_dir()):
|
||||||
if not (workspace.enabled and workspace.copy_git and workspace.has_host_git_dir):
|
|
||||||
return
|
return
|
||||||
guest_workspace_git = f"{workspace.guest_path}/.git"
|
guest_workspace_git = f"{_guest_home()}/workspace/.git"
|
||||||
host_git = str(workspace.host_path / ".git")
|
info(f"copying {plan.spec.user_cwd}/.git -> {target}:{guest_workspace_git}")
|
||||||
info(f"copying {host_git} -> {bottle.name}:{guest_workspace_git}")
|
# mkdir -p the workspace dir so `machine cp` lands the .git
|
||||||
# mkdir -p the workspace dir so cp_in lands the .git
|
|
||||||
# directly there even on first-time bottles.
|
# directly there even on first-time bottles.
|
||||||
bottle.exec(f"mkdir -p {shlex.quote(workspace.guest_path)}", user="root")
|
_smolvm.machine_exec(target, ["mkdir", "-p", f"{_guest_home()}/workspace"])
|
||||||
bottle.cp_in(host_git, guest_workspace_git)
|
_smolvm.machine_cp(
|
||||||
# cp_in lands files as root; the agent runs as node so
|
f"{plan.spec.user_cwd}/.git", f"{target}:{guest_workspace_git}",
|
||||||
|
)
|
||||||
|
# `machine cp` lands files as root; the agent runs as node so
|
||||||
# the workspace tree must be chowned over.
|
# the workspace tree must be chowned over.
|
||||||
bottle.exec(
|
_smolvm.machine_exec(
|
||||||
f"chown -R {shlex.quote(workspace.owner)} {shlex.quote(guest_workspace_git)}",
|
target, ["chown", "-R", "node:node", guest_workspace_git],
|
||||||
user="root",
|
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def _provision_git_gate_config(
|
def _provision_git_gate_config(plan: SmolmachinesBottlePlan, target: str) -> None:
|
||||||
plan: SmolmachinesBottlePlan, bottle: Bottle
|
|
||||||
) -> None:
|
|
||||||
"""Write ~/.gitconfig in the guest with the git-gate insteadOf
|
"""Write ~/.gitconfig in the guest with the git-gate insteadOf
|
||||||
rules. No-op when the bottle has no `git` entries."""
|
rules. No-op when the bottle has no `git` entries."""
|
||||||
manifest_bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
||||||
if not manifest_bottle.git:
|
if not bottle.git:
|
||||||
return
|
return
|
||||||
|
|
||||||
# `<loopback alias>:<host port>` form: the bundle's git-gate
|
# `<loopback alias>:<host port>` form: the bundle's git-gate
|
||||||
@@ -81,11 +88,11 @@ def _provision_git_gate_config(
|
|||||||
# TSI, not the docker bridge IP) can dial it. launch.py
|
# TSI, not the docker bridge IP) can dial it. launch.py
|
||||||
# populates `plan.agent_git_gate_host` after bundle bringup.
|
# populates `plan.agent_git_gate_host` after bundle bringup.
|
||||||
content = git_gate_render_gitconfig(
|
content = git_gate_render_gitconfig(
|
||||||
manifest_bottle.git, plan.agent_git_gate_host, scheme="http",
|
bottle.git, plan.agent_git_gate_host, scheme="http",
|
||||||
)
|
)
|
||||||
|
|
||||||
guest_gitconfig = f"{plan.guest_home}/.gitconfig"
|
guest_gitconfig = f"{_guest_home()}/.gitconfig"
|
||||||
# Stage the file under the plan's stage_dir so cp_in
|
# Stage the file under the plan's stage_dir so `machine cp`
|
||||||
# has a stable host path. The plan's stage_dir is cleaned up
|
# has a stable host path. The plan's stage_dir is cleaned up
|
||||||
# by start.py's session-end teardown.
|
# by start.py's session-end teardown.
|
||||||
with tempfile.NamedTemporaryFile(
|
with tempfile.NamedTemporaryFile(
|
||||||
@@ -96,38 +103,41 @@ def _provision_git_gate_config(
|
|||||||
config_file = Path(f.name)
|
config_file = Path(f.name)
|
||||||
os.chmod(config_file, 0o600)
|
os.chmod(config_file, 0o600)
|
||||||
|
|
||||||
info(f"writing {guest_gitconfig} with {len(manifest_bottle.git)} insteadOf rule(s)")
|
info(f"writing {guest_gitconfig} with {len(bottle.git)} insteadOf rule(s)")
|
||||||
bottle.cp_in(str(config_file), guest_gitconfig)
|
_smolvm.machine_cp(str(config_file), f"{target}:{guest_gitconfig}")
|
||||||
bottle.exec(
|
_smolvm.machine_exec(target, ["chown", "node:node", guest_gitconfig])
|
||||||
f"chown node:node {shlex.quote(guest_gitconfig)} && "
|
_smolvm.machine_exec(target, ["chmod", "644", guest_gitconfig])
|
||||||
f"chmod 644 {shlex.quote(guest_gitconfig)}",
|
|
||||||
user="root",
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _provision_git_user(
|
def _provision_git_user(
|
||||||
plan: SmolmachinesBottlePlan, bottle: Bottle,
|
plan: SmolmachinesBottlePlan, target: str,
|
||||||
) -> None:
|
) -> None:
|
||||||
"""Apply `git config --global user.{name,email}` inside the
|
"""Apply `git config --global user.{name,email}` inside the
|
||||||
guest as the node user so --global lands in the same
|
guest as the node user so --global lands in the same
|
||||||
`/home/node/.gitconfig` that `_provision_git_gate_config`
|
`/home/node/.gitconfig` that `_provision_git_gate_config`
|
||||||
writes to. No-op when the bottle didn't declare `git.user`.
|
writes to. No-op when the bottle didn't declare `git.user`.
|
||||||
|
|
||||||
SmolmachinesBottle.exec(user="node") automatically sets
|
Runs via `runuser -u node --`; HOME is forced via smolvm's
|
||||||
HOME=/home/node so --global writes to /home/node/.gitconfig."""
|
`-e` flag because runuser (without -l) inherits root's
|
||||||
manifest_bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
HOME=/root, which would put --global in the wrong file."""
|
||||||
gu = manifest_bottle.git_user
|
bottle = plan.spec.manifest.bottle_for(plan.spec.agent_name)
|
||||||
|
gu = bottle.git_user
|
||||||
if gu.is_empty():
|
if gu.is_empty():
|
||||||
return
|
return
|
||||||
|
env = {"HOME": _guest_home(), "USER": "node"}
|
||||||
if gu.name:
|
if gu.name:
|
||||||
info(f"git config --global user.name = {gu.name!r}")
|
info(f"git config --global user.name = {gu.name!r}")
|
||||||
bottle.exec(
|
_smolvm.machine_exec(
|
||||||
f"git config --global user.name {shlex.quote(gu.name)}",
|
target,
|
||||||
user="node",
|
["runuser", "-u", "node", "--",
|
||||||
|
"git", "config", "--global", "user.name", gu.name],
|
||||||
|
env=env,
|
||||||
)
|
)
|
||||||
if gu.email:
|
if gu.email:
|
||||||
info(f"git config --global user.email = {gu.email!r}")
|
info(f"git config --global user.email = {gu.email!r}")
|
||||||
bottle.exec(
|
_smolvm.machine_exec(
|
||||||
f"git config --global user.email {shlex.quote(gu.email)}",
|
target,
|
||||||
user="node",
|
["runuser", "-u", "node", "--",
|
||||||
|
"git", "config", "--global", "user.email", gu.email],
|
||||||
|
env=env,
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -0,0 +1,42 @@
|
|||||||
|
"""Copy the agent prompt into a running smolmachines bottle.
|
||||||
|
|
||||||
|
The prompt file is always copied (so the in-guest path always
|
||||||
|
exists) but `--append-system-prompt-file` only fires when the
|
||||||
|
agent actually has a prompt — the return value signals which
|
||||||
|
case, mirroring the docker backend's contract.
|
||||||
|
|
||||||
|
`smolvm machine cp` lands files as root inside the VM; the claude
|
||||||
|
process runs as `node`, so we chown + chmod the prompt after the
|
||||||
|
copy. Same flow as the docker backend's provision_prompt."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
|
||||||
|
from .. import smolvm as _smolvm
|
||||||
|
from ..bottle_plan import SmolmachinesBottlePlan
|
||||||
|
|
||||||
|
|
||||||
|
# `node` is the agent user from the repo Dockerfile.
|
||||||
|
# BOT_BOTTLE_GUEST_HOME mirrors the docker backend's
|
||||||
|
# BOT_BOTTLE_CONTAINER_HOME knob.
|
||||||
|
_DEFAULT_GUEST_HOME = "/home/node"
|
||||||
|
|
||||||
|
|
||||||
|
def provision_prompt(plan: SmolmachinesBottlePlan, target: str) -> str | None:
|
||||||
|
"""Copy the prompt file into the running smolvm guest, fix
|
||||||
|
ownership/mode. Returns the in-guest path if the agent has a
|
||||||
|
non-empty prompt (drives --append-system-prompt-file), else
|
||||||
|
None. The file is copied either way so the path always
|
||||||
|
exists — mirrors the docker backend's behavior."""
|
||||||
|
guest_home = os.environ.get("BOT_BOTTLE_GUEST_HOME", _DEFAULT_GUEST_HOME)
|
||||||
|
in_guest_prompt_path = f"{guest_home}/.bot-bottle-prompt.txt"
|
||||||
|
|
||||||
|
_smolvm.machine_cp(str(plan.prompt_file), f"{target}:{in_guest_prompt_path}")
|
||||||
|
# machine cp lands as root, source's 0o600 mode is preserved —
|
||||||
|
# node can't read its own prompt without these two.
|
||||||
|
_smolvm.machine_exec(target, ["chown", "node:node", in_guest_prompt_path])
|
||||||
|
_smolvm.machine_exec(target, ["chmod", "600", in_guest_prompt_path])
|
||||||
|
|
||||||
|
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||||
|
return in_guest_prompt_path if agent.prompt else None
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
"""Provision non-secret provider auth markers into a smolmachines bottle."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from ....log import die
|
||||||
|
from .. import smolvm as _smolvm
|
||||||
|
from ..bottle_plan import SmolmachinesBottlePlan
|
||||||
|
|
||||||
|
|
||||||
|
def provision_provider_auth(plan: SmolmachinesBottlePlan, target: str) -> None:
|
||||||
|
"""Apply provider-owned guest setup through smolvm primitives."""
|
||||||
|
provision = plan.agent_provision
|
||||||
|
for d in provision.dirs:
|
||||||
|
_exec(target, ["mkdir", "-p", d.guest_path], f"could not create {d.guest_path}")
|
||||||
|
_exec(target, ["chown", d.owner, d.guest_path], f"could not chown {d.guest_path}")
|
||||||
|
_exec(target, ["chmod", d.mode, d.guest_path], f"could not chmod {d.guest_path}")
|
||||||
|
for command in provision.pre_copy:
|
||||||
|
_exec(target, list(command.argv), command.error)
|
||||||
|
for f in provision.files:
|
||||||
|
_smolvm.machine_cp(str(f.host_path), f"{target}:{f.guest_path}")
|
||||||
|
_exec(target, ["chown", f.owner, f.guest_path], f"could not chown {f.guest_path}")
|
||||||
|
_exec(target, ["chmod", f.mode, f.guest_path], f"could not chmod {f.guest_path}")
|
||||||
|
for command in provision.verify:
|
||||||
|
_exec(target, list(command.argv), command.error)
|
||||||
|
|
||||||
|
|
||||||
|
def _exec(target: str, argv: list[str], error: str) -> None:
|
||||||
|
result = _smolvm.machine_exec(target, argv)
|
||||||
|
if result.returncode != 0:
|
||||||
|
detail = (result.stderr or result.stdout).strip()
|
||||||
|
if detail:
|
||||||
|
detail = f": {detail}"
|
||||||
|
die(f"agent provider provisioning: {error}{detail}")
|
||||||
@@ -0,0 +1,63 @@
|
|||||||
|
"""Copy host-side skill directories into a running smolmachines
|
||||||
|
bottle.
|
||||||
|
|
||||||
|
Skills are validated on the host before launch by
|
||||||
|
`BottleBackend._validate_skills`; this module assumes that
|
||||||
|
validation has already run. A skill that disappears between
|
||||||
|
validation and copy still dies loudly rather than silently
|
||||||
|
producing a partial guest."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
|
||||||
|
from ....log import die, info
|
||||||
|
from ...util import host_skill_dir
|
||||||
|
from .. import smolvm as _smolvm
|
||||||
|
from ..bottle_plan import SmolmachinesBottlePlan
|
||||||
|
|
||||||
|
|
||||||
|
# In-guest path mirrors the docker backend's claude-skills
|
||||||
|
# convention (~/.claude/skills/<name>/) under the node user's
|
||||||
|
# home — same path as the real bot-bottle image's
|
||||||
|
# /home/node/.claude/skills (pre-created in the Dockerfile).
|
||||||
|
_DEFAULT_SKILLS_DIR = "/home/node/.claude/skills"
|
||||||
|
|
||||||
|
|
||||||
|
def provision_skills(plan: SmolmachinesBottlePlan, target: str) -> None:
|
||||||
|
"""Copy each of the agent's named skills from the host's
|
||||||
|
~/.claude/skills/<name>/ into the guest's equivalent path.
|
||||||
|
For each skill: `mkdir -p` the destination, `smolvm machine cp`
|
||||||
|
the host source dir over, then chown the result to node:node so
|
||||||
|
the agent can read it. No-op when the agent has no skills.
|
||||||
|
|
||||||
|
smolvm machine cp on a directory copies recursively (same
|
||||||
|
semantics as `cp -r`); unlike docker cp's trailing-slash
|
||||||
|
convention, smolvm doesn't need the `/.` suffix dance.
|
||||||
|
|
||||||
|
machine cp lands files as root inside the VM, so we chown each
|
||||||
|
skill tree over to node:node after the copy — same pattern as
|
||||||
|
the docker backend's provision_prompt."""
|
||||||
|
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
||||||
|
if not agent.skills:
|
||||||
|
return
|
||||||
|
|
||||||
|
skills_dir = os.environ.get(
|
||||||
|
"BOT_BOTTLE_GUEST_SKILLS_DIR", _DEFAULT_SKILLS_DIR,
|
||||||
|
)
|
||||||
|
|
||||||
|
_smolvm.machine_exec(target, ["mkdir", "-p", skills_dir])
|
||||||
|
|
||||||
|
for name in agent.skills:
|
||||||
|
src = host_skill_dir(name)
|
||||||
|
if not os.path.isdir(src):
|
||||||
|
die(
|
||||||
|
f"skill {name!r} disappeared from host between "
|
||||||
|
f"validation and copy at {src}."
|
||||||
|
)
|
||||||
|
dst = f"{skills_dir}/{name}"
|
||||||
|
info(f"copying skill {name} into {target}:{dst}")
|
||||||
|
# Wipe any prior copy so re-runs don't accumulate.
|
||||||
|
_smolvm.machine_exec(target, ["rm", "-rf", dst])
|
||||||
|
_smolvm.machine_cp(src, f"{target}:{dst}")
|
||||||
|
_smolvm.machine_exec(target, ["chown", "-R", "node:node", dst])
|
||||||
@@ -0,0 +1,67 @@
|
|||||||
|
"""Supervise sidecar provisioning inside a running smolmachines
|
||||||
|
bottle (PRD 0023 chunk 4d; PRD 0013 supervise plane).
|
||||||
|
|
||||||
|
Registers the per-bottle supervise sidecar as an HTTP MCP server
|
||||||
|
in the agent's claude-code config so the agent discovers the
|
||||||
|
stuck-recovery MCP tools (pipelock-block, capability-block) at
|
||||||
|
startup.
|
||||||
|
|
||||||
|
Mirrors `backend.docker.provision.supervise` — same `claude mcp
|
||||||
|
add` call, just dispatched via `smolvm machine exec` instead of
|
||||||
|
`docker exec`, and against `<bundle_ip>:<port>` instead of the
|
||||||
|
short `supervise` alias (no DNS in the TSI-allowlisted guest)."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from ....log import info, warn
|
||||||
|
from .. import smolvm as _smolvm
|
||||||
|
from ..bottle_plan import SmolmachinesBottlePlan
|
||||||
|
|
||||||
|
|
||||||
|
_SUPERVISE_MCP_NAME = "supervise"
|
||||||
|
|
||||||
|
|
||||||
|
def provision_supervise(plan: SmolmachinesBottlePlan, target: str) -> None:
|
||||||
|
"""Run `claude mcp add` inside the guest to register the
|
||||||
|
supervise sidecar in claude-code's user config. No-op when
|
||||||
|
bottle.supervise is False.
|
||||||
|
|
||||||
|
The URL is the agent-side endpoint launch.py populated after
|
||||||
|
bundle bringup — `http://127.0.0.1:<host port>/` rather than
|
||||||
|
the bundle's docker bridge IP, because that bridge isn't
|
||||||
|
reachable from the smolvm guest on macOS.
|
||||||
|
|
||||||
|
Failure is logged but not fatal: the bottle still works (you
|
||||||
|
just can't call supervise tools from the agent until the entry
|
||||||
|
is added manually). The operator sees the warning at launch."""
|
||||||
|
if plan.supervise_plan is None:
|
||||||
|
return
|
||||||
|
url = plan.agent_supervise_url
|
||||||
|
info(f"registering supervise MCP server in agent claude config → {url}")
|
||||||
|
# `claude mcp add --scope user` writes to ~/.claude.json. The
|
||||||
|
# agent is the `node` user; smolvm machine_exec runs as root
|
||||||
|
# by default, so we have to switch user explicitly and set
|
||||||
|
# HOME so the config lands in /home/node/.claude.json (where
|
||||||
|
# the agent's claude actually reads it from).
|
||||||
|
r = _smolvm.machine_exec(
|
||||||
|
target,
|
||||||
|
[
|
||||||
|
"runuser", "-u", "node", "--",
|
||||||
|
"env", "HOME=/home/node",
|
||||||
|
"claude", "mcp", "add",
|
||||||
|
"--scope", "user",
|
||||||
|
"--transport", "http",
|
||||||
|
_SUPERVISE_MCP_NAME,
|
||||||
|
url,
|
||||||
|
],
|
||||||
|
)
|
||||||
|
if r.returncode != 0:
|
||||||
|
warn(
|
||||||
|
f"`claude mcp add supervise` failed (exit {r.returncode}): "
|
||||||
|
f"{(r.stderr or r.stdout or '').strip()}. Inside the bottle, "
|
||||||
|
f"register manually with: "
|
||||||
|
f"claude mcp add --scope user --transport http supervise {url}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = ["provision_supervise"]
|
||||||
@@ -1,32 +0,0 @@
|
|||||||
"""Copy the operator workspace into a smolmachines guest."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import shlex
|
|
||||||
|
|
||||||
from ....log import info
|
|
||||||
from ... import Bottle
|
|
||||||
from ..bottle_plan import SmolmachinesBottlePlan
|
|
||||||
|
|
||||||
|
|
||||||
def provision_workspace(plan: SmolmachinesBottlePlan, bottle: Bottle) -> None:
|
|
||||||
"""Copy host cwd contents to the planned guest workspace."""
|
|
||||||
workspace = plan.workspace_plan
|
|
||||||
if not (workspace.enabled and workspace.copy_contents):
|
|
||||||
return
|
|
||||||
|
|
||||||
guest_parent = workspace.guest_path.rsplit("/", 1)[0] or "/"
|
|
||||||
guest_path_q = shlex.quote(workspace.guest_path)
|
|
||||||
guest_parent_q = shlex.quote(guest_parent)
|
|
||||||
owner_q = shlex.quote(workspace.owner)
|
|
||||||
mode_q = shlex.quote(workspace.mode)
|
|
||||||
info(f"copying {workspace.host_path} -> {bottle.name}:{workspace.guest_path}")
|
|
||||||
bottle.exec(
|
|
||||||
f"rm -rf {guest_path_q} && mkdir -p {guest_parent_q}",
|
|
||||||
user="root",
|
|
||||||
)
|
|
||||||
bottle.cp_in(str(workspace.host_path), workspace.guest_path)
|
|
||||||
bottle.exec(
|
|
||||||
f"chown -R {owner_q} {guest_path_q} && chmod {mode_q} {guest_path_q}",
|
|
||||||
user="root",
|
|
||||||
)
|
|
||||||
@@ -27,13 +27,11 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import shutil
|
import shutil
|
||||||
import subprocess
|
import subprocess
|
||||||
import time
|
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Mapping, Sequence
|
from typing import Mapping, Sequence
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
_SMOLVM = "smolvm"
|
_SMOLVM = "smolvm"
|
||||||
|
|
||||||
|
|
||||||
@@ -199,34 +197,6 @@ def machine_exec(
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def wait_exec_ready(name: str, *, timeout: float = 5.0) -> None:
|
|
||||||
"""Poll `machine exec true` until exit 0 or `timeout` elapses.
|
|
||||||
|
|
||||||
Replaces `time.sleep(1.5)` after `machine_start`: libkrun's exec
|
|
||||||
channel needs a brief warm-up before back-to-back exec calls are
|
|
||||||
safe. Polling exits as soon as the channel is ready and fails
|
|
||||||
loudly if the VM never responds."""
|
|
||||||
deadline = time.monotonic() + timeout
|
|
||||||
delay = 0.1
|
|
||||||
while time.monotonic() < deadline:
|
|
||||||
r = machine_exec(name, ["true"])
|
|
||||||
if r.returncode == 0:
|
|
||||||
return
|
|
||||||
remaining = deadline - time.monotonic()
|
|
||||||
if remaining <= 0:
|
|
||||||
break
|
|
||||||
time.sleep(min(delay, remaining))
|
|
||||||
delay = min(delay * 2, 0.5)
|
|
||||||
argv = ["smolvm", "machine", "exec", "--name", name, "--", "true"]
|
|
||||||
raise SmolvmError(
|
|
||||||
argv,
|
|
||||||
subprocess.CompletedProcess(
|
|
||||||
args=argv, returncode=-1, stdout="",
|
|
||||||
stderr=f"exec channel not ready after {timeout:.0f}s — VM may have failed to boot.",
|
|
||||||
),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def machine_cp(src: str, dst: str) -> None:
|
def machine_cp(src: str, dst: str) -> None:
|
||||||
"""`smolvm machine cp SRC DST`. Path syntax: `machine:path` to
|
"""`smolvm machine cp SRC DST`. Path syntax: `machine:path` to
|
||||||
reference a path inside the VM, bare path for the host. Both
|
reference a path inside the VM, bare path for the host. Both
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
"""Main CLI dispatcher.
|
"""Main CLI dispatcher.
|
||||||
|
|
||||||
Commands: cleanup, edit, info, init, list, resume, start, supervise
|
Commands: cleanup, dashboard, edit, info, init, list, resume, start
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
@@ -12,24 +12,24 @@ from ..manifest import ManifestError
|
|||||||
from ._common import PROG
|
from ._common import PROG
|
||||||
from . import list as _list_mod
|
from . import list as _list_mod
|
||||||
from .cleanup import cmd_cleanup
|
from .cleanup import cmd_cleanup
|
||||||
|
from .dashboard import cmd_dashboard
|
||||||
from .edit import cmd_edit
|
from .edit import cmd_edit
|
||||||
from .info import cmd_info
|
from .info import cmd_info
|
||||||
from .init import cmd_init
|
from .init import cmd_init
|
||||||
from .resume import cmd_resume
|
from .resume import cmd_resume
|
||||||
from .start import cmd_start
|
from .start import cmd_start
|
||||||
from .supervise import cmd_supervise
|
|
||||||
|
|
||||||
cmd_list = _list_mod.cmd_list
|
cmd_list = _list_mod.cmd_list
|
||||||
|
|
||||||
COMMANDS = {
|
COMMANDS = {
|
||||||
"cleanup": cmd_cleanup,
|
"cleanup": cmd_cleanup,
|
||||||
|
"dashboard": cmd_dashboard,
|
||||||
"edit": cmd_edit,
|
"edit": cmd_edit,
|
||||||
"info": cmd_info,
|
"info": cmd_info,
|
||||||
"init": cmd_init,
|
"init": cmd_init,
|
||||||
"list": cmd_list,
|
"list": cmd_list,
|
||||||
"resume": cmd_resume,
|
"resume": cmd_resume,
|
||||||
"start": cmd_start,
|
"start": cmd_start,
|
||||||
"supervise": cmd_supervise,
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@@ -37,13 +37,13 @@ def usage() -> None:
|
|||||||
sys.stderr.write(f"usage: {PROG} <command> [args...]\n\n")
|
sys.stderr.write(f"usage: {PROG} <command> [args...]\n\n")
|
||||||
sys.stderr.write("Commands:\n")
|
sys.stderr.write("Commands:\n")
|
||||||
sys.stderr.write(" cleanup stop and remove all active bot-bottle containers\n")
|
sys.stderr.write(" cleanup stop and remove all active bot-bottle containers\n")
|
||||||
|
sys.stderr.write(" dashboard view + approve/modify/reject pending supervise proposals (PRD 0013)\n")
|
||||||
sys.stderr.write(" edit open an agent in vim for editing\n")
|
sys.stderr.write(" edit open an agent in vim for editing\n")
|
||||||
sys.stderr.write(" info print env, skills, and prompt details for a named agent\n")
|
sys.stderr.write(" info print env, skills, and prompt details for a named agent\n")
|
||||||
sys.stderr.write(" init interactively create a new agent and add it to bot-bottle.json\n")
|
sys.stderr.write(" init interactively create a new agent and add it to bot-bottle.json\n")
|
||||||
sys.stderr.write(" list list available agents or active containers\n")
|
sys.stderr.write(" list list available agents or active containers\n")
|
||||||
sys.stderr.write(" resume re-launch a bottle by its identity (continues state from PRD 0016)\n")
|
sys.stderr.write(" resume re-launch a bottle by its identity (continues state from PRD 0016)\n")
|
||||||
sys.stderr.write(" start boot a container for a named agent and attach an interactive session\n")
|
sys.stderr.write(" start boot a container for a named agent and attach an interactive session\n\n")
|
||||||
sys.stderr.write(" supervise view + approve/modify/reject pending supervise proposals (PRD 0013)\n\n")
|
|
||||||
sys.stderr.write(f"Run '{PROG} <command> --help' for command-specific usage.\n")
|
sys.stderr.write(f"Run '{PROG} <command> --help' for command-specific usage.\n")
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
@@ -52,10 +52,8 @@ def cmd_resume(argv: list[str]) -> int:
|
|||||||
user_cwd=metadata.cwd or USER_CWD,
|
user_cwd=metadata.cwd or USER_CWD,
|
||||||
identity=metadata.identity,
|
identity=metadata.identity,
|
||||||
)
|
)
|
||||||
backend_name = metadata.backend or None
|
|
||||||
return _launch_bottle(
|
return _launch_bottle(
|
||||||
spec,
|
spec,
|
||||||
dry_run=args.dry_run,
|
dry_run=args.dry_run,
|
||||||
remote_control=args.remote_control,
|
remote_control=args.remote_control,
|
||||||
backend_name=backend_name,
|
|
||||||
)
|
)
|
||||||
|
|||||||
+25
-11
@@ -2,8 +2,10 @@
|
|||||||
interactive claude-code session. The container is torn down when the
|
interactive claude-code session. The container is torn down when the
|
||||||
session ends.
|
session ends.
|
||||||
|
|
||||||
The launch core is shared with `cli.py resume <identity>` through
|
The launch core is shared with `cli.py resume <identity>` and (PRD
|
||||||
the private orchestrator `_launch_bottle`.
|
0020 chunk 1+) the dashboard's in-process start flow: see the
|
||||||
|
public helpers `prepare_with_preflight`, `attach_agent`, and the
|
||||||
|
private orchestrator `_launch_bottle`.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
@@ -69,7 +71,7 @@ def cmd_start(argv: list[str]) -> int:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
# --- Launch helpers ------------------------------------------------------
|
# --- Public helpers shared with the dashboard (PRD 0020) -----------------
|
||||||
|
|
||||||
|
|
||||||
def prepare_with_preflight(
|
def prepare_with_preflight(
|
||||||
@@ -82,11 +84,14 @@ def prepare_with_preflight(
|
|||||||
backend_name: str | None = None,
|
backend_name: str | None = None,
|
||||||
) -> tuple[DockerBottlePlan | None, str]:
|
) -> tuple[DockerBottlePlan | None, str]:
|
||||||
"""Run `backend.prepare`, render the preflight summary via the
|
"""Run `backend.prepare`, render the preflight summary via the
|
||||||
injected callable, prompt y/N via the injected callable.
|
injected callable, prompt y/N via the injected callable. The CLI
|
||||||
|
binds these to stderr/stdin; the dashboard binds them to a
|
||||||
|
curses modal.
|
||||||
|
|
||||||
`backend_name` selects which backend prepares the plan
|
`backend_name` selects which backend prepares the plan
|
||||||
(`None` → `$BOT_BOTTLE_BACKEND` → `docker`). The CLI passes
|
(`None` → `$BOT_BOTTLE_BACKEND` → `docker`). Dashboard
|
||||||
whatever `--backend` resolved to.
|
passes the value from its new-agent backend-picker modal; the
|
||||||
|
CLI passes whatever `--backend` resolved to.
|
||||||
|
|
||||||
Returns `(plan, identity)`. `plan` is None on dry-run or
|
Returns `(plan, identity)`. `plan` is None on dry-run or
|
||||||
operator-N, but `identity` is set as soon as `backend.prepare`
|
operator-N, but `identity` is set as soon as `backend.prepare`
|
||||||
@@ -117,10 +122,16 @@ def attach_agent(
|
|||||||
agent process's exit code.
|
agent process's exit code.
|
||||||
|
|
||||||
`resume=True` adds `--continue` so claude picks up its most
|
`resume=True` adds `--continue` so claude picks up its most
|
||||||
recent session non-interactively (no session-picker prompt).
|
recent session non-interactively (no session-picker prompt) —
|
||||||
First-attach paths (`./cli.py start`) leave it False.
|
the right shape for the dashboard's Enter re-attach (PRD 0020
|
||||||
|
chunk 3), where a bottle typically has exactly one session.
|
||||||
|
First-attach paths (`./cli.py start`, the dashboard's new-agent
|
||||||
|
flow) leave it False.
|
||||||
|
|
||||||
Used as the inner step of `./cli.py start`."""
|
Used as the inner step of `./cli.py start` (one-shot) and by the
|
||||||
|
dashboard, which calls it from inside a `curses.endwin → … →
|
||||||
|
stdscr.refresh()` handoff so the curses surface gets out of the
|
||||||
|
terminal's way while the agent has it."""
|
||||||
runtime = runtime_for(agent_provider_template)
|
runtime = runtime_for(agent_provider_template)
|
||||||
info(
|
info(
|
||||||
f"attaching interactive {agent_provider_template} session "
|
f"attaching interactive {agent_provider_template} session "
|
||||||
@@ -137,7 +148,8 @@ def attach_agent(
|
|||||||
def capture_claude_session_state(identity: str, exit_code: int) -> None:
|
def capture_claude_session_state(identity: str, exit_code: int) -> None:
|
||||||
"""Inside the launch context, while the container is still
|
"""Inside the launch context, while the container is still
|
||||||
alive: snapshot the transcript and mark for preservation if
|
alive: snapshot the transcript and mark for preservation if
|
||||||
claude crashed."""
|
claude crashed. Public for the dashboard's death-handling path
|
||||||
|
(PRD 0020 open question 3)."""
|
||||||
# FIXME: this captures Claude-specific session state. A follow-up
|
# FIXME: this captures Claude-specific session state. A follow-up
|
||||||
# spike should explore freezing provider-neutral container state
|
# spike should explore freezing provider-neutral container state
|
||||||
# instead of relying on each agent's transcript layout.
|
# instead of relying on each agent's transcript layout.
|
||||||
@@ -150,7 +162,9 @@ def capture_claude_session_state(identity: str, exit_code: int) -> None:
|
|||||||
|
|
||||||
def settle_state(identity: str) -> None:
|
def settle_state(identity: str) -> None:
|
||||||
"""Post-teardown housekeeping: print the resume hint if the
|
"""Post-teardown housekeeping: print the resume hint if the
|
||||||
state was preserved, otherwise reap the per-bottle state dir."""
|
state was preserved, otherwise reap the per-bottle state dir.
|
||||||
|
Public so the dashboard's explicit-stop path calls the same
|
||||||
|
settlement the CLI uses on context exit."""
|
||||||
if not identity:
|
if not identity:
|
||||||
return
|
return
|
||||||
if is_preserved(identity):
|
if is_preserved(identity):
|
||||||
|
|||||||
@@ -1,577 +0,0 @@
|
|||||||
"""supervise: list pending supervise proposals across all bottles and
|
|
||||||
act on them (approve / modify / reject).
|
|
||||||
|
|
||||||
Curses-based TUI; modify-then-approve shells out to $EDITOR. The
|
|
||||||
approval handlers wire to the per-tool remediation engines:
|
|
||||||
PRD 0014 (egress, retargeted from cred-proxy in PRD 0017
|
|
||||||
chunk 3) writes routes.yaml + SIGHUPs egress; PRD 0015
|
|
||||||
(pipelock) writes the allowlist + restarts pipelock; PRD 0016
|
|
||||||
(capability) rebuilds the bottle Dockerfile.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import curses
|
|
||||||
import os
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
import tempfile
|
|
||||||
import traceback
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
from .. import supervise as _supervise
|
|
||||||
from ..backend.docker.bottle_state import read_metadata
|
|
||||||
from ..backend.docker.capability_apply import (
|
|
||||||
CapabilityApplyError,
|
|
||||||
apply_capability_change,
|
|
||||||
)
|
|
||||||
from ..backend.docker.egress_apply import EgressApplyError, add_route
|
|
||||||
from ..backend.docker.pipelock_apply import (
|
|
||||||
PipelockApplyError,
|
|
||||||
apply_allowlist_change,
|
|
||||||
fetch_current_allowlist,
|
|
||||||
parse_allowlist_content,
|
|
||||||
render_allowlist_content,
|
|
||||||
)
|
|
||||||
from ..log import Die, error, info
|
|
||||||
from ..supervise import (
|
|
||||||
COMPONENT_FOR_TOOL,
|
|
||||||
AuditEntry,
|
|
||||||
Proposal,
|
|
||||||
Response,
|
|
||||||
STATUS_APPROVED,
|
|
||||||
STATUS_MODIFIED,
|
|
||||||
STATUS_REJECTED,
|
|
||||||
TOOL_CAPABILITY_BLOCK,
|
|
||||||
TOOL_EGRESS_BLOCK,
|
|
||||||
TOOL_PIPELOCK_BLOCK,
|
|
||||||
archive_proposal,
|
|
||||||
list_pending_proposals,
|
|
||||||
render_diff,
|
|
||||||
write_audit_entry,
|
|
||||||
write_response,
|
|
||||||
)
|
|
||||||
from ._common import PROG
|
|
||||||
|
|
||||||
|
|
||||||
_REFRESH_INTERVAL_MS = 1000
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class QueuedProposal:
|
|
||||||
"""A pending proposal plus the queue dir it was found in."""
|
|
||||||
|
|
||||||
proposal: Proposal
|
|
||||||
queue_dir: Path
|
|
||||||
|
|
||||||
|
|
||||||
# Errors any remediation engine may raise. Caught by the TUI key
|
|
||||||
# handlers and surfaced in the status line so a failed apply keeps
|
|
||||||
# the proposal pending rather than crashing curses.
|
|
||||||
ApplyError = (EgressApplyError, PipelockApplyError, CapabilityApplyError)
|
|
||||||
|
|
||||||
|
|
||||||
def discover_pending() -> list[QueuedProposal]:
|
|
||||||
"""Walk ~/.bot-bottle/queue/* and collect pending proposals."""
|
|
||||||
queue_root = _supervise.bot_bottle_root() / "queue"
|
|
||||||
if not queue_root.is_dir():
|
|
||||||
return []
|
|
||||||
out: list[QueuedProposal] = []
|
|
||||||
for slug_dir in sorted(queue_root.iterdir()):
|
|
||||||
if not slug_dir.is_dir():
|
|
||||||
continue
|
|
||||||
for proposal in list_pending_proposals(slug_dir):
|
|
||||||
out.append(QueuedProposal(proposal=proposal, queue_dir=slug_dir))
|
|
||||||
out.sort(key=lambda q: q.proposal.arrival_timestamp)
|
|
||||||
return out
|
|
||||||
|
|
||||||
|
|
||||||
def _approval_status(qp: QueuedProposal, verb: str) -> str:
|
|
||||||
"""Status-line text after a successful approval."""
|
|
||||||
base = f"{verb} {qp.proposal.tool} for [{qp.proposal.bottle_slug}]"
|
|
||||||
if qp.proposal.tool == TOOL_CAPABILITY_BLOCK:
|
|
||||||
return f"{base}; resume: ./cli.py resume {qp.proposal.bottle_slug}"
|
|
||||||
return base
|
|
||||||
|
|
||||||
|
|
||||||
def _detail_lines(
|
|
||||||
qp: QueuedProposal,
|
|
||||||
*,
|
|
||||||
green_attr: int = 0,
|
|
||||||
) -> list[tuple[str, int]]:
|
|
||||||
"""Return the detail-view body as (text, curses-attr) tuples."""
|
|
||||||
p = qp.proposal
|
|
||||||
out: list[tuple[str, int]] = [
|
|
||||||
(f"bottle: {p.bottle_slug}", 0),
|
|
||||||
(f"tool: {p.tool}", 0),
|
|
||||||
(f"id: {p.id}", 0),
|
|
||||||
(f"arrived: {p.arrival_timestamp}", 0),
|
|
||||||
(f"queue: {qp.queue_dir}", 0),
|
|
||||||
("", 0),
|
|
||||||
("justification:", 0),
|
|
||||||
]
|
|
||||||
out.extend((" " + line, 0) for line in p.justification.splitlines() or [""])
|
|
||||||
out.extend([
|
|
||||||
("", 0),
|
|
||||||
(_proposed_payload_label(p.tool) + ":", 0),
|
|
||||||
])
|
|
||||||
out.extend((line, 0) for line in p.proposed_file.splitlines() or [""])
|
|
||||||
if p.tool == TOOL_PIPELOCK_BLOCK:
|
|
||||||
host = _failed_url_host(p.proposed_file)
|
|
||||||
if host:
|
|
||||||
out.append(("", 0))
|
|
||||||
out.append((host, green_attr))
|
|
||||||
return out
|
|
||||||
|
|
||||||
|
|
||||||
def _failed_url_host(url: str) -> str:
|
|
||||||
"""Best-effort hostname extraction from a pipelock-block proposal."""
|
|
||||||
import urllib.parse
|
|
||||||
|
|
||||||
try:
|
|
||||||
return urllib.parse.urlsplit(url.strip()).hostname or ""
|
|
||||||
except ValueError:
|
|
||||||
return ""
|
|
||||||
|
|
||||||
|
|
||||||
def _proposed_payload_label(tool: str) -> str:
|
|
||||||
if tool == TOOL_PIPELOCK_BLOCK:
|
|
||||||
return "failed URL"
|
|
||||||
return "proposed file"
|
|
||||||
|
|
||||||
|
|
||||||
def _suffix_for_tool(tool: str) -> str:
|
|
||||||
if tool == TOOL_CAPABILITY_BLOCK:
|
|
||||||
return ".dockerfile"
|
|
||||||
return ".txt"
|
|
||||||
|
|
||||||
|
|
||||||
# --- Operator actions ------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def approve(
|
|
||||||
qp: QueuedProposal,
|
|
||||||
*,
|
|
||||||
notes: str = "",
|
|
||||||
final_file: str | None = None,
|
|
||||||
) -> None:
|
|
||||||
"""Apply the proposal, write the waiting response, and audit it."""
|
|
||||||
status = STATUS_MODIFIED if final_file is not None else STATUS_APPROVED
|
|
||||||
file_to_apply = final_file if final_file is not None else qp.proposal.proposed_file
|
|
||||||
|
|
||||||
diff_before, diff_after = "", ""
|
|
||||||
if qp.proposal.tool == TOOL_EGRESS_BLOCK:
|
|
||||||
diff_before, diff_after = add_route(
|
|
||||||
qp.proposal.bottle_slug, file_to_apply,
|
|
||||||
)
|
|
||||||
elif qp.proposal.tool == TOOL_PIPELOCK_BLOCK:
|
|
||||||
diff_before, diff_after = _apply_pipelock_url(
|
|
||||||
qp.proposal.bottle_slug, file_to_apply,
|
|
||||||
)
|
|
||||||
elif qp.proposal.tool == TOOL_CAPABILITY_BLOCK:
|
|
||||||
_meta = read_metadata(qp.proposal.bottle_slug)
|
|
||||||
if _meta is not None and not _meta.compose_project:
|
|
||||||
raise CapabilityApplyError(
|
|
||||||
"capability-block remediation is not supported for smolmachines "
|
|
||||||
"bottles. Reject this proposal or handle the capability change "
|
|
||||||
"manually, then restart the bottle."
|
|
||||||
)
|
|
||||||
diff_before, diff_after = apply_capability_change(
|
|
||||||
qp.proposal.bottle_slug, file_to_apply,
|
|
||||||
)
|
|
||||||
|
|
||||||
response = Response(
|
|
||||||
proposal_id=qp.proposal.id,
|
|
||||||
status=status,
|
|
||||||
notes=notes,
|
|
||||||
final_file=final_file,
|
|
||||||
)
|
|
||||||
write_response(qp.queue_dir, response)
|
|
||||||
_write_audit(
|
|
||||||
qp, action=status, notes=notes,
|
|
||||||
diff_before=diff_before, diff_after=diff_after,
|
|
||||||
)
|
|
||||||
if qp.proposal.tool == TOOL_CAPABILITY_BLOCK:
|
|
||||||
archive_proposal(qp.queue_dir, qp.proposal.id)
|
|
||||||
|
|
||||||
|
|
||||||
def reject(qp: QueuedProposal, *, reason: str) -> None:
|
|
||||||
"""Write a rejection response and an audit entry."""
|
|
||||||
response = Response(
|
|
||||||
proposal_id=qp.proposal.id,
|
|
||||||
status=STATUS_REJECTED,
|
|
||||||
notes=reason,
|
|
||||||
final_file=None,
|
|
||||||
)
|
|
||||||
write_response(qp.queue_dir, response)
|
|
||||||
_write_audit(qp, action=STATUS_REJECTED, notes=reason, diff_before="", diff_after="")
|
|
||||||
|
|
||||||
|
|
||||||
def _apply_pipelock_url(slug: str, failed_url: str) -> tuple[str, str]:
|
|
||||||
"""Merge a pipelock-block failed URL's host into the allowlist."""
|
|
||||||
import urllib.parse
|
|
||||||
|
|
||||||
parsed = urllib.parse.urlsplit(failed_url.strip())
|
|
||||||
host = parsed.hostname or ""
|
|
||||||
if not host:
|
|
||||||
raise PipelockApplyError(
|
|
||||||
f"proposed failed_url has no extractable host: {failed_url!r}"
|
|
||||||
)
|
|
||||||
current = fetch_current_allowlist(slug)
|
|
||||||
hosts = parse_allowlist_content(current)
|
|
||||||
if host not in hosts:
|
|
||||||
hosts.append(host)
|
|
||||||
return apply_allowlist_change(slug, render_allowlist_content(hosts))
|
|
||||||
|
|
||||||
|
|
||||||
def _write_audit(
|
|
||||||
qp: QueuedProposal,
|
|
||||||
*,
|
|
||||||
action: str,
|
|
||||||
notes: str,
|
|
||||||
diff_before: str,
|
|
||||||
diff_after: str,
|
|
||||||
) -> None:
|
|
||||||
"""Audit log for egress / pipelock tools."""
|
|
||||||
component = COMPONENT_FOR_TOOL.get(qp.proposal.tool)
|
|
||||||
if component is None:
|
|
||||||
return
|
|
||||||
write_audit_entry(AuditEntry(
|
|
||||||
timestamp=datetime.now(timezone.utc).isoformat(),
|
|
||||||
bottle_slug=qp.proposal.bottle_slug,
|
|
||||||
component=component,
|
|
||||||
operator_action=action,
|
|
||||||
operator_notes=notes,
|
|
||||||
justification=qp.proposal.justification,
|
|
||||||
diff=render_diff(diff_before, diff_after, label=component),
|
|
||||||
))
|
|
||||||
|
|
||||||
|
|
||||||
# --- $EDITOR integration --------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def edit_in_editor(content: str, *, suffix: str = ".tmp") -> str | None:
|
|
||||||
"""Open `content` in $EDITOR and return edited content, if changed."""
|
|
||||||
editor = os.environ.get("EDITOR", "vim")
|
|
||||||
with tempfile.NamedTemporaryFile(
|
|
||||||
mode="w", suffix=suffix, delete=False, prefix="supervise-modify.",
|
|
||||||
) as f:
|
|
||||||
f.write(content)
|
|
||||||
path = f.name
|
|
||||||
try:
|
|
||||||
subprocess.run([editor, path], check=False)
|
|
||||||
with open(path) as f:
|
|
||||||
edited = f.read()
|
|
||||||
return edited if edited != content else None
|
|
||||||
finally:
|
|
||||||
try:
|
|
||||||
os.unlink(path)
|
|
||||||
except OSError:
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
# --- TUI -------------------------------------------------------------------
|
|
||||||
|
|
||||||
|
|
||||||
def cmd_supervise(argv: list[str]) -> int:
|
|
||||||
parser = argparse.ArgumentParser(prog=f"{PROG} supervise", add_help=True)
|
|
||||||
parser.add_argument(
|
|
||||||
"--once", action="store_true",
|
|
||||||
help="list pending proposals once and exit (no TUI)",
|
|
||||||
)
|
|
||||||
args = parser.parse_args(argv)
|
|
||||||
|
|
||||||
if args.once:
|
|
||||||
return _list_once()
|
|
||||||
try:
|
|
||||||
curses.wrapper(_main_loop)
|
|
||||||
except KeyboardInterrupt:
|
|
||||||
return 130
|
|
||||||
except Die as e:
|
|
||||||
if e.message:
|
|
||||||
error(e.message)
|
|
||||||
else:
|
|
||||||
error("supervise exited on a fatal error (no detail captured).")
|
|
||||||
return e.code if isinstance(e.code, int) else 1
|
|
||||||
except Exception as e:
|
|
||||||
log_path = _write_crash_log(e)
|
|
||||||
error(f"supervise crashed: {type(e).__name__}: {e}")
|
|
||||||
error(f"full traceback written to {log_path}")
|
|
||||||
return 1
|
|
||||||
return 0
|
|
||||||
|
|
||||||
|
|
||||||
def _write_crash_log(exc: BaseException) -> Path:
|
|
||||||
"""Persist `exc`'s traceback to a stable file under ~/.bot-bottle/."""
|
|
||||||
stamp = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
|
||||||
body = "".join(
|
|
||||||
traceback.format_exception(type(exc), exc, exc.__traceback__)
|
|
||||||
)
|
|
||||||
entry = f"=== supervise crash {stamp} ===\n{body}\n"
|
|
||||||
try:
|
|
||||||
log_dir = _supervise.bot_bottle_root() / "logs"
|
|
||||||
log_dir.mkdir(parents=True, exist_ok=True)
|
|
||||||
path = log_dir / "supervise-crash.log"
|
|
||||||
with path.open("a", encoding="utf-8") as fh:
|
|
||||||
fh.write(entry)
|
|
||||||
return path
|
|
||||||
except OSError:
|
|
||||||
fd, tmp = tempfile.mkstemp(
|
|
||||||
prefix="bot-bottle-supervise-crash-", suffix=".log",
|
|
||||||
)
|
|
||||||
with os.fdopen(fd, "w", encoding="utf-8") as fh:
|
|
||||||
fh.write(entry)
|
|
||||||
return Path(tmp)
|
|
||||||
|
|
||||||
|
|
||||||
def _list_once() -> int:
|
|
||||||
pending = discover_pending()
|
|
||||||
if not pending:
|
|
||||||
info("no pending proposals")
|
|
||||||
return 0
|
|
||||||
for qp in pending:
|
|
||||||
sys.stdout.write(
|
|
||||||
f"{qp.proposal.arrival_timestamp} "
|
|
||||||
f"[{qp.proposal.bottle_slug}] "
|
|
||||||
f"{qp.proposal.tool} "
|
|
||||||
f"{qp.proposal.id}\n"
|
|
||||||
)
|
|
||||||
sys.stdout.write(f" {qp.proposal.justification}\n")
|
|
||||||
return 0
|
|
||||||
|
|
||||||
|
|
||||||
def _try_init_green() -> int:
|
|
||||||
"""Initialise a green color pair and return its attr, or 0."""
|
|
||||||
try:
|
|
||||||
curses.start_color()
|
|
||||||
curses.use_default_colors()
|
|
||||||
curses.init_pair(1, curses.COLOR_GREEN, -1)
|
|
||||||
return curses.color_pair(1)
|
|
||||||
except curses.error:
|
|
||||||
return 0
|
|
||||||
|
|
||||||
|
|
||||||
def _main_loop(stdscr: "curses._CursesWindow") -> None:
|
|
||||||
curses.curs_set(0)
|
|
||||||
stdscr.timeout(_REFRESH_INTERVAL_MS)
|
|
||||||
green_attr = _try_init_green()
|
|
||||||
selected = 0
|
|
||||||
status_line = ""
|
|
||||||
seen_ids: set[str] = set()
|
|
||||||
|
|
||||||
while True:
|
|
||||||
pending = discover_pending()
|
|
||||||
if selected >= len(pending):
|
|
||||||
selected = max(0, len(pending) - 1)
|
|
||||||
|
|
||||||
live_ids = {qp.proposal.id for qp in pending}
|
|
||||||
newly_arrived = live_ids - seen_ids
|
|
||||||
if seen_ids and newly_arrived:
|
|
||||||
try:
|
|
||||||
curses.beep()
|
|
||||||
except curses.error:
|
|
||||||
pass
|
|
||||||
for i, qp in enumerate(pending):
|
|
||||||
if qp.proposal.id in newly_arrived:
|
|
||||||
selected = i
|
|
||||||
break
|
|
||||||
seen_ids = live_ids
|
|
||||||
|
|
||||||
_render(
|
|
||||||
stdscr, pending, selected, status_line,
|
|
||||||
green_attr=green_attr,
|
|
||||||
)
|
|
||||||
|
|
||||||
try:
|
|
||||||
key = stdscr.getch()
|
|
||||||
except KeyboardInterrupt:
|
|
||||||
return
|
|
||||||
|
|
||||||
if key == -1:
|
|
||||||
continue
|
|
||||||
|
|
||||||
status_line = ""
|
|
||||||
|
|
||||||
if key in (ord("q"), 27):
|
|
||||||
return
|
|
||||||
|
|
||||||
if not pending:
|
|
||||||
continue
|
|
||||||
qp = pending[selected]
|
|
||||||
|
|
||||||
if key in (curses.KEY_DOWN, ord("j")):
|
|
||||||
selected = min(selected + 1, len(pending) - 1)
|
|
||||||
elif key in (curses.KEY_UP, ord("k")):
|
|
||||||
selected = max(selected - 1, 0)
|
|
||||||
elif key in (curses.KEY_ENTER, 10, 13):
|
|
||||||
_detail_view(stdscr, qp, green_attr=green_attr)
|
|
||||||
elif key == ord("a"):
|
|
||||||
try:
|
|
||||||
approve(qp)
|
|
||||||
status_line = _approval_status(qp, "approved")
|
|
||||||
except ApplyError as e:
|
|
||||||
status_line = f"apply failed: {e}"
|
|
||||||
elif key == ord("m"):
|
|
||||||
edited = _modify(stdscr, qp)
|
|
||||||
if edited is None:
|
|
||||||
status_line = "modify aborted (no change)"
|
|
||||||
else:
|
|
||||||
try:
|
|
||||||
approve(qp, final_file=edited, notes="operator modified before approving")
|
|
||||||
status_line = _approval_status(qp, "modified+approved")
|
|
||||||
except ApplyError as e:
|
|
||||||
status_line = f"apply failed: {e}"
|
|
||||||
elif key == ord("r"):
|
|
||||||
reason = _prompt(stdscr, "reject reason: ")
|
|
||||||
if reason:
|
|
||||||
reject(qp, reason=reason)
|
|
||||||
status_line = f"rejected {qp.proposal.tool} for [{qp.proposal.bottle_slug}]"
|
|
||||||
else:
|
|
||||||
status_line = "reject aborted (empty reason)"
|
|
||||||
|
|
||||||
|
|
||||||
def _render(
|
|
||||||
stdscr: "curses._CursesWindow",
|
|
||||||
pending: list[QueuedProposal],
|
|
||||||
selected: int,
|
|
||||||
status_line: str,
|
|
||||||
*,
|
|
||||||
green_attr: int = 0,
|
|
||||||
) -> None:
|
|
||||||
stdscr.erase()
|
|
||||||
h, w = stdscr.getmaxyx()
|
|
||||||
header = f"bot-bottle supervise ({len(pending)} pending)"
|
|
||||||
stdscr.addnstr(0, 0, header, w - 1, curses.A_BOLD)
|
|
||||||
stdscr.hline(1, 0, curses.ACS_HLINE, w)
|
|
||||||
|
|
||||||
row = 2
|
|
||||||
if not pending:
|
|
||||||
stdscr.addnstr(
|
|
||||||
row, 2,
|
|
||||||
"no pending proposals; agents will queue here when they call a "
|
|
||||||
"supervise tool",
|
|
||||||
w - 4,
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
for i, qp in enumerate(pending):
|
|
||||||
if row >= h - 3:
|
|
||||||
break
|
|
||||||
p = qp.proposal
|
|
||||||
ts_short = (
|
|
||||||
p.arrival_timestamp.split("T", 1)[1][:8]
|
|
||||||
if "T" in p.arrival_timestamp else p.arrival_timestamp
|
|
||||||
)
|
|
||||||
cursor = "> " if i == selected else " "
|
|
||||||
line = (
|
|
||||||
f"{cursor}{ts_short} "
|
|
||||||
f"[{p.bottle_slug}] {p.tool:<18} {p.id[:8]} "
|
|
||||||
f"{_proposed_payload_label(p.tool)}"
|
|
||||||
)
|
|
||||||
attr = curses.A_REVERSE if i == selected else curses.A_NORMAL
|
|
||||||
stdscr.addnstr(row, 0, line, w - 1, attr)
|
|
||||||
row += 1
|
|
||||||
if row >= h - 3:
|
|
||||||
break
|
|
||||||
if p.justification:
|
|
||||||
stdscr.addnstr(row, 4, p.justification[: max(0, w - 5)], w - 5)
|
|
||||||
row += 1
|
|
||||||
|
|
||||||
footer = "[j/k] move [Enter] view [a] approve [m] modify [r] reject [q] quit"
|
|
||||||
stdscr.hline(h - 2, 0, curses.ACS_HLINE, w)
|
|
||||||
stdscr.addnstr(h - 1, 0, footer, w - 1, curses.A_DIM)
|
|
||||||
if status_line:
|
|
||||||
stdscr.addnstr(h - 3, 0, status_line, w - 1, curses.A_BOLD)
|
|
||||||
stdscr.refresh()
|
|
||||||
|
|
||||||
|
|
||||||
def _detail_view(
|
|
||||||
stdscr: "curses._CursesWindow",
|
|
||||||
qp: QueuedProposal,
|
|
||||||
*,
|
|
||||||
green_attr: int = 0,
|
|
||||||
) -> None:
|
|
||||||
"""Render the full proposal. Scrollable. Press q to return."""
|
|
||||||
lines = _detail_lines(qp, green_attr=green_attr)
|
|
||||||
offset = 0
|
|
||||||
while True:
|
|
||||||
stdscr.erase()
|
|
||||||
h, w = stdscr.getmaxyx()
|
|
||||||
for i, (text, attr) in enumerate(lines[offset:offset + h - 1]):
|
|
||||||
stdscr.addnstr(i, 0, text, w - 1, attr)
|
|
||||||
stdscr.addnstr(
|
|
||||||
h - 1, 0,
|
|
||||||
"[j/k] scroll [g/G] top/bottom [a] approve [m] modify [r] reject [q] back",
|
|
||||||
w - 1, curses.A_DIM,
|
|
||||||
)
|
|
||||||
stdscr.refresh()
|
|
||||||
key = stdscr.getch()
|
|
||||||
if key in (ord("q"), 27):
|
|
||||||
return
|
|
||||||
if key in (curses.KEY_DOWN, ord("j")):
|
|
||||||
offset = min(offset + 1, max(0, len(lines) - 1))
|
|
||||||
elif key in (curses.KEY_UP, ord("k")):
|
|
||||||
offset = max(offset - 1, 0)
|
|
||||||
elif key == ord("g"):
|
|
||||||
offset = 0
|
|
||||||
elif key == ord("G"):
|
|
||||||
offset = max(0, len(lines) - 1)
|
|
||||||
elif key == ord("a"):
|
|
||||||
try:
|
|
||||||
approve(qp)
|
|
||||||
except ApplyError:
|
|
||||||
pass
|
|
||||||
return
|
|
||||||
elif key == ord("m"):
|
|
||||||
edited = _modify(stdscr, qp)
|
|
||||||
if edited is not None:
|
|
||||||
try:
|
|
||||||
approve(qp, final_file=edited, notes="operator modified before approving")
|
|
||||||
except ApplyError:
|
|
||||||
pass
|
|
||||||
return
|
|
||||||
elif key == ord("r"):
|
|
||||||
reason = _prompt(stdscr, "reject reason: ")
|
|
||||||
if reason:
|
|
||||||
reject(qp, reason=reason)
|
|
||||||
return
|
|
||||||
|
|
||||||
|
|
||||||
def _modify(stdscr: "curses._CursesWindow", qp: QueuedProposal) -> str | None:
|
|
||||||
"""Suspend curses, open $EDITOR on the proposed file, return edited content."""
|
|
||||||
suffix = _suffix_for_tool(qp.proposal.tool)
|
|
||||||
curses.endwin()
|
|
||||||
try:
|
|
||||||
edited = edit_in_editor(qp.proposal.proposed_file, suffix=suffix)
|
|
||||||
finally:
|
|
||||||
stdscr.refresh()
|
|
||||||
return edited
|
|
||||||
|
|
||||||
|
|
||||||
def _prompt(stdscr: "curses._CursesWindow", label: str) -> str:
|
|
||||||
"""One-line input at the bottom of the screen."""
|
|
||||||
curses.curs_set(1)
|
|
||||||
h, _ = stdscr.getmaxyx()
|
|
||||||
stdscr.move(h - 2, 0)
|
|
||||||
stdscr.clrtoeol()
|
|
||||||
stdscr.addstr(h - 2, 0, label)
|
|
||||||
stdscr.refresh()
|
|
||||||
curses.echo()
|
|
||||||
try:
|
|
||||||
raw = stdscr.getstr(h - 2, len(label), 200)
|
|
||||||
finally:
|
|
||||||
curses.noecho()
|
|
||||||
curses.curs_set(0)
|
|
||||||
return raw.decode("utf-8", errors="replace").strip()
|
|
||||||
|
|
||||||
|
|
||||||
__all__ = [
|
|
||||||
"QueuedProposal",
|
|
||||||
"approve",
|
|
||||||
"cmd_supervise",
|
|
||||||
"discover_pending",
|
|
||||||
"edit_in_editor",
|
|
||||||
"reject",
|
|
||||||
]
|
|
||||||
+22
-57
@@ -122,14 +122,6 @@ def _dummy_exp(now: datetime | None, exp_ts: int | None) -> int:
|
|||||||
return int(check_now.timestamp()) + 3600
|
return int(check_now.timestamp()) + 3600
|
||||||
|
|
||||||
|
|
||||||
def _dummy_timestamp(now: datetime | None = None) -> str:
|
|
||||||
check_now = now or datetime.now(timezone.utc)
|
|
||||||
if check_now.tzinfo is None:
|
|
||||||
check_now = check_now.replace(tzinfo=timezone.utc)
|
|
||||||
check_now = check_now.astimezone(timezone.utc)
|
|
||||||
return check_now.isoformat(timespec="milliseconds").replace("+00:00", "Z")
|
|
||||||
|
|
||||||
|
|
||||||
def _dummy_jwt(now: datetime | None = None, *, exp_ts: int | None = None) -> str:
|
def _dummy_jwt(now: datetime | None = None, *, exp_ts: int | None = None) -> str:
|
||||||
return _encode_dummy_jwt({
|
return _encode_dummy_jwt({
|
||||||
"exp": _dummy_exp(now, exp_ts),
|
"exp": _dummy_exp(now, exp_ts),
|
||||||
@@ -195,10 +187,8 @@ def _redact_claims(value: object) -> object:
|
|||||||
out[key] = inner if isinstance(inner, list) else []
|
out[key] = inner if isinstance(inner, list) else []
|
||||||
elif isinstance(inner, bool):
|
elif isinstance(inner, bool):
|
||||||
out[key] = inner
|
out[key] = inner
|
||||||
elif isinstance(inner, dict):
|
elif isinstance(inner, (dict, list)):
|
||||||
out[key] = {}
|
out[key] = _redact_claims(inner)
|
||||||
elif isinstance(inner, list):
|
|
||||||
out[key] = []
|
|
||||||
else:
|
else:
|
||||||
out[key] = "bot-bottle-placeholder"
|
out[key] = "bot-bottle-placeholder"
|
||||||
return out
|
return out
|
||||||
@@ -247,53 +237,28 @@ def _redact_auth_claim(value: object) -> dict:
|
|||||||
def _redact_codex_auth(
|
def _redact_codex_auth(
|
||||||
value: object, *, now: datetime | None = None, exp_ts: int | None = None,
|
value: object, *, now: datetime | None = None, exp_ts: int | None = None,
|
||||||
) -> object:
|
) -> object:
|
||||||
auth = value if isinstance(value, dict) else {}
|
|
||||||
out: dict[str, object] = {}
|
|
||||||
for key, inner in auth.items():
|
|
||||||
lower = key.lower()
|
|
||||||
if lower == "auth_mode" and isinstance(inner, str) and inner:
|
|
||||||
out[key] = inner
|
|
||||||
elif lower == "openai_api_key":
|
|
||||||
out[key] = None
|
|
||||||
elif lower == "last_refresh":
|
|
||||||
# Codex parses this as a timestamp on startup. Keep the
|
|
||||||
# schema valid without copying host-side session metadata.
|
|
||||||
out[key] = _dummy_timestamp(now)
|
|
||||||
elif lower == "tokens":
|
|
||||||
out[key] = _redact_token_block(inner, now=now, exp_ts=exp_ts)
|
|
||||||
else:
|
|
||||||
out[key] = _redact_unknown_auth_value(inner)
|
|
||||||
return out
|
|
||||||
|
|
||||||
|
|
||||||
def _redact_token_block(
|
|
||||||
value: object, *, now: datetime | None = None, exp_ts: int | None = None,
|
|
||||||
) -> dict[str, object]:
|
|
||||||
tokens = value if isinstance(value, dict) else {}
|
|
||||||
out: dict[str, object] = {}
|
|
||||||
for key, inner in tokens.items():
|
|
||||||
lower = key.lower()
|
|
||||||
if lower in {"access_token", "id_token"}:
|
|
||||||
out[key] = _dummy_jwt_from_host(inner, now=now, exp_ts=exp_ts)
|
|
||||||
elif lower == "account_id" and isinstance(inner, str) and inner:
|
|
||||||
# Current Codex uses this non-secret selected account id
|
|
||||||
# while egress owns the real bearer token.
|
|
||||||
out[key] = inner
|
|
||||||
else:
|
|
||||||
out[key] = _redact_unknown_auth_value(inner)
|
|
||||||
return out
|
|
||||||
|
|
||||||
|
|
||||||
def _redact_unknown_auth_value(value: object) -> object:
|
|
||||||
if isinstance(value, bool):
|
|
||||||
return value
|
|
||||||
if isinstance(value, dict):
|
if isinstance(value, dict):
|
||||||
return {}
|
out: dict[str, object] = {}
|
||||||
|
for key, inner in value.items():
|
||||||
|
lower = key.lower()
|
||||||
|
if lower == "openai_api_key":
|
||||||
|
out[key] = None
|
||||||
|
elif lower == "tokens":
|
||||||
|
out[key] = _redact_codex_auth(inner, now=now, exp_ts=exp_ts)
|
||||||
|
elif lower in {"access_token", "id_token"}:
|
||||||
|
out[key] = _dummy_jwt_from_host(inner, now=now, exp_ts=exp_ts)
|
||||||
|
elif "token" in lower or "secret" in lower or lower.endswith("_key"):
|
||||||
|
out[key] = "bot-bottle-placeholder"
|
||||||
|
elif lower == "account_id" and isinstance(inner, str) and inner:
|
||||||
|
out[key] = inner
|
||||||
|
elif lower in {"account_id", "user_id", "email"}:
|
||||||
|
out[key] = "bot-bottle-placeholder"
|
||||||
|
else:
|
||||||
|
out[key] = _redact_codex_auth(inner, now=now, exp_ts=exp_ts)
|
||||||
|
return out
|
||||||
if isinstance(value, list):
|
if isinstance(value, list):
|
||||||
return []
|
return [_redact_codex_auth(v, now=now, exp_ts=exp_ts) for v in value]
|
||||||
if value is None:
|
return value
|
||||||
return None
|
|
||||||
return "bot-bottle-placeholder"
|
|
||||||
|
|
||||||
|
|
||||||
def _jwt_exp(token: str) -> datetime | None:
|
def _jwt_exp(token: str) -> datetime | None:
|
||||||
|
|||||||
@@ -1,226 +0,0 @@
|
|||||||
"""Claude agent provider plugin (PRD 0050, contrib).
|
|
||||||
|
|
||||||
The Claude-specific behavior previously inlined under
|
|
||||||
`agent_provider.agent_provision_plan` (claude.json trust marker,
|
|
||||||
api.anthropic.com egress route, OAuth-token placeholder), plus
|
|
||||||
the `claude mcp add` invocation that registers the supervise
|
|
||||||
sidecar in claude-code's user config (PRD 0013)."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import shlex
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import TYPE_CHECKING
|
|
||||||
|
|
||||||
from ...agent_provider import (
|
|
||||||
AgentProvider,
|
|
||||||
AgentProviderRuntime,
|
|
||||||
AgentProvisionFile,
|
|
||||||
AgentProvisionPlan,
|
|
||||||
)
|
|
||||||
from ...egress import EgressRoute
|
|
||||||
from ...log import die, info, warn
|
|
||||||
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
|
||||||
from ...backend import Bottle, BottlePlan
|
|
||||||
|
|
||||||
|
|
||||||
_REPO_ROOT = Path(__file__).resolve().parents[3]
|
|
||||||
|
|
||||||
_SUPERVISE_MCP_NAME = "supervise"
|
|
||||||
|
|
||||||
|
|
||||||
def _skills_dir(guest_home: str) -> str:
|
|
||||||
return f"{guest_home}/.claude/skills"
|
|
||||||
|
|
||||||
|
|
||||||
def _prompt_path(guest_home: str) -> str:
|
|
||||||
return f"{guest_home}/.bot-bottle-prompt.txt"
|
|
||||||
|
|
||||||
_RUNTIME = AgentProviderRuntime(
|
|
||||||
template="claude",
|
|
||||||
command="claude",
|
|
||||||
image="bot-bottle-claude:latest",
|
|
||||||
dockerfile=str(_REPO_ROOT / "Dockerfile.claude"),
|
|
||||||
prompt_mode="append_file",
|
|
||||||
bypass_args=("--dangerously-skip-permissions",),
|
|
||||||
resume_args=("--continue",),
|
|
||||||
remote_control_args=("--remote-control",),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class ClaudeAgentProvider(AgentProvider):
|
|
||||||
@property
|
|
||||||
def runtime(self) -> AgentProviderRuntime:
|
|
||||||
return _RUNTIME
|
|
||||||
|
|
||||||
def provision_plan(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
dockerfile: str,
|
|
||||||
state_dir: Path,
|
|
||||||
guest_home: str,
|
|
||||||
guest_env: dict[str, str] | None = None,
|
|
||||||
auth_token: str = "",
|
|
||||||
forward_host_credentials: bool = False,
|
|
||||||
host_env: dict[str, str] | None = None,
|
|
||||||
trusted_project_path: str = "",
|
|
||||||
) -> AgentProvisionPlan:
|
|
||||||
del forward_host_credentials, host_env # Codex-only knobs
|
|
||||||
resolved_guest_env = dict(guest_env or {})
|
|
||||||
trusted_path = trusted_project_path or guest_home
|
|
||||||
|
|
||||||
env_vars: dict[str, str] = {
|
|
||||||
"CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC": "1",
|
|
||||||
"DISABLE_ERROR_REPORTING": "1",
|
|
||||||
}
|
|
||||||
claude_config = state_dir / "claude.json"
|
|
||||||
claude_projects = {guest_home: {"hasTrustDialogAccepted": True}}
|
|
||||||
claude_projects[trusted_path] = {"hasTrustDialogAccepted": True}
|
|
||||||
claude_config.write_text(json.dumps({
|
|
||||||
"hasCompletedOnboarding": True,
|
|
||||||
"theme": "dark",
|
|
||||||
"bypassPermissionsModeAccepted": True,
|
|
||||||
"projects": claude_projects,
|
|
||||||
}, indent=2) + "\n")
|
|
||||||
claude_config.chmod(0o600)
|
|
||||||
files = (
|
|
||||||
AgentProvisionFile(claude_config, f"{guest_home}/.claude.json"),
|
|
||||||
)
|
|
||||||
egress_routes = (EgressRoute(
|
|
||||||
host="api.anthropic.com",
|
|
||||||
auth_scheme="Bearer" if auth_token else "",
|
|
||||||
token_ref=auth_token,
|
|
||||||
tls_passthrough=True,
|
|
||||||
),)
|
|
||||||
hidden_env_names: frozenset[str] = frozenset()
|
|
||||||
if auth_token:
|
|
||||||
env_vars["CLAUDE_CODE_OAUTH_TOKEN"] = "egress-placeholder"
|
|
||||||
hidden_env_names = frozenset({"CLAUDE_CODE_OAUTH_TOKEN"})
|
|
||||||
|
|
||||||
return AgentProvisionPlan(
|
|
||||||
template=_RUNTIME.template,
|
|
||||||
command=_RUNTIME.command,
|
|
||||||
prompt_mode=_RUNTIME.prompt_mode,
|
|
||||||
image=_RUNTIME.image,
|
|
||||||
dockerfile=dockerfile,
|
|
||||||
env_vars=env_vars,
|
|
||||||
guest_env=resolved_guest_env,
|
|
||||||
files=files,
|
|
||||||
egress_routes=egress_routes,
|
|
||||||
hidden_env_names=hidden_env_names,
|
|
||||||
)
|
|
||||||
|
|
||||||
def provision_skills(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
|
||||||
"""Copy each named skill tree from `~/.claude/skills/<name>/`
|
|
||||||
on the host into the guest's claude-code skills dir. No-op
|
|
||||||
when the agent has no skills."""
|
|
||||||
from ...backend.util import host_skill_dir
|
|
||||||
|
|
||||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
|
||||||
if not agent.skills:
|
|
||||||
return
|
|
||||||
skills_dir = _skills_dir(plan.guest_home)
|
|
||||||
bottle.exec(f"mkdir -p {skills_dir}", user="root")
|
|
||||||
for name in agent.skills:
|
|
||||||
src = host_skill_dir(name)
|
|
||||||
if not os.path.isdir(src):
|
|
||||||
die(
|
|
||||||
f"skill {name!r} disappeared from host between "
|
|
||||||
f"validation and copy at {src}."
|
|
||||||
)
|
|
||||||
dst = f"{skills_dir}/{name}"
|
|
||||||
info(f"copying skill {name} into {bottle.name}:{dst}")
|
|
||||||
bottle.exec(f"rm -rf {dst} && mkdir -p {dst}", user="root")
|
|
||||||
bottle.cp_in(f"{src}/.", f"{dst}/")
|
|
||||||
bottle.exec(f"chown -R node:node {dst}", user="root")
|
|
||||||
|
|
||||||
def provision_prompt(self, plan: "BottlePlan", bottle: "Bottle") -> str | None:
|
|
||||||
"""Copy the prompt file into the guest, fix ownership/mode.
|
|
||||||
Returns the in-guest path iff the agent has a non-empty
|
|
||||||
prompt (drives `--append-system-prompt-file`); the file is
|
|
||||||
copied either way so the path always exists."""
|
|
||||||
prompt_path = _prompt_path(plan.guest_home)
|
|
||||||
bottle.cp_in(str(plan.prompt_file), prompt_path)
|
|
||||||
bottle.exec(
|
|
||||||
f"chown node:node {prompt_path} && chmod 600 {prompt_path}",
|
|
||||||
user="root",
|
|
||||||
)
|
|
||||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
|
||||||
return prompt_path if agent.prompt else None
|
|
||||||
|
|
||||||
def provision(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
|
||||||
"""Apply the claude-side declarative provision steps from
|
|
||||||
`plan.agent_provision` — today that's the `claude.json`
|
|
||||||
trust-marker file. Hot-replace this with a richer flow as
|
|
||||||
claude-code's harness shape evolves."""
|
|
||||||
provision = plan.agent_provision
|
|
||||||
for d in provision.dirs:
|
|
||||||
path = shlex.quote(d.guest_path)
|
|
||||||
_exec(bottle, f"mkdir -p {path}", f"could not create {d.guest_path}")
|
|
||||||
_exec(
|
|
||||||
bottle,
|
|
||||||
f"chown {shlex.quote(d.owner)} {path}",
|
|
||||||
f"could not chown {d.guest_path}",
|
|
||||||
)
|
|
||||||
_exec(
|
|
||||||
bottle,
|
|
||||||
f"chmod {shlex.quote(d.mode)} {path}",
|
|
||||||
f"could not chmod {d.guest_path}",
|
|
||||||
)
|
|
||||||
for command in provision.pre_copy:
|
|
||||||
_exec(bottle, shlex.join(command.argv), command.error)
|
|
||||||
for f in provision.files:
|
|
||||||
bottle.cp_in(str(f.host_path), f.guest_path)
|
|
||||||
path = shlex.quote(f.guest_path)
|
|
||||||
_exec(
|
|
||||||
bottle,
|
|
||||||
f"chown {shlex.quote(f.owner)} {path}",
|
|
||||||
f"could not chown {f.guest_path}",
|
|
||||||
)
|
|
||||||
_exec(
|
|
||||||
bottle,
|
|
||||||
f"chmod {shlex.quote(f.mode)} {path}",
|
|
||||||
f"could not chmod {f.guest_path}",
|
|
||||||
)
|
|
||||||
for command in provision.verify:
|
|
||||||
_exec(bottle, shlex.join(command.argv), command.error)
|
|
||||||
|
|
||||||
def provision_supervise_mcp(
|
|
||||||
self,
|
|
||||||
plan: "BottlePlan",
|
|
||||||
bottle: "Bottle",
|
|
||||||
supervise_url: str,
|
|
||||||
) -> None:
|
|
||||||
"""Run `claude mcp add` inside the agent guest to register the
|
|
||||||
supervise sidecar in claude-code's user config (~/.claude.json).
|
|
||||||
|
|
||||||
Failure is logged but not fatal — the bottle still works without
|
|
||||||
the entry; the operator can register it manually."""
|
|
||||||
if plan.supervise_plan is None:
|
|
||||||
return
|
|
||||||
info(f"registering supervise MCP server in agent claude config → {supervise_url}")
|
|
||||||
r = bottle.exec(
|
|
||||||
f"claude mcp add --scope user --transport http "
|
|
||||||
f"{_SUPERVISE_MCP_NAME} {supervise_url}",
|
|
||||||
user="node",
|
|
||||||
)
|
|
||||||
if r.returncode != 0:
|
|
||||||
warn(
|
|
||||||
f"`claude mcp add supervise` failed (exit {r.returncode}): "
|
|
||||||
f"{(r.stderr or r.stdout or '').strip()}. Inside the bottle, "
|
|
||||||
f"register manually with: "
|
|
||||||
f"claude mcp add --scope user --transport http supervise {supervise_url}"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _exec(bottle: "Bottle", script: str, error: str) -> None:
|
|
||||||
result = bottle.exec(script, user="root")
|
|
||||||
if result.returncode != 0:
|
|
||||||
detail = (result.stderr or result.stdout).strip()
|
|
||||||
if detail:
|
|
||||||
detail = f": {detail}"
|
|
||||||
die(f"agent provider provisioning: {error}{detail}")
|
|
||||||
@@ -1,271 +0,0 @@
|
|||||||
"""Codex agent provider plugin (PRD 0050, contrib).
|
|
||||||
|
|
||||||
The Codex-specific behavior previously inlined under
|
|
||||||
`agent_provider.agent_provision_plan` (config.toml trust marker,
|
|
||||||
chatgpt.com / api.openai.com egress routes, optional host-credential
|
|
||||||
forwarding with dummy-auth.json + verify), plus the `codex mcp add`
|
|
||||||
invocation that registers the supervise sidecar in Codex's
|
|
||||||
~/.codex/config.toml (PRD 0050)."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import os
|
|
||||||
import shlex
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import TYPE_CHECKING
|
|
||||||
|
|
||||||
from ...agent_provider import (
|
|
||||||
CODEX_HOST_CREDENTIAL_HOSTS,
|
|
||||||
AgentProvider,
|
|
||||||
AgentProviderRuntime,
|
|
||||||
AgentProvisionCommand,
|
|
||||||
AgentProvisionDir,
|
|
||||||
AgentProvisionFile,
|
|
||||||
AgentProvisionPlan,
|
|
||||||
)
|
|
||||||
from ...codex_auth import codex_host_access_token, write_codex_dummy_auth_file
|
|
||||||
from ...egress import CODEX_HOST_CREDENTIAL_TOKEN_REF, EgressRoute
|
|
||||||
from ...log import die, info, warn
|
|
||||||
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
|
||||||
from ...backend import Bottle, BottlePlan
|
|
||||||
|
|
||||||
|
|
||||||
_REPO_ROOT = Path(__file__).resolve().parents[3]
|
|
||||||
|
|
||||||
_SUPERVISE_MCP_NAME = "supervise"
|
|
||||||
|
|
||||||
|
|
||||||
def _skills_dir(guest_home: str) -> str:
|
|
||||||
# Codex agents still read skills from the claude-code convention
|
|
||||||
# (~/.claude/skills/) — the bot-bottle-codex image follows the
|
|
||||||
# same layout. If Codex grows native skill discovery later,
|
|
||||||
# change here.
|
|
||||||
return f"{guest_home}/.claude/skills"
|
|
||||||
|
|
||||||
|
|
||||||
def _prompt_path(guest_home: str) -> str:
|
|
||||||
return f"{guest_home}/.bot-bottle-prompt.txt"
|
|
||||||
|
|
||||||
_RUNTIME = AgentProviderRuntime(
|
|
||||||
template="codex",
|
|
||||||
command="codex",
|
|
||||||
image="bot-bottle-codex:latest",
|
|
||||||
dockerfile=str(_REPO_ROOT / "Dockerfile.codex"),
|
|
||||||
prompt_mode="read_prompt_file",
|
|
||||||
bypass_args=("--dangerously-bypass-approvals-and-sandbox",),
|
|
||||||
resume_args=("resume", "--last"),
|
|
||||||
remote_control_args=(),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class CodexAgentProvider(AgentProvider):
|
|
||||||
@property
|
|
||||||
def runtime(self) -> AgentProviderRuntime:
|
|
||||||
return _RUNTIME
|
|
||||||
|
|
||||||
def provision_plan(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
dockerfile: str,
|
|
||||||
state_dir: Path,
|
|
||||||
guest_home: str,
|
|
||||||
guest_env: dict[str, str] | None = None,
|
|
||||||
auth_token: str = "",
|
|
||||||
forward_host_credentials: bool = False,
|
|
||||||
host_env: dict[str, str] | None = None,
|
|
||||||
trusted_project_path: str = "",
|
|
||||||
) -> AgentProvisionPlan:
|
|
||||||
del auth_token # Claude-only knob
|
|
||||||
resolved_guest_env = dict(guest_env or {})
|
|
||||||
trusted_path = trusted_project_path or guest_home
|
|
||||||
|
|
||||||
env_vars: dict[str, str] = {
|
|
||||||
"CODEX_CA_CERTIFICATE": "/etc/ssl/certs/ca-certificates.crt",
|
|
||||||
}
|
|
||||||
auth_dir = resolved_guest_env.get("CODEX_HOME", f"{guest_home}/.codex")
|
|
||||||
if forward_host_credentials:
|
|
||||||
env_vars["CODEX_HOME"] = auth_dir
|
|
||||||
|
|
||||||
dirs = [AgentProvisionDir(auth_dir)]
|
|
||||||
files: list[AgentProvisionFile] = []
|
|
||||||
pre_copy: list[AgentProvisionCommand] = []
|
|
||||||
verify: list[AgentProvisionCommand] = []
|
|
||||||
provisioned_env: dict[str, str] = {}
|
|
||||||
|
|
||||||
config_path = f"{auth_dir}/config.toml"
|
|
||||||
config_file = state_dir / "codex-config.toml"
|
|
||||||
toml_path = trusted_path.replace("\\", "\\\\").replace('"', '\\"')
|
|
||||||
config_file.write_text(
|
|
||||||
f'[projects."{toml_path}"]\n'
|
|
||||||
'trust_level = "trusted"\n'
|
|
||||||
)
|
|
||||||
config_file.chmod(0o600)
|
|
||||||
files.append(AgentProvisionFile(config_file, config_path))
|
|
||||||
|
|
||||||
egress_routes: list[EgressRoute] = []
|
|
||||||
for host in CODEX_HOST_CREDENTIAL_HOSTS:
|
|
||||||
egress_routes.append(EgressRoute(
|
|
||||||
host=host,
|
|
||||||
auth_scheme="Bearer" if forward_host_credentials else "",
|
|
||||||
token_ref=CODEX_HOST_CREDENTIAL_TOKEN_REF if forward_host_credentials else "",
|
|
||||||
tls_passthrough=True,
|
|
||||||
))
|
|
||||||
|
|
||||||
if forward_host_credentials:
|
|
||||||
_host_env = host_env or dict(os.environ)
|
|
||||||
provisioned_env[CODEX_HOST_CREDENTIAL_TOKEN_REF] = (
|
|
||||||
codex_host_access_token(_host_env)
|
|
||||||
)
|
|
||||||
auth_file = state_dir / "codex-auth.json"
|
|
||||||
write_codex_dummy_auth_file(auth_file, _host_env)
|
|
||||||
files.append(AgentProvisionFile(auth_file, f"{auth_dir}/auth.json"))
|
|
||||||
pre_copy.append(AgentProvisionCommand((
|
|
||||||
"find", auth_dir,
|
|
||||||
"-maxdepth", "1",
|
|
||||||
"-type", "f",
|
|
||||||
"(",
|
|
||||||
"-name", "*.sqlite",
|
|
||||||
"-o", "-name", "*.sqlite-*",
|
|
||||||
"-o", "-name", "*.codex-repair-*.bak",
|
|
||||||
")",
|
|
||||||
"-delete",
|
|
||||||
), "codex host credentials: could not reset runtime db files"))
|
|
||||||
verify.append(AgentProvisionCommand((
|
|
||||||
"runuser", "-u", "node", "--",
|
|
||||||
"env",
|
|
||||||
f"HOME={guest_home}",
|
|
||||||
f"CODEX_HOME={auth_dir}",
|
|
||||||
"codex", "login", "status",
|
|
||||||
), (
|
|
||||||
"codex host credentials: dummy auth was copied into the "
|
|
||||||
"guest, but Codex did not accept it"
|
|
||||||
)))
|
|
||||||
|
|
||||||
return AgentProvisionPlan(
|
|
||||||
template=_RUNTIME.template,
|
|
||||||
command=_RUNTIME.command,
|
|
||||||
prompt_mode=_RUNTIME.prompt_mode,
|
|
||||||
image=_RUNTIME.image,
|
|
||||||
dockerfile=dockerfile,
|
|
||||||
env_vars=env_vars,
|
|
||||||
guest_env=resolved_guest_env,
|
|
||||||
dirs=tuple(dirs),
|
|
||||||
files=tuple(files),
|
|
||||||
pre_copy=tuple(pre_copy),
|
|
||||||
verify=tuple(verify),
|
|
||||||
egress_routes=tuple(egress_routes),
|
|
||||||
provisioned_env=provisioned_env,
|
|
||||||
)
|
|
||||||
|
|
||||||
def provision_skills(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
|
||||||
"""Copy each named skill tree from `~/.claude/skills/<name>/`
|
|
||||||
on the host into the guest. No-op when the agent has no
|
|
||||||
skills."""
|
|
||||||
from ...backend.util import host_skill_dir
|
|
||||||
|
|
||||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
|
||||||
if not agent.skills:
|
|
||||||
return
|
|
||||||
skills_dir = _skills_dir(plan.guest_home)
|
|
||||||
bottle.exec(f"mkdir -p {skills_dir}", user="root")
|
|
||||||
for name in agent.skills:
|
|
||||||
src = host_skill_dir(name)
|
|
||||||
if not os.path.isdir(src):
|
|
||||||
die(
|
|
||||||
f"skill {name!r} disappeared from host between "
|
|
||||||
f"validation and copy at {src}."
|
|
||||||
)
|
|
||||||
dst = f"{skills_dir}/{name}"
|
|
||||||
info(f"copying skill {name} into {bottle.name}:{dst}")
|
|
||||||
bottle.exec(f"rm -rf {dst} && mkdir -p {dst}", user="root")
|
|
||||||
bottle.cp_in(f"{src}/.", f"{dst}/")
|
|
||||||
bottle.exec(f"chown -R node:node {dst}", user="root")
|
|
||||||
|
|
||||||
def provision_prompt(self, plan: "BottlePlan", bottle: "Bottle") -> str | None:
|
|
||||||
"""Copy the prompt file into the guest, fix ownership/mode.
|
|
||||||
Codex reads it via the agent's `Read and follow the
|
|
||||||
instructions in <path>.` bootstrap (see `prompt_args`); the
|
|
||||||
file is copied either way so the path always exists."""
|
|
||||||
prompt_path = _prompt_path(plan.guest_home)
|
|
||||||
bottle.cp_in(str(plan.prompt_file), prompt_path)
|
|
||||||
bottle.exec(
|
|
||||||
f"chown node:node {prompt_path} && chmod 600 {prompt_path}",
|
|
||||||
user="root",
|
|
||||||
)
|
|
||||||
agent = plan.spec.manifest.agents[plan.spec.agent_name]
|
|
||||||
return prompt_path if agent.prompt else None
|
|
||||||
|
|
||||||
def provision(self, plan: "BottlePlan", bottle: "Bottle") -> None:
|
|
||||||
"""Apply the codex-side declarative provision steps from
|
|
||||||
`plan.agent_provision`: the `~/.codex/` dir + config.toml
|
|
||||||
trust marker, plus the dummy-auth.json drop + `codex login
|
|
||||||
status` verify when host-credential forwarding is on."""
|
|
||||||
provision = plan.agent_provision
|
|
||||||
for d in provision.dirs:
|
|
||||||
path = shlex.quote(d.guest_path)
|
|
||||||
_exec(bottle, f"mkdir -p {path}", f"could not create {d.guest_path}")
|
|
||||||
_exec(
|
|
||||||
bottle,
|
|
||||||
f"chown {shlex.quote(d.owner)} {path}",
|
|
||||||
f"could not chown {d.guest_path}",
|
|
||||||
)
|
|
||||||
_exec(
|
|
||||||
bottle,
|
|
||||||
f"chmod {shlex.quote(d.mode)} {path}",
|
|
||||||
f"could not chmod {d.guest_path}",
|
|
||||||
)
|
|
||||||
for command in provision.pre_copy:
|
|
||||||
_exec(bottle, shlex.join(command.argv), command.error)
|
|
||||||
for f in provision.files:
|
|
||||||
bottle.cp_in(str(f.host_path), f.guest_path)
|
|
||||||
path = shlex.quote(f.guest_path)
|
|
||||||
_exec(
|
|
||||||
bottle,
|
|
||||||
f"chown {shlex.quote(f.owner)} {path}",
|
|
||||||
f"could not chown {f.guest_path}",
|
|
||||||
)
|
|
||||||
_exec(
|
|
||||||
bottle,
|
|
||||||
f"chmod {shlex.quote(f.mode)} {path}",
|
|
||||||
f"could not chmod {f.guest_path}",
|
|
||||||
)
|
|
||||||
for command in provision.verify:
|
|
||||||
_exec(bottle, shlex.join(command.argv), command.error)
|
|
||||||
|
|
||||||
def provision_supervise_mcp(
|
|
||||||
self,
|
|
||||||
plan: "BottlePlan",
|
|
||||||
bottle: "Bottle",
|
|
||||||
supervise_url: str,
|
|
||||||
) -> None:
|
|
||||||
"""Run `codex mcp add` inside the agent guest to register the
|
|
||||||
supervise sidecar in Codex's user config (~/.codex/config.toml).
|
|
||||||
|
|
||||||
Mirrors the Claude provider's `claude mcp add` flow — failure
|
|
||||||
is logged but not fatal."""
|
|
||||||
if plan.supervise_plan is None:
|
|
||||||
return
|
|
||||||
info(f"registering supervise MCP server in agent codex config → {supervise_url}")
|
|
||||||
r = bottle.exec(
|
|
||||||
f"codex mcp add --transport http "
|
|
||||||
f"{_SUPERVISE_MCP_NAME} {supervise_url}",
|
|
||||||
user="node",
|
|
||||||
)
|
|
||||||
if r.returncode != 0:
|
|
||||||
warn(
|
|
||||||
f"`codex mcp add supervise` failed (exit {r.returncode}): "
|
|
||||||
f"{(r.stderr or r.stdout or '').strip()}. Inside the bottle, "
|
|
||||||
f"register manually with: "
|
|
||||||
f"codex mcp add --transport http supervise {supervise_url}"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _exec(bottle: "Bottle", script: str, error: str) -> None:
|
|
||||||
result = bottle.exec(script, user="root")
|
|
||||||
if result.returncode != 0:
|
|
||||||
detail = (result.stderr or result.stdout).strip()
|
|
||||||
if detail:
|
|
||||||
detail = f": {detail}"
|
|
||||||
die(f"agent provider provisioning: {error}{detail}")
|
|
||||||
@@ -1,121 +0,0 @@
|
|||||||
"""Gitea deploy-key provisioner (PRD 0048, contrib).
|
|
||||||
|
|
||||||
Generates ed25519 keypairs via `ssh-keygen` and registers / deletes
|
|
||||||
them using the Gitea deploy-key HTTP API. No new Python dependencies —
|
|
||||||
only stdlib `urllib.request` and `subprocess`."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import json
|
|
||||||
import subprocess
|
|
||||||
import tempfile
|
|
||||||
import urllib.error
|
|
||||||
import urllib.request
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
from ...deploy_key_provisioner import DeployKeyProvisioner
|
|
||||||
|
|
||||||
|
|
||||||
class GiteaDeployKeyProvisioner(DeployKeyProvisioner):
|
|
||||||
"""Manages deploy keys on a Gitea instance."""
|
|
||||||
|
|
||||||
def __init__(self, *, token: str, api_url: str) -> None:
|
|
||||||
self._token = token
|
|
||||||
self._api_url = api_url.rstrip("/")
|
|
||||||
|
|
||||||
def create(self, owner_repo: str, title: str) -> tuple[str, bytes]:
|
|
||||||
"""Generate an ed25519 keypair, register the public half as a
|
|
||||||
repo deploy key, and return `(key_id, private_key_bytes)`.
|
|
||||||
|
|
||||||
The key is registered with `read_only=False` because git-gate
|
|
||||||
needs push access to forward gitleaks-scanned refs upstream."""
|
|
||||||
with tempfile.TemporaryDirectory() as tmpdir:
|
|
||||||
key_path = Path(tmpdir) / "key"
|
|
||||||
subprocess.run(
|
|
||||||
[
|
|
||||||
"ssh-keygen", "-t", "ed25519",
|
|
||||||
"-f", str(key_path),
|
|
||||||
"-N", "",
|
|
||||||
],
|
|
||||||
check=True,
|
|
||||||
stdout=subprocess.DEVNULL,
|
|
||||||
stderr=subprocess.DEVNULL,
|
|
||||||
)
|
|
||||||
private_key = key_path.read_bytes()
|
|
||||||
public_key = key_path.with_suffix(".pub").read_text().strip()
|
|
||||||
|
|
||||||
owner, repo = _split_owner_repo(owner_repo)
|
|
||||||
url = f"{self._api_url}/api/v1/repos/{owner}/{repo}/keys"
|
|
||||||
payload = json.dumps({
|
|
||||||
"key": public_key,
|
|
||||||
"read_only": False,
|
|
||||||
"title": title,
|
|
||||||
}).encode()
|
|
||||||
req = urllib.request.Request(
|
|
||||||
url,
|
|
||||||
data=payload,
|
|
||||||
headers={
|
|
||||||
"Authorization": f"token {self._token}",
|
|
||||||
"Content-Type": "application/json",
|
|
||||||
},
|
|
||||||
method="POST",
|
|
||||||
)
|
|
||||||
try:
|
|
||||||
with urllib.request.urlopen(req) as resp:
|
|
||||||
body = json.loads(resp.read())
|
|
||||||
except urllib.error.HTTPError as exc:
|
|
||||||
_body = _read_error_body(exc)
|
|
||||||
raise RuntimeError(
|
|
||||||
f"failed to create deploy key for {owner_repo}: "
|
|
||||||
f"HTTP {exc.code} — {_body}"
|
|
||||||
) from exc
|
|
||||||
except urllib.error.URLError as exc:
|
|
||||||
raise RuntimeError(
|
|
||||||
f"failed to create deploy key for {owner_repo}: {exc.reason}"
|
|
||||||
) from exc
|
|
||||||
|
|
||||||
return str(body["id"]), private_key
|
|
||||||
|
|
||||||
def delete(self, owner_repo: str, key_id: str) -> None:
|
|
||||||
"""Delete the deploy key. HTTP 404 (already gone) is success.
|
|
||||||
All other errors raise RuntimeError so teardown halts."""
|
|
||||||
owner, repo = _split_owner_repo(owner_repo)
|
|
||||||
url = f"{self._api_url}/api/v1/repos/{owner}/{repo}/keys/{key_id}"
|
|
||||||
req = urllib.request.Request(
|
|
||||||
url,
|
|
||||||
headers={"Authorization": f"token {self._token}"},
|
|
||||||
method="DELETE",
|
|
||||||
)
|
|
||||||
try:
|
|
||||||
with urllib.request.urlopen(req):
|
|
||||||
pass
|
|
||||||
except urllib.error.HTTPError as exc:
|
|
||||||
if exc.code == 404:
|
|
||||||
return
|
|
||||||
_body = _read_error_body(exc)
|
|
||||||
raise RuntimeError(
|
|
||||||
f"failed to delete deploy key {key_id} for {owner_repo}: "
|
|
||||||
f"HTTP {exc.code} — {_body}"
|
|
||||||
) from exc
|
|
||||||
except urllib.error.URLError as exc:
|
|
||||||
raise RuntimeError(
|
|
||||||
f"failed to delete deploy key {key_id} for {owner_repo}: "
|
|
||||||
f"{exc.reason}"
|
|
||||||
) from exc
|
|
||||||
|
|
||||||
|
|
||||||
def _split_owner_repo(owner_repo: str) -> tuple[str, str]:
|
|
||||||
"""Split `'owner/repo'` into `('owner', 'repo')`."""
|
|
||||||
parts = owner_repo.split("/", 1)
|
|
||||||
if len(parts) != 2 or not all(parts):
|
|
||||||
raise ValueError(
|
|
||||||
f"expected 'owner/repo' format, got {owner_repo!r}"
|
|
||||||
)
|
|
||||||
return parts[0], parts[1]
|
|
||||||
|
|
||||||
|
|
||||||
def _read_error_body(exc: urllib.error.HTTPError) -> str:
|
|
||||||
try:
|
|
||||||
return exc.read().decode("utf-8", errors="replace")
|
|
||||||
except Exception:
|
|
||||||
return ""
|
|
||||||
@@ -1,52 +0,0 @@
|
|||||||
"""Deploy-key provisioner interface and factory (PRD 0048).
|
|
||||||
|
|
||||||
The core defines the abstract contract; concrete implementations live
|
|
||||||
in `bot_bottle/contrib/<provider>/deploy_key_provisioner.py`. The
|
|
||||||
factory `get_provisioner` imports contrib modules lazily so that a
|
|
||||||
missing optional dependency in one provider doesn't break unrelated
|
|
||||||
features."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from abc import ABC, abstractmethod
|
|
||||||
|
|
||||||
|
|
||||||
class DeployKeyProvisioner(ABC):
|
|
||||||
"""Manages a single deploy-key lifecycle on a remote forge."""
|
|
||||||
|
|
||||||
@abstractmethod
|
|
||||||
def create(self, owner_repo: str, title: str) -> tuple[str, bytes]:
|
|
||||||
"""Generate a keypair and register the public half as a
|
|
||||||
deploy key on the forge.
|
|
||||||
|
|
||||||
`owner_repo` is the `<owner>/<repo>` path (no `.git` suffix).
|
|
||||||
`title` is the human-readable label shown in the forge UI.
|
|
||||||
|
|
||||||
Returns `(key_id, private_key_bytes)` where `key_id` is opaque
|
|
||||||
to the caller and is only ever passed back to `delete`."""
|
|
||||||
|
|
||||||
@abstractmethod
|
|
||||||
def delete(self, owner_repo: str, key_id: str) -> None:
|
|
||||||
"""Delete the registered deploy key.
|
|
||||||
|
|
||||||
Must not raise if the key is already absent (HTTP 404 is
|
|
||||||
success). Must raise for all other failures so teardown halts."""
|
|
||||||
|
|
||||||
|
|
||||||
def get_provisioner(
|
|
||||||
provider: str, token: str, api_url: str
|
|
||||||
) -> DeployKeyProvisioner:
|
|
||||||
"""Instantiate the contrib provisioner for `provider`.
|
|
||||||
|
|
||||||
Raises `ManifestError` for unknown providers so the error surfaces
|
|
||||||
at parse time rather than at runtime."""
|
|
||||||
if provider == "gitea":
|
|
||||||
from bot_bottle.contrib.gitea.deploy_key_provisioner import (
|
|
||||||
GiteaDeployKeyProvisioner,
|
|
||||||
)
|
|
||||||
return GiteaDeployKeyProvisioner(token=token, api_url=api_url)
|
|
||||||
from .manifest_util import ManifestError
|
|
||||||
raise ManifestError(
|
|
||||||
f"unknown provisioned_key provider: {provider!r}; "
|
|
||||||
f"available: gitea"
|
|
||||||
)
|
|
||||||
+143
-79
@@ -24,13 +24,11 @@ flow (PRD 0014) at egress and renames the MCP tool.
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import dataclasses
|
|
||||||
from abc import ABC, abstractmethod
|
from abc import ABC, abstractmethod
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import TYPE_CHECKING
|
from typing import TYPE_CHECKING
|
||||||
|
|
||||||
from .egress_addon_core import Route
|
|
||||||
from .log import die
|
from .log import die
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
if TYPE_CHECKING:
|
||||||
@@ -55,17 +53,21 @@ EGRESS_ROUTES_IN_CONTAINER = "/etc/egress/routes.yaml"
|
|||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class EgressRoute(Route):
|
class EgressRoute:
|
||||||
"""Host-side extension of the addon's `Route`.
|
"""One resolved route on the egress sidecar.
|
||||||
|
|
||||||
Inherits `host`, `path_allowlist`, `auth_scheme`, and `token_env`
|
`host` matches the request's hostname (case-insensitive). The
|
||||||
from `egress_addon_core.Route` — those are the fields that cross the
|
optional `path_allowlist` constrains the URL path; empty tuple
|
||||||
YAML wire into the sidecar. The three fields below are host-only and
|
means no path-level filtering. The `auth_scheme` / `token_env` /
|
||||||
are never serialised to the addon.
|
`token_ref` triple is the credential-injection config; empty
|
||||||
|
strings mean "no auth injection" (the manifest's nested `auth`
|
||||||
|
block was omitted).
|
||||||
|
|
||||||
`token_ref` is the host env var the CLI reads at launch and forwards
|
`token_env` is the env-var slot inside the egress container
|
||||||
into the container's environ under `token_env`. Routes that share a
|
(e.g. `EGRESS_TOKEN_0`); `token_ref` is the host env var
|
||||||
`token_ref` coalesce to one `token_env` slot.
|
the CLI reads at launch and forwards into the container's environ
|
||||||
|
under `token_env`. Routes that share a `token_ref` coalesce to
|
||||||
|
one `token_env` slot.
|
||||||
|
|
||||||
`roles` carries the manifest route's role tuple (reserved for
|
`roles` carries the manifest route's role tuple (reserved for
|
||||||
future use; always empty today).
|
future use; always empty today).
|
||||||
@@ -76,6 +78,10 @@ class EgressRoute(Route):
|
|||||||
route set it (e.g. egress injects its own Bearer on that host
|
route set it (e.g. egress injects its own Bearer on that host
|
||||||
after the agent boundary and pipelock's header DLP would block it)."""
|
after the agent boundary and pipelock's header DLP would block it)."""
|
||||||
|
|
||||||
|
host: str
|
||||||
|
path_allowlist: tuple[str, ...] = ()
|
||||||
|
auth_scheme: str = ""
|
||||||
|
token_env: str = ""
|
||||||
token_ref: str = ""
|
token_ref: str = ""
|
||||||
roles: tuple[str, ...] = ()
|
roles: tuple[str, ...] = ()
|
||||||
tls_passthrough: bool = False
|
tls_passthrough: bool = False
|
||||||
@@ -135,20 +141,43 @@ class EgressPlan:
|
|||||||
def egress_manifest_routes(
|
def egress_manifest_routes(
|
||||||
bottle: Bottle,
|
bottle: Bottle,
|
||||||
) -> tuple[EgressRoute, ...]:
|
) -> tuple[EgressRoute, ...]:
|
||||||
"""Lift each `bottle.egress.routes[]` manifest entry into an EgressRoute.
|
"""Lift each `bottle.egress.routes[]` manifest entry into a
|
||||||
Order is preserved. Token slots are not assigned here — slot assignment
|
resolved EgressRoute. Order is preserved so route lookup at
|
||||||
is a final step in `egress_routes_for_bottle` after provider and manifest
|
the proxy is stable.
|
||||||
routes are merged."""
|
|
||||||
|
Token-env slots are assigned per distinct `token_ref`: the first
|
||||||
|
authenticated route with `token_ref` "GH_PAT" gets
|
||||||
|
`EGRESS_TOKEN_0`; a second route with the same `token_ref`
|
||||||
|
shares slot 0. Unauthenticated routes (`auth` omitted) contribute
|
||||||
|
no slot.
|
||||||
|
|
||||||
|
This is the effective set the addon enforces. Provider runtime
|
||||||
|
routes are intentionally not injected implicitly; every allowed
|
||||||
|
host must come from the home-owned bottle manifest."""
|
||||||
out: list[EgressRoute] = []
|
out: list[EgressRoute] = []
|
||||||
|
slot_for_token: dict[str, str] = {}
|
||||||
for r in bottle.egress.routes:
|
for r in bottle.egress.routes:
|
||||||
out.append(EgressRoute(
|
if r.AuthScheme and r.TokenRef:
|
||||||
host=r.Host,
|
token_env = slot_for_token.get(r.TokenRef)
|
||||||
path_allowlist=r.PathAllowlist,
|
if token_env is None:
|
||||||
auth_scheme=r.AuthScheme,
|
token_env = f"EGRESS_TOKEN_{len(slot_for_token)}"
|
||||||
token_ref=r.TokenRef,
|
slot_for_token[r.TokenRef] = token_env
|
||||||
roles=r.Role,
|
out.append(EgressRoute(
|
||||||
tls_passthrough=r.Pipelock.TlsPassthrough,
|
host=r.Host,
|
||||||
))
|
path_allowlist=r.PathAllowlist,
|
||||||
|
auth_scheme=r.AuthScheme,
|
||||||
|
token_env=token_env,
|
||||||
|
token_ref=r.TokenRef,
|
||||||
|
roles=r.Role,
|
||||||
|
tls_passthrough=r.Pipelock.TlsPassthrough,
|
||||||
|
))
|
||||||
|
else:
|
||||||
|
out.append(EgressRoute(
|
||||||
|
host=r.Host,
|
||||||
|
path_allowlist=r.PathAllowlist,
|
||||||
|
roles=r.Role,
|
||||||
|
tls_passthrough=r.Pipelock.TlsPassthrough,
|
||||||
|
))
|
||||||
return tuple(out)
|
return tuple(out)
|
||||||
|
|
||||||
|
|
||||||
@@ -156,39 +185,80 @@ def egress_routes_for_bottle(
|
|||||||
bottle: Bottle,
|
bottle: Bottle,
|
||||||
provider_routes: tuple[EgressRoute, ...] = (),
|
provider_routes: tuple[EgressRoute, ...] = (),
|
||||||
) -> tuple[EgressRoute, ...]:
|
) -> tuple[EgressRoute, ...]:
|
||||||
"""Effective egress routes for the agent.
|
"""Effective egress routes for the agent. This is what gets rendered
|
||||||
|
into routes.yaml and what the addon enforces.
|
||||||
|
|
||||||
Provider routes own their hosts outright; manifest routes for hosts
|
Merges manifest-declared routes with provider-owned routes. The
|
||||||
not claimed by any provider are appended. Token slots are assigned
|
manifest is the primary surface; `provider_routes` are synthesised
|
||||||
in a final pass over the merged list in order, so provisioned routes
|
by `agent_provision_plan` and may add or upgrade manifest entries.
|
||||||
get the lower slot numbers."""
|
Provider routes that conflict with an existing authenticated manifest
|
||||||
manifest = egress_manifest_routes(bottle)
|
route (different auth scheme or token ref) raise a hard error."""
|
||||||
provisioned_hosts = {pr.host.lower() for pr in provider_routes}
|
routes = list(egress_manifest_routes(bottle))
|
||||||
merged = list(provider_routes) + [
|
for pr in provider_routes:
|
||||||
r for r in manifest if r.host.lower() not in provisioned_hosts
|
routes = _merge_provider_route(routes, pr)
|
||||||
]
|
return tuple(routes)
|
||||||
return _assign_token_slots(merged)
|
|
||||||
|
|
||||||
|
|
||||||
def _assign_token_slots(
|
def _find_or_alloc_token_env(routes: list[EgressRoute], token_ref: str) -> str:
|
||||||
routes: list[EgressRoute],
|
"""Return the existing token_env slot for `token_ref`, or allocate the next one."""
|
||||||
) -> tuple[EgressRoute, ...]:
|
for route in routes:
|
||||||
"""Assign EGRESS_TOKEN_N slots to authenticated routes in order.
|
if route.token_ref == token_ref and route.token_env:
|
||||||
|
return route.token_env
|
||||||
|
return f"EGRESS_TOKEN_{len({r.token_env for r in routes if r.token_env})}"
|
||||||
|
|
||||||
Routes sharing a token_ref share a slot. Unauthenticated routes
|
|
||||||
(no auth_scheme / token_ref) keep token_env empty."""
|
def _merge_provider_route(
|
||||||
slot_for_ref: dict[str, str] = {}
|
routes: list[EgressRoute], pr: EgressRoute,
|
||||||
out: list[EgressRoute] = []
|
) -> list[EgressRoute]:
|
||||||
for r in routes:
|
"""Merge one provider-declared route into the manifest route list.
|
||||||
if r.auth_scheme and r.token_ref:
|
|
||||||
slot = slot_for_ref.get(r.token_ref)
|
Upgrade a bare-pass manifest route to authenticated if the provider
|
||||||
if slot is None:
|
declares auth for that host, or append if the host isn't in the manifest.
|
||||||
slot = f"EGRESS_TOKEN_{len(slot_for_ref)}"
|
Identical auth (same scheme + token_ref) on an existing route is a
|
||||||
slot_for_ref[r.token_ref] = slot
|
no-op, with a tls_passthrough upgrade if the provider route sets it.
|
||||||
out.append(dataclasses.replace(r, token_env=slot))
|
Conflicting auth (different scheme or token_ref) dies."""
|
||||||
else:
|
for idx, route in enumerate(routes):
|
||||||
out.append(r)
|
if route.host.lower() != pr.host.lower():
|
||||||
return tuple(out)
|
continue
|
||||||
|
if route.auth_scheme or route.token_ref:
|
||||||
|
if route.auth_scheme == pr.auth_scheme and route.token_ref == pr.token_ref:
|
||||||
|
if pr.tls_passthrough and not route.tls_passthrough:
|
||||||
|
routes[idx] = EgressRoute(
|
||||||
|
host=route.host,
|
||||||
|
path_allowlist=route.path_allowlist,
|
||||||
|
auth_scheme=route.auth_scheme,
|
||||||
|
token_env=route.token_env,
|
||||||
|
token_ref=route.token_ref,
|
||||||
|
roles=route.roles,
|
||||||
|
tls_passthrough=True,
|
||||||
|
)
|
||||||
|
return routes
|
||||||
|
die(
|
||||||
|
f"provider egress route for {pr.host!r} conflicts with an "
|
||||||
|
f"authenticated manifest route (different auth scheme or token "
|
||||||
|
f"ref). Remove the manifest route's auth block or disable the "
|
||||||
|
f"feature that adds this provider route."
|
||||||
|
)
|
||||||
|
token_env = _find_or_alloc_token_env(routes, pr.token_ref)
|
||||||
|
routes[idx] = EgressRoute(
|
||||||
|
host=route.host,
|
||||||
|
path_allowlist=route.path_allowlist,
|
||||||
|
auth_scheme=pr.auth_scheme,
|
||||||
|
token_env=token_env,
|
||||||
|
token_ref=pr.token_ref,
|
||||||
|
roles=route.roles,
|
||||||
|
tls_passthrough=pr.tls_passthrough,
|
||||||
|
)
|
||||||
|
return routes
|
||||||
|
token_env = _find_or_alloc_token_env(routes, pr.token_ref)
|
||||||
|
routes.append(EgressRoute(
|
||||||
|
host=pr.host,
|
||||||
|
auth_scheme=pr.auth_scheme,
|
||||||
|
token_env=token_env,
|
||||||
|
token_ref=pr.token_ref,
|
||||||
|
tls_passthrough=pr.tls_passthrough,
|
||||||
|
))
|
||||||
|
return routes
|
||||||
|
|
||||||
|
|
||||||
def egress_token_env_map(
|
def egress_token_env_map(
|
||||||
@@ -203,7 +273,7 @@ def egress_token_env_map(
|
|||||||
silently picking one."""
|
silently picking one."""
|
||||||
out: dict[str, str] = {}
|
out: dict[str, str] = {}
|
||||||
for r in routes:
|
for r in routes:
|
||||||
if not (r.auth_scheme and r.token_ref and r.token_env):
|
if not r.token_env:
|
||||||
continue
|
continue
|
||||||
existing = out.get(r.token_env)
|
existing = out.get(r.token_env)
|
||||||
if existing is not None and existing != r.token_ref:
|
if existing is not None and existing != r.token_ref:
|
||||||
@@ -216,43 +286,35 @@ def egress_token_env_map(
|
|||||||
return out
|
return out
|
||||||
|
|
||||||
|
|
||||||
def _route_to_yaml_fields(r: Route) -> dict:
|
|
||||||
"""Return the addon-visible fields for one route.
|
|
||||||
|
|
||||||
Single authoritative mapping between EgressRoute (host-side) and
|
|
||||||
egress_addon_core.Route (sidecar-side). When a field is added to
|
|
||||||
the addon's Route that must appear in the YAML, add it here and
|
|
||||||
in egress_addon_core._parse_one together."""
|
|
||||||
fields: dict = {"host": r.host}
|
|
||||||
if r.auth_scheme and r.token_env:
|
|
||||||
fields["auth_scheme"] = r.auth_scheme
|
|
||||||
fields["token_env"] = r.token_env
|
|
||||||
if r.path_allowlist:
|
|
||||||
fields["path_allowlist"] = list(r.path_allowlist)
|
|
||||||
return fields
|
|
||||||
|
|
||||||
|
|
||||||
def egress_render_routes(
|
def egress_render_routes(
|
||||||
routes: tuple[EgressRoute, ...],
|
routes: tuple[EgressRoute, ...],
|
||||||
) -> str:
|
) -> str:
|
||||||
"""Serialize the route table for the addon to read.
|
"""Serialize the route table for the addon to read.
|
||||||
|
|
||||||
YAML content — no token values, no host env-var names. Fields are
|
YAML content — no token values, no host env-var names. The only
|
||||||
determined by `_route_to_yaml_fields`, which is the single point of
|
thing the addon needs at runtime is the host → path_allowlist
|
||||||
truth for the EgressRoute → egress_addon_core.Route mapping."""
|
+ auth_scheme + in-container env-var mapping. The actual token
|
||||||
|
values arrive via the container's environ.
|
||||||
|
|
||||||
|
Authenticated routes carry `auth_scheme` + `token_env`;
|
||||||
|
unauthenticated routes omit both keys (the addon's parser
|
||||||
|
enforces both-or-neither). Hand-rolled YAML in the style of
|
||||||
|
`pipelock_render_yaml` so the addon's parser
|
||||||
|
(`yaml_subset.parse_yaml_subset`) round-trips it cleanly."""
|
||||||
lines: list[str] = ["routes:"]
|
lines: list[str] = ["routes:"]
|
||||||
if not routes:
|
if not routes:
|
||||||
|
# `routes:` with an empty list on the same line — the parser
|
||||||
|
# needs SOMETHING here. Empty inline list is the cleanest.
|
||||||
lines[0] = "routes: []"
|
lines[0] = "routes: []"
|
||||||
return "\n".join(lines) + "\n"
|
return "\n".join(lines) + "\n"
|
||||||
for r in routes:
|
for r in routes:
|
||||||
f = _route_to_yaml_fields(r)
|
lines.append(f' - host: "{r.host}"')
|
||||||
lines.append(f' - host: "{f["host"]}"')
|
if r.auth_scheme and r.token_env:
|
||||||
if "auth_scheme" in f:
|
lines.append(f' auth_scheme: "{r.auth_scheme}"')
|
||||||
lines.append(f' auth_scheme: "{f["auth_scheme"]}"')
|
lines.append(f' token_env: "{r.token_env}"')
|
||||||
lines.append(f' token_env: "{f["token_env"]}"')
|
if r.path_allowlist:
|
||||||
if "path_allowlist" in f:
|
|
||||||
lines.append(" path_allowlist:")
|
lines.append(" path_allowlist:")
|
||||||
for p in f["path_allowlist"]:
|
for p in r.path_allowlist:
|
||||||
lines.append(f' - "{p}"')
|
lines.append(f' - "{p}"')
|
||||||
return "\n".join(lines) + "\n"
|
return "\n".join(lines) + "\n"
|
||||||
|
|
||||||
@@ -269,6 +331,8 @@ def egress_resolve_token_values(
|
|||||||
a sealed mapping without touching `os.environ`."""
|
a sealed mapping without touching `os.environ`."""
|
||||||
out: dict[str, str] = {}
|
out: dict[str, str] = {}
|
||||||
for token_env, token_ref in token_env_map.items():
|
for token_env, token_ref in token_env_map.items():
|
||||||
|
if token_ref == CODEX_HOST_CREDENTIAL_TOKEN_REF:
|
||||||
|
continue
|
||||||
value = host_env.get(token_ref)
|
value = host_env.get(token_ref)
|
||||||
if value is None:
|
if value is None:
|
||||||
die(
|
die(
|
||||||
|
|||||||
+45
-93
@@ -29,14 +29,12 @@ backend-specific and lives on concrete subclasses (see
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import dataclasses
|
|
||||||
import os
|
|
||||||
import shlex
|
|
||||||
from abc import ABC, abstractmethod
|
from abc import ABC, abstractmethod
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass, field
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
from typing import Mapping
|
||||||
|
|
||||||
from .log import info
|
from .log import die
|
||||||
from .manifest import Bottle, GitEntry
|
from .manifest import Bottle, GitEntry
|
||||||
|
|
||||||
|
|
||||||
@@ -49,6 +47,10 @@ GIT_GATE_HOSTNAME = "git-gate"
|
|||||||
GIT_GATE_DAEMON_TIMEOUT_SECS = 15
|
GIT_GATE_DAEMON_TIMEOUT_SECS = 15
|
||||||
|
|
||||||
|
|
||||||
|
def _empty_str_map() -> dict[str, str]:
|
||||||
|
return {}
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class GitGateUpstream:
|
class GitGateUpstream:
|
||||||
"""One bare repo on the gate. `name` drives the bare-repo path
|
"""One bare repo on the gate. `name` drives the bare-repo path
|
||||||
@@ -62,7 +64,10 @@ class GitGateUpstream:
|
|||||||
KnownHostKey string from the manifest; the gate's start step
|
KnownHostKey string from the manifest; the gate's start step
|
||||||
materialises it into a known_hosts file if non-empty.
|
materialises it into a known_hosts file if non-empty.
|
||||||
|
|
||||||
the gate credential paths inside the running sidecar."""
|
`extra_hosts` is a `{hostname: ip}` map the backend injects into
|
||||||
|
the gate container's `/etc/hosts` via `--add-host` so the gate
|
||||||
|
can resolve upstream hostnames that aren't reachable via the
|
||||||
|
container's default DNS (e.g. Tailscale-only hosts)."""
|
||||||
|
|
||||||
name: str
|
name: str
|
||||||
upstream_url: str
|
upstream_url: str
|
||||||
@@ -71,6 +76,7 @@ class GitGateUpstream:
|
|||||||
identity_file: str
|
identity_file: str
|
||||||
known_host_key: str
|
known_host_key: str
|
||||||
known_hosts_file: Path = Path()
|
known_hosts_file: Path = Path()
|
||||||
|
extra_hosts: Mapping[str, str] = field(default_factory=_empty_str_map)
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
@@ -107,11 +113,38 @@ def git_gate_upstreams_for_bottle(bottle: Bottle) -> tuple[GitGateUpstream, ...]
|
|||||||
upstream_port=e.UpstreamPort,
|
upstream_port=e.UpstreamPort,
|
||||||
identity_file=e.IdentityFile,
|
identity_file=e.IdentityFile,
|
||||||
known_host_key=e.KnownHostKey,
|
known_host_key=e.KnownHostKey,
|
||||||
|
extra_hosts=dict(e.ExtraHosts),
|
||||||
)
|
)
|
||||||
for e in bottle.git
|
for e in bottle.git
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def git_gate_aggregate_extra_hosts(
|
||||||
|
upstreams: tuple[GitGateUpstream, ...],
|
||||||
|
) -> dict[str, str]:
|
||||||
|
"""Merge every upstream's `extra_hosts` into a single
|
||||||
|
`{hostname: ip}` map for `--add-host` on the gate container. Two
|
||||||
|
entries naming the same hostname with different IPs is a manifest
|
||||||
|
bug — the gate has one /etc/hosts — so die loudly with the
|
||||||
|
conflicting names rather than silently picking one."""
|
||||||
|
merged: dict[str, str] = {}
|
||||||
|
source: dict[str, str] = {}
|
||||||
|
for u in upstreams:
|
||||||
|
for host, ip in u.extra_hosts.items():
|
||||||
|
existing = merged.get(host)
|
||||||
|
if existing is None:
|
||||||
|
merged[host] = ip
|
||||||
|
source[host] = u.name
|
||||||
|
elif existing != ip:
|
||||||
|
die(
|
||||||
|
f"git-gate ExtraHosts conflict: '{host}' maps to "
|
||||||
|
f"'{existing}' in upstream '{source[host]}' and to "
|
||||||
|
f"'{ip}' in upstream '{u.name}'. The gate has one "
|
||||||
|
f"/etc/hosts; pick one IP."
|
||||||
|
)
|
||||||
|
return merged
|
||||||
|
|
||||||
|
|
||||||
def git_gate_render_gitconfig(
|
def git_gate_render_gitconfig(
|
||||||
entries: tuple[GitEntry, ...], gate_host: str, *, scheme: str = "git",
|
entries: tuple[GitEntry, ...], gate_host: str, *, scheme: str = "git",
|
||||||
) -> str:
|
) -> str:
|
||||||
@@ -211,7 +244,10 @@ def git_gate_render_entrypoint(upstreams: tuple[GitGateUpstream, ...]) -> str:
|
|||||||
"mkdir -p /git",
|
"mkdir -p /git",
|
||||||
]
|
]
|
||||||
for u in upstreams:
|
for u in upstreams:
|
||||||
lines.append(f"init_repo {shlex.quote(u.name)} {shlex.quote(u.upstream_url)}")
|
# Single-quote args so URL/path content (containing : and /)
|
||||||
|
# passes through ash unmangled. Names came through the manifest
|
||||||
|
# validator so they don't contain a single quote.
|
||||||
|
lines.append(f"init_repo '{u.name}' '{u.upstream_url}'")
|
||||||
lines.extend([
|
lines.extend([
|
||||||
"",
|
"",
|
||||||
"exec git daemon \\",
|
"exec git daemon \\",
|
||||||
@@ -360,80 +396,6 @@ exit 0
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
|
|
||||||
def _provision_dynamic_key(
|
|
||||||
entry: GitEntry,
|
|
||||||
slug: str,
|
|
||||||
stage_dir: Path,
|
|
||||||
) -> str:
|
|
||||||
"""Generate a fresh ed25519 keypair, register the public half with
|
|
||||||
the forge, and persist the private key + key ID under `stage_dir`.
|
|
||||||
|
|
||||||
Returns the host-side path to the private key file so the caller
|
|
||||||
can inject it into the GitGateUpstream as `identity_file`."""
|
|
||||||
from .deploy_key_provisioner import get_provisioner
|
|
||||||
pk = entry.ProvisionedKey
|
|
||||||
assert pk is not None
|
|
||||||
token = os.environ.get(pk.token_env)
|
|
||||||
if token is None:
|
|
||||||
raise RuntimeError(
|
|
||||||
f"git-gate.repos[{entry.Name!r}] provisioned_key.token_env"
|
|
||||||
f" = {pk.token_env!r}: env var is not set"
|
|
||||||
)
|
|
||||||
api_url = pk.api_url or f"https://{entry.UpstreamHost}"
|
|
||||||
provisioner = get_provisioner(pk.provider, token, api_url)
|
|
||||||
|
|
||||||
owner_repo = entry.UpstreamPath
|
|
||||||
if owner_repo.endswith(".git"):
|
|
||||||
owner_repo = owner_repo[:-4]
|
|
||||||
title = f"bot-bottle:{slug}:{entry.Name}"
|
|
||||||
|
|
||||||
info(f"provisioning deploy key for git-gate.repos[{entry.Name!r}]")
|
|
||||||
key_id, private_key_bytes = provisioner.create(owner_repo, title)
|
|
||||||
|
|
||||||
key_file = stage_dir / f"{entry.Name}-key"
|
|
||||||
key_file.write_bytes(private_key_bytes)
|
|
||||||
key_file.chmod(0o600)
|
|
||||||
|
|
||||||
id_file = stage_dir / f"{entry.Name}-deploy-key-id"
|
|
||||||
id_file.write_text(key_id)
|
|
||||||
id_file.chmod(0o600)
|
|
||||||
|
|
||||||
info(f"provisioned deploy key {key_id} for git-gate.repos[{entry.Name!r}]")
|
|
||||||
return str(key_file)
|
|
||||||
|
|
||||||
|
|
||||||
def revoke_git_gate_provisioned_keys(bottle: Bottle, stage_dir: Path) -> None:
|
|
||||||
"""Revoke all deploy keys provisioned for `bottle` during prepare.
|
|
||||||
|
|
||||||
Called at teardown after containers stop. Raises if any revocation
|
|
||||||
fails — a stranded key is a security concern that the operator must
|
|
||||||
address manually."""
|
|
||||||
from .deploy_key_provisioner import get_provisioner
|
|
||||||
for entry in bottle.git:
|
|
||||||
if entry.ProvisionedKey is None:
|
|
||||||
continue
|
|
||||||
pk = entry.ProvisionedKey
|
|
||||||
id_file = stage_dir / f"{entry.Name}-deploy-key-id"
|
|
||||||
if not id_file.exists():
|
|
||||||
continue
|
|
||||||
key_id = id_file.read_text().strip()
|
|
||||||
token = os.environ.get(pk.token_env)
|
|
||||||
if token is None:
|
|
||||||
raise RuntimeError(
|
|
||||||
f"git-gate.repos[{entry.Name!r}] provisioned_key.token_env"
|
|
||||||
f" = {pk.token_env!r}: env var is not set;"
|
|
||||||
f" cannot revoke deploy key {key_id}"
|
|
||||||
)
|
|
||||||
api_url = pk.api_url or f"https://{entry.UpstreamHost}"
|
|
||||||
provisioner = get_provisioner(pk.provider, token, api_url)
|
|
||||||
owner_repo = entry.UpstreamPath
|
|
||||||
if owner_repo.endswith(".git"):
|
|
||||||
owner_repo = owner_repo[:-4]
|
|
||||||
info(f"revoking deploy key {key_id} for git-gate.repos[{entry.Name!r}]")
|
|
||||||
provisioner.delete(owner_repo, key_id)
|
|
||||||
info(f"revoked deploy key {key_id} for git-gate.repos[{entry.Name!r}]")
|
|
||||||
|
|
||||||
|
|
||||||
class GitGate(ABC):
|
class GitGate(ABC):
|
||||||
"""The per-agent git-gate. Encapsulates the host-side prepare
|
"""The per-agent git-gate. Encapsulates the host-side prepare
|
||||||
(upstream lift + entrypoint/hook render); the sidecar's
|
(upstream lift + entrypoint/hook render); the sidecar's
|
||||||
@@ -445,21 +407,10 @@ class GitGate(ABC):
|
|||||||
entrypoint, pre-receive hook, and access-hook scripts (mode
|
entrypoint, pre-receive hook, and access-hook scripts (mode
|
||||||
600) under `stage_dir`. Pure host-side, no docker subprocess.
|
600) under `stage_dir`. Pure host-side, no docker subprocess.
|
||||||
|
|
||||||
For `provisioned_key` entries, also generates and registers
|
|
||||||
a fresh deploy key via the forge API and writes the private key
|
|
||||||
+ key ID to `stage_dir`.
|
|
||||||
|
|
||||||
Returned plan is incomplete: the launch step must fill
|
Returned plan is incomplete: the launch step must fill
|
||||||
`internal_network` / `egress_network` via `dataclasses.replace`
|
`internal_network` / `egress_network` via `dataclasses.replace`
|
||||||
before passing the plan to `.start`."""
|
before passing the plan to `.start`."""
|
||||||
upstreams_list = list(git_gate_upstreams_for_bottle(bottle))
|
upstreams = git_gate_upstreams_for_bottle(bottle)
|
||||||
for i, entry in enumerate(bottle.git):
|
|
||||||
if entry.ProvisionedKey is not None:
|
|
||||||
key_file = _provision_dynamic_key(entry, slug, stage_dir)
|
|
||||||
upstreams_list[i] = dataclasses.replace(
|
|
||||||
upstreams_list[i], identity_file=key_file
|
|
||||||
)
|
|
||||||
upstreams = tuple(upstreams_list)
|
|
||||||
entrypoint = stage_dir / "git_gate_entrypoint.sh"
|
entrypoint = stage_dir / "git_gate_entrypoint.sh"
|
||||||
entrypoint.write_text(git_gate_render_entrypoint(upstreams))
|
entrypoint.write_text(git_gate_render_entrypoint(upstreams))
|
||||||
entrypoint.chmod(0o600)
|
entrypoint.chmod(0o600)
|
||||||
@@ -492,6 +443,7 @@ class GitGate(ABC):
|
|||||||
identity_file=u.identity_file,
|
identity_file=u.identity_file,
|
||||||
known_host_key=u.known_host_key,
|
known_host_key=u.known_host_key,
|
||||||
known_hosts_file=known_hosts_file,
|
known_hosts_file=known_hosts_file,
|
||||||
|
extra_hosts=dict(u.extra_hosts),
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
return GitGatePlan(
|
return GitGatePlan(
|
||||||
|
|||||||
@@ -19,9 +19,6 @@ from urllib.parse import urlsplit
|
|||||||
|
|
||||||
DEFAULT_PORT = 9420
|
DEFAULT_PORT = 9420
|
||||||
|
|
||||||
# Body-size cap matching supervise_server.py's 1 MiB limit.
|
|
||||||
MAX_BODY_BYTES = 1 * 1024 * 1024
|
|
||||||
|
|
||||||
|
|
||||||
class GitHttpHandler(BaseHTTPRequestHandler):
|
class GitHttpHandler(BaseHTTPRequestHandler):
|
||||||
server_version = "bot-bottle-git-http/1"
|
server_version = "bot-bottle-git-http/1"
|
||||||
@@ -42,25 +39,13 @@ class GitHttpHandler(BaseHTTPRequestHandler):
|
|||||||
hook_path = os.environ.get(
|
hook_path = os.environ.get(
|
||||||
"GIT_GATE_ACCESS_HOOK", "/etc/git-gate/access-hook",
|
"GIT_GATE_ACCESS_HOOK", "/etc/git-gate/access-hook",
|
||||||
)
|
)
|
||||||
peer = self.client_address[0]
|
|
||||||
hook = subprocess.run(
|
hook = subprocess.run(
|
||||||
[hook_path, "upload-pack", str(repo_dir), peer, peer],
|
[hook_path, "upload-pack",
|
||||||
|
str(repo_dir), self.client_address[0], self.client_address[0]],
|
||||||
capture_output=True,
|
capture_output=True,
|
||||||
check=False,
|
check=False,
|
||||||
)
|
)
|
||||||
if hook.returncode != 0:
|
if hook.returncode != 0:
|
||||||
detail = (hook.stderr or hook.stdout).decode(
|
|
||||||
"utf-8", errors="replace",
|
|
||||||
).rstrip()
|
|
||||||
if detail:
|
|
||||||
for line in detail.splitlines():
|
|
||||||
self.log_message("access-hook denied %s: %s",
|
|
||||||
parsed.path, line)
|
|
||||||
else:
|
|
||||||
self.log_message(
|
|
||||||
"access-hook denied %s: exit=%d (no output)",
|
|
||||||
parsed.path, hook.returncode,
|
|
||||||
)
|
|
||||||
self.send_response(403)
|
self.send_response(403)
|
||||||
self.send_header("Content-Type", "text/plain; charset=utf-8")
|
self.send_header("Content-Type", "text/plain; charset=utf-8")
|
||||||
self.end_headers()
|
self.end_headers()
|
||||||
@@ -91,18 +76,7 @@ class GitHttpHandler(BaseHTTPRequestHandler):
|
|||||||
value = self.headers.get(header)
|
value = self.headers.get(header)
|
||||||
if value:
|
if value:
|
||||||
env[variable] = value
|
env[variable] = value
|
||||||
raw_length = self.headers.get("content-length", "0") or "0"
|
length = int(self.headers.get("content-length", "0") or "0")
|
||||||
try:
|
|
||||||
length = int(raw_length)
|
|
||||||
except ValueError:
|
|
||||||
self.send_error(400, "Bad Content-Length")
|
|
||||||
return
|
|
||||||
if length < 0:
|
|
||||||
self.send_error(400, "Negative Content-Length")
|
|
||||||
return
|
|
||||||
if length > MAX_BODY_BYTES:
|
|
||||||
self.send_error(413, "Request body too large")
|
|
||||||
return
|
|
||||||
body = self.rfile.read(length) if length else b""
|
body = self.rfile.read(length) if length else b""
|
||||||
proc = subprocess.run(
|
proc = subprocess.run(
|
||||||
["git", "http-backend"],
|
["git", "http-backend"],
|
||||||
|
|||||||
+1015
-70
File diff suppressed because it is too large
Load Diff
@@ -1,166 +0,0 @@
|
|||||||
"""Agent configuration manifest dataclasses."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from typing import cast
|
|
||||||
|
|
||||||
from .agent_provider import PROVIDER_TEMPLATES
|
|
||||||
from .manifest_util import ManifestError, as_json_object
|
|
||||||
from .manifest_git import GitUser
|
|
||||||
from .manifest_schema import AGENT_MODEL_KEYS
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class AgentProvider:
|
|
||||||
"""Provider/template for the agent process inside a bottle.
|
|
||||||
|
|
||||||
`template` selects a built-in launch/runtime contract. `dockerfile`
|
|
||||||
optionally points at a custom agent-image Dockerfile while leaving
|
|
||||||
bot-bottle's sidecar infrastructure intact.
|
|
||||||
|
|
||||||
`auth_token` names the host env var that holds the provider's OAuth
|
|
||||||
token (Claude only). The provisioner injects a provider-owned egress
|
|
||||||
route for api.anthropic.com that re-injects this token as the Bearer
|
|
||||||
header, and sets a placeholder CLAUDE_CODE_OAUTH_TOKEN in the agent
|
|
||||||
so the Claude Code CLI starts.
|
|
||||||
|
|
||||||
`forward_host_credentials` forwards the host Codex auth token into
|
|
||||||
the egress sidecar (Codex only).
|
|
||||||
"""
|
|
||||||
|
|
||||||
template: str = "claude"
|
|
||||||
dockerfile: str = ""
|
|
||||||
auth_token: str = ""
|
|
||||||
forward_host_credentials: bool = False
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def from_dict(cls, bottle_name: str, raw: object) -> "AgentProvider":
|
|
||||||
d = as_json_object(raw, f"bottle '{bottle_name}' agent_provider")
|
|
||||||
for k in d:
|
|
||||||
if k not in {"template", "dockerfile", "auth_token", "forward_host_credentials"}:
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' agent_provider has unknown key {k!r}; "
|
|
||||||
f"allowed: template, dockerfile, auth_token, forward_host_credentials"
|
|
||||||
)
|
|
||||||
template = d.get("template", "claude")
|
|
||||||
if not isinstance(template, str) or not template:
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' agent_provider.template must be a "
|
|
||||||
f"non-empty string"
|
|
||||||
)
|
|
||||||
if template not in PROVIDER_TEMPLATES:
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' agent_provider.template {template!r} "
|
|
||||||
f"is not one of {', '.join(sorted(PROVIDER_TEMPLATES))}"
|
|
||||||
)
|
|
||||||
dockerfile = d.get("dockerfile", "")
|
|
||||||
if not isinstance(dockerfile, str):
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' agent_provider.dockerfile must be a "
|
|
||||||
f"string (was {type(dockerfile).__name__})"
|
|
||||||
)
|
|
||||||
auth_token = d.get("auth_token", "")
|
|
||||||
if not isinstance(auth_token, str):
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' agent_provider.auth_token must be a "
|
|
||||||
f"string (was {type(auth_token).__name__})"
|
|
||||||
)
|
|
||||||
if auth_token and template != "claude":
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' agent_provider.auth_token is only "
|
|
||||||
f"supported for template 'claude'"
|
|
||||||
)
|
|
||||||
forward_host_credentials = d.get("forward_host_credentials", False)
|
|
||||||
if not isinstance(forward_host_credentials, bool):
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' agent_provider.forward_host_credentials "
|
|
||||||
f"must be a boolean (was {type(forward_host_credentials).__name__})"
|
|
||||||
)
|
|
||||||
if forward_host_credentials and template != "codex":
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' agent_provider.forward_host_credentials "
|
|
||||||
"is currently only supported for template 'codex'"
|
|
||||||
)
|
|
||||||
return cls(
|
|
||||||
template=template,
|
|
||||||
dockerfile=dockerfile,
|
|
||||||
auth_token=auth_token,
|
|
||||||
forward_host_credentials=forward_host_credentials,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class Agent:
|
|
||||||
bottle: str
|
|
||||||
skills: tuple[str, ...] = ()
|
|
||||||
prompt: str = ""
|
|
||||||
# Per-agent git identity (issue #94). Overlays the referenced
|
|
||||||
# bottle's git-gate.user per-field at `Manifest.bottle_for`. Only
|
|
||||||
# `user` is allowed at the agent level; `repos` stays bottle-only
|
|
||||||
# because it carries credentials and host trust.
|
|
||||||
git_user: GitUser = GitUser()
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def from_dict(cls, name: str, raw: object, bottle_names: set[str]) -> "Agent":
|
|
||||||
d = as_json_object(raw, f"agent '{name}'")
|
|
||||||
unknown = set(d.keys()) - AGENT_MODEL_KEYS
|
|
||||||
if unknown:
|
|
||||||
allowed = ", ".join(sorted(AGENT_MODEL_KEYS))
|
|
||||||
raise ManifestError(
|
|
||||||
f"agent '{name}' has unknown key(s) {sorted(unknown)}; "
|
|
||||||
f"allowed keys are {allowed}."
|
|
||||||
)
|
|
||||||
|
|
||||||
bottle = d.get("bottle")
|
|
||||||
if not isinstance(bottle, str) or not bottle:
|
|
||||||
raise ManifestError(f"agent '{name}' must declare a 'bottle' field naming a defined bottle")
|
|
||||||
if bottle not in bottle_names:
|
|
||||||
available = ", ".join(sorted(bottle_names)) or "(none defined)"
|
|
||||||
raise ManifestError(
|
|
||||||
f"agent '{name}' references bottle '{bottle}', which is not defined. "
|
|
||||||
f"Available: {available}"
|
|
||||||
)
|
|
||||||
|
|
||||||
skills: tuple[str, ...] = ()
|
|
||||||
skills_raw = d.get("skills")
|
|
||||||
if skills_raw is not None:
|
|
||||||
if not isinstance(skills_raw, list):
|
|
||||||
raise ManifestError(f"agent '{name}' skills must be an array (was {type(skills_raw).__name__})")
|
|
||||||
collected: list[str] = []
|
|
||||||
skills_list = cast(list[object], skills_raw)
|
|
||||||
for i, skill in enumerate(skills_list):
|
|
||||||
if not isinstance(skill, str):
|
|
||||||
raise ManifestError(
|
|
||||||
f"agent '{name}' skills[{i}] must be a string "
|
|
||||||
f"(was {type(skill).__name__})"
|
|
||||||
)
|
|
||||||
collected.append(skill)
|
|
||||||
skills = tuple(collected)
|
|
||||||
|
|
||||||
prompt_raw = d.get("prompt")
|
|
||||||
if prompt_raw is None:
|
|
||||||
prompt = ""
|
|
||||||
elif isinstance(prompt_raw, str):
|
|
||||||
prompt = prompt_raw
|
|
||||||
else:
|
|
||||||
raise ManifestError(f"agent '{name}' prompt must be a string (was {type(prompt_raw).__name__})")
|
|
||||||
|
|
||||||
# git-gate: agents may declare only `git-gate.user` (name/email).
|
|
||||||
# `git-gate.repos` is bottle-only — it carries credentials and host trust.
|
|
||||||
git_user = GitUser()
|
|
||||||
git_raw = d.get("git-gate")
|
|
||||||
if git_raw is not None:
|
|
||||||
gd = as_json_object(git_raw, f"agent '{name}' git-gate")
|
|
||||||
for k in gd.keys():
|
|
||||||
if k != "user":
|
|
||||||
raise ManifestError(
|
|
||||||
f"agent '{name}' git-gate.{k} is not allowed at the "
|
|
||||||
f"agent level; only git-gate.user (name/email) may be "
|
|
||||||
f"set on an agent. git-gate.repos is bottle-only "
|
|
||||||
f"(it carries credentials and host trust)."
|
|
||||||
)
|
|
||||||
if "user" in gd:
|
|
||||||
git_user = GitUser.from_dict(name, gd["user"])
|
|
||||||
|
|
||||||
return cls(bottle=bottle, skills=skills, prompt=prompt, git_user=git_user)
|
|
||||||
@@ -1,286 +0,0 @@
|
|||||||
"""Egress routing manifest dataclasses and helpers."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import ipaddress
|
|
||||||
from dataclasses import dataclass, field
|
|
||||||
from typing import cast
|
|
||||||
|
|
||||||
from .manifest_util import ManifestError, as_json_object
|
|
||||||
|
|
||||||
|
|
||||||
# Auth schemes for the egress route's optional `auth` block.
|
|
||||||
# Same values cred-proxy accepts today; `token` sidesteps the Gitea
|
|
||||||
# token-not-Bearer quirk (go-gitea/gitea#16734).
|
|
||||||
EGRESS_AUTH_SCHEMES = ("Bearer", "token")
|
|
||||||
|
|
||||||
|
|
||||||
def validate_egress_routes(
|
|
||||||
bottle_name: str,
|
|
||||||
routes: tuple[EgressRoute, ...],
|
|
||||||
) -> None:
|
|
||||||
"""Cross-validation for `bottle.egress.routes`: hosts must be unique.
|
|
||||||
|
|
||||||
The proxy matches by exact-host (v1); duplicate hosts leave the
|
|
||||||
route choice ambiguous so we reject them up front.
|
|
||||||
|
|
||||||
No cross-validation against `bottle.git-gate.repos` is performed.
|
|
||||||
git-gate (SSH push/fetch) and egress (HTTPS) broker different
|
|
||||||
protocols; declaring both for the same host is a legitimate dev
|
|
||||||
setup."""
|
|
||||||
seen_hosts: dict[str, None] = {}
|
|
||||||
for r in routes:
|
|
||||||
key = r.Host.lower()
|
|
||||||
if key in seen_hosts:
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' egress.routes has duplicate host "
|
|
||||||
f"{r.Host!r}; each host must be unique on the proxy."
|
|
||||||
)
|
|
||||||
seen_hosts[key] = None
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class PipelockRoutePolicy:
|
|
||||||
"""Per-route pipelock policy overrides.
|
|
||||||
|
|
||||||
`TlsPassthrough` adds the route host to pipelock's
|
|
||||||
`tls_interception.passthrough_domains`, so pipelock still enforces
|
|
||||||
the hostname allowlist but does not MITM/decrypt request bodies or
|
|
||||||
headers for that host.
|
|
||||||
|
|
||||||
`SsrfIpAllowlist` adds explicit IPs/CIDRs to pipelock's SSRF
|
|
||||||
allowlist for private/internal destinations behind this route.
|
|
||||||
"""
|
|
||||||
|
|
||||||
TlsPassthrough: bool = False
|
|
||||||
SsrfIpAllowlist: tuple[str, ...] = ()
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def from_dict(
|
|
||||||
cls, bottle_name: str, idx: int, raw: object,
|
|
||||||
) -> "PipelockRoutePolicy":
|
|
||||||
label = f"bottle '{bottle_name}' egress.routes[{idx}] pipelock"
|
|
||||||
d = as_json_object(raw, label)
|
|
||||||
for k in d:
|
|
||||||
if k not in ("tls_passthrough", "ssrf_ip_allowlist"):
|
|
||||||
raise ManifestError(
|
|
||||||
f"{label} has unknown key {k!r}; "
|
|
||||||
f"only 'tls_passthrough' and 'ssrf_ip_allowlist' "
|
|
||||||
f"are accepted"
|
|
||||||
)
|
|
||||||
tls_passthrough_raw = d.get("tls_passthrough", False)
|
|
||||||
if not isinstance(tls_passthrough_raw, bool):
|
|
||||||
raise ManifestError(
|
|
||||||
f"{label}.tls_passthrough must be a boolean "
|
|
||||||
f"(was {type(tls_passthrough_raw).__name__})"
|
|
||||||
)
|
|
||||||
ssrf_raw = d.get("ssrf_ip_allowlist", [])
|
|
||||||
if not isinstance(ssrf_raw, list):
|
|
||||||
raise ManifestError(
|
|
||||||
f"{label}.ssrf_ip_allowlist must be an array "
|
|
||||||
f"(was {type(ssrf_raw).__name__})"
|
|
||||||
)
|
|
||||||
ssrf_ip_allowlist: list[str] = []
|
|
||||||
for j, item in enumerate(ssrf_raw):
|
|
||||||
if not isinstance(item, str) or not item:
|
|
||||||
raise ManifestError(
|
|
||||||
f"{label}.ssrf_ip_allowlist[{j}] must be a non-empty "
|
|
||||||
f"string (was {type(item).__name__})"
|
|
||||||
)
|
|
||||||
try:
|
|
||||||
ipaddress.ip_network(item, strict=False)
|
|
||||||
except ValueError as e:
|
|
||||||
raise ManifestError(
|
|
||||||
f"{label}.ssrf_ip_allowlist[{j}] must be an IP address "
|
|
||||||
f"or CIDR (was {item!r}): {e}"
|
|
||||||
)
|
|
||||||
ssrf_ip_allowlist.append(item)
|
|
||||||
return cls(
|
|
||||||
TlsPassthrough=tls_passthrough_raw,
|
|
||||||
SsrfIpAllowlist=tuple(ssrf_ip_allowlist),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class EgressRoute:
|
|
||||||
"""One route on the per-bottle egress sidecar (PRD 0017).
|
|
||||||
|
|
||||||
`Host` matches the request's hostname (case-insensitive). The
|
|
||||||
optional `PathAllowlist` constrains the URL path to a set of
|
|
||||||
prefixes; empty tuple means no path-level filtering. The optional
|
|
||||||
`AuthScheme` / `TokenRef` pair drives credential injection:
|
|
||||||
when set, the proxy strips any inbound Authorization and injects
|
|
||||||
`<AuthScheme> <value-of-host-env-named-by-TokenRef>`. When the
|
|
||||||
manifest's `auth` block is omitted both fields are empty strings —
|
|
||||||
no Authorization is written, no token forwarded.
|
|
||||||
|
|
||||||
`Role` is reserved for future use; all role strings are currently
|
|
||||||
rejected by the validator.
|
|
||||||
|
|
||||||
Validation rules (enforced in `from_dict`):
|
|
||||||
- `host` required, non-empty.
|
|
||||||
- `path_allowlist` optional, list of absolute path prefixes.
|
|
||||||
- `auth` optional. If present, MUST carry both `scheme` and
|
|
||||||
`token_ref` as non-empty strings; an empty `auth: {}` is an
|
|
||||||
error rather than a synonym for "no auth" (omit `auth` for
|
|
||||||
that case).
|
|
||||||
- `role` optional, reserved — any non-empty value is rejected.
|
|
||||||
"""
|
|
||||||
|
|
||||||
Host: str
|
|
||||||
PathAllowlist: tuple[str, ...] = ()
|
|
||||||
AuthScheme: str = ""
|
|
||||||
TokenRef: str = ""
|
|
||||||
Role: tuple[str, ...] = ()
|
|
||||||
Pipelock: PipelockRoutePolicy = field(default_factory=PipelockRoutePolicy)
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def from_dict(cls, bottle_name: str, idx: int, raw: object) -> "EgressRoute":
|
|
||||||
label = f"bottle '{bottle_name}' egress.routes[{idx}]"
|
|
||||||
d = as_json_object(raw, label)
|
|
||||||
host = d.get("host")
|
|
||||||
if not isinstance(host, str) or not host:
|
|
||||||
raise ManifestError(f"{label} missing required string field 'host'")
|
|
||||||
|
|
||||||
path_allow_raw = d.get("path_allowlist")
|
|
||||||
prefixes: tuple[str, ...] = ()
|
|
||||||
if path_allow_raw is not None:
|
|
||||||
if not isinstance(path_allow_raw, list):
|
|
||||||
raise ManifestError(
|
|
||||||
f"{label} path_allowlist must be an array "
|
|
||||||
f"(was {type(path_allow_raw).__name__})"
|
|
||||||
)
|
|
||||||
path_list = cast(list[object], path_allow_raw)
|
|
||||||
collected: list[str] = []
|
|
||||||
for j, p in enumerate(path_list):
|
|
||||||
if not isinstance(p, str):
|
|
||||||
raise ManifestError(
|
|
||||||
f"{label} path_allowlist[{j}] must be a string "
|
|
||||||
f"(was {type(p).__name__})"
|
|
||||||
)
|
|
||||||
if not p.startswith("/"):
|
|
||||||
raise ManifestError(
|
|
||||||
f"{label} path_allowlist[{j}] {p!r} must be an "
|
|
||||||
f"absolute path prefix starting with '/'"
|
|
||||||
)
|
|
||||||
collected.append(p)
|
|
||||||
prefixes = tuple(collected)
|
|
||||||
|
|
||||||
auth_scheme = ""
|
|
||||||
token_ref = ""
|
|
||||||
if "auth" in d:
|
|
||||||
auth_raw = d.get("auth")
|
|
||||||
auth_d = as_json_object(auth_raw, f"{label} auth")
|
|
||||||
if not auth_d:
|
|
||||||
raise ManifestError(
|
|
||||||
f"{label} auth is empty ({{}}); omit the 'auth' key "
|
|
||||||
f"entirely if this route is unauthenticated. Otherwise "
|
|
||||||
f"both 'scheme' and 'token_ref' are required."
|
|
||||||
)
|
|
||||||
auth_scheme_raw = auth_d.get("scheme")
|
|
||||||
if not isinstance(auth_scheme_raw, str) or not auth_scheme_raw:
|
|
||||||
raise ManifestError(
|
|
||||||
f"{label} auth.scheme is required when 'auth' is set "
|
|
||||||
f"(non-empty string)"
|
|
||||||
)
|
|
||||||
if auth_scheme_raw not in EGRESS_AUTH_SCHEMES:
|
|
||||||
raise ManifestError(
|
|
||||||
f"{label} auth.scheme {auth_scheme_raw!r} is not one of "
|
|
||||||
f"{', '.join(EGRESS_AUTH_SCHEMES)}"
|
|
||||||
)
|
|
||||||
token_ref_raw = auth_d.get("token_ref")
|
|
||||||
if not isinstance(token_ref_raw, str) or not token_ref_raw:
|
|
||||||
raise ManifestError(
|
|
||||||
f"{label} auth.token_ref is required when 'auth' is set "
|
|
||||||
f"(name of the host env var holding the token value)"
|
|
||||||
)
|
|
||||||
for k in auth_d:
|
|
||||||
if k not in ("scheme", "token_ref"):
|
|
||||||
raise ManifestError(
|
|
||||||
f"{label} auth has unknown key {k!r}; "
|
|
||||||
f"only 'scheme' and 'token_ref' are accepted"
|
|
||||||
)
|
|
||||||
auth_scheme = auth_scheme_raw
|
|
||||||
token_ref = token_ref_raw
|
|
||||||
|
|
||||||
role_raw = d.get("role")
|
|
||||||
roles: tuple[str, ...] = ()
|
|
||||||
if role_raw is None:
|
|
||||||
roles = ()
|
|
||||||
elif isinstance(role_raw, str):
|
|
||||||
roles = (role_raw,)
|
|
||||||
elif isinstance(role_raw, list):
|
|
||||||
role_list = cast(list[object], role_raw)
|
|
||||||
collected_roles: list[str] = []
|
|
||||||
for r in role_list:
|
|
||||||
if not isinstance(r, str):
|
|
||||||
raise ManifestError(f"{label} role items must be strings (got {type(r).__name__})")
|
|
||||||
collected_roles.append(r)
|
|
||||||
roles = tuple(collected_roles)
|
|
||||||
else:
|
|
||||||
raise ManifestError(
|
|
||||||
f"{label} role must be a string or a list of strings "
|
|
||||||
f"(was {type(role_raw).__name__})"
|
|
||||||
)
|
|
||||||
if roles:
|
|
||||||
raise ManifestError(
|
|
||||||
f"{label} role {roles[0]!r} is not accepted; "
|
|
||||||
f"the 'role' field is reserved for future use"
|
|
||||||
)
|
|
||||||
|
|
||||||
pipelock = (
|
|
||||||
PipelockRoutePolicy.from_dict(bottle_name, idx, d["pipelock"])
|
|
||||||
if "pipelock" in d
|
|
||||||
else PipelockRoutePolicy()
|
|
||||||
)
|
|
||||||
|
|
||||||
for k in d:
|
|
||||||
if k not in ("host", "path_allowlist", "auth", "role", "pipelock"):
|
|
||||||
raise ManifestError(
|
|
||||||
f"{label} has unknown key {k!r}; accepted keys are "
|
|
||||||
f"'host', 'path_allowlist', 'auth', 'role', 'pipelock'"
|
|
||||||
)
|
|
||||||
|
|
||||||
return cls(
|
|
||||||
Host=host,
|
|
||||||
PathAllowlist=prefixes,
|
|
||||||
AuthScheme=auth_scheme,
|
|
||||||
TokenRef=token_ref,
|
|
||||||
Role=roles,
|
|
||||||
Pipelock=pipelock,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class EgressConfig:
|
|
||||||
"""Per-bottle egress configuration. Today this is just the
|
|
||||||
route table; the nesting under `egress:` leaves room for
|
|
||||||
per-bottle proxy settings (port override, log level, etc.) in
|
|
||||||
follow-ups."""
|
|
||||||
|
|
||||||
routes: tuple[EgressRoute, ...] = ()
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def from_dict(cls, bottle_name: str, raw: object) -> "EgressConfig":
|
|
||||||
d = as_json_object(raw, f"bottle '{bottle_name}' egress")
|
|
||||||
routes_raw = d.get("routes")
|
|
||||||
routes: tuple[EgressRoute, ...] = ()
|
|
||||||
if routes_raw is not None:
|
|
||||||
if not isinstance(routes_raw, list):
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' egress.routes must be an array "
|
|
||||||
f"(was {type(routes_raw).__name__})"
|
|
||||||
)
|
|
||||||
routes_list = cast(list[object], routes_raw)
|
|
||||||
routes = tuple(
|
|
||||||
EgressRoute.from_dict(bottle_name, i, entry)
|
|
||||||
for i, entry in enumerate(routes_list)
|
|
||||||
)
|
|
||||||
validate_egress_routes(bottle_name, routes)
|
|
||||||
for k in d:
|
|
||||||
if k != "routes":
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' egress has unknown key {k!r}; "
|
|
||||||
f"only 'routes' is accepted"
|
|
||||||
)
|
|
||||||
return cls(routes=routes)
|
|
||||||
@@ -1,142 +0,0 @@
|
|||||||
"""Internal bottle `extends:` resolution for manifests."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from typing import TYPE_CHECKING
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
|
||||||
from .manifest import Bottle, GitEntry
|
|
||||||
|
|
||||||
|
|
||||||
def resolve_bottles(raws: dict[str, dict[str, object]]) -> dict[str, Bottle]:
|
|
||||||
"""Apply `extends:` chains and return resolved Bottle objects."""
|
|
||||||
cache: dict[str, Bottle] = {}
|
|
||||||
for name in raws:
|
|
||||||
if name not in cache:
|
|
||||||
_resolve_one_bottle(name, raws, cache, ())
|
|
||||||
return cache
|
|
||||||
|
|
||||||
|
|
||||||
def _resolve_one_bottle(
|
|
||||||
name: str,
|
|
||||||
raws: dict[str, dict[str, object]],
|
|
||||||
cache: dict[str, Bottle],
|
|
||||||
seen: tuple[str, ...],
|
|
||||||
) -> Bottle:
|
|
||||||
from .manifest import Bottle, ManifestError
|
|
||||||
|
|
||||||
if name in cache:
|
|
||||||
return cache[name]
|
|
||||||
if name in seen:
|
|
||||||
chain = " -> ".join(seen + (name,))
|
|
||||||
raise ManifestError(f"bottle '{name}' is in an extends cycle: {chain}")
|
|
||||||
raw = raws[name]
|
|
||||||
parent_name_raw = raw.get("extends")
|
|
||||||
# Strip `extends:` before passing to Bottle.from_dict so it
|
|
||||||
# is not accidentally treated as a real Bottle field by future
|
|
||||||
# schema additions. It is only meaningful here.
|
|
||||||
child_raw = {k: v for k, v in raw.items() if k != "extends"}
|
|
||||||
|
|
||||||
if parent_name_raw is None:
|
|
||||||
bottle = Bottle.from_dict(name, child_raw)
|
|
||||||
cache[name] = bottle
|
|
||||||
return bottle
|
|
||||||
|
|
||||||
if not isinstance(parent_name_raw, str):
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{name}' extends must be a string "
|
|
||||||
f"(was {type(parent_name_raw).__name__})"
|
|
||||||
)
|
|
||||||
parent_name: str = parent_name_raw
|
|
||||||
if parent_name == name:
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{name}' extends itself; remove the "
|
|
||||||
f"self-reference"
|
|
||||||
)
|
|
||||||
if parent_name not in raws:
|
|
||||||
avail = ", ".join(sorted(raws.keys())) or "(none)"
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{name}' extends '{parent_name}' which is not "
|
|
||||||
f"defined. Available bottles: {avail}"
|
|
||||||
)
|
|
||||||
parent = _resolve_one_bottle(parent_name, raws, cache, seen + (name,))
|
|
||||||
bottle = _merge_bottles(parent, child_raw, name)
|
|
||||||
cache[name] = bottle
|
|
||||||
return bottle
|
|
||||||
|
|
||||||
|
|
||||||
def _merge_bottles(
|
|
||||||
parent: Bottle,
|
|
||||||
child_raw: dict[str, object],
|
|
||||||
name: str,
|
|
||||||
) -> Bottle:
|
|
||||||
"""Apply PRD 0025 merge rules."""
|
|
||||||
from .manifest import Bottle, GitUser
|
|
||||||
from .manifest_egress import validate_egress_routes
|
|
||||||
|
|
||||||
# Parse the child's declared fields into a Bottle (with the
|
|
||||||
# usual defaults for anything missing). Validation runs the same
|
|
||||||
# way it would for a leaf bottle: typos / wrong types die here.
|
|
||||||
child = Bottle.from_dict(name, child_raw)
|
|
||||||
|
|
||||||
# env: dict merge, child wins on collision.
|
|
||||||
merged_env = {**parent.env, **child.env}
|
|
||||||
|
|
||||||
# git-gate.user: per-field overlay. Each non-empty field on child
|
|
||||||
# wins; empties fall through to parent. The default GitUser()
|
|
||||||
# is two empty strings, so a child that omits git-gate.user
|
|
||||||
# inherits the parent's user verbatim.
|
|
||||||
merged_git_user = GitUser(
|
|
||||||
name=child.git_user.name or parent.git_user.name,
|
|
||||||
email=child.git_user.email or parent.git_user.email,
|
|
||||||
)
|
|
||||||
|
|
||||||
# git-gate.repos: missing means inherit; an explicit empty object
|
|
||||||
# clears; otherwise parent and child merge by UpstreamHost with
|
|
||||||
# child entries replacing duplicate hosts.
|
|
||||||
if _child_declares_git_gate_repos(child_raw):
|
|
||||||
merged_git = _merge_git_remotes(parent.git, child.git) if child.git else ()
|
|
||||||
else:
|
|
||||||
merged_git = parent.git
|
|
||||||
|
|
||||||
# Presence-driven full-replace for the remaining list-valued +
|
|
||||||
# scalar fields.
|
|
||||||
merged_egress = child.egress if "egress" in child_raw else parent.egress
|
|
||||||
merged_agent_provider = (
|
|
||||||
child.agent_provider
|
|
||||||
if "agent_provider" in child_raw
|
|
||||||
else parent.agent_provider
|
|
||||||
)
|
|
||||||
merged_supervise = (
|
|
||||||
child.supervise if "supervise" in child_raw else parent.supervise
|
|
||||||
)
|
|
||||||
validate_egress_routes(name, merged_egress.routes)
|
|
||||||
|
|
||||||
return Bottle(
|
|
||||||
env=merged_env,
|
|
||||||
agent_provider=merged_agent_provider,
|
|
||||||
git=merged_git,
|
|
||||||
git_user=merged_git_user,
|
|
||||||
egress=merged_egress,
|
|
||||||
supervise=merged_supervise,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _child_declares_git_gate_repos(child_raw: dict[str, object]) -> bool:
|
|
||||||
from .manifest_util import as_json_object
|
|
||||||
|
|
||||||
git_raw = child_raw.get("git-gate")
|
|
||||||
if git_raw is None:
|
|
||||||
return False
|
|
||||||
git_obj = as_json_object(git_raw, "child git-gate")
|
|
||||||
return "repos" in git_obj
|
|
||||||
|
|
||||||
|
|
||||||
def _merge_git_remotes(
|
|
||||||
parent: tuple[GitEntry, ...],
|
|
||||||
child: tuple[GitEntry, ...],
|
|
||||||
) -> tuple[GitEntry, ...]:
|
|
||||||
by_host = {entry.UpstreamHost: entry for entry in parent}
|
|
||||||
for entry in child:
|
|
||||||
by_host[entry.UpstreamHost] = entry
|
|
||||||
return tuple(by_host.values())
|
|
||||||
@@ -1,301 +0,0 @@
|
|||||||
"""Git-related manifest dataclasses and helpers."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import re
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from typing import Optional
|
|
||||||
|
|
||||||
from .manifest_util import ManifestError, as_json_object
|
|
||||||
|
|
||||||
# Shell-safe characters for git-gate repo names. Names are embedded in
|
|
||||||
# the generated entrypoint shell script (shlex.quote is the primary
|
|
||||||
# defence; this regex is belt-and-suspenders and documents intent).
|
|
||||||
_GIT_NAME_RE = re.compile(r"^[A-Za-z0-9._-]+$")
|
|
||||||
|
|
||||||
|
|
||||||
def _opt_str(value: object, label: str) -> str:
|
|
||||||
if value is None:
|
|
||||||
return ""
|
|
||||||
if not isinstance(value, str):
|
|
||||||
raise ManifestError(f"{label} must be a string (was {type(value).__name__})")
|
|
||||||
return value
|
|
||||||
|
|
||||||
|
|
||||||
def parse_git_upstream(url: str, label: str) -> tuple[str, str, str, str]:
|
|
||||||
"""Parse `ssh://user@host[:port]/path` into (user, host, port, path).
|
|
||||||
Dies if `url` doesn't match the ssh:// shape v1 supports. Default
|
|
||||||
port is 22 (matches OpenSSH)."""
|
|
||||||
if not url.startswith("ssh://"):
|
|
||||||
raise ManifestError(f"{label} must be an ssh:// URL (was {url!r})")
|
|
||||||
rest = url[len("ssh://"):]
|
|
||||||
if "@" not in rest:
|
|
||||||
raise ManifestError(f"{label} must include a user (e.g. ssh://git@host/path.git); was {url!r}")
|
|
||||||
user, _, hostpart = rest.partition("@")
|
|
||||||
if not user:
|
|
||||||
raise ManifestError(f"{label} user is empty in {url!r}")
|
|
||||||
if "/" not in hostpart:
|
|
||||||
raise ManifestError(f"{label} must include a path (e.g. ssh://git@host/path.git); was {url!r}")
|
|
||||||
hostport, _, path = hostpart.partition("/")
|
|
||||||
if not path:
|
|
||||||
raise ManifestError(f"{label} path is empty in {url!r}")
|
|
||||||
if ":" in hostport:
|
|
||||||
host, _, port = hostport.partition(":")
|
|
||||||
if not port.isdigit():
|
|
||||||
raise ManifestError(f"{label} port must be numeric in {url!r}")
|
|
||||||
else:
|
|
||||||
host = hostport
|
|
||||||
port = "22"
|
|
||||||
if not host:
|
|
||||||
raise ManifestError(f"{label} host is empty in {url!r}")
|
|
||||||
return (user, host, port, path)
|
|
||||||
|
|
||||||
|
|
||||||
def validate_unique_git_names(bottle_name: str, git: tuple[GitEntry, ...]) -> None:
|
|
||||||
seen: dict[str, None] = {}
|
|
||||||
for g in git:
|
|
||||||
if g.Name in seen:
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' git-gate.repos has duplicate name '{g.Name}'; "
|
|
||||||
f"each entry maps to a distinct bare repo on the gate."
|
|
||||||
)
|
|
||||||
seen[g.Name] = None
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class ProvisionedKeyConfig:
|
|
||||||
"""Configuration for automatic deploy-key lifecycle management
|
|
||||||
(PRD 0048). Used when a git-gate.repos entry opts out of a
|
|
||||||
static identity file and instead wants a fresh SSH keypair
|
|
||||||
generated at spin-up and revoked at teardown.
|
|
||||||
|
|
||||||
`provider` names the contrib sub-package to load (e.g. `gitea`).
|
|
||||||
`token_env` is the name of a host-side env var carrying the API
|
|
||||||
token; the value is read at provision time, never stored on the
|
|
||||||
plan. `api_url` is the forge's HTTP API root; if empty, it is
|
|
||||||
derived from the upstream URL's host at provision time."""
|
|
||||||
|
|
||||||
provider: str
|
|
||||||
token_env: str
|
|
||||||
api_url: str = ""
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class GitEntry:
|
|
||||||
"""One upstream the per-agent git-gate (PRD 0008) is allowed to
|
|
||||||
talk to. `Upstream` is the real remote URL the agent would push to
|
|
||||||
if there were no gate; the gate hosts a bare repo at /git/<Name>.git
|
|
||||||
and `IdentityFile` is the SSH key the gate uses to push that repo
|
|
||||||
upstream after gitleaks passes. The agent itself never holds the
|
|
||||||
upstream credential.
|
|
||||||
|
|
||||||
The Upstream URL is parsed once at construction and the pieces are
|
|
||||||
stashed in the `Upstream*` fields so the git-gate render step
|
|
||||||
doesn't have to re-parse.
|
|
||||||
|
|
||||||
Manifest source: `git-gate.repos.<Name>` (PRD 0047/0048). Exactly
|
|
||||||
one of `identity` (static key path) or `provisioned_key` (automatic
|
|
||||||
lifecycle) must be present. The internal field names are stable."""
|
|
||||||
|
|
||||||
Name: str
|
|
||||||
Upstream: str
|
|
||||||
IdentityFile: str = ""
|
|
||||||
KnownHostKey: str = ""
|
|
||||||
ProvisionedKey: Optional[ProvisionedKeyConfig] = None
|
|
||||||
RemoteKey: str = ""
|
|
||||||
UpstreamUser: str = ""
|
|
||||||
UpstreamHost: str = ""
|
|
||||||
UpstreamPort: str = ""
|
|
||||||
UpstreamPath: str = ""
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def from_repos_entry(
|
|
||||||
cls, bottle_name: str, repo_name: str, raw: object
|
|
||||||
) -> "GitEntry":
|
|
||||||
"""Parse one entry from `git-gate.repos.<repo_name>`.
|
|
||||||
|
|
||||||
YAML keys: `url` (required), exactly one of `identity` or
|
|
||||||
`provisioned_key` (required), `host_key` (optional).
|
|
||||||
The repo_name becomes `Name`."""
|
|
||||||
if not repo_name:
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' git-gate.repos has an empty key"
|
|
||||||
)
|
|
||||||
if not _GIT_NAME_RE.match(repo_name):
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' git-gate.repos name {repo_name!r} is invalid; "
|
|
||||||
f"allowed characters: A-Z a-z 0-9 . _ -"
|
|
||||||
)
|
|
||||||
label = f"git-gate.repos[{repo_name!r}]"
|
|
||||||
d = as_json_object(raw, f"bottle '{bottle_name}' {label}")
|
|
||||||
for k in d:
|
|
||||||
if k not in {"url", "identity", "provisioned_key", "host_key"}:
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' {label} has unknown key {k!r}; "
|
|
||||||
f"allowed: url, identity, provisioned_key, host_key"
|
|
||||||
)
|
|
||||||
upstream = d.get("url")
|
|
||||||
if not isinstance(upstream, str) or not upstream:
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' {label} missing required string field 'url'"
|
|
||||||
)
|
|
||||||
|
|
||||||
has_identity = "identity" in d
|
|
||||||
has_provisioned = "provisioned_key" in d
|
|
||||||
if has_identity and has_provisioned:
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' {label} must set exactly one of "
|
|
||||||
f"'identity' or 'provisioned_key'; got both."
|
|
||||||
)
|
|
||||||
if not has_identity and not has_provisioned:
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' {label} must set exactly one of "
|
|
||||||
f"'identity' or 'provisioned_key'; got neither."
|
|
||||||
)
|
|
||||||
|
|
||||||
ident = ""
|
|
||||||
provisioned_key: Optional[ProvisionedKeyConfig] = None
|
|
||||||
if has_identity:
|
|
||||||
raw_ident = d.get("identity")
|
|
||||||
if not isinstance(raw_ident, str) or not raw_ident:
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' {label} 'identity' must be a non-empty string"
|
|
||||||
)
|
|
||||||
ident = raw_ident
|
|
||||||
else:
|
|
||||||
provisioned_key = _parse_provisioned_key_config(
|
|
||||||
bottle_name, label, d["provisioned_key"]
|
|
||||||
)
|
|
||||||
|
|
||||||
khk = _opt_str(
|
|
||||||
d.get("host_key"),
|
|
||||||
f"bottle '{bottle_name}' {label} host_key",
|
|
||||||
)
|
|
||||||
user, host, port, path = parse_git_upstream(
|
|
||||||
upstream, f"bottle '{bottle_name}' {label} url"
|
|
||||||
)
|
|
||||||
return cls(
|
|
||||||
Name=repo_name,
|
|
||||||
Upstream=upstream,
|
|
||||||
IdentityFile=ident,
|
|
||||||
KnownHostKey=khk,
|
|
||||||
ProvisionedKey=provisioned_key,
|
|
||||||
RemoteKey=host,
|
|
||||||
UpstreamUser=user,
|
|
||||||
UpstreamHost=host,
|
|
||||||
UpstreamPort=port,
|
|
||||||
UpstreamPath=path,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _parse_provisioned_key_config(
|
|
||||||
bottle_name: str, label: str, raw: object
|
|
||||||
) -> ProvisionedKeyConfig:
|
|
||||||
d = as_json_object(raw, f"bottle '{bottle_name}' {label}.provisioned_key")
|
|
||||||
for k in d:
|
|
||||||
if k not in {"provider", "token_env", "api_url"}:
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' {label}.provisioned_key has unknown key {k!r}; "
|
|
||||||
f"allowed: provider, token_env, api_url"
|
|
||||||
)
|
|
||||||
provider = d.get("provider")
|
|
||||||
if not isinstance(provider, str) or not provider:
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' {label}.provisioned_key missing required "
|
|
||||||
f"string field 'provider'"
|
|
||||||
)
|
|
||||||
token_env = d.get("token_env")
|
|
||||||
if not isinstance(token_env, str) or not token_env:
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' {label}.provisioned_key missing required "
|
|
||||||
f"string field 'token_env'"
|
|
||||||
)
|
|
||||||
api_url_raw = d.get("api_url", "")
|
|
||||||
if not isinstance(api_url_raw, str):
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' {label}.provisioned_key 'api_url' must be a string"
|
|
||||||
)
|
|
||||||
return ProvisionedKeyConfig(
|
|
||||||
provider=provider,
|
|
||||||
token_env=token_env,
|
|
||||||
api_url=api_url_raw,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class GitUser:
|
|
||||||
"""Per-bottle `git config --global user.name` / `user.email`
|
|
||||||
pair (issue #86). The agent's commits inside the bottle are
|
|
||||||
attributed to this identity rather than the agent image's
|
|
||||||
image-baked default (no user, or whatever the image dropped
|
|
||||||
in). Either or both fields can be set independently.
|
|
||||||
|
|
||||||
`from_dict` is forgiving on shape (a single missing field is
|
|
||||||
fine — we just skip that config line at provisioning) but
|
|
||||||
strict on types (string-or-die)."""
|
|
||||||
|
|
||||||
name: str = ""
|
|
||||||
email: str = ""
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def from_dict(cls, bottle_name: str, raw: object) -> "GitUser":
|
|
||||||
d = as_json_object(raw, f"bottle '{bottle_name}' git-gate.user")
|
|
||||||
for k in d.keys():
|
|
||||||
if k not in {"name", "email"}:
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' git-gate.user has unknown key {k!r}; "
|
|
||||||
f"allowed: name, email"
|
|
||||||
)
|
|
||||||
name = d.get("name", "")
|
|
||||||
email = d.get("email", "")
|
|
||||||
if not isinstance(name, str):
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' git-gate.user.name must be a string "
|
|
||||||
f"(was {type(name).__name__})"
|
|
||||||
)
|
|
||||||
if not isinstance(email, str):
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' git-gate.user.email must be a string "
|
|
||||||
f"(was {type(email).__name__})"
|
|
||||||
)
|
|
||||||
if not name and not email:
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' git-gate.user is set but neither "
|
|
||||||
f"name nor email is non-empty; remove the block or "
|
|
||||||
f"fill at least one field."
|
|
||||||
)
|
|
||||||
return cls(name=name, email=email)
|
|
||||||
|
|
||||||
def is_empty(self) -> bool:
|
|
||||||
return not self.name and not self.email
|
|
||||||
|
|
||||||
|
|
||||||
def parse_git_gate_config(
|
|
||||||
bottle_name: str,
|
|
||||||
raw: object,
|
|
||||||
) -> tuple[tuple[GitEntry, ...], GitUser]:
|
|
||||||
d = as_json_object(raw, f"bottle '{bottle_name}' git-gate")
|
|
||||||
for k in d.keys():
|
|
||||||
if k not in {"user", "repos"}:
|
|
||||||
raise ManifestError(
|
|
||||||
f"bottle '{bottle_name}' git-gate has unknown key {k!r}; "
|
|
||||||
f"allowed: user, repos"
|
|
||||||
)
|
|
||||||
|
|
||||||
git_user = (
|
|
||||||
GitUser.from_dict(bottle_name, d["user"])
|
|
||||||
if "user" in d
|
|
||||||
else GitUser()
|
|
||||||
)
|
|
||||||
|
|
||||||
git: tuple[GitEntry, ...] = ()
|
|
||||||
repos_raw = d.get("repos")
|
|
||||||
if repos_raw is not None:
|
|
||||||
repos = as_json_object(repos_raw, f"bottle '{bottle_name}' git-gate.repos")
|
|
||||||
git = tuple(
|
|
||||||
GitEntry.from_repos_entry(bottle_name, name, entry)
|
|
||||||
for name, entry in repos.items()
|
|
||||||
)
|
|
||||||
validate_unique_git_names(bottle_name, git)
|
|
||||||
|
|
||||||
return git, git_user
|
|
||||||
@@ -1,105 +0,0 @@
|
|||||||
"""Internal per-file Markdown manifest loader."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import TYPE_CHECKING
|
|
||||||
|
|
||||||
from .log import warn
|
|
||||||
from .manifest_schema import (
|
|
||||||
entity_name_from_path,
|
|
||||||
validate_agent_frontmatter_keys,
|
|
||||||
validate_bottle_frontmatter_keys,
|
|
||||||
)
|
|
||||||
from .yaml_subset import YamlSubsetError, parse_frontmatter
|
|
||||||
|
|
||||||
if TYPE_CHECKING:
|
|
||||||
from .manifest import Agent, Bottle
|
|
||||||
|
|
||||||
|
|
||||||
def check_stale_json(dir_path: Path, md_dir: Path, label: str) -> None:
|
|
||||||
"""Die if `<dir_path>/bot-bottle.json` exists but `md_dir` does
|
|
||||||
not. The manifest format changed in PRD 0011 and we do not want
|
|
||||||
to silently leave the JSON content unused."""
|
|
||||||
from .manifest import ManifestError
|
|
||||||
|
|
||||||
legacy = dir_path / "bot-bottle.json"
|
|
||||||
if legacy.is_file() and not md_dir.exists():
|
|
||||||
raise ManifestError(
|
|
||||||
f"found {legacy} but {md_dir} does not exist. The manifest "
|
|
||||||
f"format changed in PRD 0011 — rewrite the JSON content "
|
|
||||||
f"as per-file Markdown under {md_dir}/bottles/ and "
|
|
||||||
f"{md_dir}/agents/. See README.md for the schema. "
|
|
||||||
f"({label})"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def load_bottles_from_dir(bottles_dir: Path) -> dict[str, Bottle]:
|
|
||||||
"""Walk `<bottles_dir>/*.md`, parse each as a bottle, and return
|
|
||||||
`{name: Bottle}`. Missing dir returns an empty dict."""
|
|
||||||
from .manifest import ManifestError
|
|
||||||
from .manifest_extends import resolve_bottles
|
|
||||||
|
|
||||||
raws: dict[str, dict[str, object]] = {}
|
|
||||||
if not bottles_dir.is_dir():
|
|
||||||
return {}
|
|
||||||
for path in sorted(bottles_dir.glob("*.md")):
|
|
||||||
name = entity_name_from_path(path)
|
|
||||||
if name is None:
|
|
||||||
warn(
|
|
||||||
f"skipping {path}: filename must match "
|
|
||||||
f"[a-z][a-z0-9-]*.md (got {path.name!r})"
|
|
||||||
)
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
fm, _body = parse_frontmatter(path.read_text())
|
|
||||||
except OSError as e:
|
|
||||||
raise ManifestError(f"could not read {path}: {e}")
|
|
||||||
except YamlSubsetError as e:
|
|
||||||
raise ManifestError(f"{path}: {e}")
|
|
||||||
validate_bottle_frontmatter_keys(path, fm.keys())
|
|
||||||
raws[name] = fm
|
|
||||||
return resolve_bottles(raws)
|
|
||||||
|
|
||||||
|
|
||||||
def load_agents_from_dir(
|
|
||||||
agents_dir: Path,
|
|
||||||
bottle_names: set[str],
|
|
||||||
*,
|
|
||||||
source: str,
|
|
||||||
) -> dict[str, Agent]:
|
|
||||||
"""Walk `<agents_dir>/*.md`, parse each as an agent, and return
|
|
||||||
`{name: Agent}`. The Markdown body becomes the agent's prompt.
|
|
||||||
Missing dir returns an empty dict."""
|
|
||||||
from .manifest import Agent, ManifestError
|
|
||||||
|
|
||||||
out: dict[str, Agent] = {}
|
|
||||||
if not agents_dir.is_dir():
|
|
||||||
return out
|
|
||||||
for path in sorted(agents_dir.glob("*.md")):
|
|
||||||
name = entity_name_from_path(path)
|
|
||||||
if name is None:
|
|
||||||
warn(
|
|
||||||
f"skipping {path}: filename must match "
|
|
||||||
f"[a-z][a-z0-9-]*.md (got {path.name!r})"
|
|
||||||
)
|
|
||||||
continue
|
|
||||||
try:
|
|
||||||
fm, body = parse_frontmatter(path.read_text())
|
|
||||||
except OSError as e:
|
|
||||||
raise ManifestError(f"could not read {path}: {e}")
|
|
||||||
except YamlSubsetError as e:
|
|
||||||
raise ManifestError(f"{path}: {e}")
|
|
||||||
validate_agent_frontmatter_keys(path, fm.keys())
|
|
||||||
# Build the dict Agent.from_dict expects. The body becomes
|
|
||||||
# prompt; Claude Code passthrough fields stay in fm and get
|
|
||||||
# ignored by Agent.from_dict (reads bottle/skills/git-gate/prompt).
|
|
||||||
agent_dict: dict[str, object] = {
|
|
||||||
"bottle": fm.get("bottle"),
|
|
||||||
"skills": fm.get("skills", []),
|
|
||||||
"prompt": body.strip(),
|
|
||||||
}
|
|
||||||
if "git-gate" in fm:
|
|
||||||
agent_dict["git-gate"] = fm["git-gate"]
|
|
||||||
out[name] = Agent.from_dict(name, agent_dict, bottle_names)
|
|
||||||
return out
|
|
||||||
@@ -1,70 +0,0 @@
|
|||||||
"""Internal manifest schema policy helpers."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import re
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
# Filename-as-key uses kebab-case ASCII. The first character is a
|
|
||||||
# letter so we don't conflict with hidden files / Markdown special
|
|
||||||
# names (`.md`, `_template.md`, etc.). Filenames that fail this
|
|
||||||
# pattern are skipped with a warning rather than crashing the load.
|
|
||||||
_FILENAME_RX = re.compile(r"^[a-z][a-z0-9-]*$")
|
|
||||||
|
|
||||||
|
|
||||||
# Frontmatter keys we accept on each entity. Anything not in these
|
|
||||||
# sets dies with a "did you mean" pointer: typos should not silently
|
|
||||||
# ghost into an empty config.
|
|
||||||
BOTTLE_KEYS = frozenset(
|
|
||||||
{"env", "extends", "agent_provider", "git-gate", "egress", "supervise"}
|
|
||||||
)
|
|
||||||
AGENT_KEYS_REQUIRED = frozenset({"bottle"})
|
|
||||||
AGENT_KEYS_OPTIONAL = frozenset({"skills", "git-gate"})
|
|
||||||
|
|
||||||
# Claude Code subagent fields bot-bottle ignores at launch but does
|
|
||||||
# not reject. This lets the same file double as
|
|
||||||
# `~/.claude/agents/*.md` without modification.
|
|
||||||
CLAUDE_CODE_AGENT_PASSTHROUGH_KEYS = frozenset({
|
|
||||||
"name", "description", "model", "color", "memory",
|
|
||||||
})
|
|
||||||
AGENT_KEYS = (
|
|
||||||
AGENT_KEYS_REQUIRED | AGENT_KEYS_OPTIONAL | CLAUDE_CODE_AGENT_PASSTHROUGH_KEYS
|
|
||||||
)
|
|
||||||
AGENT_MODEL_KEYS = AGENT_KEYS | frozenset({"prompt"})
|
|
||||||
|
|
||||||
|
|
||||||
def entity_name_from_path(path: Path) -> str | None:
|
|
||||||
"""Return the entity name implied by the filename, or None if the
|
|
||||||
filename does not fit the [a-z][a-z0-9-]* convention."""
|
|
||||||
if path.suffix != ".md":
|
|
||||||
return None
|
|
||||||
stem = path.stem
|
|
||||||
if not _FILENAME_RX.match(stem):
|
|
||||||
return None
|
|
||||||
return stem
|
|
||||||
|
|
||||||
|
|
||||||
def validate_bottle_frontmatter_keys(path: Path, keys: object) -> None:
|
|
||||||
_validate_frontmatter_keys("bottle", path, keys, BOTTLE_KEYS)
|
|
||||||
|
|
||||||
|
|
||||||
def validate_agent_frontmatter_keys(path: Path, keys: object) -> None:
|
|
||||||
_validate_frontmatter_keys("agent", path, keys, AGENT_KEYS)
|
|
||||||
|
|
||||||
|
|
||||||
def _validate_frontmatter_keys(
|
|
||||||
kind: str,
|
|
||||||
path: Path,
|
|
||||||
keys: object,
|
|
||||||
allowed_keys: frozenset[str],
|
|
||||||
) -> None:
|
|
||||||
from .manifest_util import ManifestError
|
|
||||||
|
|
||||||
key_set = set(keys)
|
|
||||||
unknown = key_set - allowed_keys
|
|
||||||
if unknown:
|
|
||||||
allowed = ", ".join(sorted(allowed_keys))
|
|
||||||
raise ManifestError(
|
|
||||||
f"{kind} file {path}: unknown frontmatter key(s) "
|
|
||||||
f"{sorted(unknown)}; allowed keys are {allowed}."
|
|
||||||
)
|
|
||||||
@@ -1,24 +0,0 @@
|
|||||||
"""Shared manifest primitives used by all manifest sub-modules."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from typing import cast
|
|
||||||
|
|
||||||
|
|
||||||
class ManifestError(Exception):
|
|
||||||
"""A manifest file (or the manifest tree) is invalid."""
|
|
||||||
|
|
||||||
|
|
||||||
def as_json_object(value: object, label: str) -> dict[str, object]:
|
|
||||||
"""Assert that `value` is a JSON object (str-keyed dict) and return
|
|
||||||
a view typed as `dict[str, object]` so downstream `.get(...)` calls
|
|
||||||
have a typed surface."""
|
|
||||||
if not isinstance(value, dict):
|
|
||||||
raise ManifestError(f"{label} must be a JSON object (was {type(value).__name__})")
|
|
||||||
items = cast(dict[object, object], value)
|
|
||||||
out: dict[str, object] = {}
|
|
||||||
for k, v in items.items():
|
|
||||||
if not isinstance(k, str):
|
|
||||||
raise ManifestError(f"{label} keys must be strings (found {type(k).__name__})")
|
|
||||||
out[k] = v
|
|
||||||
return out
|
|
||||||
+10
-195
@@ -19,6 +19,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
from typing import cast
|
||||||
|
|
||||||
from .egress import EGRESS_HOSTNAME, EgressRoute, egress_routes_for_bottle
|
from .egress import EGRESS_HOSTNAME, EgressRoute, egress_routes_for_bottle
|
||||||
from .supervise import SUPERVISE_HOSTNAME
|
from .supervise import SUPERVISE_HOSTNAME
|
||||||
@@ -222,180 +223,6 @@ def pipelock_build_config(
|
|||||||
return cfg
|
return cfg
|
||||||
|
|
||||||
|
|
||||||
_PIPELOCK_TOP_LEVEL_KEYS = {
|
|
||||||
"version",
|
|
||||||
"mode",
|
|
||||||
"enforce",
|
|
||||||
"api_allowlist",
|
|
||||||
"seed_phrase_detection",
|
|
||||||
"forward_proxy",
|
|
||||||
"dlp",
|
|
||||||
"request_body_scanning",
|
|
||||||
"tls_interception",
|
|
||||||
"ssrf",
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _pipelock_render_error(section: str, key: str, expected: str) -> ValueError:
|
|
||||||
return ValueError(
|
|
||||||
f"pipelock_render_yaml: {section}.{key} must be {expected}"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _reject_unknown_keys(
|
|
||||||
section: str,
|
|
||||||
obj: dict[str, object],
|
|
||||||
allowed: set[str],
|
|
||||||
) -> None:
|
|
||||||
for key in sorted(set(obj) - allowed):
|
|
||||||
raise ValueError(f"pipelock_render_yaml: {section}.{key} is unsupported")
|
|
||||||
|
|
||||||
|
|
||||||
def _required_dict(
|
|
||||||
obj: dict[str, object],
|
|
||||||
section: str,
|
|
||||||
key: str,
|
|
||||||
) -> dict[str, object]:
|
|
||||||
value = obj.get(key)
|
|
||||||
if not isinstance(value, dict):
|
|
||||||
raise _pipelock_render_error(section, key, "a mapping")
|
|
||||||
return value
|
|
||||||
|
|
||||||
|
|
||||||
def _required_bool(obj: dict[str, object], section: str, key: str) -> bool:
|
|
||||||
value = obj.get(key)
|
|
||||||
if not isinstance(value, bool):
|
|
||||||
raise _pipelock_render_error(section, key, "a boolean")
|
|
||||||
return value
|
|
||||||
|
|
||||||
|
|
||||||
def _required_int(obj: dict[str, object], section: str, key: str) -> int:
|
|
||||||
value = obj.get(key)
|
|
||||||
if isinstance(value, bool) or not isinstance(value, int):
|
|
||||||
raise _pipelock_render_error(section, key, "an integer")
|
|
||||||
return value
|
|
||||||
|
|
||||||
|
|
||||||
def _required_str(obj: dict[str, object], section: str, key: str) -> str:
|
|
||||||
value = obj.get(key)
|
|
||||||
if not isinstance(value, str):
|
|
||||||
raise _pipelock_render_error(section, key, "a string")
|
|
||||||
return value
|
|
||||||
|
|
||||||
|
|
||||||
def _required_str_list(
|
|
||||||
obj: dict[str, object],
|
|
||||||
section: str,
|
|
||||||
key: str,
|
|
||||||
) -> list[str]:
|
|
||||||
value = obj.get(key)
|
|
||||||
if not isinstance(value, list) or not all(isinstance(v, str) for v in value):
|
|
||||||
raise _pipelock_render_error(section, key, "a list of strings")
|
|
||||||
return value
|
|
||||||
|
|
||||||
|
|
||||||
def _optional_str_list(
|
|
||||||
obj: dict[str, object],
|
|
||||||
section: str,
|
|
||||||
key: str,
|
|
||||||
) -> list[str]:
|
|
||||||
if key not in obj:
|
|
||||||
return []
|
|
||||||
return _required_str_list(obj, section, key)
|
|
||||||
|
|
||||||
|
|
||||||
def _optional_bool(
|
|
||||||
obj: dict[str, object],
|
|
||||||
section: str,
|
|
||||||
key: str,
|
|
||||||
) -> bool | None:
|
|
||||||
if key not in obj:
|
|
||||||
return None
|
|
||||||
return _required_bool(obj, section, key)
|
|
||||||
|
|
||||||
|
|
||||||
def _optional_str(
|
|
||||||
obj: dict[str, object],
|
|
||||||
section: str,
|
|
||||||
key: str,
|
|
||||||
) -> str | None:
|
|
||||||
if key not in obj:
|
|
||||||
return None
|
|
||||||
return _required_str(obj, section, key)
|
|
||||||
|
|
||||||
|
|
||||||
def _validate_pipelock_render_config(cfg: dict[str, object]) -> dict[str, object]:
|
|
||||||
_reject_unknown_keys("config", cfg, _PIPELOCK_TOP_LEVEL_KEYS)
|
|
||||||
normalized: dict[str, object] = {
|
|
||||||
"version": _required_int(cfg, "config", "version"),
|
|
||||||
"mode": _required_str(cfg, "config", "mode"),
|
|
||||||
"enforce": _required_bool(cfg, "config", "enforce"),
|
|
||||||
"api_allowlist": _required_str_list(cfg, "config", "api_allowlist"),
|
|
||||||
}
|
|
||||||
|
|
||||||
if "seed_phrase_detection" in cfg:
|
|
||||||
spd = _required_dict(cfg, "config", "seed_phrase_detection")
|
|
||||||
_reject_unknown_keys("seed_phrase_detection", spd, {"enabled"})
|
|
||||||
normalized["seed_phrase_detection"] = {
|
|
||||||
"enabled": _required_bool(spd, "seed_phrase_detection", "enabled"),
|
|
||||||
}
|
|
||||||
|
|
||||||
fp = _required_dict(cfg, "config", "forward_proxy")
|
|
||||||
_reject_unknown_keys("forward_proxy", fp, {"enabled"})
|
|
||||||
normalized["forward_proxy"] = {
|
|
||||||
"enabled": _required_bool(fp, "forward_proxy", "enabled"),
|
|
||||||
}
|
|
||||||
|
|
||||||
dlp = _required_dict(cfg, "config", "dlp")
|
|
||||||
_reject_unknown_keys("dlp", dlp, {"include_defaults", "scan_env"})
|
|
||||||
normalized["dlp"] = {
|
|
||||||
"include_defaults": _required_bool(dlp, "dlp", "include_defaults"),
|
|
||||||
"scan_env": _required_bool(dlp, "dlp", "scan_env"),
|
|
||||||
}
|
|
||||||
|
|
||||||
rbs = _required_dict(cfg, "config", "request_body_scanning")
|
|
||||||
_reject_unknown_keys(
|
|
||||||
"request_body_scanning",
|
|
||||||
rbs,
|
|
||||||
{"action", "scan_headers", "header_mode"},
|
|
||||||
)
|
|
||||||
normalized_rbs: dict[str, object] = {
|
|
||||||
"action": _required_str(rbs, "request_body_scanning", "action"),
|
|
||||||
}
|
|
||||||
scan_headers = _optional_bool(rbs, "request_body_scanning", "scan_headers")
|
|
||||||
if scan_headers is not None:
|
|
||||||
normalized_rbs["scan_headers"] = scan_headers
|
|
||||||
header_mode = _optional_str(rbs, "request_body_scanning", "header_mode")
|
|
||||||
if header_mode is not None:
|
|
||||||
normalized_rbs["header_mode"] = header_mode
|
|
||||||
normalized["request_body_scanning"] = normalized_rbs
|
|
||||||
|
|
||||||
if "tls_interception" in cfg:
|
|
||||||
tls = _required_dict(cfg, "config", "tls_interception")
|
|
||||||
_reject_unknown_keys(
|
|
||||||
"tls_interception",
|
|
||||||
tls,
|
|
||||||
{"enabled", "ca_cert", "ca_key", "passthrough_domains"},
|
|
||||||
)
|
|
||||||
normalized["tls_interception"] = {
|
|
||||||
"enabled": _required_bool(tls, "tls_interception", "enabled"),
|
|
||||||
"ca_cert": _required_str(tls, "tls_interception", "ca_cert"),
|
|
||||||
"ca_key": _required_str(tls, "tls_interception", "ca_key"),
|
|
||||||
"passthrough_domains": _optional_str_list(
|
|
||||||
tls, "tls_interception", "passthrough_domains",
|
|
||||||
),
|
|
||||||
}
|
|
||||||
|
|
||||||
if "ssrf" in cfg:
|
|
||||||
ssrf = _required_dict(cfg, "config", "ssrf")
|
|
||||||
_reject_unknown_keys("ssrf", ssrf, {"ip_allowlist"})
|
|
||||||
normalized["ssrf"] = {
|
|
||||||
"ip_allowlist": _required_str_list(ssrf, "ssrf", "ip_allowlist"),
|
|
||||||
}
|
|
||||||
|
|
||||||
return normalized
|
|
||||||
|
|
||||||
|
|
||||||
def pipelock_render_yaml(cfg: dict[str, object]) -> str:
|
def pipelock_render_yaml(cfg: dict[str, object]) -> str:
|
||||||
"""Render a pipelock config dict (as produced by
|
"""Render a pipelock config dict (as produced by
|
||||||
`pipelock_build_config`) as YAML. Hand-rolled so we don't take a
|
`pipelock_build_config`) as YAML. Hand-rolled so we don't take a
|
||||||
@@ -403,38 +230,31 @@ def pipelock_render_yaml(cfg: dict[str, object]) -> str:
|
|||||||
def _bool(b: object) -> str:
|
def _bool(b: object) -> str:
|
||||||
return "true" if b else "false"
|
return "true" if b else "false"
|
||||||
|
|
||||||
cfg = _validate_pipelock_render_config(cfg)
|
|
||||||
lines: list[str] = []
|
lines: list[str] = []
|
||||||
lines.append(f"version: {cfg['version']}")
|
lines.append(f"version: {cfg['version']}")
|
||||||
lines.append(f"mode: {cfg['mode']}")
|
lines.append(f"mode: {cfg['mode']}")
|
||||||
lines.append(f"enforce: {_bool(cfg['enforce'])}")
|
lines.append(f"enforce: {_bool(cfg['enforce'])}")
|
||||||
lines.append("")
|
lines.append("")
|
||||||
lines.append("api_allowlist:")
|
lines.append("api_allowlist:")
|
||||||
api_allowlist = cfg["api_allowlist"]
|
for h in cast(list[str], cfg["api_allowlist"]):
|
||||||
assert isinstance(api_allowlist, list)
|
|
||||||
for h in api_allowlist:
|
|
||||||
lines.append(f' - "{h}"')
|
lines.append(f' - "{h}"')
|
||||||
lines.append("")
|
lines.append("")
|
||||||
if "seed_phrase_detection" in cfg:
|
if "seed_phrase_detection" in cfg:
|
||||||
lines.append("seed_phrase_detection:")
|
lines.append("seed_phrase_detection:")
|
||||||
spd = cfg["seed_phrase_detection"]
|
spd = cast(dict[str, object], cfg["seed_phrase_detection"])
|
||||||
assert isinstance(spd, dict)
|
|
||||||
lines.append(f" enabled: {_bool(spd['enabled'])}")
|
lines.append(f" enabled: {_bool(spd['enabled'])}")
|
||||||
lines.append("")
|
lines.append("")
|
||||||
lines.append("forward_proxy:")
|
lines.append("forward_proxy:")
|
||||||
fp = cfg["forward_proxy"]
|
fp = cast(dict[str, object], cfg["forward_proxy"])
|
||||||
assert isinstance(fp, dict)
|
|
||||||
lines.append(f" enabled: {_bool(fp['enabled'])}")
|
lines.append(f" enabled: {_bool(fp['enabled'])}")
|
||||||
lines.append("")
|
lines.append("")
|
||||||
lines.append("dlp:")
|
lines.append("dlp:")
|
||||||
dlp = cfg["dlp"]
|
dlp = cast(dict[str, object], cfg["dlp"])
|
||||||
assert isinstance(dlp, dict)
|
|
||||||
lines.append(f" include_defaults: {_bool(dlp['include_defaults'])}")
|
lines.append(f" include_defaults: {_bool(dlp['include_defaults'])}")
|
||||||
lines.append(f" scan_env: {_bool(dlp['scan_env'])}")
|
lines.append(f" scan_env: {_bool(dlp['scan_env'])}")
|
||||||
lines.append("")
|
lines.append("")
|
||||||
lines.append("request_body_scanning:")
|
lines.append("request_body_scanning:")
|
||||||
rbs = cfg["request_body_scanning"]
|
rbs = cast(dict[str, object], cfg["request_body_scanning"])
|
||||||
assert isinstance(rbs, dict)
|
|
||||||
lines.append(f' action: "{rbs["action"]}"')
|
lines.append(f' action: "{rbs["action"]}"')
|
||||||
if "scan_headers" in rbs:
|
if "scan_headers" in rbs:
|
||||||
lines.append(f" scan_headers: {_bool(rbs['scan_headers'])}")
|
lines.append(f" scan_headers: {_bool(rbs['scan_headers'])}")
|
||||||
@@ -443,13 +263,11 @@ def pipelock_render_yaml(cfg: dict[str, object]) -> str:
|
|||||||
if "tls_interception" in cfg:
|
if "tls_interception" in cfg:
|
||||||
lines.append("")
|
lines.append("")
|
||||||
lines.append("tls_interception:")
|
lines.append("tls_interception:")
|
||||||
tls = cfg["tls_interception"]
|
tls = cast(dict[str, object], cfg["tls_interception"])
|
||||||
assert isinstance(tls, dict)
|
|
||||||
lines.append(f" enabled: {_bool(tls['enabled'])}")
|
lines.append(f" enabled: {_bool(tls['enabled'])}")
|
||||||
lines.append(f' ca_cert: "{tls["ca_cert"]}"')
|
lines.append(f' ca_cert: "{tls["ca_cert"]}"')
|
||||||
lines.append(f' ca_key: "{tls["ca_key"]}"')
|
lines.append(f' ca_key: "{tls["ca_key"]}"')
|
||||||
passthrough = tls["passthrough_domains"]
|
passthrough = cast(list[str], tls.get("passthrough_domains", []))
|
||||||
assert isinstance(passthrough, list)
|
|
||||||
if passthrough:
|
if passthrough:
|
||||||
lines.append(" passthrough_domains:")
|
lines.append(" passthrough_domains:")
|
||||||
for d in passthrough:
|
for d in passthrough:
|
||||||
@@ -457,12 +275,9 @@ def pipelock_render_yaml(cfg: dict[str, object]) -> str:
|
|||||||
if "ssrf" in cfg:
|
if "ssrf" in cfg:
|
||||||
lines.append("")
|
lines.append("")
|
||||||
lines.append("ssrf:")
|
lines.append("ssrf:")
|
||||||
ssrf = cfg["ssrf"]
|
ssrf = cast(dict[str, object], cfg["ssrf"])
|
||||||
assert isinstance(ssrf, dict)
|
|
||||||
lines.append(" ip_allowlist:")
|
lines.append(" ip_allowlist:")
|
||||||
ip_allowlist = ssrf["ip_allowlist"]
|
for ip in cast(list[str], ssrf["ip_allowlist"]):
|
||||||
assert isinstance(ip_allowlist, list)
|
|
||||||
for ip in ip_allowlist:
|
|
||||||
lines.append(f' - "{ip}"')
|
lines.append(f' - "{ip}"')
|
||||||
return "\n".join(lines) + "\n"
|
return "\n".join(lines) + "\n"
|
||||||
|
|
||||||
|
|||||||
@@ -163,10 +163,6 @@ class _Supervisor:
|
|||||||
# Names of children that have been logged as having exited
|
# Names of children that have been logged as having exited
|
||||||
# so we only log each death once across watch-loop ticks.
|
# so we only log each death once across watch-loop ticks.
|
||||||
self._logged_dead: set[str] = set()
|
self._logged_dead: set[str] = set()
|
||||||
# Signal handlers add daemon names here and return quickly.
|
|
||||||
# The main watch loop drains the set, so repeated restart
|
|
||||||
# requests for one daemon coalesce into one restart.
|
|
||||||
self._restart_requested: set[str] = set()
|
|
||||||
|
|
||||||
def start_all(self) -> None:
|
def start_all(self) -> None:
|
||||||
for spec in self.specs:
|
for spec in self.specs:
|
||||||
@@ -177,7 +173,6 @@ class _Supervisor:
|
|||||||
if self.shutdown_at is not None:
|
if self.shutdown_at is not None:
|
||||||
return
|
return
|
||||||
self.shutdown_at = time.monotonic()
|
self.shutdown_at = time.monotonic()
|
||||||
self._restart_requested.clear()
|
|
||||||
_log(f"shutting down ({reason}); forwarding SIGTERM")
|
_log(f"shutting down ({reason}); forwarding SIGTERM")
|
||||||
for _, p in self.procs:
|
for _, p in self.procs:
|
||||||
if p.poll() is None:
|
if p.poll() is None:
|
||||||
@@ -186,24 +181,6 @@ class _Supervisor:
|
|||||||
except ProcessLookupError:
|
except ProcessLookupError:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
def request_restart(self, daemon_name: str) -> bool:
|
|
||||||
"""Queue a daemon restart for the main loop to process.
|
|
||||||
|
|
||||||
Signal handlers use this non-blocking path instead of doing
|
|
||||||
subprocess lifecycle work directly. Requests coalesce by
|
|
||||||
daemon name: one pending restart is enough to make the daemon
|
|
||||||
reread the latest config from disk.
|
|
||||||
|
|
||||||
Returns True iff a daemon by that name is known to the
|
|
||||||
supervisor and shutdown has not started."""
|
|
||||||
if self.shutdown_at is not None:
|
|
||||||
_log(f"restart {daemon_name} skipped; supervisor is shutting down")
|
|
||||||
return False
|
|
||||||
if not any(spec.name == daemon_name for spec, _ in self.procs):
|
|
||||||
return False
|
|
||||||
self._restart_requested.add(daemon_name)
|
|
||||||
return True
|
|
||||||
|
|
||||||
def tick(self) -> bool:
|
def tick(self) -> bool:
|
||||||
"""One iteration of the watch loop. Returns True when every
|
"""One iteration of the watch loop. Returns True when every
|
||||||
child has exited and the supervisor can return.
|
child has exited and the supervisor can return.
|
||||||
@@ -211,8 +188,6 @@ class _Supervisor:
|
|||||||
A child dying unexpectedly is logged but does NOT initiate
|
A child dying unexpectedly is logged but does NOT initiate
|
||||||
shutdown — see the module docstring's failure-policy
|
shutdown — see the module docstring's failure-policy
|
||||||
section. Shutdown is signal-driven only."""
|
section. Shutdown is signal-driven only."""
|
||||||
self._drain_restart_requests()
|
|
||||||
|
|
||||||
for spec, p in self.procs:
|
for spec, p in self.procs:
|
||||||
rc = p.poll()
|
rc = p.poll()
|
||||||
if rc is None or spec.name in self._logged_dead:
|
if rc is None or spec.name in self._logged_dead:
|
||||||
@@ -245,37 +220,14 @@ class _Supervisor:
|
|||||||
except ProcessLookupError:
|
except ProcessLookupError:
|
||||||
pass
|
pass
|
||||||
|
|
||||||
done = all(p.poll() is not None for _, p in self.procs)
|
return all(p.poll() is not None for _, p in self.procs)
|
||||||
if done:
|
|
||||||
for _, p in self.procs:
|
|
||||||
if p.stdout is not None:
|
|
||||||
p.stdout.close()
|
|
||||||
return done
|
|
||||||
|
|
||||||
def exit_code(self) -> int:
|
def exit_code(self) -> int:
|
||||||
"""Positive child failures win; otherwise report success.
|
"""Worst child returncode wins. On graceful shutdown every
|
||||||
|
child is signal-killed (negative returncode) and max()
|
||||||
Python represents signal-terminated children as negative
|
returns 0; if some child crashed nonzero before the signal
|
||||||
return codes. A signal-only graceful shutdown should not leak
|
the operator gets that code on container exit."""
|
||||||
that platform-specific detail into the container exit status,
|
return max((p.returncode for _, p in self.procs), default=0)
|
||||||
but a positive crash before shutdown should remain visible."""
|
|
||||||
positives = [
|
|
||||||
p.returncode for _, p in self.procs
|
|
||||||
if p.returncode is not None and p.returncode > 0
|
|
||||||
]
|
|
||||||
return max(positives, default=0)
|
|
||||||
|
|
||||||
def _drain_restart_requests(self) -> None:
|
|
||||||
if self.shutdown_at is not None:
|
|
||||||
self._restart_requested.clear()
|
|
||||||
return
|
|
||||||
requested = tuple(sorted(self._restart_requested))
|
|
||||||
self._restart_requested.clear()
|
|
||||||
for daemon_name in requested:
|
|
||||||
if self.shutdown_at is not None:
|
|
||||||
self._restart_requested.clear()
|
|
||||||
return
|
|
||||||
self.restart_daemon(daemon_name)
|
|
||||||
|
|
||||||
def forward_signal(self, sig: int, daemon_name: str) -> bool:
|
def forward_signal(self, sig: int, daemon_name: str) -> bool:
|
||||||
"""Forward a signal to one named child. Used by the SIGHUP
|
"""Forward a signal to one named child. Used by the SIGHUP
|
||||||
@@ -340,8 +292,6 @@ class _Supervisor:
|
|||||||
except ProcessLookupError:
|
except ProcessLookupError:
|
||||||
pass
|
pass
|
||||||
p.wait()
|
p.wait()
|
||||||
if p.stdout is not None:
|
|
||||||
p.stdout.close()
|
|
||||||
self._logged_dead.discard(daemon_name)
|
self._logged_dead.discard(daemon_name)
|
||||||
new_proc = _spawn(spec)
|
new_proc = _spawn(spec)
|
||||||
self.procs[idx] = (spec, new_proc)
|
self.procs[idx] = (spec, new_proc)
|
||||||
@@ -373,7 +323,7 @@ def main(argv: Sequence[str] | None = None) -> int:
|
|||||||
# supervisor restarts the pipelock daemon in place (other
|
# supervisor restarts the pipelock daemon in place (other
|
||||||
# daemons keep running — specifically supervise, whose MCP
|
# daemons keep running — specifically supervise, whose MCP
|
||||||
# socket would drop on a whole-container `docker restart`).
|
# socket would drop on a whole-container `docker restart`).
|
||||||
signal.signal(signal.SIGUSR1, lambda *_: sup.request_restart("pipelock"))
|
signal.signal(signal.SIGUSR1, lambda *_: sup.restart_daemon("pipelock"))
|
||||||
|
|
||||||
while not sup.tick():
|
while not sup.tick():
|
||||||
time.sleep(_POLL_INTERVAL)
|
time.sleep(_POLL_INTERVAL)
|
||||||
|
|||||||
@@ -12,8 +12,8 @@ agent calls when it hits a stuck-recovery category:
|
|||||||
Each tool call: the agent passes the full proposed file plus a
|
Each tool call: the agent passes the full proposed file plus a
|
||||||
justification text. The sidecar validates the proposal syntactically,
|
justification text. The sidecar validates the proposal syntactically,
|
||||||
writes it to the host's per-bottle queue dir, and holds the tool-call
|
writes it to the host's per-bottle queue dir, and holds the tool-call
|
||||||
connection open. The operator's supervise TUI
|
connection open. The operator's TUI dashboard
|
||||||
(bot_bottle.cli.supervise) sees the proposal, accepts
|
(bot_bottle.cli.dashboard) sees the proposal, accepts
|
||||||
approve / modify / reject, and writes a response file alongside the
|
approve / modify / reject, and writes a response file alongside the
|
||||||
proposal. The sidecar sees the response and returns `{status, notes}`
|
proposal. The sidecar sees the response and returns `{status, notes}`
|
||||||
to the agent.
|
to the agent.
|
||||||
|
|||||||
@@ -35,7 +35,6 @@ import json
|
|||||||
import os
|
import os
|
||||||
import socketserver
|
import socketserver
|
||||||
import sys
|
import sys
|
||||||
import time
|
|
||||||
import typing
|
import typing
|
||||||
import urllib.error
|
import urllib.error
|
||||||
import urllib.parse
|
import urllib.parse
|
||||||
@@ -64,10 +63,6 @@ ERR_METHOD_NOT_FOUND = -32601
|
|||||||
ERR_INVALID_PARAMS = -32602
|
ERR_INVALID_PARAMS = -32602
|
||||||
ERR_INTERNAL = -32603
|
ERR_INTERNAL = -32603
|
||||||
|
|
||||||
DEFAULT_RESPONSE_TIMEOUT_SECONDS = 30.0
|
|
||||||
MIN_RESPONSE_POLL_INTERVAL_SECONDS = 0.05
|
|
||||||
EGRESS_LIST_TIMEOUT_SECONDS = 5.0
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class JsonRpcRequest:
|
class JsonRpcRequest:
|
||||||
@@ -417,7 +412,6 @@ def _validate_and_bundle_egress_route(
|
|||||||
class ServerConfig:
|
class ServerConfig:
|
||||||
bottle_slug: str
|
bottle_slug: str
|
||||||
queue_dir: Path
|
queue_dir: Path
|
||||||
response_timeout_seconds: float = DEFAULT_RESPONSE_TIMEOUT_SECONDS
|
|
||||||
|
|
||||||
|
|
||||||
def handle_initialize(_params: dict[str, object]) -> dict[str, object]:
|
def handle_initialize(_params: dict[str, object]) -> dict[str, object]:
|
||||||
@@ -448,7 +442,7 @@ def handle_list_egress_routes(
|
|||||||
})
|
})
|
||||||
opener = urllib.request.build_opener(proxy_handler)
|
opener = urllib.request.build_opener(proxy_handler)
|
||||||
try:
|
try:
|
||||||
with opener.open(_sv.EGRESS_INTROSPECT_URL, timeout=EGRESS_LIST_TIMEOUT_SECONDS) as resp:
|
with opener.open(_sv.EGRESS_INTROSPECT_URL, timeout=5) as resp:
|
||||||
body = resp.read().decode("utf-8")
|
body = resp.read().decode("utf-8")
|
||||||
except (urllib.error.URLError, OSError) as e:
|
except (urllib.error.URLError, OSError) as e:
|
||||||
return {
|
return {
|
||||||
@@ -526,20 +520,7 @@ def handle_tools_call(
|
|||||||
f"for bottle {config.bottle_slug}; waiting for operator...\n"
|
f"for bottle {config.bottle_slug}; waiting for operator...\n"
|
||||||
)
|
)
|
||||||
sys.stderr.flush()
|
sys.stderr.flush()
|
||||||
deadline = time.monotonic() + config.response_timeout_seconds
|
response = _sv.wait_for_response(config.queue_dir, proposal.id)
|
||||||
try:
|
|
||||||
response = _sv.wait_for_response(
|
|
||||||
config.queue_dir,
|
|
||||||
proposal.id,
|
|
||||||
poll_interval=MIN_RESPONSE_POLL_INTERVAL_SECONDS,
|
|
||||||
deadline=deadline,
|
|
||||||
)
|
|
||||||
except TimeoutError:
|
|
||||||
text = format_pending_response_text(config.response_timeout_seconds)
|
|
||||||
return {
|
|
||||||
"content": [{"type": "text", "text": text}],
|
|
||||||
"isError": False,
|
|
||||||
}
|
|
||||||
_sv.archive_proposal(config.queue_dir, proposal.id)
|
_sv.archive_proposal(config.queue_dir, proposal.id)
|
||||||
|
|
||||||
text = format_response_text(response)
|
text = format_response_text(response)
|
||||||
@@ -561,16 +542,6 @@ def format_response_text(response: "_sv.Response") -> str:
|
|||||||
return "\n".join(lines)
|
return "\n".join(lines)
|
||||||
|
|
||||||
|
|
||||||
def format_pending_response_text(timeout_seconds: float) -> str:
|
|
||||||
return "\n".join([
|
|
||||||
"status: pending",
|
|
||||||
(
|
|
||||||
"notes: operator response timed out after "
|
|
||||||
f"{timeout_seconds:g}s; proposal remains queued"
|
|
||||||
),
|
|
||||||
])
|
|
||||||
|
|
||||||
|
|
||||||
# --- HTTP transport --------------------------------------------------------
|
# --- HTTP transport --------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
@@ -683,15 +654,10 @@ def serve(
|
|||||||
queue_dir: Path,
|
queue_dir: Path,
|
||||||
port: int = _sv.SUPERVISE_PORT,
|
port: int = _sv.SUPERVISE_PORT,
|
||||||
bind: str = "0.0.0.0",
|
bind: str = "0.0.0.0",
|
||||||
response_timeout_seconds: float = DEFAULT_RESPONSE_TIMEOUT_SECONDS,
|
|
||||||
) -> typing.NoReturn:
|
) -> typing.NoReturn:
|
||||||
queue_dir.mkdir(parents=True, exist_ok=True)
|
queue_dir.mkdir(parents=True, exist_ok=True)
|
||||||
server = MCPServer((bind, port), MCPHandler)
|
server = MCPServer((bind, port), MCPHandler)
|
||||||
server.config = ServerConfig(
|
server.config = ServerConfig(bottle_slug=bottle_slug, queue_dir=queue_dir)
|
||||||
bottle_slug=bottle_slug,
|
|
||||||
queue_dir=queue_dir,
|
|
||||||
response_timeout_seconds=response_timeout_seconds,
|
|
||||||
)
|
|
||||||
sys.stderr.write(
|
sys.stderr.write(
|
||||||
f"supervise listening on {bind}:{port}; "
|
f"supervise listening on {bind}:{port}; "
|
||||||
f"slug={bottle_slug!r}; queue={queue_dir}; "
|
f"slug={bottle_slug!r}; queue={queue_dir}; "
|
||||||
@@ -716,37 +682,9 @@ def main(argv: list[str]) -> int:
|
|||||||
queue_dir = Path(os.environ.get("SUPERVISE_QUEUE_DIR", _sv.QUEUE_DIR_IN_CONTAINER))
|
queue_dir = Path(os.environ.get("SUPERVISE_QUEUE_DIR", _sv.QUEUE_DIR_IN_CONTAINER))
|
||||||
port = int(os.environ.get("SUPERVISE_PORT", str(_sv.SUPERVISE_PORT)))
|
port = int(os.environ.get("SUPERVISE_PORT", str(_sv.SUPERVISE_PORT)))
|
||||||
bind = os.environ.get("SUPERVISE_BIND", "0.0.0.0")
|
bind = os.environ.get("SUPERVISE_BIND", "0.0.0.0")
|
||||||
try:
|
serve(bottle_slug=bottle_slug, queue_dir=queue_dir, port=port, bind=bind)
|
||||||
response_timeout_seconds = _response_timeout_from_env(os.environ)
|
|
||||||
except ValueError as e:
|
|
||||||
sys.stderr.write(f"supervise: {e}\n")
|
|
||||||
return 2
|
|
||||||
serve(
|
|
||||||
bottle_slug=bottle_slug,
|
|
||||||
queue_dir=queue_dir,
|
|
||||||
port=port,
|
|
||||||
bind=bind,
|
|
||||||
response_timeout_seconds=response_timeout_seconds,
|
|
||||||
)
|
|
||||||
return 0 # serve() does not return
|
return 0 # serve() does not return
|
||||||
|
|
||||||
|
|
||||||
def _response_timeout_from_env(env: typing.Mapping[str, str]) -> float:
|
|
||||||
raw = env.get("SUPERVISE_RESPONSE_TIMEOUT_SECONDS", "").strip()
|
|
||||||
if not raw:
|
|
||||||
return DEFAULT_RESPONSE_TIMEOUT_SECONDS
|
|
||||||
try:
|
|
||||||
value = float(raw)
|
|
||||||
except ValueError as e:
|
|
||||||
raise ValueError(
|
|
||||||
"SUPERVISE_RESPONSE_TIMEOUT_SECONDS must be a positive number"
|
|
||||||
) from e
|
|
||||||
if value <= 0:
|
|
||||||
raise ValueError(
|
|
||||||
"SUPERVISE_RESPONSE_TIMEOUT_SECONDS must be a positive number"
|
|
||||||
)
|
|
||||||
return value
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
if __name__ == "__main__":
|
||||||
raise SystemExit(main(sys.argv))
|
raise SystemExit(main(sys.argv))
|
||||||
|
|||||||
@@ -5,18 +5,9 @@ level deeper, under their backend package."""
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import ipaddress
|
|
||||||
import os
|
import os
|
||||||
|
|
||||||
|
|
||||||
def is_ip_literal(value: str) -> bool:
|
|
||||||
try:
|
|
||||||
ipaddress.ip_address(value)
|
|
||||||
except ValueError:
|
|
||||||
return False
|
|
||||||
return True
|
|
||||||
|
|
||||||
|
|
||||||
def expand_tilde(path: str) -> str:
|
def expand_tilde(path: str) -> str:
|
||||||
"""Expand a leading '~' to $HOME. Leaves paths without a leading
|
"""Expand a leading '~' to $HOME. Leaves paths without a leading
|
||||||
tilde unchanged. Falls back to the empty string if $HOME is unset
|
tilde unchanged. Falls back to the empty string if $HOME is unset
|
||||||
|
|||||||
@@ -1,52 +0,0 @@
|
|||||||
"""Backend-neutral plan for porting the operator workspace."""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from dataclasses import dataclass
|
|
||||||
from pathlib import Path
|
|
||||||
from typing import Protocol
|
|
||||||
|
|
||||||
|
|
||||||
WORKSPACE_DIRNAME = "workspace"
|
|
||||||
DEFAULT_WORKSPACE_OWNER = "node:node"
|
|
||||||
DEFAULT_WORKSPACE_MODE = "755"
|
|
||||||
|
|
||||||
|
|
||||||
class WorkspaceSpec(Protocol):
|
|
||||||
copy_cwd: bool
|
|
||||||
user_cwd: str
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class WorkspacePlan:
|
|
||||||
"""Resolved workspace contract shared by all bottle backends."""
|
|
||||||
|
|
||||||
enabled: bool
|
|
||||||
host_path: Path
|
|
||||||
guest_home: str
|
|
||||||
guest_path: str
|
|
||||||
workdir: str
|
|
||||||
owner: str = DEFAULT_WORKSPACE_OWNER
|
|
||||||
mode: str = DEFAULT_WORKSPACE_MODE
|
|
||||||
copy_contents: bool = True
|
|
||||||
copy_git: bool = True
|
|
||||||
has_host_git_dir: bool = False
|
|
||||||
|
|
||||||
|
|
||||||
def workspace_plan(spec: WorkspaceSpec, *, guest_home: str) -> WorkspacePlan:
|
|
||||||
"""Resolve the in-bottle workspace path from CLI intent."""
|
|
||||||
host_path = Path(spec.user_cwd).expanduser()
|
|
||||||
if spec.copy_cwd:
|
|
||||||
guest_path = f"{guest_home.rstrip('/')}/{WORKSPACE_DIRNAME}"
|
|
||||||
workdir = guest_path
|
|
||||||
else:
|
|
||||||
guest_path = guest_home
|
|
||||||
workdir = guest_home
|
|
||||||
return WorkspacePlan(
|
|
||||||
enabled=spec.copy_cwd,
|
|
||||||
host_path=host_path,
|
|
||||||
guest_home=guest_home,
|
|
||||||
guest_path=guest_path,
|
|
||||||
workdir=workdir,
|
|
||||||
has_host_git_dir=(host_path / ".git").is_dir(),
|
|
||||||
)
|
|
||||||
@@ -83,7 +83,12 @@ for a declared upstream:
|
|||||||
- **Manifest field.** `bottle.git` — a list of git remotes the
|
- **Manifest field.** `bottle.git` — a list of git remotes the
|
||||||
bottle is allowed to talk to, each with the credential the gate
|
bottle is allowed to talk to, each with the credential the gate
|
||||||
uses to push upstream. The agent gets no parallel `bottle.ssh`
|
uses to push upstream. The agent gets no parallel `bottle.ssh`
|
||||||
entry for those upstreams.
|
entry for those upstreams. Each entry may also carry an
|
||||||
|
`ExtraHosts: { hostname: ip }` map, surfaced to the gate as
|
||||||
|
`--add-host` so the gate can resolve upstreams whose public DNS
|
||||||
|
doesn't point at the reachable IP (e.g. Tailscale-only hosts).
|
||||||
|
The agent-side `insteadOf` rewrite keys off the original hostname,
|
||||||
|
so the manifest's `Upstream` URL stays human-readable.
|
||||||
- **Agent-side URL rewrite.** Provisioner emits `~/.gitconfig`
|
- **Agent-side URL rewrite.** Provisioner emits `~/.gitconfig`
|
||||||
with `[url "<gate-url>"] insteadOf = <real-url>` so every git
|
with `[url "<gate-url>"] insteadOf = <real-url>` so every git
|
||||||
operation against the declared upstream (push, fetch, clone,
|
operation against the declared upstream (push, fetch, clone,
|
||||||
|
|||||||
@@ -88,7 +88,8 @@ the unused path.
|
|||||||
- **Pipelock interaction.** Drop the SSH-derived branch from
|
- **Pipelock interaction.** Drop the SSH-derived branch from
|
||||||
pipelock's `ssrf.ip_allowlist` build. With no `bottle.ssh`
|
pipelock's `ssrf.ip_allowlist` build. With no `bottle.ssh`
|
||||||
there is no per-upstream IP carve-out to render; git-gate
|
there is no per-upstream IP carve-out to render; git-gate
|
||||||
has its own egress network.
|
has its own egress network and pulls in upstream resolution
|
||||||
|
via `ExtraHosts` plus DNS.
|
||||||
- **Tests.** Delete the ssh-gate unit + integration suites,
|
- **Tests.** Delete the ssh-gate unit + integration suites,
|
||||||
the ssh fixtures in `tests/fixtures.py`, and the
|
the ssh fixtures in `tests/fixtures.py`, and the
|
||||||
shadow-route assertions in `test_manifest_git.py`. Adjust
|
shadow-route assertions in `test_manifest_git.py`. Adjust
|
||||||
|
|||||||
@@ -274,6 +274,8 @@ git:
|
|||||||
Name: bot-bottle
|
Name: bot-bottle
|
||||||
Upstream: ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git
|
Upstream: ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git
|
||||||
IdentityFile: ~/.ssh/gitea-delos-2.pem
|
IdentityFile: ~/.ssh/gitea-delos-2.pem
|
||||||
|
ExtraHosts:
|
||||||
|
gitea.dideric.is: 100.78.141.42
|
||||||
KnownHostKey: ssh-rsa AAAAB3...
|
KnownHostKey: ssh-rsa AAAAB3...
|
||||||
egress:
|
egress:
|
||||||
allowlist:
|
allowlist:
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# PRD 0019: Active agents in the dashboard, agent-scoped edit verbs
|
# PRD 0019: Active agents in the dashboard, agent-scoped edit verbs
|
||||||
|
|
||||||
- **Status:** Superseded by [PRD 0049](0049-strip-dashboard-to-supervisor-tui.md)
|
- **Status:** Active
|
||||||
- **Author:** didericis
|
- **Author:** didericis
|
||||||
- **Created:** 2026-05-26
|
- **Created:** 2026-05-26
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# PRD 0020: Start and attach to agents from inside the dashboard
|
# PRD 0020: Start and attach to agents from inside the dashboard
|
||||||
|
|
||||||
- **Status:** Superseded by [PRD 0049](0049-strip-dashboard-to-supervisor-tui.md)
|
- **Status:** Active
|
||||||
- **Author:** didericis
|
- **Author:** didericis
|
||||||
- **Created:** 2026-05-26
|
- **Created:** 2026-05-26
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# PRD 0021: Dashboard as left tmux pane, selected agent as right pane
|
# PRD 0021: Dashboard as left tmux pane, selected agent as right pane
|
||||||
|
|
||||||
- **Status:** Superseded by [PRD 0049](0049-strip-dashboard-to-supervisor-tui.md)
|
- **Status:** Active
|
||||||
- **Author:** didericis
|
- **Author:** didericis
|
||||||
- **Created:** 2026-05-26
|
- **Created:** 2026-05-26
|
||||||
|
|
||||||
|
|||||||
@@ -161,7 +161,8 @@ expectation. (Same model as shell `export` precedence.)
|
|||||||
`git.remotes` is also keyed, so it follows dict-style inheritance:
|
`git.remotes` is also keyed, so it follows dict-style inheritance:
|
||||||
children can override one host without restating every remote. The
|
children can override one host without restating every remote. The
|
||||||
remote entry is replaced as a whole on host collision because
|
remote entry is replaced as a whole on host collision because
|
||||||
`Upstream`, `IdentityFile`, and `KnownHostKey` are tightly coupled.
|
`Upstream`, `IdentityFile`, `KnownHostKey`, and `ExtraHosts` are
|
||||||
|
tightly coupled.
|
||||||
|
|
||||||
The `git.user` dataclass-overlay (each non-empty field wins
|
The `git.user` dataclass-overlay (each non-empty field wins
|
||||||
individually) is so a parent can declare `git.user.name` and a
|
individually) is so a parent can declare `git.user.name` and a
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# PRD 0029: Provider auth credentials through egress
|
# PRD 0029: Provider auth credentials through egress
|
||||||
|
|
||||||
- **Status:** Active
|
- **Status:** Draft
|
||||||
- **Author:** didericis-codex
|
- **Author:** didericis-codex
|
||||||
- **Created:** 2026-05-29
|
- **Created:** 2026-05-29
|
||||||
- **Issue:** #109
|
- **Issue:** #109
|
||||||
|
|||||||
@@ -1,121 +0,0 @@
|
|||||||
# PRD 0030: Deduplicate egress token resolution across backends
|
|
||||||
|
|
||||||
- **Status:** Active
|
|
||||||
- **Author:** didericis-claude
|
|
||||||
- **Created:** 2026-06-02
|
|
||||||
- **Issue:** #118
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
Eliminate the duplicated egress token resolution block — which resolved
|
|
||||||
manifest-declared tokens and the Codex host credential — by moving
|
|
||||||
provider-specific token reading into `AgentProvisionPlan.provisioned_env`
|
|
||||||
at prepare time, and having both backends merge that map into `os.environ`
|
|
||||||
before calling the now-generic `egress_resolve_token_values`.
|
|
||||||
|
|
||||||
## Problem
|
|
||||||
|
|
||||||
The same logic block appeared in two places:
|
|
||||||
|
|
||||||
- `bot_bottle/backend/docker/launch.py` (~line 183): inline inside
|
|
||||||
`launch()`, building `token_values` before `compose_up`.
|
|
||||||
- `bot_bottle/backend/smolmachines/launch.py` (~line 422): the private
|
|
||||||
`_resolve_token_env` helper, called before `_bundle.start_bundle`.
|
|
||||||
|
|
||||||
Both blocks:
|
|
||||||
1. Short-circuit to `{}` when there are no egress routes.
|
|
||||||
2. Call `egress_resolve_token_values(token_env_map, host_env)` to
|
|
||||||
resolve ordinary manifest-declared token refs.
|
|
||||||
3. Check `agent_provider.forward_host_credentials` and, when true,
|
|
||||||
call `codex_host_access_token` and slot the result into any
|
|
||||||
`token_env` whose `token_ref` is `CODEX_HOST_CREDENTIAL_TOKEN_REF`.
|
|
||||||
|
|
||||||
The duplication means any change to step 3 must be applied twice.
|
|
||||||
PRD 0029, which introduced `forward_host_credentials`, already had to
|
|
||||||
wire both backends; the next change would too. This is a near-certain
|
|
||||||
future sync bug.
|
|
||||||
|
|
||||||
`egress_resolve_token_values` also carried a sentinel `continue` skip
|
|
||||||
for `CODEX_HOST_CREDENTIAL_TOKEN_REF`, which tied an otherwise generic
|
|
||||||
egress helper to a Codex-specific contract.
|
|
||||||
|
|
||||||
## Goals / Success Criteria
|
|
||||||
|
|
||||||
- The `forward_host_credentials` resolution logic exists in exactly one
|
|
||||||
place in the codebase.
|
|
||||||
- Both `docker/launch.py` and `smolmachines/launch.py` call
|
|
||||||
`egress_resolve_token_values` with no provider-specific arguments.
|
|
||||||
- `egress_resolve_token_values` is fully generic — it neither knows nor
|
|
||||||
cares about provider identity or the `CODEX_HOST_CREDENTIAL_TOKEN_REF`
|
|
||||||
sentinel.
|
|
||||||
- No behaviour change for either backend.
|
|
||||||
|
|
||||||
## Non-goals
|
|
||||||
|
|
||||||
- Changes to token resolution semantics. This is a pure refactor.
|
|
||||||
- Adding support for any new credential type or provider.
|
|
||||||
- Consolidating any other backend differences beyond this one block.
|
|
||||||
|
|
||||||
## Design
|
|
||||||
|
|
||||||
### `AgentProvisionPlan.provisioned_env`
|
|
||||||
|
|
||||||
Add `provisioned_env: dict[str, str]` (default empty) to
|
|
||||||
`AgentProvisionPlan`. This map holds host-side secrets that the
|
|
||||||
provisioning stage resolved and that egress needs injected into the
|
|
||||||
sidecar environ.
|
|
||||||
|
|
||||||
When `forward_host_credentials=True` for Codex, `agent_provision_plan`
|
|
||||||
calls `codex_host_access_token(host_env)` and stores the result under
|
|
||||||
`CODEX_HOST_CREDENTIAL_TOKEN_REF`. This is already the prepare-time
|
|
||||||
stage where `write_codex_dummy_auth_file` runs, so the access-token
|
|
||||||
read is colocated with all other Codex-specific provisioning.
|
|
||||||
|
|
||||||
### Backend call sites
|
|
||||||
|
|
||||||
Both backends merge `provisioned_env` over `os.environ` before calling
|
|
||||||
`egress_resolve_token_values`:
|
|
||||||
|
|
||||||
```python
|
|
||||||
effective_env = {**dict(os.environ), **plan.agent_provision.provisioned_env}
|
|
||||||
token_values = egress_resolve_token_values(plan.egress_plan.token_env_map, effective_env)
|
|
||||||
```
|
|
||||||
|
|
||||||
`provisioned_env` values take precedence over same-named host env vars,
|
|
||||||
so the Codex token slot resolves from the map written at prepare time
|
|
||||||
rather than from a raw `os.environ` lookup.
|
|
||||||
|
|
||||||
### `egress_resolve_token_values`
|
|
||||||
|
|
||||||
Remove the `CODEX_HOST_CREDENTIAL_TOKEN_REF` sentinel `continue` skip.
|
|
||||||
The function now resolves every slot in `token_env_map` from `host_env`
|
|
||||||
without special-casing any key name. The `CODEX_HOST_CREDENTIAL_TOKEN_REF`
|
|
||||||
key is present in `effective_env` (injected by `provisioned_env`) exactly
|
|
||||||
when it is needed.
|
|
||||||
|
|
||||||
## Implementation
|
|
||||||
|
|
||||||
- `bot_bottle/agent_provider.py`: add `provisioned_env` field; populate
|
|
||||||
it for Codex when `forward_host_credentials=True`.
|
|
||||||
- `bot_bottle/egress.py`: remove sentinel skip; remove
|
|
||||||
`egress_resolve_token_values_with_provider` (the intermediate function
|
|
||||||
that was introduced and then superseded by this design); drop the
|
|
||||||
`codex_auth` import.
|
|
||||||
- `bot_bottle/backend/docker/launch.py`: replace the provider-aware
|
|
||||||
resolution block with the `effective_env` merge.
|
|
||||||
- `bot_bottle/backend/smolmachines/launch.py`: same replacement in
|
|
||||||
`_resolve_token_env`.
|
|
||||||
- `tests/unit/test_agent_provider.py`: add tests verifying
|
|
||||||
`provisioned_env` is populated (or empty) for Codex with and without
|
|
||||||
`forward_host_credentials`.
|
|
||||||
- `tests/unit/test_egress.py`: remove `TestResolveTokenValuesWithProvider`;
|
|
||||||
replace the sentinel-skip test with a test verifying the Codex token
|
|
||||||
ref resolves normally when present in `host_env`.
|
|
||||||
|
|
||||||
## References
|
|
||||||
|
|
||||||
- Issue #118: Deduplicate egress token resolution across backends.
|
|
||||||
- Issue #117: Complexity hotspots in launch, egress, and auth paths
|
|
||||||
(source of the finding).
|
|
||||||
- PRD 0029: Provider auth credentials through egress (introduced the
|
|
||||||
duplicated block).
|
|
||||||
@@ -1,241 +0,0 @@
|
|||||||
# PRD 0031: Simplify egress route merge and consolidate Route types
|
|
||||||
|
|
||||||
- **Status:** Active
|
|
||||||
- **Author:** didericis-claude
|
|
||||||
- **Created:** 2026-06-02
|
|
||||||
- **Issue:** #120
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
Replace `_merge_provider_route`'s five-case nested conditional with a
|
|
||||||
flat provisioned-wins merge, and make the mapping between the host-side
|
|
||||||
`EgressRoute` and the addon's `Route` explicit in one place. Covers the
|
|
||||||
two remaining open tasks from the #117 hotspot review.
|
|
||||||
|
|
||||||
## Problem
|
|
||||||
|
|
||||||
### 1. `_merge_provider_route` branching
|
|
||||||
|
|
||||||
`_merge_provider_route` in `bot_bottle/egress.py` handles five distinct
|
|
||||||
cases in a single function with interleaved conditions:
|
|
||||||
|
|
||||||
1. **append-new** — host not in manifest; append a fresh route.
|
|
||||||
2. **upgrade-bare** — host found, no existing auth; adopt provider auth.
|
|
||||||
3. **no-op** — host found, same auth; return unchanged.
|
|
||||||
4. **tls-passthrough upgrade** — same as no-op but provider sets
|
|
||||||
`tls_passthrough=True`; flip the flag on the existing route.
|
|
||||||
5. **conflict-die** — host found, different auth; hard error.
|
|
||||||
|
|
||||||
Cases 3 and 4 share a block with no-op as the invisible fall-through.
|
|
||||||
`_find_or_alloc_token_env` is duplicated between cases 2 and 1. In-place
|
|
||||||
replacements spell out every `EgressRoute` field explicitly, so a new
|
|
||||||
field added to the dataclass silently drops its value in any replacement
|
|
||||||
site that wasn't updated.
|
|
||||||
|
|
||||||
The root cause of the complexity is that the current merge tries to be
|
|
||||||
cooperative: it lets manifest routes coexist with provider routes and
|
|
||||||
attempts to upgrade bare manifest entries. This makes sense if the
|
|
||||||
manifest is authoritative, but the actual intended hierarchy is the
|
|
||||||
opposite — provider routes claim their hosts outright and the manifest
|
|
||||||
fills in what's left.
|
|
||||||
|
|
||||||
### 2. Three-way Route type fragmentation
|
|
||||||
|
|
||||||
`EgressRoute` (in `egress.py`) and `egress_addon_core.Route` are
|
|
||||||
separate dataclasses with overlapping but not identical field sets:
|
|
||||||
|
|
||||||
| Field | `EgressRoute` | addon `Route` |
|
|
||||||
|---|---|---|
|
|
||||||
| `host` | ✓ | ✓ |
|
|
||||||
| `path_allowlist` | ✓ | ✓ |
|
|
||||||
| `auth_scheme` | ✓ | ✓ |
|
|
||||||
| `token_env` | ✓ | ✓ |
|
|
||||||
| `token_ref` | ✓ (host-side) | — |
|
|
||||||
| `roles` | ✓ (host-side) | — |
|
|
||||||
| `tls_passthrough` | ✓ (pipelock concern) | — |
|
|
||||||
|
|
||||||
`egress_render_routes` serialises `EgressRoute` fields to YAML; the
|
|
||||||
addon's `load_routes` deserialises that YAML into `Route` objects. If a
|
|
||||||
field is added to `EgressRoute` that should appear in the YAML, both
|
|
||||||
`egress_render_routes` and `_parse_one` must be updated consistently.
|
|
||||||
The render function spells the field list out inline with no reference
|
|
||||||
to the addon's parser, so divergence is silent until runtime.
|
|
||||||
|
|
||||||
`egress_addon_core.Route` cannot be replaced by `EgressRoute` — the
|
|
||||||
addon file is copied flat into the sidecar container image (`/app/`) and
|
|
||||||
has no access to the `bot_bottle` package. The types must stay separate;
|
|
||||||
the risk is that they drift.
|
|
||||||
|
|
||||||
## Goals / Success Criteria
|
|
||||||
|
|
||||||
- `egress_routes_for_bottle` implements a flat provisioned-wins merge:
|
|
||||||
provider routes claim their hosts; manifest routes for unclaimed hosts
|
|
||||||
append. No upgrade logic, no conflict detection.
|
|
||||||
- Token slot assignment is a single pass over the merged list, not
|
|
||||||
interleaved with the merge.
|
|
||||||
- `egress_render_routes` uses a single `_route_to_yaml_fields` helper
|
|
||||||
that explicitly lists the addon-visible fields, creating one place
|
|
||||||
where the `EgressRoute`→`Route` mapping is spelled out.
|
|
||||||
- All existing `TestProviderRouteMerge` and `TestRenderRoutes` tests
|
|
||||||
pass (adjusting assertions for any semantics changes described below).
|
|
||||||
- No behaviour change for existing manifests that don't trigger the
|
|
||||||
conflict-die or upgrade-bare paths.
|
|
||||||
|
|
||||||
## Non-goals
|
|
||||||
|
|
||||||
- Merging `EgressRoute` and `egress_addon_core.Route` into one class
|
|
||||||
(impossible: addon runs in a stdlib-only container environment).
|
|
||||||
- Changing what the addon does with a route once it has one.
|
|
||||||
- Changing `decide()` or `is_git_push_request()` in `egress_addon_core`
|
|
||||||
— those are already pure functions with good separation.
|
|
||||||
|
|
||||||
## Design
|
|
||||||
|
|
||||||
### Merge: provisioned wins
|
|
||||||
|
|
||||||
The new hierarchy: **provisioned routes own their hosts; manifest routes
|
|
||||||
fill the gaps.**
|
|
||||||
|
|
||||||
```
|
|
||||||
provisioned_hosts = {pr.host.lower() for pr in provider_routes}
|
|
||||||
|
|
||||||
effective = list(provider_routes)
|
|
||||||
effective += [r for r in manifest_routes if r.host.lower() not in provisioned_hosts]
|
|
||||||
```
|
|
||||||
|
|
||||||
Token slot assignment runs as a final pass over `effective` in order:
|
|
||||||
|
|
||||||
```python
|
|
||||||
def _assign_token_slots(
|
|
||||||
routes: list[EgressRoute],
|
|
||||||
) -> tuple[EgressRoute, ...]:
|
|
||||||
slot_for_ref: dict[str, str] = {}
|
|
||||||
out: list[EgressRoute] = []
|
|
||||||
for r in routes:
|
|
||||||
if r.auth_scheme and r.token_ref and not r.token_env:
|
|
||||||
token_env = slot_for_ref.get(r.token_ref)
|
|
||||||
if token_env is None:
|
|
||||||
token_env = f"EGRESS_TOKEN_{len(slot_for_ref)}"
|
|
||||||
slot_for_ref[r.token_ref] = token_env
|
|
||||||
r = dataclasses.replace(r, token_env=token_env)
|
|
||||||
out.append(r)
|
|
||||||
return tuple(out)
|
|
||||||
```
|
|
||||||
|
|
||||||
This replaces `_merge_provider_route`, `_find_or_alloc_token_env`, and
|
|
||||||
the slot-assignment loop inside `egress_manifest_routes`.
|
|
||||||
|
|
||||||
#### Semantics change
|
|
||||||
|
|
||||||
Under the old design, a manifest route for a provisioned host with a
|
|
||||||
*different* `auth_scheme` or `token_ref` raised a hard error. Under
|
|
||||||
provisioned-wins, the manifest entry is silently dropped. Operators who
|
|
||||||
relied on the conflict error to catch misconfigurations should audit
|
|
||||||
their manifests, but in practice this path was only reachable when a
|
|
||||||
manifest declared auth for `api.openai.com` or `chatgpt.com` with a
|
|
||||||
token ref other than `CODEX_HOST_CREDENTIAL_TOKEN_REF` while also
|
|
||||||
enabling `forward_host_credentials` — an unlikely combination.
|
|
||||||
|
|
||||||
Similarly, the "upgrade-bare" path (provider adopts a bare manifest
|
|
||||||
route's `path_allowlist`) is dropped: a provisioned host takes the
|
|
||||||
provider route's fields wholesale, and the manifest's `path_allowlist`
|
|
||||||
for that host is ignored.
|
|
||||||
|
|
||||||
### Route type mapping: `_route_to_yaml_fields`
|
|
||||||
|
|
||||||
Add a pure function in `egress.py`:
|
|
||||||
|
|
||||||
```python
|
|
||||||
def _route_to_yaml_fields(r: EgressRoute) -> dict:
|
|
||||||
"""Return the addon-visible fields for one route.
|
|
||||||
|
|
||||||
This is the single authoritative mapping between `EgressRoute`
|
|
||||||
(host-side) and `egress_addon_core.Route` (sidecar-side). If a
|
|
||||||
field is added to `Route` that must appear in the YAML, add it
|
|
||||||
here and in `egress_addon_core._parse_one` together."""
|
|
||||||
fields: dict = {"host": r.host}
|
|
||||||
if r.auth_scheme and r.token_env:
|
|
||||||
fields["auth_scheme"] = r.auth_scheme
|
|
||||||
fields["token_env"] = r.token_env
|
|
||||||
if r.path_allowlist:
|
|
||||||
fields["path_allowlist"] = list(r.path_allowlist)
|
|
||||||
return fields
|
|
||||||
```
|
|
||||||
|
|
||||||
`egress_render_routes` delegates to it:
|
|
||||||
|
|
||||||
```python
|
|
||||||
for r in routes:
|
|
||||||
f = _route_to_yaml_fields(r)
|
|
||||||
lines.append(f' - host: "{f["host"]}"')
|
|
||||||
if "auth_scheme" in f:
|
|
||||||
lines.append(f' auth_scheme: "{f["auth_scheme"]}"')
|
|
||||||
lines.append(f' token_env: "{f["token_env"]}"')
|
|
||||||
if "path_allowlist" in f:
|
|
||||||
lines.append(" path_allowlist:")
|
|
||||||
for p in f["path_allowlist"]:
|
|
||||||
lines.append(f' - "{p}"')
|
|
||||||
```
|
|
||||||
|
|
||||||
The docstring on `_route_to_yaml_fields` is the explicit callout to
|
|
||||||
update both it and `_parse_one` together when the schema changes.
|
|
||||||
|
|
||||||
### `egress_manifest_routes` / `egress_routes_for_bottle`
|
|
||||||
|
|
||||||
`egress_manifest_routes` becomes a pure lifter with no slot assignment:
|
|
||||||
it reads each manifest route entry and returns an `EgressRoute` with
|
|
||||||
`token_env=""` (the slot to be filled later). The function's docstring
|
|
||||||
currently promises slot assignment; that promise moves to
|
|
||||||
`egress_routes_for_bottle`.
|
|
||||||
|
|
||||||
`egress_routes_for_bottle` becomes:
|
|
||||||
|
|
||||||
```python
|
|
||||||
def egress_routes_for_bottle(
|
|
||||||
bottle: Bottle,
|
|
||||||
provider_routes: tuple[EgressRoute, ...] = (),
|
|
||||||
) -> tuple[EgressRoute, ...]:
|
|
||||||
manifest = egress_manifest_routes(bottle)
|
|
||||||
provisioned_hosts = {pr.host.lower() for pr in provider_routes}
|
|
||||||
merged = list(provider_routes) + [
|
|
||||||
r for r in manifest if r.host.lower() not in provisioned_hosts
|
|
||||||
]
|
|
||||||
return _assign_token_slots(merged)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Mitmproxy request logic
|
|
||||||
|
|
||||||
`egress_addon_core.decide()` and `is_git_push_request()` are already
|
|
||||||
pure functions; `egress_addon.EgressAddon.request()` is the minimal
|
|
||||||
mitmproxy glue (read host/path/headers from flow → call pure functions
|
|
||||||
→ apply result to flow). This split is already clean and requires no
|
|
||||||
structural change in this PRD.
|
|
||||||
|
|
||||||
## Test impact
|
|
||||||
|
|
||||||
- **`TestProviderRouteMerge`**: the `test_provider_route_upgrades_bare_manifest_route`
|
|
||||||
test asserts that a provider route preserves a bare manifest route's
|
|
||||||
`path_allowlist`. Under provisioned-wins that `path_allowlist` is
|
|
||||||
dropped. Update the test to reflect the new semantics.
|
|
||||||
- **`test_provider_route_conflicts_with_different_authed_manifest_route`**:
|
|
||||||
the conflict-die case no longer exists. Remove this test.
|
|
||||||
- All other merge and render tests should pass without modification.
|
|
||||||
|
|
||||||
## Implementation chunks
|
|
||||||
|
|
||||||
1. **PRD (this commit).** Sets the design.
|
|
||||||
2. **Merge refactor.** Replace `_merge_provider_route` and
|
|
||||||
`_find_or_alloc_token_env` with `_assign_token_slots` and the
|
|
||||||
flat provisioned-wins logic in `egress_routes_for_bottle`. Strip
|
|
||||||
slot assignment from `egress_manifest_routes`.
|
|
||||||
3. **Render consolidation.** Add `_route_to_yaml_fields`; update
|
|
||||||
`egress_render_routes` to use it.
|
|
||||||
4. **Test updates.** Adjust `TestProviderRouteMerge` for the semantics
|
|
||||||
changes above; confirm all render tests pass.
|
|
||||||
|
|
||||||
## References
|
|
||||||
|
|
||||||
- Issue #120: Refactor `_merge_provider_route` (expanded to include
|
|
||||||
Route type fragmentation).
|
|
||||||
- Issue #117: Complexity hotspots — source of both findings.
|
|
||||||
- PRD 0030: Deduplicate egress token resolution (prior egress cleanup).
|
|
||||||
@@ -1,221 +0,0 @@
|
|||||||
# PRD 0032: Decompose smolmachines launch and harden bringup sequencing
|
|
||||||
|
|
||||||
- **Status:** Active
|
|
||||||
- **Author:** didericis-claude
|
|
||||||
- **Created:** 2026-06-02
|
|
||||||
- **Issue:** #122
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
Split `launch()` into named per-step helpers, replace the empirical
|
|
||||||
`time.sleep(1.5)` with a readiness poll, and file-lock loopback alias
|
|
||||||
allocation. Addresses the three actionable issues from the #117 hotspot
|
|
||||||
review of `smolmachines/launch.py`.
|
|
||||||
|
|
||||||
## Problem
|
|
||||||
|
|
||||||
### 1. `launch()` step ordering
|
|
||||||
|
|
||||||
`launch()` in `smolmachines/launch.py` is 207 lines. Seven sequenced
|
|
||||||
steps are marked by numbered inline comments (`# 1. Reserve a loopback
|
|
||||||
alias`, `# 2. Mint per-bottle CAs`, ...) — the sequencing is
|
|
||||||
load-bearing (CA paths must be filled before the bundle spec is built;
|
|
||||||
the bundle must be running before port discovery; the VM must be created
|
|
||||||
before the allowlist is patched), but the dependencies are enforced only
|
|
||||||
by linear ordering within one function. Adding a new daemon, changing
|
|
||||||
the port-forward strategy, or debugging a bringup failure requires
|
|
||||||
reading the whole function to understand what state each step produces.
|
|
||||||
Each step is also not individually testable without mocking the entire
|
|
||||||
surrounding context.
|
|
||||||
|
|
||||||
### 2. `time.sleep(1.5)` for libkrun exec-channel race
|
|
||||||
|
|
||||||
After `machine_start`, back-to-back `machine_exec` calls occasionally
|
|
||||||
hit a SIGKILL in libkrun's exec channel at ~100ms. The sleep is
|
|
||||||
documented as "1.5s is empirically enough; provisioning already takes
|
|
||||||
seconds so the wait is amortized." The failure mode if the sleep is
|
|
||||||
insufficient: the filesystem-repair exec (`chown -R node:node /home/node`)
|
|
||||||
is SIGKILLed silently, and the agent later bails with `ENOENT`/`EPERM`
|
|
||||||
when Claude Code tries to write to `~/.claude.json`. A poll-until-ready
|
|
||||||
loop is more robust than a fixed duration: it exits as soon as the exec
|
|
||||||
channel is up, fails loudly with a timeout if the VM never becomes
|
|
||||||
responsive, and is self-documenting about what it is waiting for.
|
|
||||||
|
|
||||||
### 3. Loopback alias allocation is not concurrent-safe
|
|
||||||
|
|
||||||
`loopback_alias.allocate()` reads docker container state to determine
|
|
||||||
which aliases are already in use, then returns the lowest free alias.
|
|
||||||
There is no lock between that read and the bundle's `docker run` (which
|
|
||||||
creates the container that will appear in future `docker ps` output). Two
|
|
||||||
simultaneous bottle launches can both see the same alias as free and
|
|
||||||
claim it, causing both bundles to bind on the same loopback IP. On macOS,
|
|
||||||
where users occasionally start multiple agents in quick succession, this
|
|
||||||
is a realistic failure mode.
|
|
||||||
|
|
||||||
## Non-goals
|
|
||||||
|
|
||||||
- Removing `force_allowlist` / the `--allow-cidr` DB patch. That is a
|
|
||||||
workaround for a smolvm 0.8.0 bug; removal is a one-liner when smolvm
|
|
||||||
honors the CLI flag upstream.
|
|
||||||
- Changing the ephemeral registry / crane detour in `local_registry.py`.
|
|
||||||
Required by Docker Desktop's network topology.
|
|
||||||
- Changing `_ensure_smolmachine`'s cache design. Cache invalidation by
|
|
||||||
docker image ID works; issue #111 tracks a separate stale-sidecar
|
|
||||||
concern.
|
|
||||||
|
|
||||||
## Design
|
|
||||||
|
|
||||||
### 1. Decompose `launch()` into named helpers
|
|
||||||
|
|
||||||
Extract six focused helpers. `launch()` becomes a coordinator that calls
|
|
||||||
them in order, passing the `ExitStack` for teardown registration:
|
|
||||||
|
|
||||||
```
|
|
||||||
_allocate_resources(plan, stack) → (loopback_ip, network)
|
|
||||||
```
|
|
||||||
Reserve the loopback alias, create the docker bridge network, register
|
|
||||||
teardown callbacks for both.
|
|
||||||
|
|
||||||
```
|
|
||||||
_mint_certs(plan) → plan
|
|
||||||
```
|
|
||||||
Pipelock TLS init (always). Egress TLS init when `plan.egress_plan.routes`
|
|
||||||
is non-empty. Returns the plan with CA paths filled via
|
|
||||||
`dataclasses.replace`.
|
|
||||||
|
|
||||||
```
|
|
||||||
_start_bundle(plan, network, loopback_ip, stack) → plan
|
|
||||||
```
|
|
||||||
Build the `BundleLaunchSpec`, resolve token env, start the bundle
|
|
||||||
container, register teardown. Returns the plan with `bundle_spec` updated
|
|
||||||
(or unchanged if no plan field carries it — callers consume `bundle_spec`
|
|
||||||
directly from this call's return value if needed).
|
|
||||||
|
|
||||||
```
|
|
||||||
_discover_urls(plan, loopback_ip) → plan
|
|
||||||
```
|
|
||||||
Look up host-side ports for the published container ports; assemble
|
|
||||||
`agent_proxy_url`, `agent_git_gate_host`, `agent_supervise_url`; stamp
|
|
||||||
them onto the plan and into `guest_env`.
|
|
||||||
|
|
||||||
```
|
|
||||||
_launch_vm(plan, agent_from_path, stack) → None
|
|
||||||
```
|
|
||||||
`machine_create` + `force_allowlist` + `machine_start`. Register
|
|
||||||
`machine_stop` and `machine_delete` teardown callbacks on the stack.
|
|
||||||
|
|
||||||
```
|
|
||||||
_init_vm(plan) → None
|
|
||||||
```
|
|
||||||
Filesystem-repair exec (`chown`/`chmod`) followed by
|
|
||||||
`_wait_exec_ready()`.
|
|
||||||
|
|
||||||
`launch()` reduces to:
|
|
||||||
|
|
||||||
```python
|
|
||||||
loopback_ip, network = _allocate_resources(plan, stack)
|
|
||||||
plan = _mint_certs(plan)
|
|
||||||
plan = _start_bundle(plan, network, loopback_ip, stack)
|
|
||||||
plan = _discover_urls(plan, loopback_ip)
|
|
||||||
agent_from_path = _ensure_smolmachine(plan.agent_image_ref,
|
|
||||||
dockerfile=plan.agent_dockerfile_path)
|
|
||||||
_launch_vm(plan, agent_from_path, stack)
|
|
||||||
_init_vm(plan)
|
|
||||||
prompt_path = provision(plan, plan.machine_name)
|
|
||||||
yield SmolmachinesBottle(...)
|
|
||||||
```
|
|
||||||
|
|
||||||
Each helper's inputs and outputs are explicit; each is independently
|
|
||||||
testable with a minimal set of mocks.
|
|
||||||
|
|
||||||
### 2. Replace `time.sleep(1.5)` with `_wait_exec_ready`
|
|
||||||
|
|
||||||
Add to `smolvm.py`:
|
|
||||||
|
|
||||||
```python
|
|
||||||
def wait_exec_ready(name: str, *, timeout: float = 5.0) -> None:
|
|
||||||
"""Poll until `machine exec true` exits 0 or `timeout` elapses.
|
|
||||||
Replaces a fixed sleep after machine_start for the libkrun
|
|
||||||
exec-channel warm-up race."""
|
|
||||||
deadline = time.monotonic() + timeout
|
|
||||||
delay = 0.1
|
|
||||||
while time.monotonic() < deadline:
|
|
||||||
r = machine_exec(name, ["true"])
|
|
||||||
if r.returncode == 0:
|
|
||||||
return
|
|
||||||
remaining = deadline - time.monotonic()
|
|
||||||
if remaining <= 0:
|
|
||||||
break
|
|
||||||
time.sleep(min(delay, remaining))
|
|
||||||
delay = min(delay * 2, 0.5)
|
|
||||||
die(
|
|
||||||
f"smolvm machine {name!r}: exec channel not ready after "
|
|
||||||
f"{timeout:.0f}s — VM may have failed to boot."
|
|
||||||
)
|
|
||||||
```
|
|
||||||
|
|
||||||
`_init_vm` calls `wait_exec_ready` after the chown/chmod exec instead of
|
|
||||||
`time.sleep(1.5)`. The `time` import in `launch.py` is removed.
|
|
||||||
|
|
||||||
### 3. File-lock loopback alias allocation
|
|
||||||
|
|
||||||
Add to `loopback_alias.py`:
|
|
||||||
|
|
||||||
```python
|
|
||||||
import fcntl
|
|
||||||
|
|
||||||
_ALLOC_LOCK_PATH = Path.home() / ".cache" / "bot-bottle" / "smolmachines.lock"
|
|
||||||
|
|
||||||
def allocate(slug: str) -> str:
|
|
||||||
if not _is_macos():
|
|
||||||
return "127.0.0.1"
|
|
||||||
_ALLOC_LOCK_PATH.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
with open(_ALLOC_LOCK_PATH, "w") as lf:
|
|
||||||
fcntl.flock(lf, fcntl.LOCK_EX)
|
|
||||||
return _allocate_locked(slug)
|
|
||||||
|
|
||||||
def _allocate_locked(slug: str) -> str:
|
|
||||||
in_use = _aliases_in_use()
|
|
||||||
for ip in _pool_addresses():
|
|
||||||
if ip not in in_use:
|
|
||||||
return ip
|
|
||||||
die(...)
|
|
||||||
return ""
|
|
||||||
```
|
|
||||||
|
|
||||||
The lock is held only for the duration of `_aliases_in_use()` + the
|
|
||||||
`allocate` return. The bundle's `docker run` runs after the lock is
|
|
||||||
released. This is sufficient: once `docker run` returns, the container
|
|
||||||
is visible in docker state and future `allocate()` calls will see it.
|
|
||||||
The remaining window (lock released → container appears in docker state)
|
|
||||||
is narrowed from "the entire bringup sequence" to "a single subprocess
|
|
||||||
call," making a collision between two concurrent launches effectively
|
|
||||||
impossible in practice.
|
|
||||||
|
|
||||||
The lock is a no-op on Linux (the `_is_macos()` early-return fires
|
|
||||||
before the lock path is opened).
|
|
||||||
|
|
||||||
## Test impact
|
|
||||||
|
|
||||||
- Unit tests for each extracted helper can mock one subprocess boundary
|
|
||||||
at a time (smolvm, docker, pipelock TLS init) without wiring the full
|
|
||||||
`launch()` ExitStack.
|
|
||||||
- `wait_exec_ready` needs a test with `machine_exec` stubbed to return
|
|
||||||
non-zero N times before 0 — verifies the backoff loop and the timeout
|
|
||||||
die path.
|
|
||||||
- `allocate` tests are unchanged in shape; the lock is acquired and
|
|
||||||
released within the call so tests don't need to be aware of it.
|
|
||||||
|
|
||||||
## Implementation chunks
|
|
||||||
|
|
||||||
1. **PRD (this commit).** Sets the design.
|
|
||||||
2. **Decompose `launch()`.**
|
|
||||||
3. **Replace sleep with `wait_exec_ready`.**
|
|
||||||
4. **File-lock `allocate()`.**
|
|
||||||
5. **Tests.** Unit tests for each helper; `wait_exec_ready` backoff + timeout.
|
|
||||||
|
|
||||||
## References
|
|
||||||
|
|
||||||
- Issue #122: Decompose smolmachines launch and harden bringup sequencing.
|
|
||||||
- Issue #117: Complexity hotspots — source of the smolmachines/launch.py finding.
|
|
||||||
- Issue #111: Smolmachine sidecar doesn't reliably get refreshed (separate, not addressed here).
|
|
||||||
@@ -1,169 +0,0 @@
|
|||||||
# PRD 0033: Manifest Schema Boundaries
|
|
||||||
|
|
||||||
- **Status:** Active
|
|
||||||
- **Author:** didericis-codex
|
|
||||||
- **Created:** 2026-06-02
|
|
||||||
- **Issue:** #125
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
Split the manifest loader's schema validation, filesystem loading, `extends:`
|
|
||||||
resolution, and compatibility passthrough policy into named internal boundaries
|
|
||||||
without changing the public manifest format. The goal is to make
|
|
||||||
`bot_bottle/manifest.py` cheaper to extend and review while preserving the
|
|
||||||
strict validation behavior that keeps manifest mistakes visible.
|
|
||||||
|
|
||||||
## Problem
|
|
||||||
|
|
||||||
`bot_bottle/manifest.py` has become a broad schema surface. It owns dataclass
|
|
||||||
models, per-field validators, per-section unknown-key policy, Markdown
|
|
||||||
frontmatter loading, two-pass bottle inheritance, merge semantics, and
|
|
||||||
effective agent-to-bottle overlays in one file. The logic is deterministic and
|
|
||||||
well covered, but the number of concerns makes schema changes expensive:
|
|
||||||
reviewers have to re-derive loader behavior, parse-time validation, and
|
|
||||||
post-parse composition rules together.
|
|
||||||
|
|
||||||
One specific coupling is especially easy to miss: agent Markdown files are
|
|
||||||
allowed to double as Claude Code subagent files, so the manifest parser accepts
|
|
||||||
and ignores Claude Code frontmatter fields such as `name`, `description`,
|
|
||||||
`model`, `color`, and `memory`. That compatibility rule is encoded as a
|
|
||||||
passthrough allowlist alongside bot-bottle's own agent schema. If Claude Code
|
|
||||||
adds a frontmatter field and users start sharing files between
|
|
||||||
`~/.claude/agents/` and `.bot-bottle/agents/`, bot-bottle raises
|
|
||||||
`ManifestError` until the local passthrough policy is updated.
|
|
||||||
|
|
||||||
The current shape is workable, but it creates unnecessary risk for future
|
|
||||||
manifest features. A new field can accidentally mix parsing, inheritance, and
|
|
||||||
compatibility concerns in the same edit, or update one entry path
|
|
||||||
(`from_json_obj`) without matching the Markdown path (`from_md_dirs`).
|
|
||||||
|
|
||||||
## Goals / Success Criteria
|
|
||||||
|
|
||||||
- Preserve the existing public manifest schema and runtime behavior.
|
|
||||||
- Keep `Manifest`, `Bottle`, `Agent`, `GitEntry`, `GitUser`, `AgentProvider`,
|
|
||||||
`EgressRoute`, `EgressConfig`, and `PipelockRoutePolicy` import-compatible
|
|
||||||
from `bot_bottle.manifest`.
|
|
||||||
- Move Markdown file discovery and frontmatter loading behind a small internal
|
|
||||||
loader boundary with tests that show `$HOME` bottles, `$HOME` agents, `$CWD`
|
|
||||||
agent overrides, and ignored `$CWD` bottles still behave as before.
|
|
||||||
- Move bottle `extends:` resolution and merge rules behind a named internal
|
|
||||||
resolver boundary with tests for inheritance, replacement, cycle detection,
|
|
||||||
missing parents, and per-field `git.user` overlays.
|
|
||||||
- Centralize top-level allowed-key policy for bottle and agent schemas so
|
|
||||||
unknown-key errors remain strict and the allowed set is visible in one place
|
|
||||||
per schema.
|
|
||||||
- Make Claude Code passthrough fields a named compatibility policy with focused
|
|
||||||
tests that distinguish accepted passthrough keys from bot-bottle schema keys
|
|
||||||
and true typos.
|
|
||||||
- Keep both entry points, `Manifest.from_json_obj` and
|
|
||||||
`Manifest.from_md_dirs`, covered by tests for shared validation and shared
|
|
||||||
inheritance behavior.
|
|
||||||
|
|
||||||
## Non-goals
|
|
||||||
|
|
||||||
- No manifest format changes.
|
|
||||||
- No migration away from Markdown frontmatter or the stdlib-only YAML subset
|
|
||||||
parser.
|
|
||||||
- No dependency on Pydantic, PyYAML, JSON Schema, or another schema framework.
|
|
||||||
- No relaxation of strict unknown-key validation for bot-bottle fields.
|
|
||||||
- No provider-specific workspace, auth, launch, or egress changes.
|
|
||||||
- No user-facing CLI behavior changes.
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
In scope:
|
|
||||||
|
|
||||||
- Internal module organization for manifest loading and composition.
|
|
||||||
- Validator helpers or schema-policy helpers that reduce duplicated
|
|
||||||
unknown-key and type-checking logic.
|
|
||||||
- Focused regression tests around the two existing load paths.
|
|
||||||
- Documentation comments that clarify compatibility policy where it is encoded.
|
|
||||||
|
|
||||||
Out of scope:
|
|
||||||
|
|
||||||
- Renaming public dataclass fields or changing their capitalization.
|
|
||||||
- Reworking callers outside the manifest boundary except for import updates
|
|
||||||
that are mechanically required by an internal split.
|
|
||||||
- Adding new manifest fields.
|
|
||||||
- Changing how `bot-bottle.json` legacy-file errors are reported.
|
|
||||||
|
|
||||||
## Design
|
|
||||||
|
|
||||||
Keep `bot_bottle.manifest` as the public facade. Existing imports should
|
|
||||||
continue to work from that module, even if implementation moves into internal
|
|
||||||
modules such as:
|
|
||||||
|
|
||||||
- `bot_bottle/manifest_model.py` for dataclasses and field-level parsing.
|
|
||||||
- `bot_bottle/manifest_loader.py` for filesystem layout, Markdown
|
|
||||||
frontmatter loading, stale legacy-file checks, and `$CWD` override rules.
|
|
||||||
- `bot_bottle/manifest_extends.py` for raw-bottle inheritance, cycle checks,
|
|
||||||
and merge semantics.
|
|
||||||
- `bot_bottle/manifest_schema.py` for allowed-key sets, passthrough policy,
|
|
||||||
and small validation helpers.
|
|
||||||
|
|
||||||
The exact filenames are not required. The required boundary is conceptual:
|
|
||||||
raw input loading, schema validation, bottle inheritance, and effective
|
|
||||||
agent-to-bottle overlays should be separable when reading and testing the code.
|
|
||||||
|
|
||||||
`Manifest.from_json_obj` should continue to accept a raw JSON-like dict and
|
|
||||||
feed the same raw bottle resolver used by Markdown loading. `Manifest.from_md_dirs`
|
|
||||||
should perform only filesystem discovery and Markdown parsing before passing
|
|
||||||
the same raw sections into the same validator/composer path. That shared path
|
|
||||||
prevents a future schema field from working in one entry point but not the
|
|
||||||
other.
|
|
||||||
|
|
||||||
Claude Code passthrough fields should be represented as an explicit
|
|
||||||
compatibility allowlist, named as such, and documented near the agent schema
|
|
||||||
policy. The parser should still ignore those fields after validation. Tests
|
|
||||||
should cover every passthrough field currently accepted and at least one
|
|
||||||
unknown field that remains an error.
|
|
||||||
|
|
||||||
The `extends:` resolver should remain raw-dict based until after inheritance is
|
|
||||||
resolved. Merge rules stay unchanged:
|
|
||||||
|
|
||||||
- scalar fields use child value when present.
|
|
||||||
- `env` merges by key with child values winning.
|
|
||||||
- `git.remotes` merges by upstream host, with child entries replacing duplicate
|
|
||||||
hosts and explicit empty maps clearing inherited remotes.
|
|
||||||
- `git.user` overlays per field.
|
|
||||||
- `egress` remains full-replace when declared by the child.
|
|
||||||
- cycles, missing parents, and self-reference remain `ManifestError`s.
|
|
||||||
|
|
||||||
## Implementation Chunks
|
|
||||||
|
|
||||||
1. Add focused characterization tests for agent allowed keys, Claude Code
|
|
||||||
passthrough fields, and parity between `from_json_obj` and Markdown loading.
|
|
||||||
2. Extract allowed-key and compatibility policy helpers while keeping
|
|
||||||
`bot_bottle.manifest` as the import surface.
|
|
||||||
3. Extract raw Markdown loading into a loader boundary and rerun existing
|
|
||||||
PRD 0011 tests unchanged.
|
|
||||||
4. Extract bottle inheritance and merge rules into a resolver boundary and
|
|
||||||
rerun existing PRD 0025 tests unchanged.
|
|
||||||
5. Trim `bot_bottle.manifest` to the public facade and model composition,
|
|
||||||
leaving compatibility imports for existing callers.
|
|
||||||
|
|
||||||
Each chunk should be mergeable on its own and should keep the test suite green.
|
|
||||||
|
|
||||||
## Testing Strategy
|
|
||||||
|
|
||||||
Run the existing manifest-focused unit tests after each chunk:
|
|
||||||
|
|
||||||
- `tests/unit/test_manifest_md_load.py`
|
|
||||||
- `tests/unit/test_manifest_extends.py`
|
|
||||||
- `tests/unit/test_manifest_git.py`
|
|
||||||
- `tests/unit/test_manifest_git_user.py`
|
|
||||||
- `tests/unit/test_manifest_agent_git_user.py`
|
|
||||||
- `tests/unit/test_manifest_egress.py`
|
|
||||||
- `tests/unit/test_manifest_runtime.py`
|
|
||||||
|
|
||||||
Add new tests only where they lock down boundary behavior not already covered,
|
|
||||||
especially compatibility passthrough and entry-point parity.
|
|
||||||
|
|
||||||
## Open Questions
|
|
||||||
|
|
||||||
- Should the Claude Code passthrough allowlist intentionally track a documented
|
|
||||||
upstream schema, or should bot-bottle keep a narrow local allowlist and update
|
|
||||||
it only when users need a new shared-file field?
|
|
||||||
- Should the public facade continue exposing every helper that tests currently
|
|
||||||
import from `bot_bottle.manifest`, or should tests move to public behavior
|
|
||||||
only during this cleanup?
|
|
||||||
@@ -1,151 +0,0 @@
|
|||||||
# PRD 0034: Sidecar Restart and Shutdown Semantics
|
|
||||||
|
|
||||||
- **Status:** Active
|
|
||||||
- **Author:** didericis-codex
|
|
||||||
- **Created:** 2026-06-02
|
|
||||||
- **Issue:** #126
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
Make the sidecar bundle supervisor's signal, restart, and exit-code behavior
|
|
||||||
explicit and easier to reason about. In particular, move pipelock restart work
|
|
||||||
out of direct SIGUSR1 handler execution while preserving the caller-visible
|
|
||||||
`docker kill --signal USR1 <bundle>` contract used by pipelock apply flows.
|
|
||||||
|
|
||||||
## Problem
|
|
||||||
|
|
||||||
`bot_bottle/sidecar_init.py` is PID 1 for the bundled sidecar container. It
|
|
||||||
starts egress, pipelock, git-gate/git-http, and supervise; forwards shutdown
|
|
||||||
signals; forwards SIGHUP to egress; and restarts pipelock on SIGUSR1 after
|
|
||||||
allowlist changes.
|
|
||||||
|
|
||||||
The current SIGUSR1 handler calls `sup.restart_daemon("pipelock")` directly.
|
|
||||||
That restart path can terminate a child, wait up to a grace timeout, SIGKILL a
|
|
||||||
stubborn child, spawn a replacement with `subprocess.Popen`, and start a new
|
|
||||||
pump thread. In CPython signal handlers run between bytecodes in the main
|
|
||||||
thread, so this is not the same as POSIX async-signal-unsafe C code, but it
|
|
||||||
still means signal handling can block the supervisor loop for the restart grace
|
|
||||||
window and makes stacked signals harder to reason about.
|
|
||||||
|
|
||||||
The exit-code contract is also easy to misread. `_Supervisor.exit_code()`
|
|
||||||
returns the maximum child return code. That preserves a positive crash code
|
|
||||||
when a child failed before graceful shutdown, but the docstring currently
|
|
||||||
frames graceful shutdown as returning zero because signal-killed children have
|
|
||||||
negative return codes. The implementation is reasonable; the contract needs to
|
|
||||||
be deliberate and tested around crash-then-shutdown behavior.
|
|
||||||
|
|
||||||
## Goals / Success Criteria
|
|
||||||
|
|
||||||
- Preserve the external signal contract:
|
|
||||||
- SIGTERM/SIGINT requests bundle shutdown.
|
|
||||||
- SIGHUP still forwards to the live egress child.
|
|
||||||
- SIGUSR1 still requests an in-place pipelock restart.
|
|
||||||
- Keep signal handlers small: handlers should record intent and return, not
|
|
||||||
perform blocking subprocess lifecycle work directly.
|
|
||||||
- Process queued restart requests from the supervisor's main loop so restart
|
|
||||||
behavior is serialized with `tick()` and shutdown state.
|
|
||||||
- Avoid respawning children after shutdown has started.
|
|
||||||
- Coalesce or serialize repeated pipelock restart requests in a documented way
|
|
||||||
so stacked SIGUSR1 delivery cannot overlap restarts.
|
|
||||||
- Clarify and test aggregate exit-code semantics:
|
|
||||||
- clean unattended exits return zero when every child exits zero.
|
|
||||||
- signal-only shutdown does not invent a positive failure code.
|
|
||||||
- a positive child crash before shutdown remains visible on supervisor exit.
|
|
||||||
- Keep the implementation stdlib-only.
|
|
||||||
|
|
||||||
## Non-goals
|
|
||||||
|
|
||||||
- No new process supervisor dependency such as supervisord, s6, or runit.
|
|
||||||
- No automatic restart policy for arbitrary unexpected child death.
|
|
||||||
- No changes to the bundle's daemon set, daemon argv, env filtering, or Docker
|
|
||||||
compose contract.
|
|
||||||
- No changes to egress route reload semantics beyond preserving SIGHUP
|
|
||||||
forwarding.
|
|
||||||
- No user-facing CLI changes.
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
In scope:
|
|
||||||
|
|
||||||
- Internal signal handling and supervisor event-loop structure in
|
|
||||||
`bot_bottle/sidecar_init.py`.
|
|
||||||
- Tests in `tests/unit/test_sidecar_init.py` for queued restart behavior,
|
|
||||||
shutdown/restart ordering, repeated restart requests, and exit-code
|
|
||||||
semantics.
|
|
||||||
- Docstring/comment updates that describe the concrete signal and exit-code
|
|
||||||
contracts.
|
|
||||||
|
|
||||||
Out of scope:
|
|
||||||
|
|
||||||
- Changing pipelock itself to reload config in process.
|
|
||||||
- Restarting egress, git-gate, git-http, or supervise on demand.
|
|
||||||
- Reporting restart events to the supervise MCP plane.
|
|
||||||
- Changing the interim policy that unexpected child death leaves surviving
|
|
||||||
daemons running.
|
|
||||||
|
|
||||||
## Design
|
|
||||||
|
|
||||||
Keep `_Supervisor` as the owner of child process state, but add an explicit
|
|
||||||
pending-action boundary between signal delivery and subprocess lifecycle work.
|
|
||||||
The exact API can be small, for example:
|
|
||||||
|
|
||||||
- `request_shutdown(reason)` keeps its existing idempotent behavior.
|
|
||||||
- `request_restart(daemon_name)` records a pending restart request unless
|
|
||||||
shutdown is already in progress.
|
|
||||||
- `tick()` drains pending restart work before or after child-death logging in a
|
|
||||||
documented order.
|
|
||||||
|
|
||||||
The SIGUSR1 handler should call only the non-blocking request method. The main
|
|
||||||
loop should continue to call `tick()` and sleep on `_POLL_INTERVAL`; `tick()`
|
|
||||||
then performs the actual `restart_daemon("pipelock")` work while normal Python
|
|
||||||
control flow is in the supervisor loop.
|
|
||||||
|
|
||||||
Repeated restart requests should not overlap. Restart requests coalesce by
|
|
||||||
daemon name: if three SIGUSR1 signals arrive before the next loop turn, one
|
|
||||||
pipelock restart is enough because each restart rereads the latest
|
|
||||||
`pipelock.yaml` from disk. This treats SIGUSR1 as "make pipelock reflect the
|
|
||||||
current config" rather than "run exactly one restart per signal."
|
|
||||||
|
|
||||||
Shutdown wins over restart. If SIGTERM/SIGINT is received while a restart is
|
|
||||||
pending, the supervisor should drop the pending restart and terminate live
|
|
||||||
children. If shutdown starts while `restart_daemon` is already executing in the
|
|
||||||
main loop, the existing restart operation may finish, but no additional queued
|
|
||||||
restart should start after shutdown state is set. A simpler implementation may
|
|
||||||
check shutdown only before each queued restart, because signal handlers execute
|
|
||||||
between bytecodes and cannot interrupt a single blocking `wait()` until control
|
|
||||||
returns to Python.
|
|
||||||
|
|
||||||
Exit-code behavior should be documented as "positive failures win, otherwise
|
|
||||||
return zero." Positive process failures remain visible, while a clean shutdown
|
|
||||||
of only zero-exit or signal-terminated children returns zero instead of leaking
|
|
||||||
platform-specific negative signal return codes to the container exit status.
|
|
||||||
|
|
||||||
## Implementation Chunks
|
|
||||||
|
|
||||||
1. Add characterization tests for SIGUSR1 queuing, repeated restart coalescing,
|
|
||||||
shutdown dropping pending restarts, and crash-then-shutdown exit codes.
|
|
||||||
2. Add a pending restart request structure to `_Supervisor` and a
|
|
||||||
non-blocking request method.
|
|
||||||
3. Change the SIGUSR1 handler in `main()` to enqueue the pipelock restart
|
|
||||||
instead of calling `restart_daemon` directly.
|
|
||||||
4. Drain pending restarts from `tick()` with shutdown checks and documented
|
|
||||||
ordering.
|
|
||||||
5. Update docstrings and comments around signal handling and `exit_code()`.
|
|
||||||
|
|
||||||
## Testing Strategy
|
|
||||||
|
|
||||||
Run the existing sidecar unit tests:
|
|
||||||
|
|
||||||
- `python3 -m unittest tests.unit.test_sidecar_init`
|
|
||||||
|
|
||||||
Add focused unit tests that avoid process-wide signal handler races where
|
|
||||||
possible by driving `_Supervisor` directly. End-to-end signal tests can remain
|
|
||||||
limited to `main()` behavior that cannot be exercised otherwise.
|
|
||||||
|
|
||||||
Also run the full unit suite before merge:
|
|
||||||
|
|
||||||
- `python3 -m unittest discover -s tests/unit`
|
|
||||||
|
|
||||||
## Open Questions
|
|
||||||
|
|
||||||
None.
|
|
||||||
@@ -1,99 +0,0 @@
|
|||||||
# PRD 0035: Supervise Wait Bounds
|
|
||||||
|
|
||||||
- **Status:** Active
|
|
||||||
- **Author:** didericis-codex
|
|
||||||
- **Created:** 2026-06-02
|
|
||||||
- **Issue:** #128
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
Bound the supervise sidecar's request-thread waits so an agent tool call cannot
|
|
||||||
hold an HTTP worker forever while waiting for operator action. Preserve the MCP
|
|
||||||
tool surface, but make timeout behavior explicit, observable, and tested.
|
|
||||||
|
|
||||||
## Problem
|
|
||||||
|
|
||||||
`bot_bottle/supervise_server.py` handles MCP over a threaded stdlib HTTP
|
|
||||||
server. Tool calls validate a proposal, write it to the supervise queue, and
|
|
||||||
then wait for the operator response file. Today that wait can last forever.
|
|
||||||
Each outstanding tool call consumes one server thread until the operator acts.
|
|
||||||
|
|
||||||
The route-listing helper also performs a live HTTP request to egress inside the
|
|
||||||
request thread. It has a short timeout today, but the behavior is not described
|
|
||||||
as part of a broader request-thread budget.
|
|
||||||
|
|
||||||
This is operationally risky in multi-agent or repeated-call scenarios: a stuck
|
|
||||||
or ignored proposal can accumulate blocked threads, and callers do not get a
|
|
||||||
clear "still pending" answer they can reason about.
|
|
||||||
|
|
||||||
## Goals / Success Criteria
|
|
||||||
|
|
||||||
- Add a bounded wait for operator responses to supervise tool calls.
|
|
||||||
- Return a clear JSON-RPC tool result when the proposal remains pending after
|
|
||||||
the timeout; do not treat pending operator action as an internal server error.
|
|
||||||
- Keep the queued proposal on disk after timeout so the operator can still act.
|
|
||||||
- Make the wait duration configurable by environment with a conservative
|
|
||||||
default.
|
|
||||||
- Preserve current success and rejection result shapes for completed operator
|
|
||||||
responses.
|
|
||||||
- Keep `list-egress-routes` bounded and document its timeout behavior.
|
|
||||||
- Add focused tests for approved responses, timed-out pending responses, and
|
|
||||||
route-list timeout/error handling.
|
|
||||||
|
|
||||||
## Non-goals
|
|
||||||
|
|
||||||
- No asynchronous framework or new runtime dependency.
|
|
||||||
- No replacement of the stdlib threaded HTTP server.
|
|
||||||
- No change to the host-side supervise queue format.
|
|
||||||
- No cancellation protocol between the agent and operator UI.
|
|
||||||
- No dashboard or TUI changes.
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
In scope:
|
|
||||||
|
|
||||||
- `bot_bottle/supervise_server.py` request handling.
|
|
||||||
- Any small helper in `bot_bottle/supervise.py` needed to support a bounded
|
|
||||||
wait cleanly.
|
|
||||||
- Unit tests around tool-call response waiting and route-list behavior.
|
|
||||||
|
|
||||||
Out of scope:
|
|
||||||
|
|
||||||
- Reworking proposal persistence.
|
|
||||||
- Changing egress apply or pipelock apply flows.
|
|
||||||
- Adding background workers to complete HTTP requests after the client returns.
|
|
||||||
|
|
||||||
## Design
|
|
||||||
|
|
||||||
Introduce a supervise response wait budget,
|
|
||||||
`SUPERVISE_RESPONSE_TIMEOUT_SECONDS`, with a 30 second default. The existing
|
|
||||||
poll loop should stop after that budget and return a normal tool result such as
|
|
||||||
`{"status": "pending", "notes": "operator response timed out; proposal remains queued"}`.
|
|
||||||
The exact field names should fit the existing response schema so agents can
|
|
||||||
handle success, rejection, and pending with one result parser.
|
|
||||||
|
|
||||||
The proposal file must remain in the queue when the HTTP call times out. The
|
|
||||||
operator can still approve or reject it later, but that later response will not
|
|
||||||
resume the original HTTP request.
|
|
||||||
|
|
||||||
Route-listing should continue to use a short HTTP timeout to egress. Errors
|
|
||||||
should be returned as tool results or JSON-RPC errors consistently with the
|
|
||||||
existing server behavior; the implementation should avoid an unbounded socket
|
|
||||||
wait in the request thread.
|
|
||||||
|
|
||||||
## Testing Strategy
|
|
||||||
|
|
||||||
- Unit-test a tool call whose response appears before the timeout.
|
|
||||||
- Unit-test a tool call whose response never appears and assert the request
|
|
||||||
returns a pending result while the proposal remains queued.
|
|
||||||
- Unit-test invalid timeout env values fall back or fail clearly.
|
|
||||||
- Unit-test `list-egress-routes` timeout/error behavior with a fake URL opener.
|
|
||||||
|
|
||||||
Run:
|
|
||||||
|
|
||||||
- `python3 -m unittest tests.unit.test_supervise_server`
|
|
||||||
- `python3 -m unittest discover -s tests/unit`
|
|
||||||
|
|
||||||
## Open Questions
|
|
||||||
|
|
||||||
None.
|
|
||||||
@@ -1,111 +0,0 @@
|
|||||||
# PRD 0036: Codex Auth Redaction Policy
|
|
||||||
|
|
||||||
- **Status:** Active
|
|
||||||
- **Author:** didericis-codex
|
|
||||||
- **Created:** 2026-06-02
|
|
||||||
- **Issue:** #129
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
Make Codex host-auth redaction explicit and fixture-driven so dummy
|
|
||||||
`auth.json` generation cannot accidentally preserve future sensitive fields.
|
|
||||||
Keep forwarding only the short-lived host access token through egress, while the
|
|
||||||
guest receives a non-secret auth file whose schema remains useful to Codex.
|
|
||||||
|
|
||||||
## Problem
|
|
||||||
|
|
||||||
`bot_bottle/codex_auth.py` reads the host Codex auth file, extracts the access
|
|
||||||
token for egress, and writes a dummy guest `auth.json`. The code redacts JWT
|
|
||||||
claims and auth JSON fields with a mix of schema-specific handling and generic
|
|
||||||
placeholder behavior.
|
|
||||||
|
|
||||||
That is safer than copying raw auth, but it is still coverage-sensitive. If
|
|
||||||
Codex adds a new field that carries a token, session identifier, refresh secret,
|
|
||||||
or account metadata and the field name does not match current heuristics, the
|
|
||||||
dummy auth file could preserve more information than intended. Because this is
|
|
||||||
credential-adjacent code, the desired behavior should be allowlist-oriented and
|
|
||||||
backed by explicit fixtures.
|
|
||||||
|
|
||||||
## Goals / Success Criteria
|
|
||||||
|
|
||||||
- Define a durable redaction policy for Codex `auth.json`:
|
|
||||||
- host access token is read for egress only.
|
|
||||||
- guest dummy auth contains no bearer, refresh, session, or secret values.
|
|
||||||
- selected non-secret fields may be preserved only when needed by Codex.
|
|
||||||
- Prefer explicit per-field preservation over broad heuristic pass-through.
|
|
||||||
- Add representative fixture tests for current Codex auth shapes.
|
|
||||||
- Add regression tests for unknown nested fields, sensitive-looking field names,
|
|
||||||
lists, dictionaries, and JWT custom claims.
|
|
||||||
- Preserve dummy token expiration alignment with the host access token.
|
|
||||||
- Keep existing errors for missing, invalid, non-device, or expired auth.
|
|
||||||
|
|
||||||
## Non-goals
|
|
||||||
|
|
||||||
- No change to the egress credential-forwarding contract.
|
|
||||||
- No attempt to refresh Codex tokens inside the bottle.
|
|
||||||
- No copying of refresh tokens or raw host auth into the guest.
|
|
||||||
- No dependency on a Codex SDK or external schema package.
|
|
||||||
- No user-facing CLI changes.
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
In scope:
|
|
||||||
|
|
||||||
- `bot_bottle/codex_auth.py` redaction helpers.
|
|
||||||
- Unit tests in `tests/unit/test_codex_auth.py`.
|
|
||||||
- Small documentation comments that distinguish preserved non-secret fields from
|
|
||||||
redacted credential material.
|
|
||||||
|
|
||||||
Out of scope:
|
|
||||||
|
|
||||||
- Provider provisioning outside Codex auth file generation.
|
|
||||||
- Egress route construction for Codex.
|
|
||||||
- Runtime calls to Codex/OpenAI services.
|
|
||||||
|
|
||||||
## Design
|
|
||||||
|
|
||||||
Treat the dummy guest `auth.json` as a deliberately synthesized compatibility
|
|
||||||
file, not as a redacted copy of the host file. The implementation may continue
|
|
||||||
to start from the host object for convenience, but preserved fields should be
|
|
||||||
controlled by explicit allowlists at known schema locations.
|
|
||||||
|
|
||||||
At the top level, preserve only `auth_mode`, replace `OPENAI_API_KEY` /
|
|
||||||
`openai_api_key` with `null`, and synthesize the `tokens` block. Unknown scalar
|
|
||||||
top-level fields become placeholders, unknown lists become empty lists, and
|
|
||||||
unknown dictionaries become empty objects.
|
|
||||||
|
|
||||||
In token blocks, replace `access_token` and `id_token` with dummy JWTs, preserve
|
|
||||||
the selected non-secret `account_id`, and redact every other token-block field
|
|
||||||
with the same placeholder / empty container policy. Refresh, session, and future
|
|
||||||
token values are never copied to the guest.
|
|
||||||
|
|
||||||
In JWT payloads, preserve only claims that are known to be non-secret and
|
|
||||||
required for Codex behavior. Unknown scalar claims become placeholders, unknown
|
|
||||||
lists become empty lists, and unknown objects become empty objects.
|
|
||||||
|
|
||||||
For the OpenAI auth claim, preserve only currently necessary non-secret values
|
|
||||||
such as plan type, selected account id, and boolean localhost state. Everything
|
|
||||||
else is placeholder, empty object, or empty list according to the policy.
|
|
||||||
|
|
||||||
Tests should use fixture auth objects that include both current expected fields
|
|
||||||
and intentionally hostile future-looking fields such as `session_context`,
|
|
||||||
`bearer`, `refreshSecret`, nested `token_value`, and opaque arrays. The dummy
|
|
||||||
output must not contain the original secret strings.
|
|
||||||
|
|
||||||
## Testing Strategy
|
|
||||||
|
|
||||||
- Existing `tests/unit/test_codex_auth.py` should continue to pass.
|
|
||||||
- Add tests that assert original access/refresh/session strings do not appear in
|
|
||||||
`codex_dummy_auth_json`.
|
|
||||||
- Add tests for nested JWT and auth-claim redaction behavior.
|
|
||||||
- Add tests that the dummy access/id token `exp` still matches the host access
|
|
||||||
token expiry.
|
|
||||||
|
|
||||||
Run:
|
|
||||||
|
|
||||||
- `python3 -m unittest tests.unit.test_codex_auth`
|
|
||||||
- `python3 -m unittest discover -s tests/unit`
|
|
||||||
|
|
||||||
## Open Questions
|
|
||||||
|
|
||||||
None.
|
|
||||||
@@ -1,113 +0,0 @@
|
|||||||
# PRD 0037: Pipelock YAML Render Contract
|
|
||||||
|
|
||||||
- **Status:** Active
|
|
||||||
- **Author:** didericis-codex
|
|
||||||
- **Created:** 2026-06-02
|
|
||||||
- **Issue:** #130
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
Lock down the contract between `pipelock_build_config` and
|
|
||||||
`pipelock_render_yaml` so hand-rendered pipelock YAML stays aligned with the
|
|
||||||
structured config bot-bottle builds. Keep the stdlib-only renderer, but add
|
|
||||||
shape validation and semantic tests for every supported section.
|
|
||||||
|
|
||||||
## Problem
|
|
||||||
|
|
||||||
`bot_bottle/pipelock.py` builds a structured dict and then renders a fixed YAML
|
|
||||||
shape by hand. This avoids a runtime YAML dependency, but it also means the
|
|
||||||
renderer directly indexes expected keys. If `pipelock_build_config` adds,
|
|
||||||
renames, or conditionalizes a section, rendering can fail at runtime or emit
|
|
||||||
YAML that no longer matches the config semantics.
|
|
||||||
|
|
||||||
Existing tests assert important rendered fragments, but they do not fully lock
|
|
||||||
the build/render contract or optional-section combinations. A mismatch here can
|
|
||||||
weaken DLP enforcement or break bottle launch after a future pipelock policy
|
|
||||||
change.
|
|
||||||
|
|
||||||
## Goals / Success Criteria
|
|
||||||
|
|
||||||
- Keep the renderer stdlib-only.
|
|
||||||
- Define the supported pipelock config shape in one place.
|
|
||||||
- Fail clearly when `pipelock_render_yaml` receives an unsupported or malformed
|
|
||||||
config shape.
|
|
||||||
- Add tests covering all supported sections:
|
|
||||||
- base allowlist and forward proxy.
|
|
||||||
- seed phrase detection toggle.
|
|
||||||
- DLP and request-body/header scanning.
|
|
||||||
- TLS interception and passthrough domains.
|
|
||||||
- SSRF IP allowlist.
|
|
||||||
- Add semantic tests that compare structured config values to rendered YAML
|
|
||||||
output without relying only on brittle substring assertions.
|
|
||||||
- Preserve current rendered YAML for existing configs unless a clearer failure
|
|
||||||
path requires an error message change.
|
|
||||||
|
|
||||||
## Non-goals
|
|
||||||
|
|
||||||
- No PyYAML or other runtime dependency.
|
|
||||||
- No change to pipelock policy defaults.
|
|
||||||
- No change to egress-to-pipelock topology.
|
|
||||||
- No change to pipelock image version or config schema beyond validation of the
|
|
||||||
shape bot-bottle already emits.
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
In scope:
|
|
||||||
|
|
||||||
- `bot_bottle/pipelock.py` render helpers and validation.
|
|
||||||
- Unit tests in `tests/unit/test_pipelock_yaml.py` and related focused
|
|
||||||
pipelock tests.
|
|
||||||
- Small helper functions for typed access to config sections, if useful.
|
|
||||||
|
|
||||||
Out of scope:
|
|
||||||
|
|
||||||
- Launch/backend changes.
|
|
||||||
- Integration tests that start a real pipelock container.
|
|
||||||
- Changing the manifest schema for route-level pipelock policy.
|
|
||||||
|
|
||||||
## Design
|
|
||||||
|
|
||||||
Treat `pipelock_render_yaml` as a serializer for the narrow config shape
|
|
||||||
produced by `pipelock_build_config`, not as a generic YAML renderer. Before
|
|
||||||
rendering a section, validate that required keys exist with the expected
|
|
||||||
primitive/list/dict types. Missing or unsupported shapes should raise a clear
|
|
||||||
`ValueError` naming the section and key.
|
|
||||||
|
|
||||||
The supported top-level shape is `version`, `mode`, `enforce`,
|
|
||||||
`api_allowlist`, `seed_phrase_detection`, `forward_proxy`, `dlp`,
|
|
||||||
`request_body_scanning`, `tls_interception`, and `ssrf`. Required sections are
|
|
||||||
validated before rendering; optional sections keep the current omission
|
|
||||||
behavior. `request_body_scanning.scan_headers`,
|
|
||||||
`request_body_scanning.header_mode`, and
|
|
||||||
`tls_interception.passthrough_domains` remain optional for compatibility with
|
|
||||||
parsed running configs that only contain the older rendered subset.
|
|
||||||
|
|
||||||
Tests should cover both normal output and failure cases. Because the project is
|
|
||||||
stdlib-only, semantic tests can use a small purpose-built parser for the exact
|
|
||||||
rendered shape or compare rendered lines to values from the structured config
|
|
||||||
through helper assertions. The goal is to detect drift between config dict and
|
|
||||||
YAML without adding a general YAML dependency.
|
|
||||||
|
|
||||||
Optional sections should be exercised in combinations:
|
|
||||||
|
|
||||||
- no TLS and no SSRF.
|
|
||||||
- TLS enabled with empty and non-empty passthrough domains.
|
|
||||||
- SSRF enabled with one or more IP/CIDR entries.
|
|
||||||
- all optional sections enabled together.
|
|
||||||
|
|
||||||
## Testing Strategy
|
|
||||||
|
|
||||||
- Extend `tests/unit/test_pipelock_yaml.py` with semantic assertions tying each
|
|
||||||
rendered section back to the config dict.
|
|
||||||
- Add malformed-config tests for missing required keys and wrong section types.
|
|
||||||
- Keep existing render fragment tests where they protect exact pipelock syntax.
|
|
||||||
|
|
||||||
Run:
|
|
||||||
|
|
||||||
- `python3 -m unittest tests.unit.test_pipelock_yaml`
|
|
||||||
- `python3 -m unittest tests.unit.test_pipelock_allowlist`
|
|
||||||
- `python3 -m unittest discover -s tests/unit`
|
|
||||||
|
|
||||||
## Open Questions
|
|
||||||
|
|
||||||
None.
|
|
||||||
@@ -1,102 +0,0 @@
|
|||||||
# PRD 0038: smolmachines Env Contract and Secret-Safe Injection
|
|
||||||
|
|
||||||
- **Status:** Active
|
|
||||||
- **Author:** didericis-codex
|
|
||||||
- **Created:** 2026-06-02
|
|
||||||
- **Issue:** #135
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
Make smolmachines env handling match Docker's contract: resolve manifest env
|
|
||||||
entries through `resolve_env()`, keep secret and interpolated values out of
|
|
||||||
host argv, and document or enforce an explicit env contract for the backend.
|
|
||||||
|
|
||||||
## Problem
|
|
||||||
|
|
||||||
`bot_bottle/backend/smolmachines/prepare.py` builds the guest env from
|
|
||||||
`bottle.env` directly, bypassing `resolve_env()`. Entries like `?prompt` and
|
|
||||||
`${HOST_VAR}` can reach the guest literally rather than being prompted or
|
|
||||||
resolved. In contrast, Docker resolves env through `resolve_env()` before
|
|
||||||
writing a mode-600 env file.
|
|
||||||
|
|
||||||
`smolmachines/smolvm.py` renders env as `-e KEY=VALUE` on `smolvm machine
|
|
||||||
create` argv, and `SmolmachinesBottle.agent_argv` / `exec` prepend
|
|
||||||
`env KEY=VALUE …` onto the `smolvm machine exec` argv. Any literal or resolved
|
|
||||||
secret value is therefore visible in the host process table.
|
|
||||||
|
|
||||||
The two backends have no shared env contract document. Divergence will silently
|
|
||||||
widen as new manifest env features are added.
|
|
||||||
|
|
||||||
## Goals / Success Criteria
|
|
||||||
|
|
||||||
- Manifest env entries are resolved through `resolve_env()` before being
|
|
||||||
injected into the smolmachines guest, matching Docker behaviour.
|
|
||||||
- No manifest env value (literal or resolved) appears on host argv during
|
|
||||||
machine creation or exec.
|
|
||||||
- Define and document an explicit smolmachines env contract covering literals,
|
|
||||||
`?prompt` secrets, and `${HOST_VAR}` interpolations.
|
|
||||||
- Unit tests cover: literal passthrough, prompted-secret resolution,
|
|
||||||
host-var interpolation, and the no-argv-leak invariant.
|
|
||||||
|
|
||||||
## Non-goals
|
|
||||||
|
|
||||||
- No changes to the Docker env path.
|
|
||||||
- No changes to manifest schema or `resolve_env()` itself.
|
|
||||||
- No changes to smolmachines networking or mount handling.
|
|
||||||
- No new runtime dependencies.
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
In scope:
|
|
||||||
|
|
||||||
- `bot_bottle/backend/smolmachines/prepare.py` env resolution.
|
|
||||||
- `bot_bottle/backend/smolmachines/smolvm.py` machine-create argv.
|
|
||||||
- `bot_bottle/backend/smolmachines/bottle.py` `agent_argv` / `exec` env
|
|
||||||
injection.
|
|
||||||
- `bot_bottle/env.py` if helper changes are needed to support the smolmachines
|
|
||||||
path.
|
|
||||||
- Unit tests in `tests/unit/` covering the above.
|
|
||||||
|
|
||||||
Out of scope:
|
|
||||||
|
|
||||||
- Integration tests that start a live smolmachines VM.
|
|
||||||
- Docker backend changes.
|
|
||||||
- Dashboard or CLI changes.
|
|
||||||
|
|
||||||
## Design
|
|
||||||
|
|
||||||
Run smolmachines env through `resolve_env()` at prepare time, exactly as Docker
|
|
||||||
does. After resolution, inject env into the guest through a mechanism that does
|
|
||||||
not expose values on host argv — for example by writing a mode-600 env file
|
|
||||||
into the machine's state directory and loading it at exec time, or by passing
|
|
||||||
env through `smolvm`'s stdin if the tool supports it.
|
|
||||||
|
|
||||||
If `smolvm` provides no stdin or env-file injection path, document this as a
|
|
||||||
known limitation and at minimum move env values behind a per-invocation
|
|
||||||
tmpfile rather than inline argv.
|
|
||||||
|
|
||||||
The env contract for smolmachines should mirror Docker's:
|
|
||||||
|
|
||||||
- Literals: passed as-is after resolution.
|
|
||||||
- `?prompt` entries: prompted at prepare time; resolved value injected, never
|
|
||||||
on argv.
|
|
||||||
- `${HOST_VAR}` entries: interpolated from the operator's env at prepare time;
|
|
||||||
resolved value injected, never on argv.
|
|
||||||
|
|
||||||
## Testing Strategy
|
|
||||||
|
|
||||||
- Unit tests for `prepare.py` asserting `resolve_env()` is called and that
|
|
||||||
resolution results are used rather than raw `bottle.env` values.
|
|
||||||
- Unit tests for `smolvm.py` machine-create argv asserting no env value appears
|
|
||||||
inline.
|
|
||||||
- Unit tests for `bottle.py` exec path asserting the same argv invariant.
|
|
||||||
|
|
||||||
Run:
|
|
||||||
|
|
||||||
- `python3 -m unittest tests.unit.test_smolmachines_prepare`
|
|
||||||
- `python3 -m unittest discover -s tests/unit`
|
|
||||||
|
|
||||||
## Open Questions
|
|
||||||
|
|
||||||
- Does `smolvm machine create` support an env-file flag or stdin injection that
|
|
||||||
avoids `-e KEY=VALUE` argv?
|
|
||||||
@@ -1,87 +0,0 @@
|
|||||||
# PRD 0039: smolmachines Capability-Block Remediation
|
|
||||||
|
|
||||||
- **Status:** Active
|
|
||||||
- **Author:** didericis-codex
|
|
||||||
- **Created:** 2026-06-02
|
|
||||||
- **Issue:** #136
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
Make capability-block remediation backend-aware. Today the dashboard approval
|
|
||||||
path calls Docker-only teardown and apply code regardless of which backend
|
|
||||||
created the bottle. Either implement smolmachines remediation or add a clean
|
|
||||||
disable/unsupported path so operators never get a partial Docker teardown
|
|
||||||
against a smolmachines slug.
|
|
||||||
|
|
||||||
## Problem
|
|
||||||
|
|
||||||
`bot_bottle/cli/dashboard.py` dispatches every capability-block approval to
|
|
||||||
`bot_bottle/backend/docker/capability_apply.py`. That code snapshots with
|
|
||||||
`docker cp`, pushes via `docker exec`, rewrites a Dockerfile override, and
|
|
||||||
removes Docker containers and networks. It does not stop or delete a smolvm
|
|
||||||
machine.
|
|
||||||
|
|
||||||
smolmachines bottles still receive the capability-block supervise tool through
|
|
||||||
`backend/smolmachines/provision/supervise.py`, so agents can queue a
|
|
||||||
remediation the host cannot correctly apply. A partial Docker teardown against
|
|
||||||
a smolmachines slug corrupts neither backend cleanly.
|
|
||||||
|
|
||||||
## Goals / Success Criteria
|
|
||||||
|
|
||||||
- Capability-block approval is routed to backend-specific code.
|
|
||||||
- For the smolmachines backend, either:
|
|
||||||
a. A real remediation implementation that stops the VM, applies the
|
|
||||||
capability change, and restarts correctly; or
|
|
||||||
b. A clean unsupported response that tells the operator the action cannot
|
|
||||||
be taken and leaves the bottle in a consistent state.
|
|
||||||
- If option (b): smolmachines agents do not receive the capability-block tool,
|
|
||||||
so the operator is never prompted for an action that will fail.
|
|
||||||
- Unit tests cover the dispatch logic and the smolmachines path.
|
|
||||||
|
|
||||||
## Non-goals
|
|
||||||
|
|
||||||
- No changes to the Docker capability-apply path.
|
|
||||||
- No changes to other supervise tools (cred-block, pipelock-block).
|
|
||||||
- No changes to manifest or egress configuration.
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
In scope:
|
|
||||||
|
|
||||||
- `bot_bottle/cli/dashboard.py` approval dispatch.
|
|
||||||
- `bot_bottle/backend/smolmachines/provision/supervise.py` tool registration.
|
|
||||||
- New or updated backend-specific capability apply/disable module for
|
|
||||||
smolmachines.
|
|
||||||
- Unit tests for dispatch routing and smolmachines path.
|
|
||||||
|
|
||||||
Out of scope:
|
|
||||||
|
|
||||||
- Changes to `backend/docker/capability_apply.py` internals.
|
|
||||||
- Integration tests that exercise a live smolmachines VM remediation.
|
|
||||||
|
|
||||||
## Design
|
|
||||||
|
|
||||||
Introduce a backend-aware dispatch at the approval call site. Each backend
|
|
||||||
exposes a capability remediation entry point; the dashboard calls the one that
|
|
||||||
matches the bottle's backend. If the backend does not support remediation,
|
|
||||||
the entry point returns a structured error that the dashboard surfaces as an
|
|
||||||
operator message without attempting any teardown.
|
|
||||||
|
|
||||||
If option (b) is chosen initially, suppress capability-block registration in
|
|
||||||
`smolmachines/provision/supervise.py` so agents never see the tool.
|
|
||||||
|
|
||||||
## Testing Strategy
|
|
||||||
|
|
||||||
- Unit test that approval dispatch selects the smolmachines path for a
|
|
||||||
smolmachines bottle and the Docker path for a Docker bottle.
|
|
||||||
- Unit test for the smolmachines path (unsupported response or real apply).
|
|
||||||
- Regression test that Docker approval still calls `capability_apply.py`.
|
|
||||||
|
|
||||||
Run:
|
|
||||||
|
|
||||||
- `python3 -m unittest discover -s tests/unit`
|
|
||||||
|
|
||||||
## Open Questions
|
|
||||||
|
|
||||||
- Is a real smolmachines capability-apply implementation in scope for this PRD,
|
|
||||||
or should it be deferred to a follow-on after PRD 0040 lands?
|
|
||||||
@@ -1,87 +0,0 @@
|
|||||||
# PRD 0040: Backend-Aware Resume and Dashboard Reattach
|
|
||||||
|
|
||||||
- **Status:** Active
|
|
||||||
- **Author:** didericis-codex
|
|
||||||
- **Created:** 2026-06-02
|
|
||||||
- **Issue:** #137
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
Persist the backend name in `BottleMetadata` and thread it through `resume` and
|
|
||||||
dashboard reattach so both flows construct the correct backend bottle without
|
|
||||||
relying on env overrides or defaulting to Docker.
|
|
||||||
|
|
||||||
## Problem
|
|
||||||
|
|
||||||
`BottleMetadata` records identity, agent, cwd, started_at, and compose project,
|
|
||||||
but not the backend name. Without it:
|
|
||||||
|
|
||||||
- `cli/resume.py` cannot select the right backend from a preserved state dir
|
|
||||||
alone; operators must remember to set `BOT_BOTTLE_BACKEND=smolmachines`
|
|
||||||
separately.
|
|
||||||
- `cli/dashboard.py` `_bottle_for_slug` constructs a `DockerBottle` for any
|
|
||||||
externally discovered slug, so reattaching to a live smolmachines agent
|
|
||||||
from the dashboard sends Docker commands to a smolvm machine.
|
|
||||||
|
|
||||||
## Goals / Success Criteria
|
|
||||||
|
|
||||||
- `BottleMetadata` includes the backend name, written at bottle creation time
|
|
||||||
for both Docker and smolmachines.
|
|
||||||
- `cli resume` reads the persisted backend name and constructs the correct
|
|
||||||
bottle type without requiring an env override.
|
|
||||||
- Dashboard reattach (`_bottle_for_slug`) reads the persisted backend name and
|
|
||||||
constructs the correct bottle type.
|
|
||||||
- Existing Docker bottles without a persisted backend name fall back to Docker
|
|
||||||
(backward-compatible default).
|
|
||||||
- Unit tests cover write, read, backward-compatible fallback, and both
|
|
||||||
resume/reattach code paths.
|
|
||||||
|
|
||||||
## Non-goals
|
|
||||||
|
|
||||||
- No changes to manifest or egress configuration.
|
|
||||||
- No new CLI flags (backend selection at resume time should be automatic).
|
|
||||||
- No smolmachines capability-apply implementation (see PRD 0039).
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
In scope:
|
|
||||||
|
|
||||||
- `bot_bottle/backend/docker/bottle_state.py` `BottleMetadata` schema and
|
|
||||||
write path.
|
|
||||||
- `bot_bottle/backend/docker/bottle.py` and
|
|
||||||
`bot_bottle/backend/smolmachines/bottle.py` metadata write at creation.
|
|
||||||
- `bot_bottle/cli/resume.py` backend selection from metadata.
|
|
||||||
- `bot_bottle/cli/dashboard.py` `_bottle_for_slug` backend selection.
|
|
||||||
- Unit tests covering the above.
|
|
||||||
|
|
||||||
Out of scope:
|
|
||||||
|
|
||||||
- Migration tooling for existing state dirs.
|
|
||||||
- Integration tests that exercise full resume across process restarts.
|
|
||||||
|
|
||||||
## Design
|
|
||||||
|
|
||||||
Add a `backend` field to `BottleMetadata` with a default of `"docker"` for
|
|
||||||
backward compatibility. Both `DockerBottle` and `SmolmachinesBottle` write
|
|
||||||
their backend name into metadata at creation time.
|
|
||||||
|
|
||||||
`resume` reads the metadata before constructing the bottle object and selects
|
|
||||||
the appropriate backend class. `_bottle_for_slug` does the same. A helper
|
|
||||||
function in the metadata module can encapsulate the backend-name-to-class
|
|
||||||
mapping so the logic is not duplicated.
|
|
||||||
|
|
||||||
## Testing Strategy
|
|
||||||
|
|
||||||
- Unit tests for `BottleMetadata` serialisation with and without the backend
|
|
||||||
field.
|
|
||||||
- Unit tests for the backward-compatible default.
|
|
||||||
- Unit tests for `resume` selecting smolmachines vs Docker from metadata.
|
|
||||||
- Unit tests for `_bottle_for_slug` selecting smolmachines vs Docker.
|
|
||||||
|
|
||||||
Run:
|
|
||||||
|
|
||||||
- `python3 -m unittest discover -s tests/unit`
|
|
||||||
|
|
||||||
## Open Questions
|
|
||||||
|
|
||||||
None.
|
|
||||||
@@ -1,60 +0,0 @@
|
|||||||
# PRD 0041: Git HTTP Request Bounds
|
|
||||||
|
|
||||||
- **Status:** Active
|
|
||||||
- **Author:** didericis-codex
|
|
||||||
- **Created:** 2026-06-02
|
|
||||||
- **Issue:** #138
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
Add Content-Length validation and a body-size cap to `git_http_backend.py` so malformed or oversized smart-HTTP requests fail cleanly rather than crashing the handler or exhausting memory.
|
|
||||||
|
|
||||||
## Problem
|
|
||||||
|
|
||||||
`bot_bottle/git_http_backend.py` calls `int(self.headers.get("Content-Length", 0))` without catching `ValueError`. A request with a non-numeric Content-Length raises an unhandled exception in the request handler.
|
|
||||||
|
|
||||||
The handler reads the full declared length into memory before passing the body to `git http-backend` with no upper bound. A local or compromised client can force arbitrarily high memory use. For comparison, `supervise_server.py` caps request bodies at 1 MiB.
|
|
||||||
|
|
||||||
## Goals / Success Criteria
|
|
||||||
|
|
||||||
- A missing or non-numeric Content-Length returns HTTP 400.
|
|
||||||
- A negative Content-Length returns HTTP 400.
|
|
||||||
- A body larger than the cap (1 MiB, matching `supervise_server.py`) returns HTTP 413.
|
|
||||||
- Valid Git smart-HTTP pushes and fetches continue to work.
|
|
||||||
- Unit tests cover: missing length, non-numeric length, negative length, over-cap length, and a valid push/fetch passthrough.
|
|
||||||
|
|
||||||
## Non-goals
|
|
||||||
|
|
||||||
- No changes to git-gate authentication or route logic.
|
|
||||||
- No changes to `supervise_server.py`.
|
|
||||||
- No streaming / chunked-transfer-encoding support.
|
|
||||||
- No TLS changes.
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
In scope:
|
|
||||||
|
|
||||||
- `bot_bottle/git_http_backend.py` request parsing and body reading.
|
|
||||||
- Unit tests in `tests/unit/test_git_http_backend.py`.
|
|
||||||
|
|
||||||
Out of scope:
|
|
||||||
|
|
||||||
- Integration tests that drive a real Git client through the handler.
|
|
||||||
|
|
||||||
## Design
|
|
||||||
|
|
||||||
Wrap the Content-Length parse in a try/except and return 400 on `ValueError`. Add an explicit check for negative values. After parsing, compare the declared length against a module-level `MAX_BODY_BYTES` constant (default 1 MiB) and return 413 if exceeded. Read exactly `min(content_length, MAX_BODY_BYTES)` bytes.
|
|
||||||
|
|
||||||
## Testing Strategy
|
|
||||||
|
|
||||||
- Unit tests using `unittest.mock` to drive the handler with crafted headers.
|
|
||||||
- Test cases: no Content-Length header, `Content-Length: abc`, `Content-Length: -1`, `Content-Length: 2097152` (over cap), and a normal small POST body.
|
|
||||||
|
|
||||||
Run:
|
|
||||||
|
|
||||||
- `python3 -m unittest tests.unit.test_git_http_backend`
|
|
||||||
- `python3 -m unittest discover -s tests/unit`
|
|
||||||
|
|
||||||
## Open Questions
|
|
||||||
|
|
||||||
None.
|
|
||||||
@@ -1,85 +0,0 @@
|
|||||||
# PRD 0042: smolmachines Cross-Backend Parity Tests
|
|
||||||
|
|
||||||
- **Status:** Active
|
|
||||||
- **Author:** didericis-codex
|
|
||||||
- **Created:** 2026-06-02
|
|
||||||
- **Issue:** #139
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
Add tests that prove secrets, forwarded env, resume, and remediation behave
|
|
||||||
equivalently across Docker and smolmachines backends. The fixes in PRDs
|
|
||||||
0038–0040 are unverifiable without this coverage.
|
|
||||||
|
|
||||||
## Problem
|
|
||||||
|
|
||||||
The existing unit suite is broad but backend-specific. There are no tests that
|
|
||||||
run the same scenario against both Docker and smolmachines and assert the
|
|
||||||
outcomes match. A regression in one backend goes undetected until a live run,
|
|
||||||
and PRDs 0038–0040 can each pass their own unit tests while the backends still
|
|
||||||
diverge at the integration boundary.
|
|
||||||
|
|
||||||
## Goals / Success Criteria
|
|
||||||
|
|
||||||
- A parity test suite that covers at least:
|
|
||||||
- Secret env injection: `?prompt` and `${HOST_VAR}` entries produce the same
|
|
||||||
guest env on both backends.
|
|
||||||
- Forwarded env: literal manifest env values reach the guest on both backends.
|
|
||||||
- Resume: a preserved bottle state dir round-trips correctly on both backends
|
|
||||||
(relies on PRD 0040 metadata).
|
|
||||||
- Remediation: capability-block approval routes to the correct backend handler
|
|
||||||
(relies on PRD 0039 dispatch).
|
|
||||||
- Each scenario is parameterised so a failure names the backend that regressed.
|
|
||||||
- Tests run without a live VM or Docker daemon (mock or stub backends).
|
|
||||||
|
|
||||||
## Non-goals
|
|
||||||
|
|
||||||
- No end-to-end agent execution tests.
|
|
||||||
- No performance or load tests.
|
|
||||||
- No changes to production code (test-only PRD).
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
In scope:
|
|
||||||
|
|
||||||
- New test file(s) under `tests/unit/` for parity scenarios.
|
|
||||||
- Stub or mock implementations of smolmachines and Docker backends as needed.
|
|
||||||
|
|
||||||
Out of scope:
|
|
||||||
|
|
||||||
- Changes to `bot_bottle/` production code.
|
|
||||||
- CI infrastructure changes beyond adding the new test file to the discover
|
|
||||||
invocation.
|
|
||||||
|
|
||||||
## Dependencies
|
|
||||||
|
|
||||||
- PRD 0038 should land before the env parity tests are finalised.
|
|
||||||
- PRDs 0039 and 0040 should land before the remediation and resume scenarios
|
|
||||||
are finalised; stubs can be written speculatively beforehand.
|
|
||||||
|
|
||||||
## Design
|
|
||||||
|
|
||||||
Parameterise each scenario over a list of backend factory functions. Each
|
|
||||||
factory returns a bottle instance wired to a stub subprocess layer. The test
|
|
||||||
body is backend-agnostic: it calls the same public API, captures the same
|
|
||||||
observable output, and asserts equality.
|
|
||||||
|
|
||||||
For env scenarios, capture the argv or env-file content passed to the guest
|
|
||||||
and compare against resolved manifest values. For resume, write metadata with
|
|
||||||
one backend class and read it back to verify correct selection. For remediation,
|
|
||||||
assert dispatch selects the per-backend handler.
|
|
||||||
|
|
||||||
## Testing Strategy
|
|
||||||
|
|
||||||
Run as part of the standard unit discover:
|
|
||||||
|
|
||||||
- `python3 -m unittest discover -s tests/unit`
|
|
||||||
|
|
||||||
Or directly:
|
|
||||||
|
|
||||||
- `python3 -m unittest tests.unit.test_backend_parity`
|
|
||||||
|
|
||||||
## Open Questions
|
|
||||||
|
|
||||||
- Should parity tests live under `tests/unit/` (mock-based) or
|
|
||||||
`tests/integration/` (live infra)? Mock-based is preferred to keep CI simple.
|
|
||||||
@@ -1,74 +0,0 @@
|
|||||||
# PRD 0043: Sidecar Pipe Lifecycle Cleanup
|
|
||||||
|
|
||||||
- **Status:** Active
|
|
||||||
- **Author:** didericis-codex
|
|
||||||
- **Created:** 2026-06-02
|
|
||||||
- **Issue:** #140
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
Close the unclosed child stdout pipe file descriptors that `sidecar_init.py`
|
|
||||||
leaks during restart and shutdown paths, eliminating `ResourceWarning` noise
|
|
||||||
and tightening the process lifecycle.
|
|
||||||
|
|
||||||
## Problem
|
|
||||||
|
|
||||||
Unit tests for `sidecar_init.py` pass, but restart and shutdown cases emit
|
|
||||||
`ResourceWarning: unclosed file <_io.BufferedReader …>` for child stdout pipes,
|
|
||||||
originating around lines 141 and 273. The warnings indicate the restart path
|
|
||||||
leaks pipe file descriptors: a pipe opened for a stopped or replaced child is
|
|
||||||
not explicitly closed before the next child is spawned or before the supervisor
|
|
||||||
exits.
|
|
||||||
|
|
||||||
## Goals / Success Criteria
|
|
||||||
|
|
||||||
- `python3 -m unittest tests.unit.test_sidecar_init` produces no
|
|
||||||
`ResourceWarning` output.
|
|
||||||
- Pipe file descriptors for stopped or replaced child processes are explicitly
|
|
||||||
closed in the restart path.
|
|
||||||
- Pipe file descriptors for all children are explicitly closed in the shutdown
|
|
||||||
path.
|
|
||||||
- No change to the external signal or exit-code contract from PRD 0034.
|
|
||||||
|
|
||||||
## Non-goals
|
|
||||||
|
|
||||||
- No changes to restart or shutdown policy (coalescing, ordering, timeout).
|
|
||||||
- No changes to egress, pipelock, git-gate, or supervise daemon argv.
|
|
||||||
- No new runtime dependencies.
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
In scope:
|
|
||||||
|
|
||||||
- `bot_bottle/sidecar_init.py` pipe open/close lifecycle in `_Supervisor`.
|
|
||||||
- Unit tests in `tests/unit/test_sidecar_init.py` asserting no leaked pipes.
|
|
||||||
|
|
||||||
Out of scope:
|
|
||||||
|
|
||||||
- Changing how pumping threads read from pipes.
|
|
||||||
- Integration tests that start a live sidecar container.
|
|
||||||
|
|
||||||
## Design
|
|
||||||
|
|
||||||
Audit every code path in `_Supervisor` where a child process is stopped,
|
|
||||||
replaced, or reaches end-of-life, and ensure the corresponding stdout pipe is
|
|
||||||
explicitly closed before spawning a replacement or exiting the supervisor loop.
|
|
||||||
|
|
||||||
Where a pumping thread holds a reference to the pipe, coordinate closure so the
|
|
||||||
thread sees EOF and exits cleanly rather than blocking indefinitely.
|
|
||||||
|
|
||||||
## Testing Strategy
|
|
||||||
|
|
||||||
- Enable `ResourceWarning` as an error in test setUp:
|
|
||||||
`warnings.simplefilter("error", ResourceWarning)`.
|
|
||||||
- Run existing restart and shutdown test cases under this stricter setting.
|
|
||||||
- Add tests for restart-then-shutdown if not already covered.
|
|
||||||
|
|
||||||
Run:
|
|
||||||
|
|
||||||
- `python3 -m unittest tests.unit.test_sidecar_init`
|
|
||||||
- `python3 -m unittest discover -s tests/unit`
|
|
||||||
|
|
||||||
## Open Questions
|
|
||||||
|
|
||||||
None.
|
|
||||||
@@ -1,119 +0,0 @@
|
|||||||
# PRD 0044: Print Parity Across Backends
|
|
||||||
|
|
||||||
- **Status:** Active
|
|
||||||
- **Author:** didericis-claude
|
|
||||||
- **Created:** 2026-06-02
|
|
||||||
- **Issue:** #96
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
Hoist `git_gate_plan`, `egress_plan`, `agent_provision`, and `supervise_plan`
|
|
||||||
from the concrete `BottlePlan` subclasses up to `BottlePlan`, and implement
|
|
||||||
`print` concretely there. This eliminates the two per-backend output divergences
|
|
||||||
and ensures any future backend gets correct preflight rendering for free.
|
|
||||||
|
|
||||||
## Problem
|
|
||||||
|
|
||||||
`BottlePlan.print` is `@abstractmethod`, so each backend provides its own
|
|
||||||
implementation. The two current implementations have drifted:
|
|
||||||
|
|
||||||
| Field | Docker | smolmachines |
|
|
||||||
|---|---|---|
|
|
||||||
| git gate lines | `upstream_host:upstream_port` from resolved `git_gate_plan.upstreams` | `Name → Upstream` from manifest `bottle.git` |
|
|
||||||
| egress lines | `host [auth:scheme]` | `host` only (auth dropped) |
|
|
||||||
|
|
||||||
The smolmachines docstring says "same shape as the Docker backend's so operators
|
|
||||||
see one format across backends" — that intent is real but nothing enforces it.
|
|
||||||
|
|
||||||
The env_names divergence previously noted in this issue was resolved by PRD 0038
|
|
||||||
(smolmachines env contract): `resolved.forwarded` is now merged into
|
|
||||||
`agent_provision.guest_env` at prepare time on both backends, so displayed env
|
|
||||||
names are equivalent.
|
|
||||||
|
|
||||||
## Goals / Success Criteria
|
|
||||||
|
|
||||||
- `BottlePlan` carries `git_gate_plan`, `egress_plan`, `agent_provision`, and
|
|
||||||
`supervise_plan` as concrete fields; subclasses no longer declare them
|
|
||||||
independently.
|
|
||||||
- `BottlePlan.print` is a concrete method; subclasses have no `print`
|
|
||||||
implementation of their own.
|
|
||||||
- Both backends render git gate lines as `name → upstream_host:upstream_port`
|
|
||||||
(using `git_gate_plan.upstreams`), not the manifest-level URL.
|
|
||||||
- Both backends render egress lines as `host [auth:scheme]` (dropping the
|
|
||||||
annotation only when `auth_scheme` is empty).
|
|
||||||
- Unit tests assert the unified output for both backends from a single shared
|
|
||||||
test helper.
|
|
||||||
|
|
||||||
## Non-goals
|
|
||||||
|
|
||||||
- No changes to the Docker or smolmachines launch, prepare, or cleanup paths.
|
|
||||||
- No changes to how env values are resolved or injected (that is PRD 0038).
|
|
||||||
- No changes to the manifest schema or `GitEntry`.
|
|
||||||
- No new CLI flags or dashboard changes.
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
In scope:
|
|
||||||
|
|
||||||
- `bot_bottle/backend/__init__.py` — add `git_gate_plan`, `egress_plan`,
|
|
||||||
`agent_provision`, and `supervise_plan` fields to `BottlePlan`; replace
|
|
||||||
`@abstractmethod print` with a concrete implementation.
|
|
||||||
- `bot_bottle/backend/docker/bottle_plan.py` — remove the four hoisted fields
|
|
||||||
and the `print` method.
|
|
||||||
- `bot_bottle/backend/smolmachines/bottle_plan.py` — remove the four hoisted
|
|
||||||
fields and the `print` method.
|
|
||||||
- `tests/unit/` — add or update tests asserting unified preflight output; a
|
|
||||||
shared helper can build a minimal plan fixture for each backend and assert
|
|
||||||
the same lines appear.
|
|
||||||
|
|
||||||
Out of scope:
|
|
||||||
|
|
||||||
- Changes to `bot_bottle/backend/print_util.py` beyond what the new `print`
|
|
||||||
implementation requires.
|
|
||||||
- Changes to `BottleCleanupPlan.print` or any other print method.
|
|
||||||
- Integration tests that launch a real bottle.
|
|
||||||
|
|
||||||
## Design
|
|
||||||
|
|
||||||
Move the four fields that both concrete subclasses already declare —
|
|
||||||
`git_gate_plan: GitGatePlan`, `egress_plan: EgressPlan`,
|
|
||||||
`agent_provision: AgentProvisionPlan`, `supervise_plan: SupervisePlan | None`
|
|
||||||
— up to `BottlePlan`. Both backends' `prepare` paths already produce these with
|
|
||||||
the same types, so no prepare-time changes are needed.
|
|
||||||
|
|
||||||
Replace the `@abstractmethod` `print` with a concrete implementation on
|
|
||||||
`BottlePlan` that:
|
|
||||||
|
|
||||||
1. Builds `env_names` from `bottle.env.keys() | agent_provision.guest_env.keys()`
|
|
||||||
filtered through `agent_provision.hidden_env_names`.
|
|
||||||
2. Builds git gate lines from `git_gate_plan.upstreams` as
|
|
||||||
`f"{u.name} → {u.upstream_host}:{u.upstream_port}"`.
|
|
||||||
3. Builds egress lines from `egress_plan.routes` as
|
|
||||||
`f"{r.host} [auth:{r.auth_scheme}]"` when `r.auth_scheme` is non-empty,
|
|
||||||
else `r.host`.
|
|
||||||
4. Renders the standard two-column preflight block (leading blank line, agent,
|
|
||||||
provider, env, skills, bottle, git identity, git gate, egress, trailing blank
|
|
||||||
line).
|
|
||||||
|
|
||||||
Docker's `forwarded_env` keys are already merged into `agent_provision.guest_env`
|
|
||||||
via the `agent_provision_plan` builder, so no special handling is needed for
|
|
||||||
env_names.
|
|
||||||
|
|
||||||
## Testing Strategy
|
|
||||||
|
|
||||||
- Add a shared fixture builder (e.g. `make_plan(backend)`) in a new or existing
|
|
||||||
unit test module that constructs a minimal `DockerBottlePlan` and
|
|
||||||
`SmolmachinesBottlePlan` from the same spec and plan fields.
|
|
||||||
- Assert that `plan.print(remote_control=False)` produces identical git gate and
|
|
||||||
egress lines for both backends given the same `git_gate_plan` and
|
|
||||||
`egress_plan`.
|
|
||||||
- Test the `auth_scheme` annotation: present when non-empty, absent otherwise.
|
|
||||||
- Test git gate rendering: `name → host:port` format.
|
|
||||||
|
|
||||||
Run:
|
|
||||||
|
|
||||||
- `python3 -m unittest discover -s tests/unit`
|
|
||||||
|
|
||||||
## Open Questions
|
|
||||||
|
|
||||||
None.
|
|
||||||
@@ -1,167 +0,0 @@
|
|||||||
# PRD 0045: Workspace Porting Plan
|
|
||||||
|
|
||||||
- **Status:** Active
|
|
||||||
- **Author:** didericis-codex
|
|
||||||
- **Created:** 2026-06-02
|
|
||||||
- **Issue:** #116
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
Add a backend-neutral `WorkspacePlan` that describes how the operator's current
|
|
||||||
workspace is represented inside a bottle. Docker and smolmachines should both
|
|
||||||
use this plan for workspace path, working directory, content copy, `.git` copy,
|
|
||||||
ownership, and provider trust configuration instead of rediscovering
|
|
||||||
`/home/node/workspace` in separate launch and provisioning code paths.
|
|
||||||
|
|
||||||
## Problem
|
|
||||||
|
|
||||||
The current `--cwd` behavior is spread across backend-specific code:
|
|
||||||
|
|
||||||
- Docker builds a derived image that copies the host cwd to
|
|
||||||
`/home/node/workspace`, sets that as `WORKDIR`, and patches Claude trust in
|
|
||||||
the generated Dockerfile.
|
|
||||||
- Docker git provisioning separately copies `.git` into
|
|
||||||
`/home/node/workspace/.git`.
|
|
||||||
- smolmachines git provisioning reconstructs `<guest_home>/workspace/.git`, but
|
|
||||||
does not copy the full working tree.
|
|
||||||
- Codex provider setup trusts `guest_home`, not the copied workspace path.
|
|
||||||
|
|
||||||
These details create backend drift and make provider-specific workspace fixes
|
|
||||||
easy to hard-code in the wrong layer.
|
|
||||||
|
|
||||||
## Goals / Success Criteria
|
|
||||||
|
|
||||||
- `BottleSpec` remains the CLI intent shape (`copy_cwd`, `user_cwd`), while a
|
|
||||||
resolved `WorkspacePlan` carries the backend-neutral guest workspace contract.
|
|
||||||
- `BottlePlan` exposes `workspace_plan` so shared and backend-specific
|
|
||||||
provisioning paths consume one resolved object.
|
|
||||||
- The default in-bottle workspace path remains `/home/node/workspace` when
|
|
||||||
`--cwd` is enabled.
|
|
||||||
- Docker uses `WorkspacePlan` when building the derived cwd image and when
|
|
||||||
provisioning cwd `.git` state.
|
|
||||||
- smolmachines copies the host cwd contents into the same logical workspace
|
|
||||||
path and uses `WorkspacePlan` when provisioning cwd `.git` state.
|
|
||||||
- Provider trust configuration is written for the workspace path when `--cwd`
|
|
||||||
is enabled, and for the guest home when `--cwd` is disabled.
|
|
||||||
- Unit tests cover plan resolution, provider trust path selection, Docker
|
|
||||||
derived image rendering, and both backends' `.git` copy targets.
|
|
||||||
|
|
||||||
## Non-goals
|
|
||||||
|
|
||||||
- No new user-facing flags for custom workspace paths.
|
|
||||||
- No manifest schema changes.
|
|
||||||
- No redesign of git-gate or `bottle.git` entries.
|
|
||||||
- No switch from Docker image-copy to bind-mount.
|
|
||||||
- No unrelated provider auth changes.
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
In scope:
|
|
||||||
|
|
||||||
- Add a small workspace planning module.
|
|
||||||
- Add `workspace_plan` to `BottlePlan` and populate it in Docker and
|
|
||||||
smolmachines prepare paths.
|
|
||||||
- Thread the trusted project path into provider provisioning.
|
|
||||||
- Replace hard-coded `/home/node/workspace` cwd copy and `.git` copy sites with
|
|
||||||
`WorkspacePlan` values.
|
|
||||||
- Copy full host cwd contents for smolmachines `--cwd` parity.
|
|
||||||
- Update focused unit tests.
|
|
||||||
|
|
||||||
Out of scope:
|
|
||||||
|
|
||||||
- Integration tests that launch real Docker containers or smolmachines VMs.
|
|
||||||
- Path customization in the bottle manifest or CLI.
|
|
||||||
- Runtime synchronization after bottle launch; this remains a launch-time copy.
|
|
||||||
|
|
||||||
## Design
|
|
||||||
|
|
||||||
Add `bot_bottle/workspace.py`:
|
|
||||||
|
|
||||||
```python
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class WorkspacePlan:
|
|
||||||
enabled: bool
|
|
||||||
host_path: Path
|
|
||||||
guest_home: str
|
|
||||||
guest_path: str
|
|
||||||
workdir: str
|
|
||||||
owner: str = "node:node"
|
|
||||||
mode: str = "755"
|
|
||||||
copy_contents: bool = True
|
|
||||||
copy_git: bool = True
|
|
||||||
has_host_git_dir: bool = False
|
|
||||||
```
|
|
||||||
|
|
||||||
`workspace_plan(spec, guest_home)` resolves:
|
|
||||||
|
|
||||||
- `enabled` from `spec.copy_cwd`.
|
|
||||||
- `host_path` from `spec.user_cwd`.
|
|
||||||
- `guest_path` as `<guest_home>/workspace` when enabled, else `guest_home`.
|
|
||||||
- `workdir` as `guest_path` when enabled, else `guest_home`.
|
|
||||||
- `has_host_git_dir` from `<host_path>/.git`.
|
|
||||||
|
|
||||||
Backends resolve this in `prepare` using their existing guest-home knobs:
|
|
||||||
|
|
||||||
- Docker: `BOT_BOTTLE_CONTAINER_HOME`, default `/home/node`.
|
|
||||||
- smolmachines: `BOT_BOTTLE_GUEST_HOME`, default `/home/node`.
|
|
||||||
|
|
||||||
`BottlePlan` carries the result so launch, git provisioning, and provider
|
|
||||||
provisioning stop consulting `spec.copy_cwd` and hard-coded paths directly.
|
|
||||||
|
|
||||||
### Docker
|
|
||||||
|
|
||||||
Keep the current derived-image transport. Change
|
|
||||||
`build_image_with_cwd(derived, base, cwd)` to accept a `WorkspacePlan` or
|
|
||||||
explicit guest path/workdir fields, then render:
|
|
||||||
|
|
||||||
- `COPY --chown=node:node . <workspace_plan.guest_path>`
|
|
||||||
- `WORKDIR <workspace_plan.workdir>`
|
|
||||||
|
|
||||||
Claude trust should move out of the generated cwd Dockerfile and into provider
|
|
||||||
provisioning so Docker and smolmachines share the same provider trust behavior.
|
|
||||||
|
|
||||||
### smolmachines
|
|
||||||
|
|
||||||
Copy host cwd contents into `workspace_plan.guest_path` during provisioning or
|
|
||||||
VM initialization, then chown the resulting workspace to `node:node`. Continue
|
|
||||||
to copy `.git` through the existing smolvm transport, but target
|
|
||||||
`<workspace_plan.guest_path>/.git`.
|
|
||||||
|
|
||||||
This intentionally closes the current parity gap where smolmachines receives
|
|
||||||
repo metadata without the working tree.
|
|
||||||
|
|
||||||
### Provider Trust
|
|
||||||
|
|
||||||
Extend provider planning with a `trusted_project_path` argument. Callers pass
|
|
||||||
`workspace_plan.workdir`.
|
|
||||||
|
|
||||||
Codex writes:
|
|
||||||
|
|
||||||
```toml
|
|
||||||
[projects."<trusted_project_path>"]
|
|
||||||
trust_level = "trusted"
|
|
||||||
```
|
|
||||||
|
|
||||||
Claude writes or updates `.claude.json` so `projects` includes
|
|
||||||
`trusted_project_path` with `hasTrustDialogAccepted: true`. This provisioning
|
|
||||||
belongs in `AgentProvisionPlan` so both backends apply it through their existing
|
|
||||||
provider file-copy primitives.
|
|
||||||
|
|
||||||
## Testing Strategy
|
|
||||||
|
|
||||||
- Unit-test `workspace_plan()` for enabled and disabled cwd, guest-home
|
|
||||||
overrides, and `.git` detection.
|
|
||||||
- Unit-test Docker cwd image rendering to prove it uses the plan's guest path
|
|
||||||
and workdir.
|
|
||||||
- Unit-test provider planning for Codex and Claude trusted project paths.
|
|
||||||
- Unit-test Docker and smolmachines git provisioning targets using mocked copy
|
|
||||||
and exec primitives.
|
|
||||||
- Unit-test smolmachines workspace content copy target and ownership command.
|
|
||||||
|
|
||||||
Run:
|
|
||||||
|
|
||||||
- `python3 -m unittest discover -s tests/unit`
|
|
||||||
|
|
||||||
## Open Questions
|
|
||||||
|
|
||||||
None.
|
|
||||||
@@ -1,64 +0,0 @@
|
|||||||
# PRD 0046: Remove Git Remote Host Overrides
|
|
||||||
|
|
||||||
- **Status:** Active
|
|
||||||
- **Author:** didericis-codex
|
|
||||||
- **Created:** 2026-06-02
|
|
||||||
- **Issue:** #152
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
Remove git remote host override plumbing from bottle manifests and git-gate
|
|
||||||
startup. Git remote declarations should describe upstream repositories and the
|
|
||||||
git-gate credential material needed to mirror them; they should not also
|
|
||||||
configure hosts-file behavior for sidecars.
|
|
||||||
|
|
||||||
## Problem
|
|
||||||
|
|
||||||
The git remote model currently has a hosts override path that can make a git
|
|
||||||
upstream resolve differently inside the git-gate sidecar. That is surprising
|
|
||||||
because the same hostname may also be used for HTTP/API traffic that should keep
|
|
||||||
using the normal egress DNS and policy path.
|
|
||||||
|
|
||||||
Keeping host resolution in the git remote model makes repository routing,
|
|
||||||
sidecar hosts files, and egress behavior feel coupled even when the operator
|
|
||||||
only meant to configure git-gate.
|
|
||||||
|
|
||||||
## Goals / Success Criteria
|
|
||||||
|
|
||||||
- Git remote manifest parsing no longer stores host override data.
|
|
||||||
- Git-gate upstream plans no longer carry host override data.
|
|
||||||
- Docker compose rendering no longer emits sidecar `extra_hosts` entries from
|
|
||||||
git remote declarations.
|
|
||||||
- Smolmachines bundle launch planning has no unused host override path for
|
|
||||||
git-gate.
|
|
||||||
- Focused unit tests cover the absence of sidecar `extra_hosts` for git
|
|
||||||
upstreams.
|
|
||||||
- Current user-facing documentation no longer advertises git remote host
|
|
||||||
overrides.
|
|
||||||
|
|
||||||
## Non-goals
|
|
||||||
|
|
||||||
- No replacement hosts-file override feature.
|
|
||||||
- No SSH client config provisioning.
|
|
||||||
- No change to git-gate's SSH credential or known-host handling.
|
|
||||||
- No change to egress DNS, HTTP auth, or pipelock routing semantics.
|
|
||||||
|
|
||||||
## Design
|
|
||||||
|
|
||||||
Remove the host override field from the internal `GitEntry` and
|
|
||||||
`GitGateUpstream` models. Remove the git-gate aggregation helper and the Docker
|
|
||||||
compose code that converted those values into sidecar `extra_hosts`.
|
|
||||||
|
|
||||||
The manifest parser does not need a migration-specific error path. After this
|
|
||||||
change, the old hosts override key has no internal model field and no runtime
|
|
||||||
effect.
|
|
||||||
|
|
||||||
## Testing Strategy
|
|
||||||
|
|
||||||
Run:
|
|
||||||
|
|
||||||
- `python3 -m unittest discover -s tests/unit`
|
|
||||||
|
|
||||||
## Open Questions
|
|
||||||
|
|
||||||
None.
|
|
||||||
@@ -1,170 +0,0 @@
|
|||||||
# PRD 0047: Git-gate Manifest Redesign
|
|
||||||
|
|
||||||
- **Status:** Active
|
|
||||||
- **Author:** didericis
|
|
||||||
- **Created:** 2026-06-03
|
|
||||||
- **Issue:** #160
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
Replace the `git` top-level key in bottle and agent manifests with `git-gate`,
|
|
||||||
consolidating git-identity configuration (`user`) and git-gate sidecar
|
|
||||||
configuration (`repos`) under a single section. Within `repos`, field names
|
|
||||||
move to lowercase snake_case and the local repo name is promoted to the YAML
|
|
||||||
key. The change removes the ambiguity in the current `git` block: its fields
|
|
||||||
are not generic git or SSH config — they are specifically the credential,
|
|
||||||
host-trust, and identity material that is managed in relation to git-gate.
|
|
||||||
|
|
||||||
## Problem
|
|
||||||
|
|
||||||
The current bottle manifest uses a `git` top-level key that mixes two concerns:
|
|
||||||
|
|
||||||
- `git.user` — `git config --global user.name / user.email` identity, which
|
|
||||||
the provisioner injects into the agent's shell.
|
|
||||||
- `git.remotes` — upstream URL, identity file, and host key material that the
|
|
||||||
git-gate sidecar consumes; the agent never sees these values.
|
|
||||||
|
|
||||||
That grouping suggests the `remotes` entries behave like an SSH config or a
|
|
||||||
generic `.gitconfig` remote declaration. They do not. The gate reads the
|
|
||||||
credential material to push upstream after gitleaks passes; the agent's
|
|
||||||
`.gitconfig` receives only the `insteadOf` rewrite that redirects traffic
|
|
||||||
through the gate. Nothing in the current key name or field names signals this.
|
|
||||||
|
|
||||||
Splitting `git.user` into a separate section from `git.remotes` also doesn't
|
|
||||||
help: both concepts exist because of git-gate, and keeping them under a single
|
|
||||||
`git-gate` key makes their relationship and purpose explicit.
|
|
||||||
|
|
||||||
The field names inside each remote entry also use PascalCase (`Name`,
|
|
||||||
`Upstream`, `IdentityFile`, `KnownHostKey`), inconsistent with every other
|
|
||||||
manifest section, which uses snake_case.
|
|
||||||
|
|
||||||
The current `git.remotes` dict is keyed by upstream host, which works for
|
|
||||||
simple remotes but forces a separate `Name` field to give the gate's bare repo
|
|
||||||
a local label. The host key and `Name` field are often redundant or confusing
|
|
||||||
(e.g., IP-literal upstreams where the key carries no semantic meaning).
|
|
||||||
|
|
||||||
## Goals / Success Criteria
|
|
||||||
|
|
||||||
- `git-gate` is accepted as a top-level bottle and agent key; `git` is removed
|
|
||||||
from both allowed-key sets.
|
|
||||||
- `git-gate.repos` is a named map where each key is the local repo name
|
|
||||||
exposed by the gate (bottle-only; rejected at the agent level).
|
|
||||||
- Each entry in `git-gate.repos` accepts exactly: `url` (required), `identity`
|
|
||||||
(required), `host_key` (optional).
|
|
||||||
- `git-gate.user` replaces `git.user` on both bottles and agents, with the
|
|
||||||
same `name` / `email` fields and overlay semantics.
|
|
||||||
- The manifest parser rejects `git.remotes` and `git.user` with errors that
|
|
||||||
point to the new keys.
|
|
||||||
- `GitEntry` internal fields are updated to match the new names; all callers
|
|
||||||
(provisioner, git-gate render, plan, tests) compile and pass.
|
|
||||||
- Existing unit tests in `tests/unit/test_manifest_git.py` and
|
|
||||||
`tests/unit/test_manifest_git_user.py` are rewritten to use the new YAML
|
|
||||||
shape; all other manifest unit tests remain green.
|
|
||||||
- The demo manifest (`bot-bottle.demo.json`) and any examples using the old
|
|
||||||
shape are updated.
|
|
||||||
|
|
||||||
## Non-goals
|
|
||||||
|
|
||||||
- No change to `git.user` / `git-gate.user` semantics or field names (`name`,
|
|
||||||
`email`).
|
|
||||||
- No change to git-gate runtime behavior (mirroring, gitleaks, access-hook
|
|
||||||
refresh).
|
|
||||||
- No change to the `insteadOf` rewrite the provisioner emits.
|
|
||||||
- No migration shim: the old `git.*` shape is rejected immediately with clear
|
|
||||||
error messages pointing to the new keys.
|
|
||||||
- No change to how agent-level user config overlays the bottle-level value.
|
|
||||||
|
|
||||||
## Design
|
|
||||||
|
|
||||||
### New manifest shape
|
|
||||||
|
|
||||||
**Before** (bottle frontmatter):
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
git:
|
|
||||||
user:
|
|
||||||
name: implementer-bot
|
|
||||||
email: eric+implementer@dideric.is
|
|
||||||
remotes:
|
|
||||||
gitea.dideric.is:
|
|
||||||
Name: bot-bottle
|
|
||||||
Upstream: ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git
|
|
||||||
IdentityFile: ~/.ssh/gitea-delos-2.pem
|
|
||||||
KnownHostKey: "ssh-rsa AAAA..."
|
|
||||||
```
|
|
||||||
|
|
||||||
**After**:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
git-gate:
|
|
||||||
user:
|
|
||||||
name: implementer-bot
|
|
||||||
email: eric+implementer@dideric.is
|
|
||||||
repos:
|
|
||||||
bot-bottle:
|
|
||||||
url: ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git
|
|
||||||
identity: ~/.ssh/gitea-delos-2.pem
|
|
||||||
host_key: "ssh-rsa AAAA..."
|
|
||||||
```
|
|
||||||
|
|
||||||
`git-gate` is the single optional top-level key for all git configuration.
|
|
||||||
Bottles that previously used only `git.user` now use only `git-gate.user`;
|
|
||||||
those that used only `git.remotes` now use only `git-gate.repos`.
|
|
||||||
|
|
||||||
### Key-name-as-repo-name
|
|
||||||
|
|
||||||
The YAML key in `git-gate.repos` becomes the local repo name (previously
|
|
||||||
`Name`). The upstream host is no longer the primary key; the provisioner and
|
|
||||||
gate derive it from the `url` field during parse. IP-literal upstreams work
|
|
||||||
without an artificial host-as-key constraint.
|
|
||||||
|
|
||||||
### Field renames
|
|
||||||
|
|
||||||
| Old field | New field |
|
|
||||||
|-----------|-----------|
|
|
||||||
| `Name` (from dict key) | YAML key in `repos` |
|
|
||||||
| `Upstream` | `url` |
|
|
||||||
| `IdentityFile` | `identity` |
|
|
||||||
| `KnownHostKey` | `host_key` |
|
|
||||||
|
|
||||||
### Parser changes
|
|
||||||
|
|
||||||
- `manifest_schema.py`: replace `"git"` with `"git-gate"` in `BOTTLE_KEYS`
|
|
||||||
and `AGENT_KEYS_OPTIONAL`.
|
|
||||||
- `manifest.py`: replace `_parse_git_config` with `_parse_git_gate_config`
|
|
||||||
that validates both `user` and `repos` subkeys. Update `Bottle.from_dict`
|
|
||||||
and `Agent.from_dict` to call it for the `"git-gate"` key.
|
|
||||||
- `Agent.from_dict` continues to reject `repos` at the agent level with a
|
|
||||||
clear error.
|
|
||||||
- Remove `from_remote_dict` and update `GitEntry._from_object` to accept the
|
|
||||||
new field names. Internal dataclass field names (`UpstreamUser`, etc.) are
|
|
||||||
unchanged — they are internal plumbing, not user-facing.
|
|
||||||
- Any existing `"git"` key raises a targeted error:
|
|
||||||
|
|
||||||
```
|
|
||||||
bottle 'dev' uses 'git' which has been replaced by 'git-gate' (PRD 0047).
|
|
||||||
Move git.user → git-gate.user and git.remotes → git-gate.repos.
|
|
||||||
```
|
|
||||||
|
|
||||||
## Testing Strategy
|
|
||||||
|
|
||||||
Run:
|
|
||||||
|
|
||||||
```
|
|
||||||
python3 -m unittest discover -s tests/unit
|
|
||||||
```
|
|
||||||
|
|
||||||
Test files to update:
|
|
||||||
|
|
||||||
- `tests/unit/test_manifest_git.py` — rewrite fixtures and assertions to use
|
|
||||||
`git-gate.repos` / lowercase fields. Cover: minimal entry, optional
|
|
||||||
`host_key`, missing `url`, missing `identity`, unknown key, IP-literal
|
|
||||||
upstreams, duplicate name rejection, old `git.remotes` and bare `git` key
|
|
||||||
both rejected.
|
|
||||||
- `tests/unit/test_manifest_git_user.py` and
|
|
||||||
`tests/unit/test_manifest_agent_git_user.py` — update fixtures to use
|
|
||||||
`git-gate.user` at both bottle and agent level.
|
|
||||||
|
|
||||||
## Open Questions
|
|
||||||
|
|
||||||
None.
|
|
||||||
@@ -1,296 +0,0 @@
|
|||||||
# PRD 0048: SSH Deploy-Key Provisioning
|
|
||||||
|
|
||||||
- **Status:** Active
|
|
||||||
- **Author:** didericis-claude
|
|
||||||
- **Created:** 2026-06-03
|
|
||||||
- **Issue:** #169
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
Replace per-repo static SSH identity files with short-lived ed25519 deploy
|
|
||||||
keys that are generated at spin-up and revoked at teardown. Introduce
|
|
||||||
`bot_bottle/contrib/` as the package for platform-specific provisioners and
|
|
||||||
ship the first contrib sub-package: `bot_bottle/contrib/gitea/` with
|
|
||||||
`GiteaDeployKeyProvisioner`. A new `provisioned_key:` block in `git-gate.repos`
|
|
||||||
entries opts a repo into automatic key lifecycle management; `identity:` stays
|
|
||||||
valid for operators who supply their own key material.
|
|
||||||
|
|
||||||
## Problem
|
|
||||||
|
|
||||||
The current `git-gate.repos` entries require an `identity:` field pointing to
|
|
||||||
a host-side SSH private key (PRD 0047). Keys are static: the operator generates
|
|
||||||
them once, registers them with the upstream forge, and the same key is reused
|
|
||||||
across every bottle spin-up. This has several consequences:
|
|
||||||
|
|
||||||
- **No automatic revocation.** If a bottle misbehaves or a key leaks, the
|
|
||||||
operator must notice and manually delete the key from the forge. There is no
|
|
||||||
teardown hook that does it.
|
|
||||||
- **Broad blast radius.** A forge deploy key typically grants write access for
|
|
||||||
the lifetime of the key. A static key that survives bottle teardown continues
|
|
||||||
to grant that access.
|
|
||||||
- **Manual rotation burden.** Operators must manage key files on disk, keeping
|
|
||||||
them secure, rotating them on a schedule, and distributing them across hosts
|
|
||||||
that run `./cli.py start`.
|
|
||||||
|
|
||||||
## Goals / Success Criteria
|
|
||||||
|
|
||||||
- `git-gate.repos` entries accept `provisioned_key:` as an alternative to
|
|
||||||
`identity:`. The parser rejects entries that have both, or neither.
|
|
||||||
- `provisioned_key.provider: gitea` provisions and revokes deploy keys via the
|
|
||||||
Gitea HTTP API.
|
|
||||||
- At prepare time the provisioner generates a fresh ed25519 keypair, registers
|
|
||||||
the public half as a repo-scoped deploy key, and makes the private key
|
|
||||||
available to git-gate at the path it expects — the rest of the pipeline is
|
|
||||||
unchanged.
|
|
||||||
- At teardown the provisioner deletes the registered deploy key. Failure to
|
|
||||||
delete halts teardown and propagates the error loudly.
|
|
||||||
- `bot_bottle/contrib/` is introduced as the package for platform-specific
|
|
||||||
implementations; the core defines the abstract interface; contrib sub-packages
|
|
||||||
provide concrete implementations.
|
|
||||||
- Existing `identity:`-based repos continue to work without change.
|
|
||||||
- The unit test suite passes unchanged for `identity:` paths; new tests cover
|
|
||||||
`provisioned_key:` parse, validation, and provisioner dispatch.
|
|
||||||
|
|
||||||
## Non-goals
|
|
||||||
|
|
||||||
- GitHub, GitLab, or other forge providers (a future contrib sub-package each).
|
|
||||||
- Dashboard UI for listing or revoking orphaned deploy keys.
|
|
||||||
- SSH CA certificate approach (rejected in the issue thread in favour of
|
|
||||||
per-repo deploy keys for simpler revocation, smaller blast radius, and forge
|
|
||||||
compatibility).
|
|
||||||
- Key rotation mid-session (keys live for exactly one spin-up / teardown cycle).
|
|
||||||
- Any change to how `identity:` repos are provisioned.
|
|
||||||
|
|
||||||
## Design
|
|
||||||
|
|
||||||
### Manifest changes (builds on PRD 0047)
|
|
||||||
|
|
||||||
`git-gate.repos.<name>` currently accepts exactly:
|
|
||||||
|
|
||||||
```
|
|
||||||
url (required string)
|
|
||||||
identity (required string)
|
|
||||||
host_key (optional string)
|
|
||||||
```
|
|
||||||
|
|
||||||
After this PRD:
|
|
||||||
|
|
||||||
```
|
|
||||||
url (required string)
|
|
||||||
identity (optional string — mutually exclusive with provisioned_key)
|
|
||||||
provisioned_key (optional object — mutually exclusive with identity)
|
|
||||||
host_key (optional string)
|
|
||||||
```
|
|
||||||
|
|
||||||
Exactly one of `identity` or `provisioned_key` must be present. The parser
|
|
||||||
emits a targeted error for each violation:
|
|
||||||
|
|
||||||
```
|
|
||||||
bottle 'dev' git-gate.repos['bot-bottle'] must set exactly one of
|
|
||||||
'identity' or 'provisioned_key'; got neither.
|
|
||||||
|
|
||||||
bottle 'dev' git-gate.repos['bot-bottle'] must set exactly one of
|
|
||||||
'identity' or 'provisioned_key'; got both.
|
|
||||||
```
|
|
||||||
|
|
||||||
`provisioned_key` object schema:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
provisioned_key:
|
|
||||||
provider: gitea # required; names the contrib module to load
|
|
||||||
token_env: GITEA_TOKEN # required; name of a host env var holding the API token
|
|
||||||
api_url: https://... # optional; defaults to https://<host from url>
|
|
||||||
```
|
|
||||||
|
|
||||||
| Field | Type | Notes |
|
|
||||||
|-------|------|-------|
|
|
||||||
| `provider` | required string | Must match a sub-package under `bot_bottle/contrib/` |
|
|
||||||
| `token_env` | required string | Resolved at provision time via `os.environ`; never stored in plan |
|
|
||||||
| `api_url` | optional string | Override when the API endpoint differs from the git host |
|
|
||||||
|
|
||||||
**Example bottle manifest:**
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
git-gate:
|
|
||||||
user:
|
|
||||||
name: implementer-bot
|
|
||||||
email: eric+implementer@dideric.is
|
|
||||||
repos:
|
|
||||||
bot-bottle:
|
|
||||||
url: ssh://git@gitea.dideric.is:30009/didericis/bot-bottle.git
|
|
||||||
provisioned_key:
|
|
||||||
provider: gitea
|
|
||||||
token_env: GITEA_DEPLOY_TOKEN
|
|
||||||
host_key: "ssh-rsa AAAA..."
|
|
||||||
```
|
|
||||||
|
|
||||||
### `contrib` package structure
|
|
||||||
|
|
||||||
```
|
|
||||||
bot_bottle/
|
|
||||||
contrib/
|
|
||||||
__init__.py # empty; no core symbols
|
|
||||||
gitea/
|
|
||||||
__init__.py # empty
|
|
||||||
deploy_key_provisioner.py
|
|
||||||
```
|
|
||||||
|
|
||||||
`contrib` is a flat namespace of forge/platform sub-packages. Each sub-package
|
|
||||||
is self-contained; the core imports from contrib lazily (inside factory
|
|
||||||
functions) so that missing optional dependencies in a contrib sub-package don't
|
|
||||||
break unrelated features.
|
|
||||||
|
|
||||||
### Core interface
|
|
||||||
|
|
||||||
New file: `bot_bottle/deploy_key_provisioner.py`
|
|
||||||
|
|
||||||
```python
|
|
||||||
from abc import ABC, abstractmethod
|
|
||||||
|
|
||||||
class DeployKeyProvisioner(ABC):
|
|
||||||
@abstractmethod
|
|
||||||
def create(self, owner_repo: str, title: str) -> tuple[str, bytes]:
|
|
||||||
"""Generate a keypair and register the public half.
|
|
||||||
|
|
||||||
owner_repo: '<owner>/<repo>' portion of the git upstream URL.
|
|
||||||
title: human-readable label shown in the forge key list.
|
|
||||||
|
|
||||||
Returns (key_id, private_key_pem) where key_id is opaque to
|
|
||||||
the caller and is only passed back to delete()."""
|
|
||||||
|
|
||||||
@abstractmethod
|
|
||||||
def delete(self, owner_repo: str, key_id: str) -> None:
|
|
||||||
"""Delete the registered deploy key.
|
|
||||||
|
|
||||||
Must not raise if the key is already absent (HTTP 404 is success).
|
|
||||||
Must raise for all other failures so that teardown halts."""
|
|
||||||
|
|
||||||
|
|
||||||
def get_provisioner(provider: str, token: str, api_url: str) -> DeployKeyProvisioner:
|
|
||||||
"""Instantiate the named contrib provisioner.
|
|
||||||
|
|
||||||
Raises ManifestError for unknown providers so the error is caught
|
|
||||||
at parse time rather than at runtime."""
|
|
||||||
if provider == "gitea":
|
|
||||||
from bot_bottle.contrib.gitea.deploy_key_provisioner import (
|
|
||||||
GiteaDeployKeyProvisioner,
|
|
||||||
)
|
|
||||||
return GiteaDeployKeyProvisioner(token=token, api_url=api_url)
|
|
||||||
from .manifest_util import ManifestError
|
|
||||||
raise ManifestError(f"unknown provisioned_key provider: {provider!r}")
|
|
||||||
```
|
|
||||||
|
|
||||||
### Gitea contrib implementation
|
|
||||||
|
|
||||||
`bot_bottle/contrib/gitea/deploy_key_provisioner.py`:
|
|
||||||
|
|
||||||
`create(owner_repo, title)`:
|
|
||||||
1. Generate an ed25519 keypair via `ssh-keygen -t ed25519 -f <tmpfile> -N ''`
|
|
||||||
(uses the SSH tooling already required by git-gate; no new Python dependency).
|
|
||||||
2. Read the private key bytes and the `.pub` file.
|
|
||||||
3. `POST /api/v1/repos/{owner}/{repo}/keys` with the public key, `title`, and
|
|
||||||
`read_only: false` (deploy keys always need push access for git-gate).
|
|
||||||
4. Return `(str(response["id"]), private_key_bytes)`.
|
|
||||||
|
|
||||||
`delete(owner_repo, key_id)`:
|
|
||||||
1. `DELETE /api/v1/repos/{owner}/{repo}/keys/{id}`.
|
|
||||||
2. Treat HTTP 404 as success (key already gone).
|
|
||||||
3. Raise `RuntimeError` for any other non-2xx response or network error,
|
|
||||||
including the status code and response body in the message.
|
|
||||||
|
|
||||||
HTTP calls use `urllib.request` from the stdlib; no new runtime dependency.
|
|
||||||
|
|
||||||
### `GitEntry` dataclass changes
|
|
||||||
|
|
||||||
`bot_bottle/manifest_git.py`:
|
|
||||||
|
|
||||||
- Add `ProvisionedKeyConfig` dataclass:
|
|
||||||
|
|
||||||
```python
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class ProvisionedKeyConfig:
|
|
||||||
provider: str
|
|
||||||
token_env: str
|
|
||||||
api_url: str # empty string means "derive from UpstreamHost"
|
|
||||||
```
|
|
||||||
|
|
||||||
- `GitEntry`:
|
|
||||||
- `IdentityFile: str` unchanged internally; empty string when
|
|
||||||
`provisioned_key` is used; set at provision time, not parse time.
|
|
||||||
- New field: `ProvisionedKey: ProvisionedKeyConfig | None = None`
|
|
||||||
- `from_repos_entry` validates the mutually-exclusive constraint and parses
|
|
||||||
the `provisioned_key` block when present.
|
|
||||||
|
|
||||||
### `GitGateUpstream` / prepare-time changes
|
|
||||||
|
|
||||||
`bot_bottle/git_gate.py` and `bot_bottle/backend/docker/provision/git.py`:
|
|
||||||
|
|
||||||
The existing path writes the identity file path into `GitGateUpstream.IdentityFile`
|
|
||||||
and docker-cp's it into `/git-gate/creds/<name>-key`. That path stays unchanged
|
|
||||||
for `identity:` repos.
|
|
||||||
|
|
||||||
For `provisioned_key:` repos, a new helper `provision_deploy_key(entry,
|
|
||||||
stage_dir, bottle_name)` runs before the git-gate sidecar starts:
|
|
||||||
|
|
||||||
1. Resolve `token = os.environ[entry.ProvisionedKey.token_env]`. Missing key
|
|
||||||
raises `RuntimeError` with a clear message naming the env var.
|
|
||||||
2. Resolve `api_url = entry.ProvisionedKey.api_url or f"https://{entry.UpstreamHost}"`.
|
|
||||||
3. Instantiate `get_provisioner(entry.ProvisionedKey.provider, token, api_url)`.
|
|
||||||
4. Call `provisioner.create(entry.UpstreamPath.lstrip("/"), title)` where
|
|
||||||
`title = f"bot-bottle:{bottle_name}:{entry.Name}"`.
|
|
||||||
5. Write private key to `stage_dir / f"{entry.Name}-key"` (mode 0o600).
|
|
||||||
6. Write key ID to `stage_dir / f"{entry.Name}-deploy-key-id"` (plain text).
|
|
||||||
7. Return the key file path; caller sets `GitGateUpstream.IdentityFile` to it.
|
|
||||||
|
|
||||||
`owner_repo` is extracted from `entry.UpstreamPath` (the path component of the
|
|
||||||
`ssh://` URL, e.g. `/didericis/bot-bottle.git` → `didericis/bot-bottle`).
|
|
||||||
|
|
||||||
### Teardown changes
|
|
||||||
|
|
||||||
`bot_bottle/backend/docker/cleanup.py` (or the equivalent teardown path):
|
|
||||||
|
|
||||||
After the git-gate sidecar stops, for each `GitEntry` with `ProvisionedKey`
|
|
||||||
set:
|
|
||||||
|
|
||||||
1. Check that `stage_dir / f"{entry.Name}-deploy-key-id"` exists; skip if
|
|
||||||
absent (provision never ran or already cleaned up).
|
|
||||||
2. Resolve token and API URL as above.
|
|
||||||
3. Instantiate provisioner and call `provisioner.delete(owner_repo, key_id)`.
|
|
||||||
4. On success, log at INFO. On failure, allow the exception to propagate —
|
|
||||||
teardown halts and the error surfaces to the operator.
|
|
||||||
|
|
||||||
A stranded deploy key is a security concern: the operator must know about it
|
|
||||||
and address it manually. Silent continuation is not acceptable.
|
|
||||||
|
|
||||||
The private key file in `stage_dir` is cleaned up as part of normal stage-dir
|
|
||||||
teardown (no extra step needed).
|
|
||||||
|
|
||||||
## Testing strategy
|
|
||||||
|
|
||||||
```
|
|
||||||
python3 -m unittest discover -s tests/unit
|
|
||||||
```
|
|
||||||
|
|
||||||
New / modified test files:
|
|
||||||
|
|
||||||
- `tests/unit/test_manifest_git.py` — add cases for:
|
|
||||||
- `provisioned_key:` accepted with valid `provider`, `token_env`, optional `api_url`
|
|
||||||
- Both `identity` and `provisioned_key` present → `ManifestError`
|
|
||||||
- Neither `identity` nor `provisioned_key` present → `ManifestError`
|
|
||||||
- Unknown key inside `provisioned_key` block → `ManifestError`
|
|
||||||
- Missing `provider` or `token_env` inside `provisioned_key` → `ManifestError`
|
|
||||||
|
|
||||||
- `tests/unit/test_deploy_key_provisioner.py` — new:
|
|
||||||
- `get_provisioner("gitea", ...)` returns `GiteaDeployKeyProvisioner`
|
|
||||||
- `get_provisioner("unknown", ...)` raises `ManifestError`
|
|
||||||
|
|
||||||
- `tests/unit/test_contrib_gitea_deploy_key.py` — new (using `unittest.mock`
|
|
||||||
to stub `urllib.request.urlopen` and `subprocess.run`):
|
|
||||||
- `create()` calls `ssh-keygen`, POSTs to correct endpoint, returns key ID
|
|
||||||
- `delete()` DELETEs to correct endpoint
|
|
||||||
- `delete()` tolerates HTTP 404 (already-deleted key)
|
|
||||||
- `delete()` raises `RuntimeError` on non-404 HTTP error
|
|
||||||
|
|
||||||
## Open questions
|
|
||||||
|
|
||||||
None.
|
|
||||||
@@ -1,283 +0,0 @@
|
|||||||
# PRD 0049: Named / Labelled Agents
|
|
||||||
|
|
||||||
- **Status:** Draft
|
|
||||||
- **Author:** didericis
|
|
||||||
- **Created:** 2026-06-03
|
|
||||||
- **Issue:** #171
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
At agent launch time, prompt the operator for a short human-readable label
|
|
||||||
(defaulting to the manifest agent key) and an optional color from the 16-color
|
|
||||||
ANSI palette. Store both in the bottle's `metadata.json`. Display the label —
|
|
||||||
rendered in the chosen color — in the dashboard's active-agents pane, replacing
|
|
||||||
the bare manifest key. Inject the label and color into the in-container
|
|
||||||
`claude.json` as `name` / `color` so Claude Code can surface them in its own
|
|
||||||
harness when upstream support lands.
|
|
||||||
|
|
||||||
## Problem
|
|
||||||
|
|
||||||
The dashboard's agents pane identifies each running instance by its manifest
|
|
||||||
agent key (e.g., `implementer`) plus a random slug suffix. When an operator
|
|
||||||
runs three `implementer` bottles simultaneously — one each for three different
|
|
||||||
repos — the pane shows:
|
|
||||||
|
|
||||||
```
|
|
||||||
[docker] a3f9 implementer started 14:02:11 [egress,pipelock]
|
|
||||||
[docker] b81c implementer started 14:03:45 [egress,pipelock]
|
|
||||||
[docker] d220 implementer started 14:05:01 [egress,pipelock]
|
|
||||||
```
|
|
||||||
|
|
||||||
There is no way to tell which bottle is working on which task without attaching
|
|
||||||
to each one in turn. The slug is opaque; the manifest key is shared. Operators
|
|
||||||
working a multi-bottle session resort to keeping a mental map of slug→task,
|
|
||||||
which breaks the moment they switch windows.
|
|
||||||
|
|
||||||
## Goals / Success Criteria
|
|
||||||
|
|
||||||
1. After the operator selects an agent name (dashboard picker or CLI argument),
|
|
||||||
they are prompted for a label. The prompt suggests the manifest key as the
|
|
||||||
default; pressing Enter (or providing no input) accepts it. The label may
|
|
||||||
contain any printable characters up to 64 bytes.
|
|
||||||
2. After the label prompt, the operator is optionally prompted for a color from
|
|
||||||
the 16-color ANSI palette (names: `black`, `red`, `green`, `yellow`, `blue`,
|
|
||||||
`magenta`, `cyan`, `white`, `bright-black`, `bright-red`, `bright-green`,
|
|
||||||
`bright-yellow`, `bright-blue`, `bright-magenta`, `bright-cyan`,
|
|
||||||
`bright-white`). Pressing Enter without a selection skips color entirely.
|
|
||||||
3. `label` and `color` are stored in `BottleMetadata` and written to the
|
|
||||||
bottle's `metadata.json`. Both fields default to `""` (empty / unset).
|
|
||||||
4. `ActiveAgent` carries `label` and `color`; `enumerate_active()` reads them
|
|
||||||
from `metadata.json`.
|
|
||||||
5. `_format_agent_row` uses the label when non-empty (falling back to
|
|
||||||
`agent_name`). If a non-empty color is set and the terminal supports it, the
|
|
||||||
label substring is rendered in that color.
|
|
||||||
6. `BottleSpec` carries `label` and `color`; the docker backend's `prepare`
|
|
||||||
step copies them into `BottleMetadata`.
|
|
||||||
7. `agent_provider.py` writes `label` → `"name"` and `color` → `"color"` into
|
|
||||||
the generated `claude.json`, alongside the existing fields. Fields are
|
|
||||||
omitted when empty.
|
|
||||||
8. The dashboard's `_new_agent_flow` (PRD 0020) includes the label+color step
|
|
||||||
between agent selection and the backend picker.
|
|
||||||
9. `cmd_start` (CLI) includes the label+color step after argument validation
|
|
||||||
and before prepare-with-preflight.
|
|
||||||
10. All existing unit tests stay green; no new tests are required for this
|
|
||||||
change (the label/color fields are thin plumbing with no branching logic
|
|
||||||
worth unit-testing beyond the already-tested metadata read/write path).
|
|
||||||
|
|
||||||
## Non-goals
|
|
||||||
|
|
||||||
- Showing the agent label inside the Claude Code TUI (status line, terminal
|
|
||||||
title, custom header). That requires upstream Claude Code / codex support.
|
|
||||||
Writing to `claude.json` is best-effort scaffolding for when that lands.
|
|
||||||
- Per-bottle color affecting anything outside the dashboard agents pane (e.g.,
|
|
||||||
proposal-pane highlights, log prefixes).
|
|
||||||
- Validating or constraining label content beyond the 64-byte printable cap.
|
|
||||||
- Persisting color-pair state across dashboard restarts (color pairs are
|
|
||||||
initialized fresh each session).
|
|
||||||
- Editing the label or color of an already-running bottle.
|
|
||||||
- Exposing label/color via `./cli.py list` (out of scope for v1; trivial to
|
|
||||||
add later since the field will be in metadata).
|
|
||||||
|
|
||||||
## Design
|
|
||||||
|
|
||||||
### Data flow
|
|
||||||
|
|
||||||
```
|
|
||||||
operator input
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
BottleSpec.label, BottleSpec.color
|
|
||||||
│
|
|
||||||
├─► docker/prepare.py → BottleMetadata.label / .color → metadata.json
|
|
||||||
│
|
|
||||||
└─► agent_provider.py → claude.json {"name": label, "color": color}
|
|
||||||
(omitted when empty)
|
|
||||||
|
|
||||||
dashboard refresh
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
enumerate_active() → read_metadata(slug) → ActiveAgent.label / .color
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
_format_agent_row → label (colored) in the row string
|
|
||||||
```
|
|
||||||
|
|
||||||
### BottleSpec changes
|
|
||||||
|
|
||||||
```python
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class BottleSpec:
|
|
||||||
manifest: Manifest
|
|
||||||
agent_name: str
|
|
||||||
copy_cwd: bool
|
|
||||||
user_cwd: str
|
|
||||||
identity: str = ""
|
|
||||||
label: str = "" # operator-chosen display name; defaults to agent_name at render time
|
|
||||||
color: str = "" # one of the 16 ANSI color names, or "" for terminal default
|
|
||||||
```
|
|
||||||
|
|
||||||
`label` and `color` default to `""` so all existing callers remain valid with
|
|
||||||
no changes.
|
|
||||||
|
|
||||||
### BottleMetadata changes
|
|
||||||
|
|
||||||
Add two new fields with backward-compatible defaults:
|
|
||||||
|
|
||||||
```python
|
|
||||||
@dataclass
|
|
||||||
class BottleMetadata:
|
|
||||||
identity: str
|
|
||||||
agent_name: str
|
|
||||||
cwd: str
|
|
||||||
copy_cwd: bool
|
|
||||||
started_at: str
|
|
||||||
compose_project: str
|
|
||||||
backend: str
|
|
||||||
label: str = ""
|
|
||||||
color: str = ""
|
|
||||||
```
|
|
||||||
|
|
||||||
`metadata.json` written by older bot-bottle versions won't have these keys;
|
|
||||||
`read_metadata` already uses `dict.get` with defaults, so existing slugs load
|
|
||||||
cleanly with `label=""`, `color=""`.
|
|
||||||
|
|
||||||
### ActiveAgent changes
|
|
||||||
|
|
||||||
```python
|
|
||||||
@dataclass(frozen=True)
|
|
||||||
class ActiveAgent:
|
|
||||||
backend_name: str
|
|
||||||
slug: str
|
|
||||||
agent_name: str
|
|
||||||
started_at: str
|
|
||||||
services: tuple[str, ...]
|
|
||||||
label: str = ""
|
|
||||||
color: str = ""
|
|
||||||
```
|
|
||||||
|
|
||||||
`enumerate_active()` copies `label` and `color` out of `BottleMetadata` when
|
|
||||||
constructing each `ActiveAgent`. The smolmachines backend gets the same
|
|
||||||
additions for symmetry; it reads from its own metadata path.
|
|
||||||
|
|
||||||
### Dashboard row rendering
|
|
||||||
|
|
||||||
`_format_agent_row` already falls through cleanly on missing fields. The
|
|
||||||
change is:
|
|
||||||
|
|
||||||
```python
|
|
||||||
display_name = a.label if a.label else a.agent_name
|
|
||||||
```
|
|
||||||
|
|
||||||
Color rendering uses the existing `_try_init_green()` pattern as a model.
|
|
||||||
A `_color_pair_for(color_name)` helper initialises a fresh curses color pair
|
|
||||||
for the requested named color and returns its attr (or 0 on failure). Each
|
|
||||||
unique color in the active agent list gets its own pair index. Color pairs are
|
|
||||||
allocated lazily and cached in a `dict[str, int]` that lives for the duration
|
|
||||||
of the dashboard session.
|
|
||||||
|
|
||||||
The 16 ANSI color name → curses constant mapping:
|
|
||||||
|
|
||||||
| Name | curses constant |
|
|
||||||
|------|----------------|
|
|
||||||
| `black` | `curses.COLOR_BLACK` |
|
|
||||||
| `red` | `curses.COLOR_RED` |
|
|
||||||
| `green` | `curses.COLOR_GREEN` |
|
|
||||||
| `yellow` | `curses.COLOR_YELLOW` |
|
|
||||||
| `blue` | `curses.COLOR_BLUE` |
|
|
||||||
| `magenta` | `curses.COLOR_MAGENTA` |
|
|
||||||
| `cyan` | `curses.COLOR_CYAN` |
|
|
||||||
| `white` | `curses.COLOR_WHITE` |
|
|
||||||
| `bright-*` | same constant + `curses.A_BOLD` |
|
|
||||||
|
|
||||||
Terminals that don't support color fall back to plain text (the helper returns
|
|
||||||
0, which ORed in is a no-op — same pattern as `_try_init_green`).
|
|
||||||
|
|
||||||
### Label + color prompt — dashboard
|
|
||||||
|
|
||||||
In `_new_agent_flow`, after `_picker_modal` returns a non-None name and before
|
|
||||||
`_backend_picker_modal`:
|
|
||||||
|
|
||||||
```python
|
|
||||||
label, color = _label_color_modal(stdscr, default_label=picked)
|
|
||||||
```
|
|
||||||
|
|
||||||
`_label_color_modal` uses `curses.endwin()` → text-mode prompts → restore
|
|
||||||
(the same drop-and-resume pattern as the existing editor flow and preflight
|
|
||||||
Y/N). Two sequential prompts:
|
|
||||||
|
|
||||||
```
|
|
||||||
bot-bottle: agent label [implementer]: <operator types>
|
|
||||||
bot-bottle: color (red/green/blue/… or Enter to skip): <operator types>
|
|
||||||
```
|
|
||||||
|
|
||||||
Invalid color names are silently ignored (treated as empty). The function
|
|
||||||
returns `(label, color)` — both strings, both possibly `""`.
|
|
||||||
|
|
||||||
### Label + color prompt — CLI
|
|
||||||
|
|
||||||
In `cmd_start`, after argument parsing and before `_launch_bottle`:
|
|
||||||
|
|
||||||
```python
|
|
||||||
label = _text_prompt_label(args.name)
|
|
||||||
color = _text_prompt_color()
|
|
||||||
```
|
|
||||||
|
|
||||||
`_text_prompt_label(default)` writes `"bot-bottle: agent label [{default}]: "`
|
|
||||||
to stderr and returns the stripped input (or `default` if blank).
|
|
||||||
`_text_prompt_color()` writes the color prompt and returns the stripped input
|
|
||||||
(or `""` if blank or invalid).
|
|
||||||
|
|
||||||
Both use `read_tty_line()` (already in `start.py`) for the read.
|
|
||||||
|
|
||||||
### Claude Code config injection
|
|
||||||
|
|
||||||
In `agent_provider.py`, where `claude_config.write_text(...)` is called,
|
|
||||||
expand the JSON dict conditionally:
|
|
||||||
|
|
||||||
```python
|
|
||||||
payload = {
|
|
||||||
"hasCompletedOnboarding": True,
|
|
||||||
"theme": "dark",
|
|
||||||
"bypassPermissionsModeAccepted": True,
|
|
||||||
"projects": claude_projects,
|
|
||||||
}
|
|
||||||
if spec.label:
|
|
||||||
payload["name"] = spec.label
|
|
||||||
if spec.color:
|
|
||||||
payload["color"] = spec.color
|
|
||||||
claude_config.write_text(json.dumps(payload, indent=2) + "\n")
|
|
||||||
```
|
|
||||||
|
|
||||||
`spec` here is the `AgentProvisionSpec` (or equivalent) that `agent_provider`
|
|
||||||
already receives; it needs `label` and `color` threaded in from `BottleSpec`
|
|
||||||
through whatever plan/provision object the provider operates on.
|
|
||||||
|
|
||||||
## Implementation chunks
|
|
||||||
|
|
||||||
Two PRs, each independently mergeable.
|
|
||||||
|
|
||||||
### Chunk 1 — schema + storage
|
|
||||||
|
|
||||||
- Add `label: str = ""` and `color: str = ""` to `BottleSpec`,
|
|
||||||
`BottleMetadata`, and `ActiveAgent`.
|
|
||||||
- `docker/prepare.py`: copy `spec.label` / `spec.color` into `BottleMetadata`.
|
|
||||||
- `docker/enumerate.py`: copy `metadata.label` / `metadata.color` into
|
|
||||||
`ActiveAgent`.
|
|
||||||
- `agent_provider.py` (or the plan object it reads): thread label/color through
|
|
||||||
to `claude.json` write.
|
|
||||||
- Smolmachines backend: parallel changes to metadata read/write and
|
|
||||||
`ActiveAgent` construction.
|
|
||||||
- No prompt changes; no UI changes. All existing behavior is identical.
|
|
||||||
|
|
||||||
### Chunk 2 — prompts + display
|
|
||||||
|
|
||||||
- `start.py`: add `_text_prompt_label` and `_text_prompt_color`; call them in
|
|
||||||
`cmd_start` before `_launch_bottle`; pass `label` / `color` into `BottleSpec`.
|
|
||||||
- `dashboard.py`: add `_label_color_modal` (drop-and-resume); call it in
|
|
||||||
`_new_agent_flow`; pass label/color into `BottleSpec`; add
|
|
||||||
`_color_pair_for` helper; update `_format_agent_row` to use `a.label` with
|
|
||||||
color rendering.
|
|
||||||
|
|
||||||
## Open questions
|
|
||||||
|
|
||||||
None.
|
|
||||||
@@ -1,343 +0,0 @@
|
|||||||
|
|
||||||
- **Status:** Active
|
|
||||||
- **Author:** didericis
|
|
||||||
- **Created:** 2026-06-03
|
|
||||||
- **Issue:** #174
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
The `./cli.py dashboard` command has grown from its PRD 0013 roots
|
|
||||||
(triage supervise proposals) into a parallel-agent control surface
|
|
||||||
(PRDs 0019/0020/0021): an active-agents pane, agent picker + start,
|
|
||||||
re-attach, per-bottle stop, tmux split-pane handoff, operator-
|
|
||||||
initiated `routes`/`pipelock` edits. Each chunk is reasonable on its
|
|
||||||
own; together they make the dashboard the largest CLI file in the
|
|
||||||
repo and the thing most likely to break on a rough edge (curses /
|
|
||||||
tmux / docker-exec / metadata-discovery interactions).
|
|
||||||
|
|
||||||
This PRD reverses that scope creep. The dashboard is reduced to the
|
|
||||||
**supervise-plane triage TUI** it was in PRDs 0013–0016: list pending
|
|
||||||
proposals, approve / modify / reject each one, write audit entries,
|
|
||||||
deliver the response that unblocks the agent's tool call. Everything
|
|
||||||
that's about *starting / re-entering / stopping* bottles, or about
|
|
||||||
*operator-initiated* config edits, comes out. The command is renamed
|
|
||||||
`./cli.py supervise` so the name matches what it does after the cut.
|
|
||||||
|
|
||||||
Future agent-management UX is explicitly punted: if and when a
|
|
||||||
control surface for parallel agents resurfaces, the working
|
|
||||||
assumption (per the issue) is that a web GUI — usable from mobile
|
|
||||||
— is a better second pass than another round of curses iteration.
|
|
||||||
That decision is not in this PRD's scope; this PRD only removes the
|
|
||||||
half-built local-curses path so we stop maintaining it.
|
|
||||||
|
|
||||||
## Problem
|
|
||||||
|
|
||||||
Three concrete pains, all downstream of the dashboard's growth:
|
|
||||||
|
|
||||||
1. **Surface area vs. polish.** `dashboard.py` is ~1740 lines;
|
|
||||||
`dashboard_model.py` adds another ~420. The interactions among
|
|
||||||
curses, modals, tmux split-pane, docker-exec handoff, agent
|
|
||||||
provider templates, metadata-driven re-attach, and
|
|
||||||
ExitStack-free bottle ownership are intricate enough that
|
|
||||||
shipping the next polish increment costs more than it returns.
|
|
||||||
2. **No clear ownership of "starts and stops bottles".** Today
|
|
||||||
that responsibility is split: `./cli.py start` owns one-shot
|
|
||||||
sessions; the dashboard owns multi-session bottles it started
|
|
||||||
itself; `./cli.py cleanup` owns everything else. The dashboard
|
|
||||||
tracking its own `bottles: dict[str, (cm, bottle, identity)]`
|
|
||||||
that doesn't survive a quit is a confusing third lane.
|
|
||||||
3. **Wrong target shape for a "manage many agents" UI.** The
|
|
||||||
parallel-agent experience the dashboard reaches for is mobile-
|
|
||||||
meaningful — checking in on agents from a phone is the high-
|
|
||||||
value case — and curses inside an SSH session is the wrong
|
|
||||||
tool for that. Continuing to polish a local-only TUI delays
|
|
||||||
the right next investment.
|
|
||||||
|
|
||||||
The triage half of the dashboard isn't suffering from any of these.
|
|
||||||
Pending proposals are a small, well-scoped, real workload, and the
|
|
||||||
PRD 0013–0016 surface for handling them is the right shape. The
|
|
||||||
problem is everything that got bolted onto that core after.
|
|
||||||
|
|
||||||
## Goals / Success Criteria
|
|
||||||
|
|
||||||
1. The supervise TUI starts up, lists pending proposals across all
|
|
||||||
running bottles, and supports approve / modify / reject + the
|
|
||||||
`--once` non-interactive mode — exactly as PRDs 0013–0016
|
|
||||||
specified, minus everything 0019/0020/0021 added.
|
|
||||||
2. The CLI subcommand is renamed `supervise` (was `dashboard`). The
|
|
||||||
old name is not aliased — this PRD is intentionally a
|
|
||||||
compat/breaking change (the issue carries the
|
|
||||||
`Compat/Breaking` label).
|
|
||||||
3. `dashboard.py` shrinks to a single proposal-triage curses loop:
|
|
||||||
no agents pane, no Tab pane switching, no agent picker, no
|
|
||||||
start / re-attach / stop verbs, no tmux split-pane, no
|
|
||||||
`e`/`p` operator-edit verbs, no per-process `bottles` dict.
|
|
||||||
4. `dashboard_model.py` is collapsed into whatever
|
|
||||||
`supervise.py` (CLI) needs; the model module is removed if it
|
|
||||||
has no purpose after the cut.
|
|
||||||
5. The proposal-side apply paths in `bot_bottle/backend/docker/
|
|
||||||
egress_apply.py`, `pipelock_apply.py`, and `capability_apply.py`
|
|
||||||
are unchanged — they are still called by the approve path.
|
|
||||||
6. The supervise-sidecar / proposal-queue protocol (PRD 0013) is
|
|
||||||
unchanged: the agent's experience is identical.
|
|
||||||
7. The previously-active PRDs that this one undoes are marked
|
|
||||||
`Superseded by PRD 0049`:
|
|
||||||
- PRD 0019 — active-agents pane + agent-scoped edit verbs
|
|
||||||
- PRD 0020 — start / re-attach / stop from the dashboard
|
|
||||||
- PRD 0021 — tmux split-pane
|
|
||||||
|
|
||||||
## Non-goals
|
|
||||||
|
|
||||||
- **A web GUI for managing agents.** The issue floats this as a
|
|
||||||
second pass; this PRD does not design or commit to it. The cut
|
|
||||||
is "remove the path we no longer want to invest in", not
|
|
||||||
"build the replacement".
|
|
||||||
- **A separate CLI for operator-initiated routes / pipelock
|
|
||||||
edits.** Today those edits live as `e` / `p` keys inside the
|
|
||||||
dashboard. After this PRD they don't exist anywhere — operators
|
|
||||||
who need ad-hoc edits use the same path the agents do (call the
|
|
||||||
supervise tool from inside the bottle) or hand-edit the host-
|
|
||||||
side files and restart the sidecar. Adding a `./cli.py routes
|
|
||||||
edit <slug>` verb is a follow-up if the loss bites.
|
|
||||||
- **Removing `./cli.py start` or changing its semantics.** Start
|
|
||||||
remains the one-shot launch path. PRD 0020's bottle-outlives-
|
|
||||||
process model is removed; the only path to a long-running
|
|
||||||
bottle is `./cli.py start` (foreground) plus `cli.py cleanup`
|
|
||||||
for teardown.
|
|
||||||
- **Removing the supervise-sidecar protocol or any of the three
|
|
||||||
block-remediation engines.** PRDs 0013–0016 stay Active. The
|
|
||||||
agent's view of the world doesn't change.
|
|
||||||
- **Renaming `dashboard` anywhere other than the CLI entry
|
|
||||||
point.** The dashboard-related docs (PRDs, decision records,
|
|
||||||
research notes) keep their historical references — they
|
|
||||||
describe the state of the world at the time they were written,
|
|
||||||
and the Status: Superseded line is the marker that the world
|
|
||||||
has moved on.
|
|
||||||
- **Migrating the proposal-queue file layout.** The queue still
|
|
||||||
lives at `~/.bot-bottle/queue/<slug>/`; the audit log still
|
|
||||||
lives at `~/.bot-bottle/audit/<component>-<slug>.log`. The CLI
|
|
||||||
surface changes; the on-disk surface does not.
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
### In scope
|
|
||||||
|
|
||||||
- **Rename the subcommand.** `./cli.py dashboard` becomes
|
|
||||||
`./cli.py supervise`. The module moves from `bot_bottle/cli/
|
|
||||||
dashboard.py` to `bot_bottle/cli/supervise.py`. The dispatcher
|
|
||||||
in `bot_bottle/cli/__init__.py` and the help text both update.
|
|
||||||
- **Strip the curses loop to proposal-only.** The remaining
|
|
||||||
surface is: list pending proposals (with the new-arrival bell
|
|
||||||
from PRD 0013), Enter for detail view,
|
|
||||||
`a`/`m`/`r` for approve / modify / reject, `q` to quit. No
|
|
||||||
agents pane, no Tab, no agent picker, no `n`/`x`/`e`/`p`, no
|
|
||||||
tmux dispatch, no `bottles` dict on the main loop.
|
|
||||||
- **Drop unused helpers.** `_picker_modal`, `_preflight_modal`,
|
|
||||||
`_backend_picker_modal`, `_new_agent_flow`, `_attach_to_bottle`,
|
|
||||||
`_attach_in_tmux`, `_attach_via_handoff`, `_tmux_*`,
|
|
||||||
`_ensure_right_pane`, `_redirect_stderr_to_file`,
|
|
||||||
`_route_op_to_right_pane`, `_stop_bottle_flow`,
|
|
||||||
`_operator_edit_*_flow`, `operator_edit_routes`,
|
|
||||||
`operator_edit_allowlist`, and their imports come out.
|
|
||||||
- **Collapse the model module.** `dashboard_model.py`'s
|
|
||||||
proposal-side helpers (`QueuedProposal`, `discover_pending`,
|
|
||||||
`_approval_status`, `_detail_lines`,
|
|
||||||
`_failed_url_host`, `_proposed_payload_label`,
|
|
||||||
`_suffix_for_tool`, `_REFRESH_INTERVAL_MS`) move back into
|
|
||||||
`supervise.py` (CLI) or into `bot_bottle/supervise.py`
|
|
||||||
(the daemon-side module) — wherever they fit. The agents /
|
|
||||||
picker / tmux helpers in that module (`PANE_*`,
|
|
||||||
`_filter_agents`, `_running_counts`, `_format_agent_row`,
|
|
||||||
`_selection_status`, `_selected_agent`, `_bottle_for_slug`,
|
|
||||||
`_pick_next_after_stop`, `_agent_runtime_args`,
|
|
||||||
`_build_resume_argv_with_fallback`, `_build_split_pane_argv`,
|
|
||||||
`_build_respawn_pane_argv`, `_in_tmux`,
|
|
||||||
`discover_active_agents`) are deleted.
|
|
||||||
- **Mark superseded PRDs.** The Status line on PRDs 0019, 0020,
|
|
||||||
and 0021 changes to `Superseded by [PRD 0049](0049-strip-
|
|
||||||
dashboard-to-supervisor-tui.md)`.
|
|
||||||
- **Test cleanup.** Any test that targets a removed surface (the
|
|
||||||
agent picker, the tmux split helpers, the start-from-dashboard
|
|
||||||
flow, the operator-edit flows, `discover_active_agents`)
|
|
||||||
comes out. Tests covering proposal triage stay.
|
|
||||||
- **Help / usage strings.** `bot_bottle/cli/__init__.py`'s usage
|
|
||||||
block updates the command name and one-liner.
|
|
||||||
|
|
||||||
### Out of scope
|
|
||||||
|
|
||||||
- Any new feature in the supervise TUI. The cut is purely
|
|
||||||
subtractive (except for the rename).
|
|
||||||
- Behavior changes in `./cli.py start`, `cli.py cleanup`,
|
|
||||||
`cli.py resume`, `cli.py list`, `cli.py info`, `cli.py edit`,
|
|
||||||
`cli.py init` — unchanged.
|
|
||||||
- Changes to the supervise sidecar (`supervise_server.py`,
|
|
||||||
`supervise.py` daemon module). The wire protocol stays.
|
|
||||||
- Changes to the routes / pipelock / capability apply engines.
|
|
||||||
- Migration helpers, deprecation warnings, or a transitional
|
|
||||||
`dashboard` alias for `supervise`. The label on the issue says
|
|
||||||
Compat/Breaking; the rename is a hard cutover.
|
|
||||||
|
|
||||||
## Proposed design
|
|
||||||
|
|
||||||
### Final shape of the TUI
|
|
||||||
|
|
||||||
After this PRD the `./cli.py supervise` curses surface is:
|
|
||||||
|
|
||||||
```
|
|
||||||
bot-bottle supervise (3 pending)
|
|
||||||
─────────────────────────────────────────────────────────
|
|
||||||
> 03:14:22 [implementer-cy7a6] egress-block abc123… add
|
|
||||||
github.com/foo
|
|
||||||
03:13:55 [researcher-9xqs1] pipelock-block def456… allow
|
|
||||||
registry.npmjs.org
|
|
||||||
03:13:10 [implementer-cy7a6] capability-block ghi789… install
|
|
||||||
ripgrep
|
|
||||||
|
|
||||||
─────────────────────────────────────────────────────────
|
|
||||||
[j/k] move [Enter] view [a] approve [m] modify [r] reject [q] quit
|
|
||||||
```
|
|
||||||
|
|
||||||
- One pane. No Tab. `j` / `k` / arrows move through the queue.
|
|
||||||
- Enter opens the existing detail view (justification +
|
|
||||||
proposed-file body + the green pipelock host-extraction hint).
|
|
||||||
`a` / `m` / `r` work from both the list view and the detail
|
|
||||||
view, same as today.
|
|
||||||
- `q` / Esc quits. There are no dashboard-owned bottles, so no
|
|
||||||
per-process teardown decision — `q` just exits.
|
|
||||||
- The new-arrival bell stays, because it is a real win for the
|
|
||||||
operator's "I was typing at claude and a proposal landed" case.
|
|
||||||
No tmux-specific focus management remains.
|
|
||||||
|
|
||||||
### Code organisation
|
|
||||||
|
|
||||||
After the cut, the CLI module looks roughly like:
|
|
||||||
|
|
||||||
```
|
|
||||||
bot_bottle/cli/supervise.py
|
|
||||||
- cmd_supervise(argv)
|
|
||||||
- _list_once() # --once mode
|
|
||||||
- _main_loop(stdscr) # proposal-only
|
|
||||||
- _render(stdscr, pending, ...)
|
|
||||||
- _detail_view(stdscr, qp, ...)
|
|
||||||
- _modify(stdscr, qp)
|
|
||||||
- _prompt(stdscr, label)
|
|
||||||
- _write_crash_log(exc)
|
|
||||||
- approve(qp, *, notes, final_file)
|
|
||||||
- reject(qp, *, reason)
|
|
||||||
- QueuedProposal, discover_pending
|
|
||||||
- _detail_lines, _approval_status,
|
|
||||||
_failed_url_host,
|
|
||||||
_proposed_payload_label,
|
|
||||||
_suffix_for_tool
|
|
||||||
```
|
|
||||||
|
|
||||||
`dashboard_model.py` has no purpose once the agents / picker /
|
|
||||||
tmux helpers are gone, so it is removed and the surviving
|
|
||||||
proposal-side helpers move into `supervise.py` directly. The
|
|
||||||
PRD-0013 refactor that split model out (`refactor: extract
|
|
||||||
dashboard state/model layer into dashboard_model.py`) was
|
|
||||||
load-bearing for the bigger dashboard surface; with the surface
|
|
||||||
shrunk back, the split is no longer justified.
|
|
||||||
|
|
||||||
### Removed PRDs: how to mark them
|
|
||||||
|
|
||||||
The three superseded PRDs keep their bodies intact. Only the
|
|
||||||
Status line at the top changes:
|
|
||||||
|
|
||||||
```
|
|
||||||
- **Status:** Superseded by [PRD
|
|
||||||
0049](0049-strip-dashboard-to-supervisor-tui.md)
|
|
||||||
```
|
|
||||||
|
|
||||||
The PRD's own Goals / Success Criteria are left as the historical
|
|
||||||
record of what the feature shipped — readers tracing back from the
|
|
||||||
code or the git log land in a PRD that explains what once was, with
|
|
||||||
a clear pointer forward. No PRD body is rewritten.
|
|
||||||
|
|
||||||
### Tests to keep, tests to remove
|
|
||||||
|
|
||||||
Keep:
|
|
||||||
- `tests/cli/test_dashboard*.py` cases that exercise
|
|
||||||
`discover_pending`, `approve`, `reject`, `_detail_lines`,
|
|
||||||
`_approval_status`, `_failed_url_host`,
|
|
||||||
`_proposed_payload_label`, `_suffix_for_tool`,
|
|
||||||
`_modify` / `edit_in_editor`.
|
|
||||||
- `tests/cli/test_dashboard_once.py` (or equivalent) — the
|
|
||||||
`--once` listing mode.
|
|
||||||
|
|
||||||
Remove:
|
|
||||||
- Any test of `_picker_modal`, `_preflight_modal`,
|
|
||||||
`_backend_picker_modal`, `_new_agent_flow`, `_attach_*`,
|
|
||||||
`_tmux_*`, `_route_op_to_right_pane`,
|
|
||||||
`_redirect_stderr_to_file`, `_stop_bottle_flow`,
|
|
||||||
`_operator_edit_*`, `_filter_agents`, `_running_counts`,
|
|
||||||
`_format_agent_row`, `_selection_status`,
|
|
||||||
`_selected_agent`, `_bottle_for_slug`,
|
|
||||||
`_pick_next_after_stop`, `_agent_runtime_args`,
|
|
||||||
`_build_*_argv`, `discover_active_agents`.
|
|
||||||
- The test files that exist solely to cover those (e.g.,
|
|
||||||
`test_dashboard_picker.py`, `test_dashboard_tmux.py`,
|
|
||||||
`test_dashboard_attach.py`, `test_dashboard_agents.py` —
|
|
||||||
whichever of these exist after the file walk).
|
|
||||||
|
|
||||||
Files are renamed `test_supervise_*.py` to mirror the module
|
|
||||||
rename. The rename is mechanical; no test logic changes.
|
|
||||||
|
|
||||||
## Implementation chunks
|
|
||||||
|
|
||||||
Sized for a single PR each.
|
|
||||||
|
|
||||||
1. **Strip + rename in one cut.** Move `bot_bottle/cli/
|
|
||||||
dashboard.py` to `bot_bottle/cli/supervise.py`, delete the
|
|
||||||
removed helpers, delete `dashboard_model.py`, inline the
|
|
||||||
surviving helpers, update the dispatcher + usage in
|
|
||||||
`bot_bottle/cli/__init__.py`, rename tests to match, mark
|
|
||||||
PRDs 0019/0020/0021 as superseded. One commit per logical
|
|
||||||
piece inside the PR (rename, strip, supersede notes,
|
|
||||||
tests).
|
|
||||||
2. **Activate PRD 0049.** Flip this PRD's Status line from
|
|
||||||
Draft to Active in the same PR as chunk 1 once the
|
|
||||||
implementation lands. (The repo convention is that a PRD's
|
|
||||||
shipping commit is also the Status flip — see the recent
|
|
||||||
`docs(prd): activate PRD 0048…` commit shape.)
|
|
||||||
|
|
||||||
The PR closes issue #174.
|
|
||||||
|
|
||||||
## Open questions
|
|
||||||
|
|
||||||
1. **`e` / `p` operator-initiated edits — gone for good or
|
|
||||||
moved to a separate CLI verb?** The PRD removes them with no
|
|
||||||
replacement. The simplest replacement is `./cli.py routes
|
|
||||||
edit <slug>` and `./cli.py pipelock edit <slug>`, sharing
|
|
||||||
the existing `apply_routes_change` / `apply_allowlist_change`
|
|
||||||
engines. If the loss is felt within the first parallel
|
|
||||||
run after this lands, that follow-up is a small PR. Leaving
|
|
||||||
it for a separate PRD so this one stays subtractive.
|
|
||||||
|
|
||||||
2. **`--once` output shape.** The text listing today emits one
|
|
||||||
proposal per line. Worth keeping exactly as-is for
|
|
||||||
scripting consumers; this PRD does not change it. Flagging
|
|
||||||
only because the rename could tempt a tweak.
|
|
||||||
|
|
||||||
3. **Audit-log entry shape for an unprompted edit applied via
|
|
||||||
a future `routes edit` CLI verb.** Today's
|
|
||||||
`operator_edit_routes` writes an `ACTION_OPERATOR_EDIT`
|
|
||||||
audit entry. With those flows removed the constant has no
|
|
||||||
callers inside this PRD's scope. Keep the constant exported
|
|
||||||
from `supervise.py` (it's already an `__all__` member) so a
|
|
||||||
follow-up CLI verb can re-use the same audit shape without
|
|
||||||
re-introducing dead code first.
|
|
||||||
|
|
||||||
## References
|
|
||||||
|
|
||||||
- Issue
|
|
||||||
[#174](https://gitea.dideric.is/didericis/bot-bottle/issues/174)
|
|
||||||
— the request: "strip the dashboard down into just a TUI for
|
|
||||||
managing agent requests for new egress routes and new
|
|
||||||
capabilities."
|
|
||||||
- PRD 0013 — supervise plane foundation (the floor this PRD
|
|
||||||
reverts the dashboard to).
|
|
||||||
- PRDs 0014 / 0015 / 0016 — block-remediation engines that the
|
|
||||||
supervise TUI continues to drive on approve.
|
|
||||||
- PRDs 0019 / 0020 / 0021 — the bolted-on capabilities this PRD
|
|
||||||
removes.
|
|
||||||
@@ -1,401 +0,0 @@
|
|||||||
# PRD 0050: Move provider-specific agent logic into contrib
|
|
||||||
|
|
||||||
- **Status:** Active
|
|
||||||
- **Author:** claude
|
|
||||||
- **Created:** 2026-06-03
|
|
||||||
- **Issue:** #177
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
The agent provider module (`bot_bottle/agent_provider.py`) hard-codes
|
|
||||||
the Claude- and Codex-specific provisioning rules — auth file shapes,
|
|
||||||
trust-dialog markers, egress routes, dummy-auth dance, env vars — in a
|
|
||||||
single `if template == "codex": ... if template == "claude": ...`
|
|
||||||
chain (lines 154–230 today). Other pieces of provider behavior live in
|
|
||||||
each backend's `provision/` directory (`provision_skills`,
|
|
||||||
`provision_prompt`, `provision_provider_auth`, `provision_supervise`),
|
|
||||||
duplicated once per backend, even though almost none of what they do
|
|
||||||
is actually backend-specific.
|
|
||||||
|
|
||||||
This PRD reshapes the agent provider into a proper plugin boundary.
|
|
||||||
The two existing providers (Claude, Codex) move out of `agent_provider`
|
|
||||||
into `bot_bottle/contrib/claude/` and `bot_bottle/contrib/codex/` —
|
|
||||||
the same `contrib/` layout PRD 0048 established for the Gitea
|
|
||||||
deploy-key provisioner. The four provisioner methods backends
|
|
||||||
currently duplicate move into the provider plugin itself; the backend
|
|
||||||
keeps only the bottle-side primitives (`cp_in`, `exec`) the plugin
|
|
||||||
calls through. MCP server registration becomes a first-class part of
|
|
||||||
the provider contract so Codex finally gets the supervise sidecar
|
|
||||||
wired in alongside Claude.
|
|
||||||
|
|
||||||
The shipping artifact is two new provider plugins under `contrib/`, a
|
|
||||||
narrower `AgentProvider` ABC in `bot_bottle/agent_provider.py`, four
|
|
||||||
fewer provisioner hooks on `BottleBackend`, and a supervise-MCP entry
|
|
||||||
visible from the Codex agent at launch.
|
|
||||||
|
|
||||||
## Problem
|
|
||||||
|
|
||||||
Three concrete pains, all downstream of the provider abstraction not
|
|
||||||
being where the work happens:
|
|
||||||
|
|
||||||
1. **Adding a third provider is a five-file edit.** A hypothetical
|
|
||||||
Gemini or Aider provider has to: (a) add a branch in
|
|
||||||
`agent_provision_plan`, (b) add a runtime entry in `_RUNTIMES`,
|
|
||||||
(c) thread a `prompt_mode` enum value, (d) potentially extend
|
|
||||||
`provision_provider_auth` per backend, (e) wire MCP registration
|
|
||||||
into both `backend/docker/provision/supervise.py` and
|
|
||||||
`backend/smolmachines/provision/supervise.py`. Nothing about that
|
|
||||||
spread is load-bearing; it's leftover from when there was one
|
|
||||||
provider.
|
|
||||||
|
|
||||||
2. **MCP server registration is Claude-only.** Both
|
|
||||||
`backend/docker/provision/supervise.py` and
|
|
||||||
`backend/smolmachines/provision/supervise.py` run `claude mcp add`
|
|
||||||
verbatim. Codex bottles silently get no MCP entry — the sidecar
|
|
||||||
is running, the routes are open, but the agent can't see the
|
|
||||||
tools because nothing wrote them into Codex's TOML config. Today
|
|
||||||
this is a latent gap. The provider plugin is the only layer that
|
|
||||||
knows how a given agent discovers MCP servers, so that's where
|
|
||||||
the registration belongs.
|
|
||||||
|
|
||||||
3. **`provision_skills` / `provision_prompt` / `provision_provider_auth`
|
|
||||||
are duplicated between backends.** Each backend has its own
|
|
||||||
~50-line copy. The differences are entirely about which path the
|
|
||||||
backend uses for `cp_in` and what user it `chown`s to. Same
|
|
||||||
business logic, two implementations, two test surfaces, two
|
|
||||||
places to update when the rules change.
|
|
||||||
|
|
||||||
The agent_provider module is the right home for all of this. It already
|
|
||||||
owns the `AgentProvisionPlan` (the declarative description of what
|
|
||||||
needs to land in the guest); extending it to own the imperative
|
|
||||||
"actually land it" step is the natural next move. Putting
|
|
||||||
provider-specific code under `contrib/` mirrors the convention PRD 0048
|
|
||||||
established and keeps the core package provider-agnostic.
|
|
||||||
|
|
||||||
## Goals / Success Criteria
|
|
||||||
|
|
||||||
1. `bot_bottle/agent_provider.py` contains no Claude- or
|
|
||||||
Codex-specific branches. The Claude and Codex template strings
|
|
||||||
themselves still live in the core module (they're the public
|
|
||||||
manifest values), but everything keyed off them moves out.
|
|
||||||
2. `bot_bottle/contrib/claude/agent_provider.py` and
|
|
||||||
`bot_bottle/contrib/codex/agent_provider.py` exist and contain
|
|
||||||
the provider-specific behavior previously in lines 154–230 of
|
|
||||||
`agent_provider.py`. Each is reachable from the core registry via
|
|
||||||
a lazy import (the same pattern PRD 0048 used for
|
|
||||||
`GiteaDeployKeyProvisioner`).
|
|
||||||
3. `AgentProvider` is an ABC (or protocol) with at minimum:
|
|
||||||
- `provision_plan(...) -> AgentProvisionPlan` — what the existing
|
|
||||||
`agent_provision_plan` produces today, scoped to one provider.
|
|
||||||
- `provision_skills(bottle, plan)` — copy host skills into the guest.
|
|
||||||
- `provision_prompt(bottle, plan)` — copy the prompt file, return
|
|
||||||
the in-guest path (or None).
|
|
||||||
- `provision_supervise_mcp(bottle, plan, supervise_url)` — register
|
|
||||||
the supervise sidecar in the provider's MCP config. No-op when
|
|
||||||
the bottle has no supervise sidecar.
|
|
||||||
- The Claude implementation runs `claude mcp add`. The Codex
|
|
||||||
implementation writes the corresponding entry into
|
|
||||||
`~/.codex/config.toml`'s `[mcp_servers.supervise]` table.
|
|
||||||
4. `BottleBackend` loses the four abstract methods being moved
|
|
||||||
(`provision_skills`, `provision_prompt`, `provision_provider_auth`,
|
|
||||||
`provision_supervise`). `BottleBackend.provision_in_bottle` calls
|
|
||||||
the provider plugin directly via the bottle and plan it already
|
|
||||||
has. `provision_ca`, `provision_workspace`, and `provision_git`
|
|
||||||
stay on the backend — they're backend infrastructure, not
|
|
||||||
provider behavior.
|
|
||||||
5. `bot_bottle/backend/docker/provision/{skills,prompt,provider_auth,
|
|
||||||
supervise}.py` and `bot_bottle/backend/smolmachines/provision/{skills,
|
|
||||||
prompt,provider_auth,supervise}.py` are deleted. The
|
|
||||||
backend-specific provisioners that remain (`ca`, `git`,
|
|
||||||
`workspace`) stay.
|
|
||||||
6. A Codex bottle launched with `--supervise` shows the
|
|
||||||
supervise MCP server entry in its Codex config and can call
|
|
||||||
supervise tools from inside the bottle (egress-block,
|
|
||||||
pipelock-block, capability-block).
|
|
||||||
7. Existing tests for the moved logic move with the code:
|
|
||||||
provider-specific tests under `tests/unit/test_contrib_claude_*.py`
|
|
||||||
and `tests/unit/test_contrib_codex_*.py`, mirroring
|
|
||||||
`tests/unit/test_contrib_gitea_deploy_key.py`.
|
|
||||||
8. PRD 0050's Status flips Draft → Active in the same commit that
|
|
||||||
removes the last `if template == "claude"` branch from
|
|
||||||
`agent_provider.py`.
|
|
||||||
|
|
||||||
## Non-goals
|
|
||||||
|
|
||||||
- **A third agent provider.** This PRD reshapes the boundary so a
|
|
||||||
third provider is cheap to add. It does not add one.
|
|
||||||
- **Changing the manifest surface.** The `agent.provider`
|
|
||||||
manifest field still takes `"claude"` or `"codex"`. The set of
|
|
||||||
valid strings is unchanged.
|
|
||||||
- **Changing `AgentProvisionPlan`'s shape.** The dataclasses
|
|
||||||
(`AgentProvisionDir`, `AgentProvisionFile`, `AgentProvisionCommand`,
|
|
||||||
`AgentProvisionPlan` itself) stay in the core module and keep their
|
|
||||||
current fields. Provider plugins produce the same plan shape; only
|
|
||||||
the producer moves.
|
|
||||||
- **Changing the supervise sidecar protocol or the supervise tool
|
|
||||||
surface.** PRDs 0013–0016 stay Active. What changes is how the
|
|
||||||
agent discovers the sidecar's MCP endpoint, not what it does once
|
|
||||||
connected.
|
|
||||||
- **Per-skill provider differences.** A Codex agent and a Claude
|
|
||||||
agent see the same `~/.claude/skills/<name>/` tree today (Codex
|
|
||||||
reads it via its own skills mechanism). This PRD does not change
|
|
||||||
that — `provision_skills` lands the same content for both.
|
|
||||||
- **Removing the `prompt_args` helper from `agent_provider.py`.** It
|
|
||||||
stays at module scope; it's already a pure dispatch on `prompt_mode`
|
|
||||||
and has no Claude/Codex `if` chain to extract.
|
|
||||||
- **`provision_provider_auth` migration.** The issue notes this method
|
|
||||||
is "probably not needed anymore" once each provider owns its own
|
|
||||||
provisioning. After the move, the work that
|
|
||||||
`provision_provider_auth` did (apply `dirs` / `files` / `pre_copy` /
|
|
||||||
`verify` from the plan) becomes a shared helper the per-provider
|
|
||||||
`provision_skills` / `provision_prompt` calls dispatch through —
|
|
||||||
or, more likely, a single `provision(bottle)` entry point on the
|
|
||||||
provider. The hook is removed from `BottleBackend`; whether the
|
|
||||||
underlying loop lives on `AgentProvider` as a default
|
|
||||||
implementation or as a free function in `contrib/_apply.py` is
|
|
||||||
decided at implementation time, not in this PRD.
|
|
||||||
|
|
||||||
## Scope
|
|
||||||
|
|
||||||
### In scope
|
|
||||||
|
|
||||||
- New `AgentProvider` ABC in `bot_bottle/agent_provider.py` with the
|
|
||||||
five methods listed under Goal 3. Existing `agent_provision_plan`
|
|
||||||
becomes `AgentProvider.provision_plan`.
|
|
||||||
- New `bot_bottle/contrib/claude/__init__.py`,
|
|
||||||
`bot_bottle/contrib/claude/agent_provider.py`,
|
|
||||||
`bot_bottle/contrib/codex/__init__.py`,
|
|
||||||
`bot_bottle/contrib/codex/agent_provider.py`. Each defines a
|
|
||||||
`ClaudeAgentProvider` / `CodexAgentProvider` class.
|
|
||||||
- A `get_provider(template) -> AgentProvider` registry in
|
|
||||||
`bot_bottle/agent_provider.py`, lazy-imported from `contrib/`,
|
|
||||||
mirroring `get_provisioner(provider, ...)` in
|
|
||||||
`bot_bottle/deploy_key_provisioner.py`.
|
|
||||||
- Backend changes:
|
|
||||||
- `BottleBackend.provision_in_bottle` resolves the provider once
|
|
||||||
and calls `provider.provision_skills(bottle, plan)`,
|
|
||||||
`provider.provision_prompt(bottle, plan)`, and
|
|
||||||
`provider.provision_supervise_mcp(bottle, plan, url)` in place
|
|
||||||
of the current four abstract hooks.
|
|
||||||
- `BottleBackend.provision_skills`, `provision_prompt`,
|
|
||||||
`provision_provider_auth`, `provision_supervise` are removed.
|
|
||||||
- Docker and smolmachines backends remove their corresponding
|
|
||||||
`provision_*` implementations and the
|
|
||||||
`backend/<name>/provision/{skills,prompt,provider_auth,
|
|
||||||
supervise}.py` modules.
|
|
||||||
- Codex MCP wiring: `CodexAgentProvider.provision_supervise_mcp`
|
|
||||||
writes a `[mcp_servers.supervise]` block into
|
|
||||||
`~/.codex/config.toml` pointing at the same agent-side supervise
|
|
||||||
URL the Claude provider uses. The file already exists from the
|
|
||||||
trust-dialog step; the MCP entry is appended (or the file is
|
|
||||||
rewritten in a single shot, whichever's simpler).
|
|
||||||
- Tests migrate. Backend tests that targeted the four moved
|
|
||||||
provisioners are rewritten against the provider plugin, with one
|
|
||||||
test file per provider mirroring `tests/unit/test_contrib_gitea_*.py`.
|
|
||||||
|
|
||||||
### Out of scope
|
|
||||||
|
|
||||||
- Adding a manifest field for "extra MCP servers the agent should
|
|
||||||
see". The supervise sidecar is the only MCP server provisioned
|
|
||||||
today, and the issue's "Add mcp server configuring into agent
|
|
||||||
provision" line is about the supervise sidecar specifically. A
|
|
||||||
general-purpose user-declared MCP list is a follow-up if and when
|
|
||||||
the need surfaces.
|
|
||||||
- Refactoring `AgentProvisionPlan`'s dataclasses. They stay byte-
|
|
||||||
for-byte the same so the diff is purely "who owns the producer".
|
|
||||||
- A `BottleBackend.provision_provider_auth` shim during transition.
|
|
||||||
The hook is removed in one cut; the only caller is the backend
|
|
||||||
itself, no manifest consumers reference it.
|
|
||||||
- Renaming `agent_provider.py` → `agent_providers/`. The module
|
|
||||||
still has core dataclasses + the ABC + the registry; it's a single
|
|
||||||
file's worth of code.
|
|
||||||
|
|
||||||
## Proposed design
|
|
||||||
|
|
||||||
### Module shape after the cut
|
|
||||||
|
|
||||||
```
|
|
||||||
bot_bottle/agent_provider.py
|
|
||||||
PROVIDER_CLAUDE, PROVIDER_CODEX, PROVIDER_TEMPLATES
|
|
||||||
PromptMode (Literal)
|
|
||||||
AgentProvisionDir, AgentProvisionFile, AgentProvisionCommand,
|
|
||||||
AgentProvisionPlan (dataclasses, unchanged)
|
|
||||||
AgentProviderRuntime (dataclass — template/command/image/etc.)
|
|
||||||
AgentProvider (ABC)
|
|
||||||
.runtime() -> AgentProviderRuntime
|
|
||||||
.provision_plan(state_dir, ..., trusted_project_path, ...) -> AgentProvisionPlan
|
|
||||||
.provision_skills(bottle, plan) -> None
|
|
||||||
.provision_prompt(bottle, plan) -> str | None
|
|
||||||
.provision_supervise_mcp(bottle, plan, supervise_url) -> None
|
|
||||||
get_provider(template: str) -> AgentProvider # lazy-imports contrib
|
|
||||||
prompt_args(prompt_mode, prompt_path, *, argv) # unchanged
|
|
||||||
|
|
||||||
bot_bottle/contrib/claude/agent_provider.py
|
|
||||||
ClaudeAgentProvider(AgentProvider)
|
|
||||||
_RUNTIME = AgentProviderRuntime(template="claude", ...)
|
|
||||||
.provision_plan(...) # owns the lines-204–230 chunk
|
|
||||||
.provision_skills(...) # was backend/<name>/provision/skills.py
|
|
||||||
.provision_prompt(...) # was backend/<name>/provision/prompt.py
|
|
||||||
.provision_supervise_mcp(...)# was backend/<name>/provision/supervise.py
|
|
||||||
|
|
||||||
bot_bottle/contrib/codex/agent_provider.py
|
|
||||||
CodexAgentProvider(AgentProvider)
|
|
||||||
_RUNTIME = AgentProviderRuntime(template="codex", ...)
|
|
||||||
.provision_plan(...) # owns the lines-154–204 chunk
|
|
||||||
.provision_skills(...) # same as Claude impl, factored to shared helper
|
|
||||||
.provision_prompt(...) # same as Claude impl, factored to shared helper
|
|
||||||
.provision_supervise_mcp(...)# writes [mcp_servers.supervise] to config.toml
|
|
||||||
```
|
|
||||||
|
|
||||||
The skills / prompt / provider-auth-apply implementations are 99%
|
|
||||||
identical across providers — `cp_in` then `chown` / `chmod`. They are
|
|
||||||
extracted to small free functions in
|
|
||||||
`bot_bottle/contrib/_provision_apply.py` (or kept as default
|
|
||||||
implementations on `AgentProvider` if every concrete subclass would
|
|
||||||
just call them). Picked at implementation time; both options match
|
|
||||||
PRD 0048's contrib convention. The visible contract is that
|
|
||||||
provisioning lives on the provider plugin.
|
|
||||||
|
|
||||||
### MCP registration for Codex
|
|
||||||
|
|
||||||
Codex reads MCP servers from `~/.codex/config.toml` (or whatever
|
|
||||||
`CODEX_HOME/config.toml` resolves to). The provider already writes
|
|
||||||
this file once during `provision_plan` to set the project trust
|
|
||||||
level. `CodexAgentProvider.provision_supervise_mcp` extends the
|
|
||||||
existing write: same path, append a `[mcp_servers.supervise]` table
|
|
||||||
pointing at the agent-side supervise URL.
|
|
||||||
|
|
||||||
Two implementation routes worth flagging:
|
|
||||||
|
|
||||||
- **Option A:** Pre-bake the MCP entry in the same config-write that
|
|
||||||
happens during `provision_plan`, before bottle launch. Simpler;
|
|
||||||
the supervise URL has to be known at plan time, which means
|
|
||||||
`provision_plan` needs the supervise URL (or a sentinel that means
|
|
||||||
"fill this in"). The smolmachines backend already plumbs
|
|
||||||
`agent_supervise_url` through to its provision_supervise step, so
|
|
||||||
the value is available.
|
|
||||||
- **Option B:** Append at bottle-launch time via a `bottle.exec`
|
|
||||||
that writes to the file inside the guest, matching the
|
|
||||||
`claude mcp add` flow. Slower but uniform with how
|
|
||||||
`ClaudeAgentProvider.provision_supervise_mcp` works.
|
|
||||||
|
|
||||||
Option B is the symmetric choice and the one this PRD assumes.
|
|
||||||
The implementer can switch to A if Option B turns out to need a
|
|
||||||
TOML-merge primitive the codebase doesn't already have.
|
|
||||||
|
|
||||||
### Backend after the cut
|
|
||||||
|
|
||||||
```python
|
|
||||||
class BottleBackend:
|
|
||||||
def provision_in_bottle(self, plan, bottle, supervise_url):
|
|
||||||
provider = get_provider(plan.spec.manifest.agents[
|
|
||||||
plan.spec.agent_name].provider)
|
|
||||||
self.provision_ca(plan, bottle)
|
|
||||||
prompt_path = provider.provision_prompt(bottle, plan)
|
|
||||||
provider.provision_skills(bottle, plan)
|
|
||||||
self.provision_workspace(plan, bottle)
|
|
||||||
self.provision_git(plan, bottle)
|
|
||||||
provider.provision_supervise_mcp(bottle, plan, supervise_url)
|
|
||||||
return prompt_path
|
|
||||||
```
|
|
||||||
|
|
||||||
`supervise_url` is the existing per-backend "where does the agent
|
|
||||||
reach the sidecar from inside the guest" value. The Docker backend
|
|
||||||
passes `http://supervise:<port>/`; smolmachines passes the
|
|
||||||
`http://127.0.0.1:<port>/` it already computed. The backend's only
|
|
||||||
remaining provider-touching duty is "tell the provider what the
|
|
||||||
sidecar URL is".
|
|
||||||
|
|
||||||
### Registry
|
|
||||||
|
|
||||||
```python
|
|
||||||
# bot_bottle/agent_provider.py
|
|
||||||
def get_provider(template: str) -> AgentProvider:
|
|
||||||
if template == PROVIDER_CLAUDE:
|
|
||||||
from bot_bottle.contrib.claude.agent_provider import (
|
|
||||||
ClaudeAgentProvider,
|
|
||||||
)
|
|
||||||
return ClaudeAgentProvider()
|
|
||||||
if template == PROVIDER_CODEX:
|
|
||||||
from bot_bottle.contrib.codex.agent_provider import (
|
|
||||||
CodexAgentProvider,
|
|
||||||
)
|
|
||||||
return CodexAgentProvider()
|
|
||||||
raise ValueError(f"unknown agent provider template: {template!r}")
|
|
||||||
```
|
|
||||||
|
|
||||||
Lazy imports keep core import-time graph small and match PRD 0048.
|
|
||||||
|
|
||||||
## Implementation chunks
|
|
||||||
|
|
||||||
Each chunk is one commit on the PR; the PR ships as one cut.
|
|
||||||
|
|
||||||
1. **Lift `AgentProvider` ABC + registry.** Add the ABC and
|
|
||||||
`get_provider` next to the existing `agent_provision_plan`
|
|
||||||
function. Have `agent_provision_plan` delegate to
|
|
||||||
`get_provider(template).provision_plan(...)` so callers keep
|
|
||||||
working through the transition.
|
|
||||||
2. **Move provider-specific `provision_plan` content into
|
|
||||||
contrib.** Create `contrib/claude/` and `contrib/codex/`. The
|
|
||||||
Claude and Codex branches of `agent_provision_plan` move into
|
|
||||||
the respective provider classes. The shared scaffolding
|
|
||||||
(initial dict setup, final `AgentProvisionPlan(...)` return)
|
|
||||||
stays in the ABC as a template method or moves into each
|
|
||||||
subclass — whichever needs less indirection.
|
|
||||||
3. **Move backend provisioners onto the provider.** Add
|
|
||||||
`provision_skills`, `provision_prompt`, `provision_supervise_mcp`
|
|
||||||
to `AgentProvider` (with a shared apply helper for skills /
|
|
||||||
prompt). Update `BottleBackend.provision_in_bottle` to call them.
|
|
||||||
Delete the four backend hook methods and the eight
|
|
||||||
`backend/<name>/provision/{skills,prompt,provider_auth,supervise}.py`
|
|
||||||
modules.
|
|
||||||
4. **Add Codex MCP support.** Implement
|
|
||||||
`CodexAgentProvider.provision_supervise_mcp` against
|
|
||||||
`~/.codex/config.toml`. Add a unit test that runs the method
|
|
||||||
against an in-memory FakeBottle and asserts the
|
|
||||||
`[mcp_servers.supervise]` block is present.
|
|
||||||
5. **Migrate tests.** Per-backend tests for the moved
|
|
||||||
provisioners turn into per-provider tests under
|
|
||||||
`tests/unit/test_contrib_claude_*.py` and
|
|
||||||
`tests/unit/test_contrib_codex_*.py`. Keep one integration-style
|
|
||||||
test per backend that confirms `provision_in_bottle` still
|
|
||||||
reaches every step.
|
|
||||||
6. **Activate.** Flip Status: Draft → Active in this PRD; close
|
|
||||||
#177 on merge.
|
|
||||||
|
|
||||||
## Open questions (resolved)
|
|
||||||
|
|
||||||
1. **`codex mcp add` exists.** Implementation calls
|
|
||||||
`codex mcp add --transport http supervise <url>` as `node` —
|
|
||||||
symmetric with `claude mcp add` (no `--scope user`; Codex writes
|
|
||||||
`~/.codex/config.toml` by default). Failure logs a warning; the
|
|
||||||
bottle still works without the entry.
|
|
||||||
2. **Each provider owns its apply steps end-to-end.** The base
|
|
||||||
ABC declares `provision_skills` / `provision_prompt` /
|
|
||||||
`provision` as abstract; each concrete provider implements its
|
|
||||||
own copy loop. No shared `_provision_apply.py`. The apply
|
|
||||||
sequences look similar today, but Claude and Codex harnesses
|
|
||||||
diverge over time (codex already grew a dummy-auth dance + a
|
|
||||||
`codex login status` verify with no Claude analogue) and the
|
|
||||||
"shared because both happen to call cp_in then chown" coupling
|
|
||||||
would just rot. Duplication is intentional.
|
|
||||||
3. **Env knobs removed.** `BOT_BOTTLE_CONTAINER_HOME`,
|
|
||||||
`BOT_BOTTLE_GUEST_HOME`, `BOT_BOTTLE_CONTAINER_SKILLS_DIR`, and
|
|
||||||
`BOT_BOTTLE_GUEST_SKILLS_DIR` are gone; `/home/node` is hardcoded
|
|
||||||
everywhere it was read. The values were effectively constants;
|
|
||||||
the knobs added surface area for no real flexibility.
|
|
||||||
|
|
||||||
## References
|
|
||||||
|
|
||||||
- Issue
|
|
||||||
[#177](https://gitea.dideric.is/didericis/bot-bottle/issues/177)
|
|
||||||
— the request: move provider logic into contrib, add MCP
|
|
||||||
configuration to agent provision, rename provision_supervise →
|
|
||||||
provision_supervise_mcp, ensure Codex gets MCP provisioned.
|
|
||||||
- PRD 0013 — supervise plane foundation (defines the MCP-discoverable
|
|
||||||
block-remediation tools this PRD makes available to Codex).
|
|
||||||
- PRD 0048 — SSH deploy key provisioning (the `contrib/` convention
|
|
||||||
this PRD follows).
|
|
||||||
- Current source:
|
|
||||||
[agent_provider.py L154-L230](https://gitea.dideric.is/didericis/bot-bottle/src/branch/main/bot_bottle/agent_provider.py#L154-L230)
|
|
||||||
— the provider-specific block this PRD relocates to contrib.
|
|
||||||
@@ -1,151 +0,0 @@
|
|||||||
# Gitea Webhook Agent Dispatch
|
|
||||||
|
|
||||||
## Question
|
|
||||||
|
|
||||||
How should bot-bottle spawn and manage agents in response to Gitea PR events — and how do we reuse the same agent (with its full session context) across every event in a PR's lifecycle?
|
|
||||||
|
|
||||||
## Summary
|
|
||||||
|
|
||||||
A lightweight webhook receiver maps Gitea PR events to `cli.py` invocations. Spawning is straightforward: the existing work on non-interactive run mode (see [host-dispatch-to-container-agents.md](host-dispatch-to-container-agents.md)) is the missing piece. Session continuity is harder: it requires tracking two identifiers per open PR — the **bottle identity** (bot-bottle's slug for the container state dir) and the **Claude session ID** (the UUID Claude writes to its JSONL transcript). The transcript snapshot mechanism already used by capability-block is the right foundation; it just needs a non-interactive path and a PR-keyed store.
|
|
||||||
|
|
||||||
## Gitea Webhook Events for PR Lifecycle
|
|
||||||
|
|
||||||
Gitea fires `X-Gitea-Event: pull_request` (with an `action` field) for most PR state changes. The payload always includes `pull_request.number`, which is the stable key for correlating events to a running agent.
|
|
||||||
|
|
||||||
| `X-Gitea-Event` value | Relevant `action` values | When it fires |
|
|
||||||
|---|---|---|
|
|
||||||
| `pull_request` | `opened`, `reopened`, `closed`, `synchronized` | PR created, closed, or pushed to |
|
|
||||||
| `pull_request_comment` | `created`, `edited` | Timeline comment posted |
|
|
||||||
| `pull_request_review_approved` | — | Review submitted with approval |
|
|
||||||
| `pull_request_review_rejected` | — | Review submitted requesting changes |
|
|
||||||
| `pull_request_review_comment` | — | Inline code review comment |
|
|
||||||
| `pull_request_sync` | — | New commits pushed to the PR branch |
|
|
||||||
|
|
||||||
`pull_request` with `action: synchronized` and `pull_request_sync` both fire on push; they carry the same information but are separate subscriptions in the webhook config UI. Subscribe to `pull_request` and `pull_request_review` (the umbrella) plus `pull_request_comment` to cover the full lifecycle.
|
|
||||||
|
|
||||||
The webhook receiver validates the `X-Gitea-Signature-256` HMAC header (SHA-256 of the raw body, keyed by the configured secret) before dispatching.
|
|
||||||
|
|
||||||
## Spawning an Agent From a Webhook
|
|
||||||
|
|
||||||
### What we need from bot-bottle
|
|
||||||
|
|
||||||
The current `cli.py start` is interactive — it prompts y/N and attaches a tty. A webhook handler needs a non-interactive mode that:
|
|
||||||
|
|
||||||
1. Starts the container for a named agent.
|
|
||||||
2. Runs `claude -p "<task>" --output-format json --dangerously-skip-permissions` inside it (no tty, no session picker).
|
|
||||||
3. Captures stdout as JSON, extracts `session_id`.
|
|
||||||
4. Blocks until Claude exits, then tears down.
|
|
||||||
|
|
||||||
The [host-dispatch-to-container-agents](host-dispatch-to-container-agents.md) research proposes `cli.py run <agent> <task>` for exactly this. That command is the prerequisite for everything below. It should return the Claude JSON output so callers can extract `session_id`.
|
|
||||||
|
|
||||||
### Webhook receiver sketch
|
|
||||||
|
|
||||||
The receiver is a small HTTP service (Flask, FastAPI, or a Go net/http handler) running alongside bot-bottle on the host. It:
|
|
||||||
|
|
||||||
1. Validates the HMAC signature.
|
|
||||||
2. Extracts `pull_request.number` and `X-Gitea-Event` / `action`.
|
|
||||||
3. Looks up whether a bottle already exists for this PR number.
|
|
||||||
4. Spawns or resumes accordingly (see next section).
|
|
||||||
5. Optionally posts a comment back to the PR via Gitea API once Claude finishes.
|
|
||||||
|
|
||||||
The receiver does not need to be async or queue-based for a single-repo bot, but should at minimum serialize events for the same PR number (a per-PR lock) to avoid two concurrent sessions clobbering each other's transcript.
|
|
||||||
|
|
||||||
## Reusing the Same Agent Across a PR
|
|
||||||
|
|
||||||
This is the harder problem. Two separate identities need to be tracked and connected:
|
|
||||||
|
|
||||||
### Identity 1: bottle identity (bot-bottle slug)
|
|
||||||
|
|
||||||
The slug is the per-bottle state directory name (`~/.bot-bottle/state/<slug>/`). It's what `cli.py resume <slug>` uses to relaunch a container and mount the preserved state — including the transcript snapshot. This already works for the capability-block flow.
|
|
||||||
|
|
||||||
### Identity 2: Claude session ID
|
|
||||||
|
|
||||||
Claude Code's `--output-format json` response includes a `session_id` UUID. Passing `--resume <session_id>` on a subsequent non-interactive run makes Claude continue from exactly that conversation, with full memory of prior tool calls. `--continue` (which maps to `resume_args` in `agent_provider.py`) only picks up the *most recent* session in the project directory — unsafe when multiple sessions may be running concurrently.
|
|
||||||
|
|
||||||
The session JSONL lives at `~/.claude/projects/<encoded-cwd>/<session_id>.jsonl` inside the container guest. The transcript snapshot (`snapshot_transcript(slug)` in `capability_apply.py`) copies all of `~/.claude` out of the container before teardown, so the JSONL is preserved in `~/.bot-bottle/state/<slug>/transcript/.claude/`. When the bottle is relaunched and the transcript remounted, `claude --resume <session_id>` can find the JSONL at the right path.
|
|
||||||
|
|
||||||
### Per-PR session registry
|
|
||||||
|
|
||||||
The receiver needs a small persistent map:
|
|
||||||
|
|
||||||
```
|
|
||||||
PR number → { bottle_identity: str, claude_session_id: str, agent_name: str }
|
|
||||||
```
|
|
||||||
|
|
||||||
The simplest implementation is a JSON file at `~/.bot-bottle/pr-sessions.json`, written after each successful first-run and updated with each resume. A sqlite database is better if concurrent multi-repo support is needed.
|
|
||||||
|
|
||||||
### Full lifecycle flow
|
|
||||||
|
|
||||||
```
|
|
||||||
PR opened
|
|
||||||
→ webhook: action=opened
|
|
||||||
→ no entry in pr-sessions.json
|
|
||||||
→ cli.py run <agent> "Review PR #N: <title>\n<diff URL>"
|
|
||||||
→ starts container, runs claude -p ... --output-format json
|
|
||||||
→ on success: captures session_id from JSON output
|
|
||||||
→ snapshot_transcript(slug)
|
|
||||||
→ tears down container
|
|
||||||
→ write pr-sessions.json: { pr: N, slug: <slug>, session_id: <uuid> }
|
|
||||||
|
|
||||||
PR gets new commit
|
|
||||||
→ webhook: action=synchronized OR pull_request_sync
|
|
||||||
→ look up pr-sessions.json: found slug + session_id
|
|
||||||
→ cli.py run-resume <slug> --claude-session <session_id> "New commits pushed. Review the diff."
|
|
||||||
→ relaunches container with transcript snapshot mounted
|
|
||||||
→ runs claude -p ... --resume <session_id> --output-format json
|
|
||||||
→ captures new session_id (same or rotated)
|
|
||||||
→ snapshot_transcript(slug) again
|
|
||||||
→ update pr-sessions.json with latest session_id
|
|
||||||
|
|
||||||
Comment @-mentions bot
|
|
||||||
→ webhook: pull_request_comment, action=created
|
|
||||||
→ extract comment body, check for bot mention
|
|
||||||
→ same resume flow as above with comment as the prompt
|
|
||||||
|
|
||||||
PR closed / merged
|
|
||||||
→ webhook: action=closed
|
|
||||||
→ cli.py cleanup <slug> (or equivalent)
|
|
||||||
→ remove from pr-sessions.json
|
|
||||||
```
|
|
||||||
|
|
||||||
### What needs to be built
|
|
||||||
|
|
||||||
| Piece | Status | Notes |
|
|
||||||
|---|---|---|
|
|
||||||
| `cli.py run <agent> <task>` | Missing | Non-interactive start; see host-dispatch research |
|
|
||||||
| `cli.py run-resume <slug> --claude-session <id> <task>` | Missing | Like `resume` but non-interactive, passes `--resume <id>` to claude |
|
|
||||||
| `snapshot_transcript` on clean exit | Exists (PRD 0012) | Already called from `start.py`'s session-end path |
|
|
||||||
| Transcript remount on resume | Exists | `bottle_state.py::transcript_snapshot_dir` → docker cp in on launch |
|
|
||||||
| PR session registry | Missing | Needs to be designed; `~/.bot-bottle/pr-sessions.json` is the simplest start |
|
|
||||||
| Webhook receiver service | Missing | New service; needs to be a declared bottle or run as a host process |
|
|
||||||
|
|
||||||
## Known Rough Edges
|
|
||||||
|
|
||||||
**Session ID is not available from within the session.** The ID is only in the `--output-format json` result, readable after the process exits. There is no env var or hook that exposes it mid-session ([upstream issue #44607](https://github.com/anthropics/claude-code/issues/44607)). For the webhook bot this is fine — the outer receiver reads it from the subprocess result.
|
|
||||||
|
|
||||||
**`--continue` vs `--resume <id>`:** The existing `resume_args = ("--continue",)` in `agent_provider.py` picks up the *most recent* session. For an interactive single-user resume this is fine. For a webhook bot that may have multiple open PRs, it is not safe — two PRs' transcripts would collide if they share a project directory encoding. Use `--resume <session_id>` explicitly.
|
|
||||||
|
|
||||||
**Project directory encoding.** Claude stores sessions keyed by the absolute cwd, encoded as a path. Inside the container the cwd is always `/home/node` or a subdir. As long as every run for the same PR uses the same cwd, `--resume <session_id>` will find the right JSONL. The cwd should be pinned per PR entry in the session registry.
|
|
||||||
|
|
||||||
**Concurrent events for the same PR.** If two webhooks arrive close together (e.g., push + CI comment), the receiver must serialize them. A per-PR asyncio lock or a simple file lock on the session registry entry is enough.
|
|
||||||
|
|
||||||
**Context window growth.** Each resume appends to the same session. A PR with many round trips will eventually hit the context limit. Mitigation options: start a fresh Claude session (new `cli.py run`) periodically and carry forward a summary; or rely on Claude's built-in compaction. The session registry could include a turn count to trigger rotation.
|
|
||||||
|
|
||||||
**Webhook delivery ordering.** Gitea does not guarantee ordered delivery or exactly-once delivery. The receiver should be idempotent (same PR event processed twice should not create two bottles) and should ignore events for closed PRs.
|
|
||||||
|
|
||||||
## Relationship to Existing Bot-Bottle Infrastructure
|
|
||||||
|
|
||||||
The transcript snapshot + bottle identity system (PRD 0012, `capability_apply.py`) was designed for the capability-block flow: an operator-triggered resume after a security event. The webhook flow is the same mechanism on a faster loop driven by Gitea events instead of operator action. The implementation delta is:
|
|
||||||
|
|
||||||
1. Non-interactive run mode (the `cli.py run` gap already identified in host-dispatch research).
|
|
||||||
2. Passing `--resume <session_id>` explicitly rather than `--continue`.
|
|
||||||
3. A PR-keyed registry to connect PR numbers to bottle identities and session IDs.
|
|
||||||
4. A webhook receiver to drive the loop.
|
|
||||||
|
|
||||||
These are additive changes that sit on top of the existing transcript preservation machinery without altering it.
|
|
||||||
|
|
||||||
## Recommendation
|
|
||||||
|
|
||||||
Start with the non-interactive run mode (`cli.py run`) since everything else depends on it. Once that exists, the webhook receiver and session registry are straightforward glue. The receiver should run as a host process (not inside a bottle) since it needs to call `cli.py` and manage the session registry file. Serialize per-PR to avoid concurrency bugs. Use `--resume <session_id>` (not `--continue`) for all resume paths.
|
|
||||||
|
|
||||||
The PR session registry is deliberately minimal to start — a JSON file is fine. If multi-repo or multi-agent scenarios appear, migrating to sqlite is a one-file change.
|
|
||||||
@@ -1,278 +0,0 @@
|
|||||||
# Local Ollama: Deployment Topology, Harness Selection, and Model Sizing
|
|
||||||
|
|
||||||
Research notes on running Ollama locally for a bot-bottle coding agent workflow.
|
|
||||||
Covers the native-vs-VM question, which harness integrates best with an agent loop,
|
|
||||||
and which models make sense on an RTX 3070 (8 GB VRAM / 30 GB RAM) machine.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 1. Deployment topology: native, container, or VM?
|
|
||||||
|
|
||||||
The core question is whether running Ollama in a VM significantly degrades inference
|
|
||||||
performance. The short answer: a full KVM/QEMU VM with GPU passthrough adds roughly
|
|
||||||
2–5% overhead, Docker on Linux adds roughly 1–2%, and LXC containers add sub-1%. None
|
|
||||||
of these are significant for interactive coding use.
|
|
||||||
|
|
||||||
### Native (bare metal)
|
|
||||||
|
|
||||||
Zero overhead, immediate GPU access, simplest setup. The right default for a solo
|
|
||||||
developer doing inference on their own workstation.
|
|
||||||
|
|
||||||
### Docker containers on Linux + NVIDIA
|
|
||||||
|
|
||||||
With `nvidia-container-toolkit` and `--gpus all`, containerized Ollama runs at
|
|
||||||
essentially native speed (~1–2% overhead on Linux). The dramatic exception is macOS,
|
|
||||||
where Docker Desktop runs a Linux VM with no access to Apple's Metal/GPU — inference
|
|
||||||
is 5–6× slower. On Linux/Windows with NVIDIA hardware, Docker is fine.
|
|
||||||
|
|
||||||
Common pitfall: if `docker exec ollama ollama ps` shows 0 GPU layers, the container
|
|
||||||
fell back to CPU. Usual causes: stale VRAM allocation, missing `nvidia-container-toolkit`,
|
|
||||||
or a host driver too old for the container's CUDA version.
|
|
||||||
|
|
||||||
### KVM/QEMU VM with full PCIe passthrough
|
|
||||||
|
|
||||||
Full GPU passthrough makes the GPU invisible to the host while the VM owns it. Overhead
|
|
||||||
from the IOMMU translation layer and virtualized PCIe bus is ~2–5%. This is viable if
|
|
||||||
you need VM-level isolation (snapshotting, migration, separate kernel). Setup complexity
|
|
||||||
is non-trivial: BIOS IOMMU, IOMMU group management, VFIO driver binding. Once configured
|
|
||||||
it is stable.
|
|
||||||
|
|
||||||
**Critical gotcha:** set the VM's CPU type to `host`. If left at the default
|
|
||||||
(`x86-64-v2-AES` / "QEMU Virtual CPU version 2.5+"), Ollama may silently disable GPU
|
|
||||||
support even when drivers appear correct.
|
|
||||||
|
|
||||||
### LXC containers (Proxmox et al.)
|
|
||||||
|
|
||||||
The sweet spot for isolation without overhead. Sub-1% performance difference from bare
|
|
||||||
metal because LXC shares the host kernel; GPU device files are bind-mounted into the
|
|
||||||
container. The tradeoff is weaker isolation (shared kernel) and the requirement that
|
|
||||||
host and container driver versions match. Not suitable if you need VM-level snapshots
|
|
||||||
or live migration.
|
|
||||||
|
|
||||||
### Summary
|
|
||||||
|
|
||||||
| Topology | GPU overhead | Isolation | Complexity |
|
|
||||||
|---|---|---|---|
|
|
||||||
| Native | 0% | None | Low |
|
|
||||||
| Docker (Linux) | ~1–2% | Process | Low |
|
|
||||||
| LXC | <1% | Namespace | Medium |
|
|
||||||
| KVM passthrough | 2–5% | Full VM | High |
|
|
||||||
| VM no passthrough | CPU-only | Full VM | Medium |
|
|
||||||
|
|
||||||
Running Ollama in a VM will **not** significantly slow inference as long as GPU passthrough
|
|
||||||
is configured. Without passthrough (software rendering / CPU fallback) performance
|
|
||||||
collapses — that is what the user is rightly worried about.
|
|
||||||
|
|
||||||
### Local vs. remote server
|
|
||||||
|
|
||||||
| Factor | Local machine | Remote server |
|
|
||||||
|---|---|---|
|
|
||||||
| Latency | Near-zero | Network round-trip; cumulative in agent loops |
|
|
||||||
| Cost | Zero after hardware | Per-token or subscription |
|
|
||||||
| Privacy | 100% on-device | Data leaves the machine |
|
|
||||||
| Model size ceiling | VRAM-limited | No hard limit (671B+ feasible) |
|
|
||||||
| Offline use | Yes | No |
|
|
||||||
| Concurrency under load | Sequential by default | Scales horizontally |
|
|
||||||
|
|
||||||
For agentic coding workflows making 20–50 tool calls per session, network latency
|
|
||||||
accumulates quickly. Local inference eliminates this. A practical hybrid pattern:
|
|
||||||
use the local GPU for routine coding loops; route only to a remote API for tasks
|
|
||||||
requiring a 70B+ model or very long context (>128K tokens).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2. Harness selection
|
|
||||||
|
|
||||||
The landscape in 2026 has settled into three categories: IDE plugins, terminal agents,
|
|
||||||
and chat UIs.
|
|
||||||
|
|
||||||
### Continue.dev — recommended IDE plugin
|
|
||||||
|
|
||||||
Open-source VS Code / JetBrains / Zed / Vim extension. Routes autocomplete, chat, and
|
|
||||||
refactoring commands to any configured LLM backend (Ollama, cloud APIs). The recommended
|
|
||||||
setup uses two models: a small FIM-capable model for inline autocomplete (Qwen2.5-Coder 7B)
|
|
||||||
and a larger model for chat/edit. Handles inline completions, multi-file edits, and
|
|
||||||
codebase-aware chat. No API key, no data leaving the machine.
|
|
||||||
|
|
||||||
### Aider — recommended for git-native terminal workflows
|
|
||||||
|
|
||||||
Terminal-based coding agent. Builds a codebase map before editing, makes changes
|
|
||||||
directly, and auto-commits to git with readable messages. Every change is one
|
|
||||||
`git revert` away. Supports 100+ languages; connects to any Ollama-served model
|
|
||||||
via the OpenAI-compatible API. Best for terminal-first developers who want
|
|
||||||
version-controlled agent interactions. Does not do inline autocomplete.
|
|
||||||
|
|
||||||
### OpenCode — recommended for bot-bottle–style agent loops
|
|
||||||
|
|
||||||
Terminal-based coding agent with 15 built-in tools (bash execution, file read/write/edit,
|
|
||||||
grep, glob, web fetch, MCP support) and connections to 75+ model providers including
|
|
||||||
local Ollama models. This is the closest open-source equivalent to a Claude Code–style
|
|
||||||
plan → tool-call → execute → observe → loop. Native Ollama integration.
|
|
||||||
|
|
||||||
**Critical setup note:** Ollama defaults to a 4096-token context window, which is
|
|
||||||
completely insufficient for an agent loop carrying conversation history, tool schemas,
|
|
||||||
a system prompt, and code simultaneously. Configure at least 64K tokens explicitly
|
|
||||||
in the model's context settings.
|
|
||||||
|
|
||||||
### Cline — agentic VS Code assistant
|
|
||||||
|
|
||||||
VS Code extension that operates as an autonomous agent: plans, edits files, runs commands
|
|
||||||
in a loop, connects to Ollama's local endpoint. Compared to OpenCode it lives inside the
|
|
||||||
IDE rather than the terminal; compared to Continue.dev it is a full agent rather than a
|
|
||||||
plugin. Its system prompt overhead is higher (~7,000–10,000 tokens) than minimal harnesses.
|
|
||||||
|
|
||||||
### Open WebUI / Jan / LM Studio — chat UIs, not coding harnesses
|
|
||||||
|
|
||||||
These are browser or desktop chat interfaces useful for ad-hoc conversations (explaining
|
|
||||||
APIs, drafting documentation, exploring ideas) but without IDE integration, autocomplete,
|
|
||||||
or git integration. LM Studio offers the smoothest onboarding (visual model browser with
|
|
||||||
VRAM estimates). Jan is the most privacy-auditable (fully open-source, Apache 2.0, no
|
|
||||||
telemetry). Neither is a replacement for a coding harness.
|
|
||||||
|
|
||||||
### Harness comparison
|
|
||||||
|
|
||||||
| Harness | Type | Autocomplete | Agent loop | Ollama | Git integration |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| Continue.dev | IDE plugin | Yes (FIM) | Basic | Native | No |
|
|
||||||
| Aider | Terminal agent | No | Multi-turn | Via API | Auto-commit |
|
|
||||||
| OpenCode | Terminal agent | No | Full tools | Native | Via bash |
|
|
||||||
| Cline | IDE agent | No | Full tools | Via API | Via bash |
|
|
||||||
| Open WebUI | Chat UI | No | No | Native | No |
|
|
||||||
| Jan | Chat UI | No | No | Native | No |
|
|
||||||
|
|
||||||
For a bot-bottle workflow (an isolated sandbox running an agentic loop with tool access),
|
|
||||||
**OpenCode** is the closest open-source match. For an IDE-first developer who wants
|
|
||||||
autocomplete + chat, **Continue.dev + Qwen2.5-Coder 7B** is the recommended pair.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 3. Model selection: RTX 3070 (8 GB VRAM / 30 GB RAM)
|
|
||||||
|
|
||||||
### VRAM hard limits at Q4_K_M quantization
|
|
||||||
|
|
||||||
| Model size | Approx. VRAM (Q4_K_M) | Fits in 8 GB? | Tokens/sec (RTX 3070) |
|
|
||||||
|---|---|---|---|
|
|
||||||
| 3–4B | 2.5–3.5 GB | Yes, with headroom | 60–90 |
|
|
||||||
| 7–8B | 5–6 GB | Yes | 35–55 |
|
|
||||||
| 12–14B | 7.5–9 GB | Edge / RAM offload | 8–18 |
|
|
||||||
| 22B+ | 14+ GB | No | — |
|
|
||||||
|
|
||||||
The RTX 3070 has high memory bandwidth for its VRAM tier and consistently outperforms
|
|
||||||
the newer RTX 4060 Ti on token generation speed. Bandwidth matters more than raw compute
|
|
||||||
for inference.
|
|
||||||
|
|
||||||
### Does Gemma 4 exist?
|
|
||||||
|
|
||||||
Yes. Google released **Gemma 4** on 2 April 2026 (Apache 2.0). The family includes
|
|
||||||
E2B (2B), E4B (4B), a 26B MoE, and a 31B Dense. A 12B multimodal variant was announced
|
|
||||||
2026-06-04. The 31B scores 80.0% on LiveCodeBench v6 — a major jump from Gemma 3 27B
|
|
||||||
at 29.1%. However, only the E4B fits comfortably within 8 GB VRAM:
|
|
||||||
|
|
||||||
| Variant | VRAM (approx.) | Fits? |
|
|
||||||
|---|---|---|
|
|
||||||
| Gemma 4 E2B | ~2 GB | Yes |
|
|
||||||
| Gemma 4 E4B | ~5 GB | Yes |
|
|
||||||
| Gemma 4 12B | ~8–9 GB (Q4) | Edge |
|
|
||||||
| Gemma 4 26B MoE | 14–18 GB | No |
|
|
||||||
| Gemma 4 31B Dense | ~20 GB | No |
|
|
||||||
|
|
||||||
### Model-by-model evaluation
|
|
||||||
|
|
||||||
**Qwen2.5-Coder 7B — primary recommendation**
|
|
||||||
|
|
||||||
The strongest purpose-built coding model that fits fully within 8 GB VRAM. Leads
|
|
||||||
HumanEval among 7–8B-class models. Strong on Python, JavaScript, TypeScript. Has
|
|
||||||
FIM (fill-in-the-middle) support for inline autocomplete. 35–55 tok/sec on RTX 3070.
|
|
||||||
|
|
||||||
```
|
|
||||||
ollama pull qwen2.5-coder:7b
|
|
||||||
```
|
|
||||||
|
|
||||||
**Qwen2.5-Coder 14B — secondary, with RAM offloading**
|
|
||||||
|
|
||||||
At Q4_K_M this needs ~8.7 GB, just over the 8 GB limit. With 30 GB system RAM, Ollama
|
|
||||||
automatically offloads the overflow layers to CPU. Performance drops to ~8–18 tok/sec
|
|
||||||
versus 35–55 tok/sec for the 7B fully in VRAM. Quality is noticeably better for complex
|
|
||||||
multi-file reasoning. Viable for chat-based coding tasks where quality matters more than
|
|
||||||
speed; too slow for live autocomplete. Keep context window at 8K tokens to minimize
|
|
||||||
VRAM pressure during offloaded inference.
|
|
||||||
|
|
||||||
```
|
|
||||||
ollama pull qwen2.5-coder:14b
|
|
||||||
```
|
|
||||||
|
|
||||||
**Gemma 4 E4B (~5 GB VRAM)**
|
|
||||||
|
|
||||||
Fits comfortably with 3 GB to spare. Strong on reasoning, multimodal, and general-purpose
|
|
||||||
tasks. Less specialized for coding than Qwen2.5-Coder 7B. Good choice for one model that
|
|
||||||
covers coding + general reasoning + image analysis. The E4B outperforms Gemma 3 equivalents
|
|
||||||
significantly on coding benchmarks.
|
|
||||||
|
|
||||||
```
|
|
||||||
ollama pull gemma4:e4b
|
|
||||||
```
|
|
||||||
|
|
||||||
**Phi-4 Mini 3.8B (~3 GB VRAM)**
|
|
||||||
|
|
||||||
Best reasoning-per-VRAM model; leaves ~5 GB free for other applications. Strong on math,
|
|
||||||
logic, and structured output. Good for agentic sub-tasks requiring tight reasoning. Not the
|
|
||||||
strongest at raw code synthesis but excellent for reasoning-heavy parts of a coding loop.
|
|
||||||
Viable as the autocomplete model in a two-model Continue.dev setup.
|
|
||||||
|
|
||||||
```
|
|
||||||
ollama pull phi4-mini
|
|
||||||
```
|
|
||||||
|
|
||||||
**DeepSeek-R1 8B (~5–6 GB VRAM)**
|
|
||||||
|
|
||||||
Strong reasoning model for logic-heavy code (algorithms, correctness proofs). The full
|
|
||||||
DeepSeek-Coder-V2 (236B MoE) is impractical here — only the 8B distilled variants are
|
|
||||||
relevant. Outperforms Gemma 4 E4B on reasoning-heavy benchmarks; weaker on raw code
|
|
||||||
generation than Qwen2.5-Coder 7B.
|
|
||||||
|
|
||||||
**Codestral — not viable at 8 GB**
|
|
||||||
|
|
||||||
The top FIM autocomplete model on HumanEval-FIM benchmarks, but requires 12–16 GB VRAM
|
|
||||||
minimum. Not an option here. Worth revisiting if upgrading to a 12 GB+ card (RTX 4070
|
|
||||||
Super or newer).
|
|
||||||
|
|
||||||
### RAM offloading: does 30 GB help?
|
|
||||||
|
|
||||||
Yes, meaningfully. Ollama automatically splits layers between GPU and system RAM when
|
|
||||||
VRAM is exceeded. With 30 GB RAM, models up to ~14B at Q4_K_M run with partial offloading.
|
|
||||||
The tradeoff is a 2–5× throughput penalty (8–18 tok/sec vs 35–55 tok/sec). Acceptable
|
|
||||||
for batch tasks (reviewing a PR, generating an algorithm); too slow for live autocomplete.
|
|
||||||
|
|
||||||
### Recommended setup
|
|
||||||
|
|
||||||
**Autocomplete (fast, always-in-VRAM):** `qwen2.5-coder:7b`
|
|
||||||
- Configure in Continue.dev as the tab-completion model
|
|
||||||
- FIM-capable; 35–55 tok/sec; fits with 2–3 GB VRAM to spare
|
|
||||||
|
|
||||||
**Chat / agent loop (quality-first):** `qwen2.5-coder:14b` or `gemma4:e4b`
|
|
||||||
- 14B for strongest multi-file coding; expect 8–18 tok/sec with RAM offload
|
|
||||||
- Gemma 4 E4B if you want vision + general reasoning + coding in one model; ~60 tok/sec
|
|
||||||
|
|
||||||
**Two-model Continue.dev config (lower VRAM pressure):**
|
|
||||||
`phi4-mini` (autocomplete) + `qwen2.5-coder:7b` (chat) — both fit simultaneously with
|
|
||||||
~1–2 GB to spare, keeping the OS and IDE from contending for VRAM.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Sources
|
|
||||||
|
|
||||||
- [Ollama on Proxmox: GPU Passthrough for LXC and VM AI Workloads](https://linuxprofessional.ie/article.php?slug=ollama-proxmox-gpu-passthrough-lxc-vm)
|
|
||||||
- [Run Ollama with NVIDIA GPU in Proxmox VMs and LXC containers](https://www.virtualizationhowto.com/2025/05/run-ollama-with-nvidia-gpu-in-proxmox-vms-and-lxc-containers/)
|
|
||||||
- [Ollama Performance Tuning: Getting Maximum Speed from Local LLMs](https://dasroot.net/posts/2026/01/ollama-performance-tuning-gpu-acceleration-model-quantization/)
|
|
||||||
- [Pros and Cons: Containerized Ollama vs. Local Setup](https://alain-airom.medium.com/pros-and-cons-using-containerized-ollama-vs-local-setup-d9bdf225bbb5)
|
|
||||||
- [Best Local Coding Models Ranked: Every VRAM Tier (2026)](https://insiderllm.com/guides/best-local-coding-models-2026/)
|
|
||||||
- [Best Local LLMs for RTX 4060, RTX 3070, and RTX 5060](https://aiagentskit.com/blog/best-local-llms-rtx-4060-3070-5060/)
|
|
||||||
- [Best Local LLMs for 8GB VRAM: Real Hardware Benchmarks (2026)](https://localllm.in/blog/best-local-llms-8gb-vram-2025)
|
|
||||||
- [Self-Hosted AI Coding Agent: Ollama + Continue + Open WebUI Setup in 2026](https://www.web3aiblog.com/blog/self-hosted-ai-coding-agent-ollama-continue-2026)
|
|
||||||
- [Best Local-First AI Coding Tools 2026: 14 Compared](https://nimbalyst.com/blog/best-local-first-ai-coding-tools-2026/)
|
|
||||||
- [OpenCode + Ollama: Private Local AI Coding Agent Setup](https://lushbinary.com/blog/opencode-ollama-local-ai-coding-privacy-guide/)
|
|
||||||
- [Gemma 4: Google DeepMind](https://deepmind.google/models/gemma/gemma-4/)
|
|
||||||
- [Running Gemma 4 Locally: VRAM Requirements](https://knightli.com/en/2026/05/01/gemma-4-local-vram-quantization-table/)
|
|
||||||
- [Phi-4 Mini vs. Gemma 3 vs. Qwen 2.5: Best SLM for Coding Tasks in 2026](https://botmonster.com/ai/phi-4-mini-vs-gemma-3-vs-qwen-25-best-slm-coding-2026/)
|
|
||||||
- [Qwen2.5-Coder 14B VRAM Requirements Guide](https://willitrunai.com/blog/qwen-2-5-coder-14b-vram-requirements)
|
|
||||||
- [Comparing AI Harnesses: OpenCode, Ollama, LM Studio, Claude Code, Open WebUI, and VS Code](https://jace.pro/blog/comparing-ai-harnesses-opencode-ollama-lm-studio-claude-code-open-webui-and-vs-code/)
|
|
||||||
@@ -5,7 +5,7 @@ model: opus
|
|||||||
bottle: dev
|
bottle: dev
|
||||||
skills:
|
skills:
|
||||||
- init-prd
|
- init-prd
|
||||||
git-gate:
|
git:
|
||||||
user:
|
user:
|
||||||
name: implementer-bot
|
name: implementer-bot
|
||||||
email: eric+implementer@dideric.is
|
email: eric+implementer@dideric.is
|
||||||
|
|||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user